From Accuracy to Diversity in Product Recommendations: Relationship Between Diversity and Customer Retention

Transcription

1 From Accuracy to Diversity in Product Recommendations: Relationship Between Diversity and Customer Retention Sung-Hyuk Park and Sang Pil Han ABSTRACT: Recommending diverse products to consumers is a new strategy for the next generation of recommender systems. However, no existing studies have empirically identified the impact of product diversity on consumer behavior. The aim of this study is to explain how product category diversity affects customer retention rates. To answer this research question, we examine how the number of product categories purchased by consumers is related to customer retention rates at a large digital content distributor. We use panel data consisting of product characteristics, purchase transactions, and customer retention rates from the company. Through segment-level and individual-level panel data analyses, we find that purchase quantity is positively associated with customer retention rates, and that variety of purchased digital content categories is positively associated with customer retention rates. That is, customers who have purchased digital content from multiple categories are more likely to stay longer than those who purchased digital content from a single category or from fewer categories. Put differently, as a complement to the conventional wisdom that just recommending products with similar features that a customer values highly (i.e., similar content from the same category) is important, our results imply that recommending products with different features (i.e., different content across different categories) is also important. KEY WORDS AND PHRASES: Cross-selling, customer retention, econometrics, product diversity, recommender systems. In today s Long Tail markets, the main effect of filters is to help people move from the world they know ( hits ) to the world they don t ( niches ) via a route that is both comfortable and tailored to their tastes. [5, p. 109] Information technology such as recommender systems (on the software side) and smart devices (on the hardware side) have positively influenced consumers usage of niche products by lowering their search costs [9, 16]. The recommender systems research community, which has traditionally accepted an accuracy-based recommendation strategy, is paying increasing attention to diversity (i.e., offering not merely a few best-selling products to improve recommendation accuracy but also many niche products, according to the long-tail sales strategy) as a key feature in addition to accuracy-oriented predictions in real-world recommendation scenarios [1, 26, 29, 36, 37]. An earlier version of this paper was presented at the 2012 International Conference on Electronic Commerce. The authors are grateful to the special issue editors, Karl Reiner Lang and Ting Li, and the anonymous reviewers for their valuable suggestions and extremely helpful comments. Any remaining errors are the authors. The work described in this paper was partially supported by a research fund from Recobell. International Journal of Electronic Commerce / Winter , Vol. 18, No. 2, pp Copyright 2014 M.E. Sharpe, Inc. All rights reserved. Permissions: ISSN (print) / ISSN (online) DOI: /JEC park.indd 51 11/14/ :16:23 AM

2 52 Park and Han One of the claims of these recent studies on the impact of information technology on consumers product purchases is that the consumers who have purchased more diverse (or less concentrated) products should perform better than those who have purchased concentrated products from fewer categories [1, 9, 11, 30, 31]. In contrast, most of the traditional studies on recommender systems have focused on the design of recommendation engines to predict the most appropriate (or accurate) next product for a given user. For example, the well-known Netflix contest that awarded a US$1 million prize to the eventual winner showcased a real-world, advanced recommendation technology competition to predict a user s preference (i.e., the next item that will be ordered) with high accuracy [6, 7]. Moreover, although recent studies have highlighted the importance of sales diversity, no study has examined the economic impact of product diversity on consumer behavior. If product diversity positively influences consumer behavior, for example, in terms of consumer retention rates or increased consumption, then it allows marketing managers to consider not only an accuracy-based recommendation strategy (i.e., short-term performance) but also a product-diversity-based recommendation strategy (i.e., relatively long-term performance). In part because of the adoption of recommendation systems by major online retailers, the last ten years have seen a substantial increase in the number of products available in both online markets and offline markets (e.g., [10]). Consumers have also taken advantage of the increased number of products that previously had not been made widely available. New marketing strategies may need to be implemented in this environment in lieu of, or in addition to, traditional marketing methods. This requires a deeper understanding of the impact of product diversity on consumer behavior, such as retention. In customer relationship management, consumers retention rates can be a salient determinant to evaluate consumers long-term performance [18]. In addition, this metric is directly associated with consumers lifetime monetary value. In marketing, customer lifetime value (CLV) has been a well-defined, long-term customer profitability metric that effectively calculates an individual customer s worth as a monetary value [14, 20]. CLV is defined as the sum of the net present value of a customer s future cash flows. This metric has also been used to manage customer relationships as a firm s core asset. In most CLV models, the CLV is positively correlated with customer retention rates [13]. Thus, if we can show a positive relationship between product diversity and customer retention rates, then we believe that this study will bridge the gap between diversity and consumers long-term performances, such as CLV. This study focuses on previously unexplored research questions: How is product diversity associated (e.g., negatively or positively) with customer retention likelihood? Moreover, does a diversity-based recommendation strategy perform better than a concentrated product-recommendation strategy in terms of customer retention rates? To answer these questions, we perform an in-depth descriptive analysis as well as an individual-level and aggregated segment-level econometric model analysis. We analyze a large panel data set (6.74 million paid content-transaction data records across approximately 35,000 customers for the most recent 21-month period) consisting of product 02 park.indd 52 11/14/ :16:23 AM

3 International journal of electronic commerce 53 category, revenues, and customer churn (or retention) incidence information collected from a nationwide online e-book company in East Asia. The results allow us to explore the impact of product category diversity on customer retention rates. We found that as the number of purchased product categories increased, customer retention rates dramatically increased, and the effectiveness (e.g., changes in retention rates) from an increase in diversity was larger than the effectiveness from an increase in the number of purchases within the same (single) product category. In this study, we examined the effect of consumer choices for new versus existing categories. Although this study does not clearly establish a causal relationship between product category diversity and customer retention rates, we believe that we found strong empirical evidence to suggest that customers who purchase more products from diverse categories exhibit a higher retention rate than others, even when the user s shopping preferences (e.g., average price, category concentration level) and demographic information (e.g., age, gender, prior experience) are controlled. In addition, based on the results of the econometric model analysis, we conducted some scenario analyses to compare the changes in retention rates in each of two different recommendation strategies: (1) when customers purchase a new item in existing categories versus (2) when customers purchase a new category item. Our results show that when customers purchase a new category item, increasing category diversity, their retention rates dramatically increase. Our results are consistent with existing theoretical studies [9, 26] and support their claim that product diversity is positively associated with customer retention rates. More specifically, the impact of a one-unit increase in product category diversity (i.e., a new category) is larger than a one-unit increase in the number of existing product category purchases. As for managerial implications, we believe that marketing managers should adopt the diversity-based recommendation strategy to improve their customers retention rates and future profits. The remainder of this paper is organized as follows. The next section introduces an overview of related work. The third section describes the data set used in this study with summary statistics. The fourth section explains econometric models and data analysis results with key findings, and the fifth explains the scenario analysis results with implications. The last three sections conclude with a discussion of managerial implications, limitations, and suggestions for future research. Literature Review This study is related to the active research area in recommender systems (see [2, 34] for a review). Traditional recommender systems have used data on consumer purchases, product ratings, and consumer profiles to predict the preference that a user gives to a product that the user has not yet considered. Recommendations do influence consumer choices, and even online recommendations can be more influential than in-person recommendations [35]. These 02 park.indd 53 11/14/ :16:23 AM

4 54 Park and Han systems are standard marketing components at companies such as Amazon, Netflix, Travelocity, and many others [22, 25]. There is an argument concerning whether recommender systems positively influence the diversity of sales. Some scholars argue that recommender systems can lead to a reduction in sales diversity, as they only reinforce the popularity of already-popular products [3, 11, 27]. Others believe that because of lower search costs, users tend to discover new (or unseen products) and more diverse products by using these systems [8, 9, 33]. Whereas existing studies examine how recommender systems affect immediate consumer choices, our interest is to examine the impact of product diversity on customer retention rates. In addition, data sets used in prior studies did not include individual-level data. For example, in recent studies, simulation experiments [11] and product-level data analyses [9] were conducted to investigate the economic impact of recommender systems on sales diversity. We use individual-level data collected from a nationwide electronic commerce company to show the impact of sales diversity on customer retention rates at the individual level. In addition, our study is related to the emerging stream of literature on cross-selling efforts through recommender systems. For example, Pathak et al. [31] showed that recommender systems help to reinforce the long-tail phenomenon in electronic commerce and that obscure recommendations positively affect cross-selling. This is because the predictable purchase cycle of customers (e.g., some items are purchased before others) allows companies to cross-sell additional items to their customers [24]. Some studies have empirically examined how product diversity resulting from cross-selling is positively associated with customer retention rates. According to Kamakura et al. [17], customers who purchase multiple-category products are more willing to stay with the company because as the number of points where the company and customer connect increases, the customer s switching cost increases also. In addition, they argued that the customer company relationship is developed through customers endeavors to explore various products from a company and is further reinforced by the company s cross-selling efforts. In marketing, some models have indicated that customer retention rates are associated with the customer s long-term profitability. According to these models, if the repeated business generates a positive profit, then customer retention rates positively influence long-term profitability. Since Kotler [20] introduced the concept of long-term customer profitability as the net present value of a customer s present and future profits, CLV has been widely accepted by marketing practitioners and scholars. For example, CLV has contributed to solving business problems such as marketing resource allocation (e.g., identifying more profitable customer segments and providing marketing offers) [18, 32] and company valuation [13]. As explained, CLV is positively associated with retention rates under the assumption that additional business generates a positive profit (i.e., excluding sales that generate losses). Therefore, if we show that product diversity is positively associated with customer retention rates, it would be evidence to support the importance of diversity-based recommendations in terms of profitability. In short, this literature review reveals three critical issues. First, the recommender system literature has focused on system design issues, and a 02 park.indd 54 11/14/ :16:23 AM

5 International journal of electronic commerce 55 few studies have investigated the impact of recommender systems on sales diversity, although not with individual-level data sets. Second, cross-selling literature has successfully highlighted that as customers purchase diverse products, their customer retention time increases, but no empirical evidence for this has been provided. Third, although marketing studies have identified the relationship between the retention rate and CLV, the relationship between product diversity and retention rates (and customer long-term performance) has not yet been examined. Data Description We collected consumer purchase data from one of the largest e-book companies in East Asia. As discussed in prior studies on the (electronic) book market, books represent a commodity product, which means that brand- and productspecific effects are minimal [12, 16]. Since e-book prices are relatively inexpensive and more than one hundred thousand e-books have been published in the company s online market, we were able to have a variety of products in our panel data set. We believe that our data set is a good sample with which to evaluate the main effects (i.e., category diversity and purchase quantity effects) in our research questions because brand- and product-specific effects are negligible, allowing us to estimate the effects of the two main factors more precisely. Moreover, since e-books in an online market are much more homogeneous than many real products in an offline market (e.g., millions of different products offered in a department store), we believe that using e-book data is appropriate to show whether category diversity plays a key role in lowering consumers churn probability, even when a company sells homogeneous products or has less firm-level product diversity. Our data set includes about 6.74 million transaction records from 34,967 customers over a twenty-one-month period (from January 1, 2011, to September 31, 2012). We divide our sample into two subsamples (i.e., first-year and second-year sample data). We use the first-year (from January 1 to December 31, 2011) data to derive our main variables, category diversity and purchase quantity, similar to a prior study [29]. Specifically, category diversity denotes the number of different product categories purchased among the eighteen predefined top-level product categories that the particular e-book business carries (business, cartoon, education, essay, fantasy, health, foreign language, kids, living, novel, magazine, religion, romance, self-development, science, social science, and travel). These eighteen top-level product categories were created by the company, adopting a book industry standard structure. In this case, for example, a possible cross-selling campaign could be recommending a travel book to a self-development book buyer. Although some categories such as romance, novel, and self-development are more popular than other categories, our data set contains enough records to provide various category diversity values. During the study period, the e-book company used a recommendation strategy to show best sellers (on the top line) and selected newly released e-books (on the next line) to customers on the first page. This is a generally 02 park.indd 55 11/14/ :16:23 AM

6 56 Park and Han Table 1. Descriptive Statistics. Variable Mean SD Min. Max. Purchase quantity (number of e-books purchased during the first year) Category diversity (number of different book categories purchased during the first year) Retention status (1 = retention; 0 = churn [in the second year]) Age (years) Gender (1 = male; 0 = female) Days since subscribed (i.e., prior experience) ,235 Note: Number of observations: 34,967. used recommendation strategy for many online retailers. In this setting, some customers may buy different e-books depending on their needs, independent of any particular recommendation strategy being used. Next, we explain our key variables. Purchase quantity denotes the total number of purchase incidences during the first year. In our e-book setting, no one orders multiple items of the same e-book for himself or herself, and thus purchase quantity is defined as the frequency of the observed purchases for a user. Last, retention status indicates whether a customer churns (denoted by 0), meaning that the customer makes no purchase in the second year, or whether the customer is still active (denoted by 1), meaning that the customer makes at least one purchase during the second year. Table 1 summarizes the descriptive statistics of the variables used in this study. Using our e-book data set, we conduct a natural experiment. In this setting, the treatment assignment (e.g., recommending multiple-category products to a user) process was exogenous. In other words, we only analyzed the archival data, which means that we did not conduct a randomized field experiment consisting of treatment and control group test comparisons. We established that customers in the retention status = 0 group have similar profiles (e.g., age, gender, and tenure) to those of customers in the retention status = 1 group. The results of mean comparison tests (t-tests) are provided in Table 2. As summarized in Table 2, none of the t-statistics are statistically significant (at the 5 percent level), and the results support that customers in the two groups have no difference in terms of user profiles. Although the mean-comparison test results indicate that the demographic user profiles are not significant when we define our model, since user-specific variables, such as user s age or gender, and prior experiences are expected to be related to customer churn rates [4], we decided to control users demographic profiles to more precisely estimate the effect of category diversity and purchase quantity (i.e., how category diversity and purchase quantity influence customer retention rates). Table 3 illustrates how customer retention rates change as the number of product categories purchased increases. The key finding is that category diversity is positively associated with customer retention rates. For example, 02 park.indd 56 11/14/ :16:23 AM

7 International journal of electronic commerce 57 Table 2. Mean Comparison Tests. Control variable Group Gender Age Days since subscribed (prior experience) Retention status = (churn) Retention status = (retention) t-test result G1 G2 μ gender = μ gender G1 G2 μ age = μ age G1 G2 μ experience = μ experience p-value customers who purchased items from multiple categories are less likely to churn than those who purchased items from a single category. The last column in Table 3 shows that the customer retention rate increases from 24.4 percent to 37.0 percent (+12.6 percent) as the number of content categories purchased increases from 1 to 2. Moreover, from our calculation, the retention rates dramatically increase when the number of content categories is high (e.g., to 77.4 percent at 7, to 80.0 percent at 8, and to 87.2 percent at 9, the highest number of categories present in the data). This result qualitatively remains the same, even after controlling for the number of customers content purchase quantities. For example, when the purchase quantity is 2 (see column 2), the one-unit increase in content category diversity results in a retention rate increase of 2.3 percent (31.7 percent 29.4 percent), and when the purchase quantity is 5, the one-unit increase in category diversity from 1 to 2 results in a retention rate increase of 19 percent (48.6 percent 29.6 percent). In addition, Figure 1 illustrates that as customers purchase multiple-category items, their retention rates increase, and the marginal effects are more dramatic than when customers do not purchase multiple-category items. Hence, as a complement to the conventional practice of just recommending products with similar features that a customer values (i.e., products within the same category), our results imply that recommending products with different features (i.e., products across different categories) is important as well. This finding suggests that a systematic cross-selling strategy encouraging a user to purchase a variety of books from different product categories is effective in decreasing (increasing) the user s churn (retention) probability. Our results imply that recommending products that customers might otherwise not have purchased serves as an effective form of advertisement, increasing customer awareness about unfamiliar products [5]. This kind of recommendation may serve to reduce customer search costs and the uncertainty associated with the purchase of products that customers usually purchase from other retailers. Thus, this could result in an increase in the share-of-wallet for products from one retailer, possibly at the expense of other retailers. 02 park.indd 57 11/14/ :16:23 AM

8 58 Park and Han Table 3. Customer Retention Rate Changes by Category Diversity and Purchase Quantity. Purchase quantity (PQ = q) Category diversity (CD = d) P(Retention status = 1 Category diversity = d and Purchase quantity = q) P(RS = 1 CD = d) % 29.4% 31.0% 33.7% 29.6% 31.7% 34.4% 32.3% 23.5% 32.8% 24.4% % 34.7% 42.6% 48.6% 49.3% 37.7% 45.0% 50.0% 48.8% 37.0% % 48.3% 53.7% 55.3% 49.0% 56.9% 54.0% 55.0% 52.2% % 49.1% 57.8% 51.8% 60.0% 59.6% 62.2% 60.0% % 54.3% 53.3% 70.5% 70.8% 59.2% 67.8% 6 N/A N/A N/A 76.2% 43.3% 71.7% P(RS = 1 PQ = q) 18.7% 30.6% 35.1% 42.3% 45.4% 47.6% 44.3% 50.2% 41.4% 49.9% 33.24% Notes: Cells that have fewer than 30 cases have missing values (e.g., N/A). Each cell shows the conditional probability (e.g., P[retention status = 1 conditions]) of retention for a given condition described as category diversity = d and/or purchase quantity = q. 02 park.indd 58 11/14/ :16:23 AM

9 International journal of electronic commerce 59 Figure 1. Customer Retention Rate Comparisons: Category Diversity vs. Purchase Quantity The Impact of Category Diversity on Customer Retention Rate Our estimation approach is to identify the relationship between product diversity and customers long-term performance, such as retention rates. We first run a multiple linear regression of customer retention rates on purchase quantity and category diversity. To estimate the regression coefficients, we use customer segment level panel data for which each segment is defined by the number of purchases from different product categories and the total purchase amount during the first year. For example, the customers who had the same number of product categories and an equal number of total purchase amount during the first year were categorized into the same segment (e.g., each cell in Table 3). In addition, we computed the customers retention rate for each segment to set up a customer segment level panel data set. The model we estimated can be written as follows: RetentionRate i = β 0 + β 1 CategoryDiversity i + β 2 PurchaseQuantity i + ε i, where i denotes the customer segment index and ε i is an unobserved error term representing all causes of RetentionRate i other than our two main variables, CategoryDiversity i and PurchaseQuantity i. Table 4 summarizes the results from the linear regression. We analyze three different models that vary according to the parameter selection. All the models show consistent results: for example, our main variables, category diversity and purchase quantity, are statistically significant at the 5 percent level with no exception. Specifically, in Model 1, the impact of a one-unit increase in category diversity on customer retention rates is higher than the impact of a one-unit increase in purchase quantity on 02 park.indd 59 11/14/ :16:23 AM

10 60 Park and Han Table 4. Basic Model Linear Regression Results: Coefficients (Standard Errors). Variable Model 1 Model 2 Model 3 Constant 0.228** (0.046) 0.327** (0.033) 0.270** (0.050) Category diversity 0.031** (0.009) 0.041** (0.009) Purchase quantity 0.020** (0.007) 0.029** (0.007) F-statistic ** ** ** R Notes: Dependent variable: customer retention rates. ** p < 0.01; * p < customer retention rates. In addition, the regression coefficient of category diversity in Model 2 (the category diversity effect only) is higher than the regression coefficient of purchase quantity in Model 3 (the purchase quantity effect only). The results support that category diversity more strongly influences the customer retention rate than purchase quantity does. In addition, we analyze the individual-level panel data to increase the robustness of our results. Our segment-level panel data used in the first regression and the individual-level panel data are pooled as cross-sectional data. We define an extended model as shown below, and we adopt logistic regression analysis to estimate the regression coefficient. RetentionStatus j = β 0 + β 1 CategoryDiversity j + β 2 PurchaseQuantity j + γ 1 Price j + γ 2 CategoryConcentration j + γ 3 Age j + γ 4 Gender j + γ 5 ln(tenure i ) + ε j, where the dependent variable, RetentionStatus j, is a binary (e.g., 1: no churn during the second year; otherwise 0) variable and j denotes an individual user index. In addition, the extended model consists of not only the purchase quantity and category diversity variables but also average purchase price, level of category concentration, age, gender, and tenure variables, which control user-characteristic-specific effects. Particularly, the Price j variable is used to control for a user s size of purchase amount, and the CategoryConcentration j variable is designed to control for a user s level of category concentration (or shopping category preference), where category concentration shows whether a user has a per se shopping preference to buy multiple-category products or not (i.e., readers with wide interests vs. readers with specialized interests). For example, consider the following two users who have purchased ten items from two different categories: user A (five business books, five novels) and user B (nine business books, one novel). It is trivial that user B has a more concentrated shopping preference than user A, because user B has a clear category preference for business books (90 percent), whereas user A has no priority between business books (50 percent) and novels (50 percent). We adopt the well-known Gini impurity measure to define our CategoryConcentration j variable. Uncertainty measures such as the Gini impurity measure have been 02 park.indd 60 11/14/ :16:23 AM

11 International journal of electronic commerce 61 widely used to calculate weather a set of different items are unequally distributed or not in the field of data mining (e.g., classification and regression trees) [15, 30]. The Gini measure reaches its minimum (zero) when all items fall in a single case (or category), and it reaches its maximum (one) when all items are categorized into different cases. We defined our CategoryConcentration j variable as follows: CategoryConcentration j = 1 Gini impurity = 1 k CDj f k2, where k is the user j s category index, and CD j is the category diversity of the jth user. Thus, our measure reaches its maximum (one) when all the items fall in a single category. For example, user A s CategoryConcentration A = (5/10) 2 + (5/10) 2 = 0.5, and user B s CategoryConcentration B = (9/10) 2 + (1/10) 2 = Hence, user B has a higher concentrated category preference than user A. By controlling a user s level of category concentration, we believe that our advanced econometric model provides results that are more reliable. Model 1, as indicated in Table 5, reports the most comprehensive results. We include both the category diversity and purchase quantity terms to measure the main effects and user profiles (shopping preference and demographics) to control for user-characteristic-specific effects. The estimated regression coefficients of category diversity and purchase quantity are all positive, which means that a one-unit increase in either category diversity or purchase quantity positively influences customers retention rates. In addition, a one-unit increase in category diversity seems to have a much larger marginal effect than a oneunit increase in purchase quantity. For example, the marginal effect sizes of category diversity and purchase quantity are and 0.059, respectively. In terms of practical significance, our results strongly suggest that a sales diversity strategy plays a more effective role in customer retention than a simple repurchase strategy does. Interestingly, the regression coefficient for price is positive, which means that as a user purchases more expensive items, the user s survival rate (in the next year) increases accordingly. Moreover, the category concentration coefficient shows that as a user develops a highly concentrated category preference, the user s retention rate decreases. In summary, we can say that the ideal customer should exhibit larger category diversity with equally distributed category preferences. According to the variance inflation factor (VIF) diagnostic results, no explanatory variables exceeded the maximum acceptable value of 10 [28], which revealed no multicollinearity. In Models 2, 3, and 4, we checked the robustness of the main effects in Model 1 for alternative specifications. Models 2, 3, and 4 had fewer controls than our main model. First, in Model 2, the main effects do not change when the price control is omitted. Second, in Model 3, in which we exclude the category concentration control, the category diversity effect is much higher than the effect in Model 1. Other results remain qualitatively consistent main results (e.g., Model 1 and Model 2). Third, when we exclude the three user demographic variables in Model 4, the main effects are consistent with the main effects in Models 1, 2, and 3. Therefore, we believe our results strongly support that (1) both category diversity and purchase quantity positively 02 park.indd 61 11/14/ :16:23 AM

12 62 Park and Han Table 5. Logistic Regression Results with Robustness Checks. Variable Model 1 Extended model Model 2 No price Model 3 No category concentration Model 4 No demographic profiles Model 5 No controls Model 6 Resampling with new subscribers Constant (0.150) Category diversity 0.259** (0.026) Purchase quantity 0.059** (0.002) Price 0.076** (0.006) Category concentration 0.588** (0.101) Age 0.018** (0.001) Gender 0.137** (0.027) ln (Tenure) 0.077** (0.012) Goodness of fit (Hosmer & Lemeshow) R 2 (Cox & Snell) (0.146) 0.260** (0.026) 0.053** (0.002) 0.645** (0.101) 0.017** (0.001) 0.167** (0.027) 0.087** (0.012) 0.994** (0.083) 0.395** (0.013) 0.056** (0.002) 0.077** (0.006) 0.018** (0.001) 0.140** (0.027) 0.074** (0.012) 1.306** (0.123) 0.259** (0.026) 0.056** (0.002) 0.076** (0.006) 0.486** (0.101) 1.504** (0.023) 0.389** (0.012) 0.047** (0.002) 0.758** (0.171) 0.265** (0.030) 0.061** (0.003) 0.069** (0.006) 0.704** (0.114) 0.019** (0.002) 0.078* (0.030) 0.280** (0.015) 81.01** 83.84** ** ** ** 80.46** Notes: Dependent variable: binary retention status variable (1 = retention; 0 = churn)..regression coefficients (and standard errors) are summarized..** p < 0.01; * p < park.indd 62 11/14/ :16:23 AM

13 International journal of electronic commerce 63 influence customer retention rates and (2) the category diversity effect is much larger than the purchase quantity effect. Consequently, we also checked the robustness of the main effects in Model 1 for an alternative specification that had no control variables. As shown in Model 5 our results do not change from those in Model 1 if we exclude both shopping preference controls for average shopping price and category concentration and demographic controls for age, gender, and ln(tenure). Moreover, in Model 6 we show that our results are robust to our definition of individual user. We selected sample users as new subscribers who registered and purchased at least one book during the first year. This allowed us to check whether our results are robust to our definition of retention status as well. As for survival analysis, time to event is not observable [19]. For example, some users who subscribed before the beginning day of our panel data (i.e., first year) had missing information, and this might cause left censoring. However, in Model 6 we used resampled data with new subscribers who had no missing information in the past. The regression coefficients in Model 6 show the same pattern as before (i.e., with or without left censoring). Thus, our analysis results are robust under different assumptions. Scenario Analyses To compare customer retention rate when customers purchase a product in an existing category with the retention rate whey they purchase a product in a new category, we carry out a scenario analysis. Since user demographic variables (e.g., age and gender) are related to customer churn rates [4], we divided a market into several homogeneous submarkets [21] and followed the same approach. In addition, price is a key factor in economics and a representative variable that can be used for market segmentation. Since our data set does not include users income history, we use price instead of income. We define 12 customer segments wherein each segment is a unique combination of age group (e.g., 20s, 30s, or 40s), gender (male vs. female), and average purchase price (i.e., high vs. low). This approach is practical and useful because it provides intuitive and easily understandable prediction results across different user segments, so marketing managers can see how they make money from customers in each segment when they recommend a new category product to customers. In addition, the marketing managers can choose the more efficient customer segments when they allocate their marketing budget to lower their customers churn probability. Columns 1, 2, and 3 in Table 6 show how we defined the twelve customer segments. For each segment, we randomly selected one thousand new users who were not selected in the modeling procedure. First, we calculated an individual user s retention rate by multiplying the user s recent transactions and the regression coefficients of Model 1 in Table 5 (i.e., the main model). Then, we calculated the average customer retention rates for the selected one thousand individuals in each segment. Column 4 in Table 6 shows the average customer retention rate. Next, we constructed two scenarios: (1) customers purchase an item, but in an existing category, and (2) customers purchase an item in a new 02 park.indd 63 11/14/ :16:23 AM

14 64 Park and Han Table 6. Scenario Analysis Results (Example). Average customer retention rate at time T (observed) Average customer retention rate at time T + 1 (predicted) (1) (2) Age group Gender Average price P(retention PQ =q, CD = d, ϕ) Purchasing an item in existing categories: P(retention PQ = q + 1, CD = d, ϕ) Purchasing a new category item: P(retention PQ = q + 1, CD = d + 1, ϕ) 20s Male High 40.06% 41.39% 47.36% Low 34.81% 36.06% 41.73% Female High 35.62% 36.89% 42.69% Low 34.98% 36.15% 41.50% 30s Male High 38.23% 39.52% 45.36% Low 33.60% 34.78% 40.16% Female High 34.38% 35.61% 41.25% Low 37.49% 38.57% 43.52% 40s Male High 36.40% 37.65% 43.36% Low 31.46% 32.57% 37.72% Female High 32.41% 33.60% 39.07% Low 36.27% 37.29% 41.99% Average 35.48% 36.67% 42.14% Note. ϕ denotes user-specific conditions consisting of shopping preferences and demographic profiles. 02 park.indd 64 11/14/ :16:23 AM

15 International journal of electronic commerce 65 category. We then predicted the customer retention rate for each scenario (see column 5 of Table 6). The scenario analysis results report the initial (before purchasing a product) and the improved (after purchasing a product) customer retention rates (see Table 6). Without any exceptions, the improvement ratio when users purchase a new category item (+6.66 percent from T: percent to T + 1: percent) is significantly higher than the improvement ratio when users purchase a new item, but in existing categories (+1.19 percent from T: percent to T + 1: percent). These findings are consistent with our results from the econometric model analyses. The dramatic improvement in customer retention rate clearly supports the proposition that purchasing a new category item can be an effective strategy to improve a company s long-term performance. As mentioned previously, since customer profitability is directly associated with retention rate, companies earn more profit as their customer retention rate increases. According to our calculation (see the last row in Table 6), companies can expect more profit when their customers purchase a new category item than when customers purchase an existing category item. In addition, regarding marketing resource allocation, marketing managers should allocate their marketing budget toward increasing the retention rate of the relatively poorly performing segments (e.g., 40s, male, low CLV) as their first priority. We believe that if marketing managers focus on offering a selection of personalized new category items to their customers by applying category-level association and rule mining techniques for cross-selling (e.g., see [23]), they can expect higher profits. Discussion and Implications Through econometric analyses, we find that the impact of a one-unit increase in category diversity on retention rates is much larger than the impact of a one-unit increase in purchase quantity on retention rates. Put differently, as a complement to the conventional strategy of recommending products with similar features (i.e., products within the same category), our results imply that recommending products with different features (i.e., products across different categories) is important as well. Moreover, through additional scenario analyses, we also show that recommending a new category item is much more effective than recommending an existing category item in improving customer retention rates (and therefore long-term profitability). Hence, we suggest that recommender designs that incorporate explicit promotion of product category diversity should be considered by digital content providers. For example, in a recent empirical study, Brynjolfsson et al. [9] found that consumers usage of recommendation or search tools led to more demand for niche products, earning companies more revenue from the extra demand. As for long-term profit, by the theoretical definition of CLV, the customer retention rate is positively associated with CLV. Although our study does not provide clear evidence of the causal relationship between category diversity and customer retention rates, if buying from different categories leads to a higher retention rate, as we conclude in our analysis results, then a 02 park.indd 65 11/14/ :16:23 AM

16 66 Park and Han new profitable path from category diversity to CLV (through retention rates) can also be considered. Under the assumption that buying different categories leads to higher retention rates and that this benefit is directly converted into future profits, we believe that category diversity can positively influence not only customer retention rates but also long-term profits (or CLV). Existing studies have focused on accuracy-based recommendations, and it is well known that these approaches are effective in maximizing short-term performance. In contrast, this study highlights how diversity-based recommendations are effective in improving customer retention rates. In addition, the product diversity-based strategy should also be effective in terms of companies potential long-term profits. This study provides significant managerial implications for the evolving marketing strategies in electronic commerce. First, marketing managers aiming to increase long-term profits by reducing customer churn through crossselling efforts involving offering diverse products can use our research model to build their marketing strategy. In marketing, RFM (recency, frequency, and monetary) measures are widely accepted for evaluating a customer based on purchase history. We believe that category diversity can be used as a simple and well-defined measure to predict a customer s retention (or churn) likelihood, which is directly (and positively) associated with the customer s lifetime value. Since the hypothesis that the product diversity effect is much larger than the product quantity (or frequency) effect is supported, we suggest that marketing managers should not only use the traditional RFM model but also consider extensions to an RFDM (recency, frequency, diversity, and monetary) or RDM (recency, diversity, and monetary) model. In addition, if marketing managers put more effort into making their customers select from more diverse product categories, they can expect better performance. Regarding cross-selling, as customers purchase more diverse category products, companies have more opportunities to understand their customers needs and will be in a better position than their competitors [17]. Thus, diversity-based recommendations allow companies to learn more about consumer behavior, including personalized preferences, when the customers accept their offers. For example, an e-book company that can provide more specific, individualized offers to a customer who has multiple interests (e.g., fantasy, romance, and science fiction categories) can gain competitive advantage because the company can satisfy such a customer better than other companies. Similarly, if e commerce companies other than e book companies learn about the multiple needs of their customers, rather than single needs, they will also gain competitive advantage. According to our findings, sales (category) diversity positively influences customer retention rates. However, this does not mean that companies should have recommender systems that only present diverse products to their customers when they have recommendation opportunities. For example, Amazon.com provides hybrid recommendations blending accuracy-based and diversity-based predictions. The company already has expertise in mixing two opposite strategies according to given user characteristics and product information. When a customer has a specific need to purchase an MP3-player, for example, Amazon shows MP3-player products to the customer in terms of 02 park.indd 66 11/14/ :16:23 AM

17 International journal of electronic commerce 67 accuracy-based recommendations. However, when a customer has less specific needs, the company shows new offers with diverse products predicated on the customer s shopping behavior data. The key message here is that if more practitioners realize the importance of product category diversity when they design and implement their recommender systems, they can create better sales performance with more flexible recommendation strategies. For example, consider an online fashion shopping mall that has a goal to enhance the relationship between the company and its most profitable customers. These customers have a higher purchase quantity for existing category products. It could therefore be a good strategy to provide something like a discount coupon to be used only when the customers purchase a new category product (e.g., shoes or accessories). By making such an offer on a new product that the customer has not purchased in the past, the company obtains more detailed customer information, and the data can be used to provide more specific offers in the future. In addition, these customers would likely have better retention rates than customers who have not accepted the new offers. Interestingly, some scholars have designed and implemented a diversitybased recommender system that provides diverse products to users while keeping accuracy loss to a minimum. In one study, the authors adopted a strategy to lower the probability of showing best-seller products in the top recommendation ranking so that more diverse (and less popular) products could be displayed on the top page [1]. We strongly agree that this idea could be a great way to develop a diversity-based recommender system. According to Vargas and Castells [36], inherent uncertainty is involved in userpreference-based predictions because the prediction results (recommended items) are based on incomplete and implicit evidence provided by users. Thus, they suggested that offering a narrow array of products is generally not a good approach. Moreover, we believe that when online retailers adopt a diversity-based recommendation strategy, they can enhance their business by expanding into long-tail sales [11], thereby effectively increasing customer retention rates as well. Limitations and Future Research Our study has several limitations. We suggest that diversity-oriented recommendations might be more effective than traditional accuracy-oriented ones in terms of increasing customer retention rates. However, technically, this is not fully implied in our study. Thus, we would like to encourage others to formally test this hypothesis in future studies. For example, by systematically offering diversity-based recommendations to the customers in a treatment group and by offering accuracy-based recommendations (i.e., the traditional approach), one could explore the possible existence of a causal relationship between product category diversity and customer retention rates. Specifically, since we did not have such an opportunity with our research design (field experiment), we could not claim any causal relationship because consumers with a higher 02 park.indd 67 11/14/ :16:23 AM

18 68 Park and Han repurchase intention also tend to purchase products that are more diverse. We do plan to carry out a randomized field experiment in another study. Because of restrictions concerning the data source, we could only obtain the e-book data set for a specific period, and this did not allow us to use the data longitudinally. With a longer data set involving several periods, we plan to use a time series model approach to examine causality. In addition, our study uses a two-period model (e.g., the first and second year) in which customers retention is defined and derived by observing the second period. In our future research, a multiperiod model should be considered to draw conclusions to provide generalized theories and findings. In this study, we assume that CLV is positively associated with customer retention rates. Although we believe that this is not an impractical assumption, it is possible that some customers could have negative profits in their future business with a company. When additional business generates losses, CLV for some customers may actually decrease with retention. Thus, one should be careful when applying the strategy of recommending new category products when the presence of unprofitable customers is an issue for a business. When we develop econometric models, we use control variables such as age, gender, category concentration, and prior experience (tenure) to minimize user-specific effects. However, other factors such as user inclinations and previous experiences are not fully controlled due to the lack of data. Thus, when possible, additional control variables should be used to estimate more accurate regression coefficients in future studies. To provide more generalized results, we should apply our model to various kinds of data sets. In fact, the e-book data set could be too narrow in scope to generalize our findings. It was difficult to collect a new, better data set. Consequently, only the e-book data set was available to us for the present study. Thus, we do plan to collect a new data set from another electronic commerce company that sells more diverse products than e-books. Conclusion We study how product category diversity affects customer retention rates. To answer the question, we examine how the number of different product categories offered to customers is related to customer retention rates at a nationwide online e-book company. We find that not only purchase quantity but also product category diversity are positively associated with customer retention rates. Our finding supports the supposition that a diversity-based recommendation strategy is effective in reducing a user s churn probability, or in increasing the user s long-term performance. We believe that this strategy can be applied to loyal customers who are already profitable in order to achieve higher retention rates. This work has two main academic contributions: (1) we develop an econometric model to examine the impact of sales category diversity on customer retention likelihood with aggregate and individual-level panel data sets; and (2) concerning the results of the empirical analyses, we find that although purchase quantity positively influences customer retention rates, the impact 02 park.indd 68 11/14/ :16:23 AM

19 International journal of electronic commerce 69 of category diversity is more effective in decreasing customer churn probability (or increasing customer retention rates). Although our study does not suggest that there is strong evidence to show a causal relationship between product diversity and retention rates, we can claim that customers who buy a greater diversity of products tend to remain active customers, and hence diversity-based recommendations may be more effective than accuracy-based ones. Thus, we believe that our findings help in bridging the gap between the two independent concepts. As a minor contribution, we use a Gini impurity measure for category concentration as a control variable and show that customers with concentrated category preferences have lower retention rates than others. REFERENCES 1. Adomavicius, G., and Kwon, Y. Toward more diverse recommendations: Item re-ranking methods for recommender systems. Paper presented at the 19th Workshop on Information Technologies and Systems, Phoenix, AZ, December 14 15, Adomavicius, G., and Tuzhilin, A. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17, 6 (2005), Ahn, H.J. Utilizing popularity characteristics for product recommendation. International Journal of Electronic Commerce, 11, 2 (winter ), Ahn, J.; Han, S.; and Lee, Y. Customer churn analysis: Churn determinants and mediation effects of partial defection in the Korean mobile telecommunications service industry. Telecommunications Policy, 30, 10/11 (2006), Anderson, C. The Long Tail: Why the Future of Business Is Selling Less of More. New York: Hyperion, Bell, R.; Koren, Y.; and Volinsky, C. The BellKor 2008 solution to the Netflix prize. Research report, AT&T Labs and Yahoo! Research, 2008 (available at www2.research.att.com/~volinsky/netflix/bellkor2008.pdf). 7. Bennett, J., and Lanning, S. The Netflix prize. Paper presented at the ACM Knowledge Discovery and Data Mining (KDD) Cup and Workshop, San Jose, CA, August 12, Bogers, T., and van den Bosch, A. Fusing recommendations for social bookmarking Web sites. International Journal of Electronic Commerce, 15, 3 (spring 2011), Brynjolfsson, E.; Hu, Y.; and Simester, D. Goodbye Pareto principle, hello long tail: The effect of search costs on the concentration of product sales. Management Science, 57, 8 (2011), Brynjolfsson, E.; Hu, Y.; and Smith, M. From niches to riches: The anatomy of the long tail. Sloan Management Review, 47, 4 (2006), Fleder, D., and Hosanagar, K. Blockbuster culture s next rise or fall: The impact of recommender systems on sales diversity. Management Science, 55, 5 (2009), park.indd 69 11/14/ :16:23 AM

20 70 Park and Han 12. Forman, C.; Ghose, A.; and Goldfarb, A. Competition between local and electronic markets: How the benefit of buying online depends on where you live. Management Science, 55, 1 (2009), Gupta, S.; Hanssens, D.; Hardie, B.; Kahn, W.; Kumar, V.; Lin, N.; and Sriram, S. Modeling customer lifetime value. Journal of Service Research, 9, 2 (2006), Haenlein, M.; Kaplan, A.M.; and Schoder, D. Valuing the real option of abandoning unprofitable customers when calculating customer lifetime value. Journal of Marketing, 70, 3 (2006), Han, J.; Kamber, M.; and Pei, J. Data Mining: Concept and Techniques. Waltham, MA: Morgan Kaufmann, Hwang, K.; Park, S.; and Han, I. Multi-screen strategy for selling mobile content to customers. In M.-H. Huang, G. Piccoli, and V. Sambamurthy (eds.), Proceedings of 33rd International Conference of Information Systems. Orlando: Association for Information Systems, 2012 (available at aisnet.org/icis2012/proceedings/ebusinessstrategy/2). 17. Kamakura, W.A.; Wedel, M.; Rosa, F.D.; and Mazzon, J.A. Cross-selling through database marketing: A mixed data factor analyzer for data augmentation and prediction. International Journal of Research in Marketing, 20, 1 (2003), Kim, H., and Kim, Y. A CRM performance measurement framework: Its development process and application. Industrial Marketing Management, 38, 4 (2009), Klein, J.J., and Moeschberger M.L. Survival Analysis: Techniques for Censored and Truncated Data. New York: Springer, Kotler, P. Marketing during periods of shortage. Journal of Marketing, 38, 3 (1974), Kotler, P., and Armstrong, G. Principles of Marketing. New York: Prentice Hall, Köhler, C.F.; Breugelmans, E.; and Dellaert, B.G.C. Consumer acceptance of recommendations by interactive decision aids: The joint role of temporal distance and concrete versus abstract communications. Journal of Management Information Systems, 27, 4 (spring 2011), Lee, D.; Park, S.; and Moon, S. Utility-based association rule mining: A marketing solution for cross-selling. Expert Systems with Applications, 40, 7 (2013), Li, S., Sun, B.; and Wilcox, R.T. Cross-selling sequentially ordered products: An application to customer banking services. Journal of Marketing Research, 42, 2 (2005), Linden, G.; Smith, B.; and York, J. Amazon.com recommendations: Itemto-item collaborative filtering. IEEE Internet Computing, 7, 1 (2003), McNee, S.M.; Riedl, J.; and Konstan. J.A. Being accurate is not enough: How accuracy metrics have hurt recommender systems. In Proceedings of CHI 06 Extended Abstracts on Human Factors in Computing Systems, New York: ACM, 2006, pp Mooney, R.J., and Roy, L. Content-based book recommending using learning for text categorization. In Proceedings of the 5th ACM Conference on Digital Libraries. New York: ACM, 2000, pp park.indd 70 11/14/ :16:23 AM