LoyalTracker: Visualizing Loyalty Dynamics in Search Engines

Transcription

1 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. 12, DECEMBER LoyalTracker: Vsualzng Loyalty Dynamcs n Search Engnes Congle Sh, Yngca Wu, Member, IEEE, Shxa Lu, Senor Member, IEEE, Hong Zhou and Huamn Qu, Member, IEEE nflow branch outflow branch selected layer flow engne B engne C 1~7 8~15 16~25 26~ selected branch flow Fg. 1: LoyalTracker llustrates loyalty dynamcs of the users usng search engne A. Top and bottom show the same flow vew that hghlghts two dfferent flowng patterns of the users (n orange) selected from a layer flow (top) and a branch flow (bottom) across multple loyalty categores (layers) over tme. The swtchng hstogram on the top shows a vsual summary of swtchng behavor. Abstract The huge amount of user log data collected by search engne provders creates new opportuntes to understand user loyalty and defecton behavor at an unprecedented scale. However, ths also poses a great challenge to analyze the behavor and glean nsghts nto the complex, large data. In ths paper, we ntroduce LoyalTracker, a vsual analytcs system to track user loyalty and swtchng behavor towards multple search engnes from the vast amount of user log data. We propose a new nteractve vsualzaton technque (flow vew) based on a flow metaphor, whch conveys a proper vsual summary of the dynamcs of user loyalty of thousands of users over tme. Two other vsualzaton technques, a densty map and a word cloud, are ntegrated to enable analysts to gan further nsghts nto the patterns dentfed by the flow vew. Case studes and the ntervew wth doman experts are conducted to demonstrate the usefulness of our technque n understandng user loyalty and swtchng behavor n search engnes. Index Terms Tme-seres vsualzaton, stacked graphs, log data vsualzaton, text vsualzaton 1 INTRODUCTION Search engnes have become a necessty n our daly lfe, as the search engnes empower us to fnd our desred nformaton rapdly from the vast volume of web pages. Search engne busness has thus become one of the most proftable busnesses on Internet. Meanwhle, search engne provders also face ferce market competton [17, 27]. Pror studes show that 70% of users rely on multple search engnes and due to the low swtchng cost, engne swtchng happens frequently [44]. Earnng loyalty can be crtcally mportant for search engne provders. Customer loyalty has long been regarded as a vtal source of sustaned profts, whch enables a company to develop a sustanable advantage over compettors [1]. Retanng merely 5% of customers can mprove profts by almost 100% [28]. Therefore, effectvely trackng user loyalty and understandng how and why loyalty changes and users leave become partcularly ndspensable for search engne provders. As onlne servce provders, search engne companes can collect large-scale log data from ther users who consent to provde ther search data, offerng a rcher source of data for an n-depth analyss of user behavor. Ths capablty attracts consderable research attenton from dfferent research areas such as data mnng and vsualzaton. Extensve studes have been conducted to analyze the user behavor wth respects to search engne swtchng [9, 42, 43, 44]. However, most exstng work manly focuses on short-term search engne swtchng behavor [43] and uses smple statstcal methods to valdate some C. Sh and H. Qu are wth the Hong Kong Unversty of Scence and Technology. E-mal: {clsh,huamn}@cse.ust.hk Y. Wu and S. Lu are wth Mcrosoft Research Asa. E-mal: {yngca.wu, shxa.lu}@mcrosoft.com. S. Lu s the correspondng author. H. Zhou s wth Shenzhen Unversty. E-mal: [email protected] Manuscrpt receved 31 Mar. 2014; accepted 1 Aug Date of publcaton 11Aug. 2014; date of current verson 9 Nov For nformaton on obtanng reprnts of ths artcle, please send e-mal to: [email protected]. Dgtal Object Identfer /TVCG gven assumptons rather than detect abnormal or unexpected patterns. Vsualzaton practtoners have also developed varous systems [26, 40, 21, 34] usng vsualzaton technques such as graphs to explore user behavor from the collected user log data. Nevertheless, most systems am at vsualzng Web traffc and/or user navgaton paths through a webste, but are ncapable of trackng user retenton rgorously or conductng a systematc analyss of defecton patterns. A system capable of trackng customer loyalty s crtcally mportant for companes to prevent customer defecton. For nstance, customers usually have a hgher probablty of leavng when loyalty s found to be contnuously decreasng. Wth an effectve vsualzaton system, a company can make an nformed decson to customze ts offerngs or to take other necessary actons to retan customers. However, there are few proper vsualzaton technques that can be used to solve the problem. One partcular challenge s the desgn of an ntutve and nteractve vsual representaton to understand long-term behavor better, as characterzed by dynamc loyalty varaton and frequent defecton over tme [27, 43]. In addton, the faster word-of-mouth and lower swtchng costs on the Internet change the pace at whch companes must mprove ther products and servces to keep users loyal. These goals requre tmely detecton and a thorough analyss of user loyalty and defecton [27]. The growng scalablty of the search log data makes tmely detecton and analyss dffcult. Despte the dscovery of patterns, the manner by whch to convey the fndngs successfully to managers s another obstacle. Therefore, developng an effectve and ntutve system to glean nsghts nto large scale data through the vsualzaton of user loyalty and defecton over tme s mportant. To address these challenges, we develop LoyalTracker, a vsual analytcs system to analyze and track user loyalty and swtchng behavor towards multple search engnes usng large-scale user log data. LoyalTracker enables analysts to defne multple loyalty categores such as hard-core loyals and swtchers [18], followng the general practce n marketng. The analysts can then vsually trace how users change ther loyalty across categores over tme. The system has three lnked IEEE. Personal use s permtted, but republcaton/redstrbuton requres IEEE permsson. See for more nformaton.

2 1734 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. 12, DECEMBER 2014 vews: a flow vew, a densty map vew, and a word cloud vew. The flow vew uses a new nteractve vsualzaton technque nspred by a chart drawn by Munroe [24], whch can vsualze the dynamcs n each predefned loyalty category and ther aggregaton over tme clearly. In the densty map vew and word cloud vew, we show more detaled nformaton on demand, whch ads analysts n quckly detectng user behavor, determnng the underlyng reasons, and ntutvely conveyng the fndngs to a wde audence. LoyalTracker targets the trackng and analyss of the user loyalty on search engnes, but technques used can be easly adapted to other smlar problems on user loyalty and defecton, because the characterstcs of loyalty and swtchng behavor are shared across dfferent areas. Our contrbutons are descrbed as follows: a set of desgn prncples for analyzng the long-term and dynamc user loyalty of a large number of users at dfferent levels of detals. an nteractve vsualzaton system to enable analysts to understand better user loyalty devatons wthn a search engne and the defectons between multple search engnes. an augmented stacked graph to show the flows between layers and the flows that enter or leave the graph. 2 RELATED WORK Past research related to ths work can be classfed as follows: search engne swtchng, stacked graphs, and log data vsualzaton. 2.1 Search Engne Swtchng Web search engnes ad users n quckly locatng nformaton on the Internet, makng the tools a necessty n daly lfe. A large percentage of daly users (up to 70%) frequently swtch between dfferent engnes because of the low swtchng cost and other factors such as satsfacton, effectveness, and famlarty [44]. In addton, 4% of search tasks nvolve multple search engne usages. Ths percentage ncreases to over 10% for longer search tasks [42]. Extensve studes were conducted to understand how, when, and why users swtch engnes [9, 42, 43, 44]. Pror studes classfed users nto dfferent categores based on ther swtchng behavor [16], examned multple search usage [44], developed models to explan engne swtchng behavor based on brand loyalty [23], predcted when users wll swtch [9], or employed large-scale log analyss and user survey data to understand user motvatons for swtchng [42]. Researchers have recently suggested the development of a better understandng of long-term engne swtchng behavor [43], but exstng studes only use smple statstcal methods to valdate the gven assumptons. To our knowledge, nteractve vsualzaton systems capable of trackng and analyzng user behavor based on loyalty and defecton reman lackng. 2.2 Stacked Graphs A stacked graph s constructed by stackng one tme seres on top of another, such that each tme seres s represented by a stacked layer. Havre et al. [13] ntroduced ThemeRver that uses a rver metaphor to vsually depct the thematc varatons n documents over tme. Byron and Wattenberg [2] presented Streamgraph, whch sgnfcantly mproves the qualty of standard stacked graphs. Streamgraph has been extended to help analysts better understand text corpora [5, 6, 22, 31]. TIARA [22] and ts varant [31] are nteractve vsual text analyss tools that seamlessly ntegrate text summarzaton technques for explorng large collectons of documents. Dörk et al. [6] descrbed a hghly nteractve system based on talored stacked graphs to vsualze a contnuously updatng nformaton stream. These technques can be used to vsualze multple tme seres but cannot reveal how dfferent tme seres exchange quanttes over tme. TextFlow [5] employs a flow-based metaphor to enhance stacked graphs, such that the mergng and splttng relatonshps between evolvng topcs n text corpora can be revealed. RankExplorer [30] extends stacked graphs by embeddng color bars and changng glyphs to help users analyze rankng changes. TextFlow and RankExplorer were developed for dfferent applcatons, but both could be used to track how quanttes of tme seres flow to one another usng ether the flow-based vsual metaphor [5] or the color bars [30]. However, the two methods are napproprate for trackng user loyalty because of a varety of lmtatons such as dstorton and ntutveness ssues. We analyze and compare the technques n more detal n Secton Log Data Vsualzaton Human-generated log data, such as query logs [48], usablty logs [10, 8], and Web clckstreams [41], records varous actvtes of users. SDSS Log Vewer [48] helps analysts quckly dentfy the data-seekng behavor of users from SQL query logs. Gray et al. [8] used a set of vertcal strped bars to show the usage patterns from usablty logs collected from a graphcal user nterface. Clckstreams vsualzaton examnes Web traffc and/or user navgaton paths through a Web ste for user behavor analyses [3, 20] and usablty mprovement [4]. Numerous systems such as WebQult [40] and Webvz [26] commonly use trees, treemaps, or node-lnk graphs to vsualze clckstreams. Lee et al. [20] used parallel coordnates and star felds to vsualze user paths and product performance. We et al. [41] ntroduced an nteractve clusterng method to reveal user behavor patterns from Web clckstream data. TralExplorer2 [29] uses stacked bars and pe charts to dscover valuable nformaton from large-scale Web clckstreams. Behavor Graphs (WBG) [3], based on state dagrams, were used to vsualze the search structure on the Web and to dscover usage behavor patterns. Lam et al. [19] developed Sesson Vewer to help analysts understand Web search usage behavor wth multple coordnated vews, such as state transton dagrams, stacked bars, and tables. Outflow vsualzaton [45] used a flow based desgn to show the aggregated multple event sequences and ther outcome, whch can be used to analyze event progresson pathways. Masrur et al. [36] mproved the drected graphs to summarze people s preference transton. The mproved graphs can show more nformaton lke latency and frequency, but t can hardly show long tme temporal patterns. Compared wth exstng systems, we manly focus on analyzng the dynamcs of user loyalty and defecton usng mllons of Web log entres, whch s not supported by prevous systems. 3 BACKGROUND Ths secton ntroduces the background knowledge on user loyalty and search engne swtchng, and then summarzes the analyss tasks. 3.1 Customer Loyalty and Defecton A company can sgnfcantly boost ts profts by buldng customer loyalty and reducng customer defecton [28]. Thus, companes need to gather nformaton about customers to track ther loyalty, analyze ther behavors, and dentfy why they are leavng [28]. Customer loyalty s a complex phenomenon and s therefore dffcult to defne and measure [46]. From a behavoral vew, customers are consdered loyal to a frm f they consstently purchase products or servces from the frm. Behavoral measures contan crtera such as repeat purchase and word-of-mouth referrals. From an atttudnal vew, customers are consdered loyal to a frm f they have a strong desre to mantan a relatonshp wth the frm. Atttudnal measures nclude crtera such as commtment and trust. Atttudnal measures may better explan how and why loyalty changes but are more dffcult to evaluate quanttatvely and may need addtonal efforts to conduct questonnares. Loyalty analyss of search engnes has receved consderable attenton [9]. Search engnes have a sgnfcantly lower swtchng barrer, and users can easly change ther search engnes [9, 43]. Loyalty change and swtchng behavor are also easly observed, thus provdng relatvely complete data. Frequent varatons and the rch data create a good opportunty to analyze the loyalty and derve desgn prncples. Therefore, we base our work on the loyalty analyss of search engnes. Nevertheless, our vsualzaton desgn can be easly extended to analyze loyalty and swtchng behavor of other products and servces, because of smlar data characterstcs, analyss tasks, and goals. 3.2 Data Characterzaton and Task Analyss We collected search log data from consentng users of a wdelydstrbuted Web browser and store t n a MapReduce system. All personally dentfable nformaton from the logs had been removed.

3 SHI ET AL.: LOYALTRACKER: VISUALIZING LOYALTY DYNAMICS IN SEARCH ENGINES 1735 Every log entry ncludes: a unque user dentfer for each user, a query performed by the user, the search engne used for the query, the tmestamp when the query was ssued, and the URL and dwell tme of the result page clcked. We followed an teratve user-centered desgn process to develop our vsualzaton technques. We worked closely wth three doman experts (two appled scentsts (AS) and a software development engneer (SDE)), from the search department of a corporaton for eght months. The ASs focus on analyzng and understandng onlne user behavor and user experence. The SDE mantans a system called XSystem. XSystem uses the 2.5% sampled data from the collected raw data. It enables analysts to access large-scale search log data quckly usng carefully-constructed queres and returns the results n a table. The feedback collected by the SDE suggests that a vsual system that enables analysts to nteract wth the complete data and explore searchng behavor s urgently needed. We held bweekly meetngs and exchanged emals wth the experts to gather and refne desgn requrements, present prototypes, and collect feedback to mprove the system teratvely. We defne some terms formally below. A search sesson conssts of a seres of user search actvtes. A sesson ends f the user s dle for more than 30 mnutes, whch s wdely adopted [33, 42]. A swtchng event s defned as a par of consecutve queres ssued on dstnct search engnes wthn a sngle search sesson [9]. Accordng to the suggestons of the doman experts, we employ wellestablshed measures to estmate user loyalty and satsfacton. User loyalty n search s evaluated usng a behavoral measure of user engagement, the frequency of usng the search engne. In general, there are two wdely used metrcs to quantfy the user engagement: the number of queres [14] (at query level) and the number of sessons [33] (at sesson level) performed by users n certan tme perods. The user engagement at query level s manly used to descrbe the short-term user behavor, whereas the user engagement at sesson level s focusng more on descrbng the long-term user behavor. In our case, the analysts pay more attenton to long-term behavor analyss. Therefore, the loyalty s defned at sesson level. User satsfacton n search s measured based on how long a user stays on the destnaton page. A query s vewed as satsfactory f the searcher clcks a search result followed by a dwell tme of more than 30 seconds [7, 11, 15, 42]. Although Hassan et al. proposed a sophstcated metrc to measure user satsfacton [12], we chose the smpler one because of two concerns: 1) due to the large sze of the data and the computng complexty of the sophstcated metrc, n the current computng nfrastructure, t s hard to get the result n reasonable tme when applyng the sophstcated metrc; 2) the smpler metrc s wdely used n the company and t has been proved to be effectve. Nevertheless, n our system, we have carefully decoupled the calculaton module wth other modules such that t wll be easy to replace current measure wth more advanced one wthout affectng other modules. We compled a lst of analyss tasks through a seres of ntervews wth them. Ths process aded us n better understandng the problem doman and dentfyng the challenges faced by the target users. Q.1 How does the loyalty of searchers usng a partcular search engne change over tme? Groups of users wth a smlar loyalty-changng trend are of partcular nterest for analyss. Detectng not only the sudden and dramatc changes n user loyalty, but also the long-term and gradual loyalty changes s mportant. Q.2 How do searchers usng a partcular search engne swtch to other search engnes over tme? How s the user-swtchng behavor related to the dynamc varaton of user loyalty over tme? The analyss and dentfcaton of the user-swtchng behavor pattern s crucal for formulatng effectve strateges to retan users. Q.3 Where do the new users come from and have these users used the search engne before? How does the loyalty of swtchers or new users change over tme? Our collaborators need to analyze the dstrbuton of swtchers or new users to gan a better understandng of user behavor. Q.4 What are the dfferences n the dynamc loyalty varaton or userswtchng behavor among multple search engnes? Our collaborators want to analyze the dfferences among multple search engnes to dentfy the strengths and weaknesses of a search engne better based on loyalty and defecton. Q.5 What are the reasons for the swtchng behavor or loyalty change? Our collaborators want to determne the reasons behnd the dentfed user behavor pattern. These experts are nterested n knowng whether a change n user satsfacton could lead to the behavor or f any keywords that easly trgger the pattern exst. 4 SYSTEM DESIGN In ths secton, we dscuss the vsualzaton challenges and desgn ratonale for the system. 4.1 Analyss Challenges and Desgn Ratonale Durng our collaboraton wth the experts, we dentfed a few analyss challenges. Before we proposed the LoyalTracker system, when analysts want to analyze the complete data, they have to formulate ther assumptons and construct scrpts to test these assumptons and then submt the scrpts to a MapReduce system to dentfy user behavor patterns. Runnng scrpts on the system may take hours or days dependng on the complexty of the tasks and the resources avalable on the system before the analysts can retreve the results n tables. The analysts often use Mcrosoft Excel to analyze the retreved table data and to create smple charts for demonstraton. Obtanng the desred results to valdate ther assumptons s a tedous tral-and-error procedure. Testng assumptons s dffcult, but detectng unexpected user behavor patterns from a vast amount of data s sgnfcantly more dffcult and tme-consumng. Although nterestng results may be detected, effectvely presentng these results would be another obstacle. A vsualzaton system that enables analysts to analyze user loyalty nteractvely s urgently needed by the analysts n the company. Exstng vsualzaton technques are unsutable for the effectve trackng of dynamc loyalty varaton and analyzng user-swtchng behavor n one coherent vew. We worked closely wth doman experts and dentfed a set of desgn goals to address these challenges. A. User Flow Revelaton. In marketng and loyalty analyss, analysts often classfy users nto dfferent loyalty categores, such as hardcore loyals [18]. The users behavor s then analyzed and compared across dfferent loyalty categores. The doman experts are concerned wth the vsual trackng of the dynamc varatons of dfferent loyalty categores and the summaton of all the categores (Q1). More mportantly, the desgn could effectvely reveal how users flow between dfferent loyalty categores as well as how they flow n and out of a typcal search engne over tme, thus enablng the accomplshment of the analyss tasks (Q1-Q3) related to the user flow of loyalty varaton. B. Intutve Storytellng Metaphor. A vsual metaphor capable of tellng a story ntutvely s desred by our collaborators for the analyss tasks (Q1-Q5). An approprate vsual storytellng metaphor enables them to convey ther fndngs more effectvely wth the support of vsual evdence or related detals to product teams and senor managers. Therefore, our work employs a vsual representaton based on an ntutve flow metaphor to facltate storytellng. C. Mult-Scale Vsual Representaton. Detectng both short-term and long-term patterns s mportant. A sudden and dramatc change n customer loyalty should be mmedately detected, especally for onlne servce provders wth a sgnfcantly lower swtchng barrer, to take prompt actons. It s also crucal for companes to dentfy long-term user behavor patterns, whch can shed more lght on user preferences and usage patterns [43]. Knowledge of the key trends n user loyalty and swtchng behavor s nvaluable to companes. Therefore, the desgn should naturally support mult-scale analyses (Q1-Q5). D. Interactve Pattern Unfoldng. A vsual system that enables analysts to nteract wth the data drectly and see the results mmedately s always preferred by doman experts to complete the descrbed tasks, partcularly for Q3-Q5. The system should provde a vsual overvew of how user loyalty changes over tme to dentfy nterestng patterns, and enable analysts to gan further nsght nto the patterns and determne

4 !"##!"#$ %"& 1736 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. 12, DECEMBER 2014 User Extracton Raw Data Loyalty Calculaton Data Processng Satsfacton Calculaton Keyword Extracton Flow Vew Densty Map Vew Word Cloud Vew Vsual Encodng Fg. 2: Our system has a data processng part and a vsualzaton part wth three coordnated vews. A flow vew s used to provde a quck overvew of the dynamc loyalty varaton and swtchng behavor. A word cloud vew and a densty map vew can help analysts nterpret the user behavor patterns dscovered n the flow vew. the causes of such changes (.e., the relatonshps between satsfacton and loyalty, and keywords that trgger the patterns). E. Comparatve Analyss. A system that enables comparatve analyss can ad n accomplshng the task of Q4. A desrable system should naturally support nteractve brushng and vsually hghlght the smlartes and dfferences n the dynamc user loyalty flow among dfferent search engnes. Furthermore, a statstcal graph showng the overall dstrbuton of the loyalty varaton and flow n a search engne s also needed to facltate the comparatve analyss. 4.2 System Overvew LoyalTracker begns wth the data processng part, whch submts a scrpt to the MapReduce system and retreves the results once the tasks complete. After that, LoyalTracker can perform nteractvely. LoyalTracker conssts of three vews: a flow vew, a densty map vew, and a word cloud vew (Fg. 2). The flow vew s used to provde a vsual summary of dynamc loyalty change and user-swtchng pattern over tme, such that analysts can track the dynamc varaton readly. It uses a compact flow-based vsual metaphor to enhance the stacked graphs for ntutvely showng not only how users flow between layers but also how users enter or ext the search engne over tme. The analysts can flexbly and vsually specfy queres n the flow vew to perform an n-depth analyss n other vews. The densty map vew can help analyze the relatonshps between user loyalty and satsfacton of a group of users. The word cloud vew vsually summarzes the query keywords that trgger engne swtchng for a group of users selected n the flow vew to facltate further understandng of the correlaton between keywords and flow varaton. The three vews are well-coordnated to help analysts accomplsh the varous of tasks descrbed n Secton VISUAL ENCODING METHODS In ths secton, we descrbe a set of vsualzaton technques for analyzng user loyalty. User nteractons are subsequently presented. 5.1 Flow Vew In the flow vew, we propose a new desgn and an nteracton technque based on a flow metaphor for vsualzng the dynamcs of user loyalty. The flow metaphor s nspred by an nfographcs chart drawn by an artst [24] to llustrate the hstory of the Unted States congressonal electons. Fg. 3(a) shows the chart placed horzontally to ft the wde screen of modern dsplays better, wth the tmelne startng from left to rght. The Congress members are categorzed nto dfferent poltcal groups, such as left and rght, based on ther poltcal belefs. The chart vsually traces the evolvng composton of the US congress usng a stacked graph layout. Each group s encoded usng a layer n a dstnct color. The graph, compared wth general stacked graphs, also shows how dfferent layers exchange members wth one another usng a flow metaphor (.e., branches) over tme. The chart shows two types of branches on top of a layer at each tme pont. * Inflow branch along the top-left to bottom-rght drecton (Fg. 3(a)) ndcates that a group of members s enterng the layer from the above. The nflow comng from the top empty space (Fg. 3(b)) represents the new members. * Outflow branch along the bottom-left to top-rght drecton (Fg. 3(c)) ndcates that a group of members are leavng the layer to go to the layers above. The outflow departng from a layer to the top empty space (Fg. 3(d)) represents the leavng members. An outflow branch lnked to an nflow branch (Fg. 3(e)) ndcates the returnng members who left for a short tme. The wdth of a branch represents the number of members n t. The endponts of multple nflow (or outflow) branches are bundled together f the branches meet n a layer durng a tme frame. The chart s excellent for storytellng and has facltated numerous dscussons on the Web (approxmately 430,000 results returned from the Google by searchng xkcd 1127 and xkcd congress as of March 2014). The stable poltcal belefs of members yeld the chart wth less edge crossngs, resultng n a clear, legble layout. The tasks of loyalty analyss are very smlar to the tracng of poltcal belefs of the US congress members. A loyalty analyss also defnes multple loyalty categores and traces the dynamc change n dfferent categores as dscussed n Secton 4.1. The customer loyalty does not dramatcally change n most cases on the Web or n the real world [27]. The storytellng characterstc of the metaphor and ts capablty to reveal the flow between multple categores drectly satsfy desgn ratonaltes A and B. Therefore, we use a smlar vsual metaphor to desgn the flow vew. We also extend the basc layout to enable multlevel representaton, nteractve pattern unfoldng, and comparatve analyss (meetng desgn ratonalty C, D, and E) Comparson wth Alternatve Solutons The doman experts need to vsually track the dynamcs of dfferent loyalty categores as well as the summaton of all the categores (Q1). Accordng to the desgn ratonals of user flow revelaton (A), t s reasonable to desgn a vsualzaton smlar to stacked graphs to acheve the goal. It also demands an advanced layout that can convey how users flow across multple loyalty categores over tme. Two exstng stacked graph technques, TextFlow [5] and RankExplorer [30], can accomplsh ths goal. TextFlow also uses a flow-based metaphor to convey the flowng pattern (see Fg. 4(a)). Ths technque draws a complete flow branch between two consecutve tme ponts to represent the streams of customers, whch mght ntroduce sgnfcant vsual clutter caused by the edge crossngs of multple flows. TextFlow optmzes the layer orderng to reduce vsual clutter. However, the orderng n loyalty analyss should be preserved because the order nherently mples a semantc meanng (.e., the loyalty level). By contrast, LoyalTracker only draws a partal flow rather than a complete flow to mtgate the problem of edge crossngs. TextFlow produces rregular whte gaps between layers to show the flow, whch dstort the layers and present an ncorrect aggregate pattern (sum of ndvdual tme seres). Our flow-based vsualzaton does not produce whte gaps. RankExplorer [30] uses color bars rather than the flow metaphor to convey the flowng pattern (see Fg. 4(b)). A flow from a layer m to another layer n s represented by a color bar n m, wheren the heght of the bar encodes the flow sze and the color of the bar encodes the flow drecton. Compared wth TextFlow, the color bars can elmnate the problem of edge crossngs and help create a more compact layout wthout the dstorton. However, ths method s generally less ntutve than the flow-based methods. We present all three canddate desgns n Fg. 4 to our collaborators and the feedback from them also confrmed our desgn choce Vsual Encodng We use the flow metaphor descrbed n Secton 5.1 to desgn the flow vew to show the dynamc loyalty varaton. Fg. 4(c) shows the vsual encodng scheme. We defne a cell as the part of a layer n a tme frame (.e., from tme pont t to t + 1).

5 SHI ET AL.: LOYALTRACKER: VISUALIZING LOYALTY DYNAMICS IN SEARCH ENGINES 1737 b d a c e Fg. 3: Left: Layout extracted from a chart drawn by an artst [24] where (a) and (b) are the nflow branches; (c) and (d) are the outflow branches; and (e) s a group of returnng members. Rght: Layout created through our method usng the data manually extracted from the nfographcs chart. Both layouts can show not only the dynamc varaton of each group but also the dynamc exchange of members across dfferent groups. Wdth of the flows heght out flow n flow +1 t Wdth of the theme t t+1 t+1 (a) t (b) t (c) Fg. 4: (a) TextFlow based on a flow-based metaphor, (b) RankExplorer based on color bars, and (c) The flow vew based on a new flow metaphor. Each layer n the flow vew represents a certan degree of user loyalty. The wdth of the layer at a tme pont ndcates the respectve number of the users at that tme. The branches represent the users that flow n or out of the layers. The wdth of a branch encodes the number of users. Each cell contans two types of branches: the outflow branches on the left and nflow branches on the rght (see Fg. 4(c)). The branches of the same type (e.g., nflow) n a cell are bundled to create a neat layout. The bundlng order s determned by the order of the layers that the branches flow from (nflow) or to (outflow). The vertcal poston of a bundled group encodes the average loyalty level of users. The branches n the empty regon above ndcate new users who have just started usng the search engne, or leavng users who stop usng the current search engne. We place a swtchng hstogram above the flow desgn to provde an overvew of user swtchng behavor at each tme pont. The color ndcates where the new users come from or where the leavng users go to (see Fg. 1). Flow based vsualzaton could also show the user swtchng behavor across multple search engnes. However, after some experments, we found that ths soluton suffered from severe vsual clutter and can hardly handle comparatve analyss and exploraton wth multple search engnes. The layout algorthm s descrbed n detals as follows Layout Algorthm We follow four desgn prncples, ncludng legblty, aesthetcs, neatness, and fathfulness, so as to create a good layout. Legblty and aesthetcs prncples have been commonly employed n creatng engagng, storytellng vsualzatons [2, 35]. Our work employs the legblty prncple to create a proper layout, such that we can clearly reveal the user flow nformaton (desgn ratonalty A). Specfcally, we avod puttng the branches n a small regon. A symmetrc vsual desgn s often utlzed to enhance the aesthetcs prncple [2, 13] to generate an engagng vsualzaton (desgn ratonalty B). Moreover, pror research proves that symmetrcal objects can be perceved much more strongly [39]. Thus, we enforce the nflow and outflow branches to be symmetrcal at each tme pont (see Fg. 5(a) and (d)). The neatness and fathfulness prncples ensure that we can effectvely and fathfully reveal the nformaton n the layout. The neatness prncple s used to reduce vsual clutter by algnng objects n an organzed manner, because excess and dsorganzed tems usually degrade vsual task performance [37]. Thus, we algn the endponts of outflow branches to the rght (Fg. 5(a), (c), and (f)), and algn the endponts of nflow branches to the left (Fg. 5(b), (g), and (h)) n each cell. The fathfulness prncple ensures that the vertcal poston of each group of bundled branches should be unchanged, consderng that the vertcal poston represents the average loyalty level of the branches. Thus, the branches can only move along a vrtual track (see the dashed lne n Fg. 5(e)) wthn the cell. d root pont a f Fg. 5: Illustraton for the force-drected model. Black nodes represent the group of the bundled branches. Nodes are connected by sprngs to acheve our layout goals. The dashed curve n the top left corner shows the track that the node can move along. Followng these prncples, a force-drected model s developed to create the layout. We assume that a node placed n the mddle of the endpont of the group of bundled branches (the black nodes n Fg. 5) can represent the group. The model has three basc forces: a sprng force, a repulsve force, and a symmetrc force. To meets the legblty prncple, a sprng force (the green sprngs n Fg. 5) s used to cause attracton between the nflow and outflow branches n a cell, whch enables the branches to have proper lengths to be legble. The sprng force between nodes a and b (see Fg. 5) s defned as follows c f s (a,b)=k s ( p a p b l) (p a p b )/ p a p b (1) where k s s a gven weght, p a and p b represent the postons of a and b, respectvely; and l s the orgnal length of the sprng. We also employ a repulsve force to make sure that the branches n a cell do not overlap when attractve force s used. The repulsve force follows Coulomb s g b h

6 1738 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. 12, DECEMBER 2014 law, whch descrbes the electrostatc nteracton between electrcally charged partcles and s defned as: f e (a,b)=k e q a q b (p a p b )/ p a p b 3 (2) where k e s the physcal constant, and q a and q b are the charge of the nodes determned by the wdths of the branches. The sprng force s also used to create a neat layout and algn the nodes of the outflow branches to the rght or the nodes of the nflow branches to the left across the layers (red sprngs n Fg. 5). We defne a symmetrc horzontal force for the nflow and outflow branches sharng the same root n the same layer at a tme pont (see Fg. 5 (a) and (d)) to satsfy the aesthetcs prncple. f sym (a)=k sym (x t +(x a x d )/2 x a ) (3) f sym (d)=k sym (x t (x a x d )/2 x d ) (4) where k sym s a gven weght, x a and x d represents the postons of nodes a and d along the horzontal axs, respectvely, x t s the horzontal poston of the root pont of a and d at tme pont t. When the algorthm starts, n each cell, the branches are placed along the horzontal axs. The algorthm then aggregates all the forces for each node and teratvely moves the partcles based on the aggregated force. To satsfy the fathfulness prncple, each node can only move along the vrtual track wthn the tme frame. Ths process repeats untl the entre partcle system acheves a stable state. Notce that all prncples are equally mportant and thus k e and k sym are assgned approprately to reflect the property User Flow Tracng the mechansm of flow of a selected group of users across layers over tme n the flow vew s very mportant n analyzng user behavor (desgn ratonalty A). Therefore, we use a desgn smlar to FlowMap [25] to vsually track the contnuous evoluton of a group of selected users n the flow vew. One smple method to show ths pece of nformaton s by drectly drawng a set of flows n the flow vew to show the dstrbuton of the users (see the hghlghted orange flows n Fg. 6(a)). The wdth of a flow encodes the number of the users n a layer or branch. Each flow s ntally drawn n the mddle poston n a branch (.e., branch flow, representng the users swtchng ther category) or n a layer (.e., layer flow, representng the users stayng n the same loyalty category between two consecutve tme ponts) (see Fg. 6(a)) to smplfy the problem. Although the user dstrbuton s shown, the dsconnected flows prevent the analysts trackng the flow patterns readly. Therefore, we smoothly lnk the flows together. In a branch (see Fg. 6(a)) n layer, we move down the root pont of the branch flow as small as possble to jon the layer flow (below the branch n layer 1) representng the users stayng n the same layer. We also extend the end pont of the branch flow to jon the layer flow n the current layer. The dashed red lne n Fg. 6(a) shows the adjusted flow. Ths process s repeated for each branch, such that both ends of all the branches jon n the layer flows (n neghborng layers and 1). To avod ambguty, we should make sure that a layer flow s n parallel wth the layer, and should not devate from the mddle of the layer too much. Otherwse, the flow may show a wrong ncreasng or decreasng trend n the layer, whch can mslead analysts. We desgn a method to optmze the postons of flows layer by layer. For layer, we ntally select the maxmum layer flow and set ts poston to be the mddle of the layer (see the yellow layer flows hghlghted n Fg. 6(b) n each layer). Next, we adjust other flows to lnk them smoothly to the maxmum flow n layer and obtan an ntal layout (Fg. 6(b)). Our goal s to fnd a vertcal offset d to adjust the ntal layout to mnmze the ambguty. We defne the cost as follows. C (d)= n l j=1 n b w l j (y l j y l j d) 2 + w bk (y bk y b k d) 2 (5) k=1 where n l and n b ndcate the number of layer flows and branch flows, respectvely; y and y denote the orgnal (Fg. 6(a)) and ntal (Fg. 6(b)) vertcal postons, respectvely; w l j and w bk represent the wdths of the branches. We can easly derve the optmal offset d satsfyng C (d)=0. Fg. 6(c) shows the adjusted smooth user flows from Fg. 6(b). (a) (b) (c) layer flow maxmum layer flow root pont branch flow end pont maxmum layer flow maxmum layer flow Fg. 6: Illustraton of generatng the user flow. (a) Straghtforward method that shows the dstrbuton of users over tme, (b) the ntal layout created by fxng the layer flow wth the maxmum wdth and adjustng other flows accordngly for the smoothng purpose, and (c) optmal result by mnmzng the total cost C. 5.2 Densty Map and Word Cloud We use two addtonal vsualzatons, densty map and world cloud, to facltate the n-depth analyss of user loyalty (desgn ratonalty D). One mportant task of loyalty analyss s to nvestgate the relatonshp between user loyalty and satsfacton. We use a kernel densty estmaton (KDE) technque to create a densty-based scatterplot called densty map that shows the relatonshp (see Fg. 9 mddle). KDE s partcularly effectve n plottng large datasets n scatterplots to crcumvent the overdraw problem and acheve a truthful assessment of dstrbutonal data characterstcs. Our collaborators are also nterested n examnng the keywords to trgger an engne swtch. Thus, we also ntroduce a vsualzaton (see Fg. 9) to vsually summarze the keywords that trgger the swtchng behavor. Word clouds are chosen because of the excellent storytellng and engagng capablty [38] (desgn ratonalty B). Addtonally, we place a color bar under each keyword to ntutvely reveal the dstrbuton of dfferent swtchng types, as requested by our collaborators (See Fg. 9). Each color bar under a word s dvded vertcally nto multple parts. Each part represents one type of engne swtches and contans two color blocks (see the legend above the word cloud n Fg. 9): the upper color encodes the prevously used search engne and the lower color encodes the newly swtched search engne. Also, the rato between the sze of each part and the total sze of the color bar vsually encodes the percentage of each type of engne swtches. 5.3 User Interactons Desgn ratonalty D requests that the system enables analysts to nteract wth the data. Apart from basc nteractons such as pan and zoom, the system supports a set of user nteractons. Mult-scale exploraton s supported by LoyalTracker to facltate vsual detecton and analyss of patterns at dfferent levels of detal (desgn ratonalty C). LoyalTracker enables analysts to select a tme perod and a scale (day, week, or month). The flow vew wll be updated automatcally when the analysts choose a new tme perod or a scale. Comparatve vsualzaton can help analysts dentfy the smlarty and dfference between multple search engnes from the perspectve of dynamc loyalty varaton (desgn ratonalty E). Our system naturally enables nteractve comparatve vsualzaton by showng multple lnked LoyalTrackers (see Fg. 10). Flterng enables analysts to focus on mportant nformaton. In a LoyalTracker, not all branches are mportant. Generally, the larger the

7 SHI ET AL.: LOYALTRACKER: VISUALIZING LOYALTY DYNAMICS IN SEARCH ENGINES 1739 branches, the more mportant they are. Our system allows an analyst to nteractvely remove less mportant branches. Brushng enables analysts to nteractvely choose ther nterested data n the vsualzaton drectly for further analyss. All vsualzatons are lnked to allow for data exploraton from dfferent perspectves. An analyst can easly specfy a farly complex query by brushng multple branches or layers n the graph. The branches are then combned by takng the unon or ntersecton of the branches, thus allowng the analyst to vsually construct queres n a very flexble manner. 6 EVALUATION AND DISCUSSION We mplemented the system usng Java. After data preprocessng, nteractve performance can be acheved usng a PC wth Intel(R) Core(TM) CPU and 8 GB memory. To further evaluate LoyalTracker system, we conducted ntervews wth fve doman experts. 6.1 Comparson wth Munroe s Chart In the experment, we compare Munroe s chart [24] wth the result of our automatc method. We extracted a part of the orgnal chart (see Fg. 3 left) and then manually labeled the data used for our experment. Fg. 3 rght shows the result. Although the data we used s not exactly the same wth the orgnal one (we were unable to obtan the same data and thus only estmated the wdth of the branches n the chart), we can fnd that our result can preserve the orgnal nformaton n Munroe s chart. Both fgures enable an analyst to easly trace and dentfy the dynamcs of the evoluton of dfferent poltcal groups. Compared wth the orgnal chart, our result appears clean, neat, and smple. 6.2 Case Studes We extracted the top 100,000 actve customers n the US market durng the frst week of July 2012 from the massve search logs, as suggested by the doman experts. We then acqured the search behavor data on these users from July 2012 through December The doman experts were allowed freely to classfy users nto dfferent categores accordng to the loyalty levels of the users. Based on ther background knowledge, after several trals, they settled down four categores: more than 26, 16-25, 8-15, and less than 7 for the weekly level, or more than 6, 5, 3-4, and 1-2 for the daly level Tracng Customer Loyalty n Indvdual Engnes The frst case study was to demonstrate the usefulness of LoyalTracker on analyzng the dynamcs of user loyalty n engne A. Choosng a tme scale and a segmentaton s the frst step for analysts to explore data usng LoyalTracker. In ths step, they can try dfferent scales n order to see both long-term behavor and short-term behavor. In ths case study, we explored the data usng two scale levels: daly level (Fg. 7) and weekly level (Fg. 1). The flow vews can provde an overvew of the dynamcs of user loyalty. From the flow vew at daly level (Fg. 7), we can clearly see a pattern: many customers dd not use engne A on the weekend. From the flow vew at weekly level (Fg. 1), we can see no sgnfcant change n the szes of dfferent loyalty categores over tme. The nflow and outflow branches have nearly the same sze at each tme pont, ndcatng that there are almost equal numbers of customers enterng or leavng a layer at each tme pont. Thus, each layer appears flat over tme. Addtonally, the fgure also reveals that most customers tend to change ther loyalty level between adjacent layers. Gven that the vertcal poston of the endpont of a branch nsde a layer encodes the average level of customer loyalty, we see that most branches le n lower parts of the layers, ndcatng that the changes n loyalty are usually ~2 3~4 5 6~ Fg. 7: The flow vew shows the loyalty change of the engne A at the daly level. Many customers stop usng t on the weekend (July 14th). slght. The stable nature of customer loyalty has also been reported n pror research [27]. From the swtchng hstogram (Fg. 1), we can see the statstcs of engne swtchng behavor. More users who left current engne A swtched to engne C than to engne B. By trackng the overall heght n the flow vew, we can observe that the number of total users s decreasng gradually. After examnng all the engnes, we found that the decreasng number was not caused by engne swtchng, gven that all search engnes exhbted the same phenomenon. The doman experts dentfed two possble reasons: 1) customers started to use other Web browsers; 2) customers cleaned the cookes of the browser. In such crcumstance, we cannot track them any more. When a proper tme scale and a sutable segmentaton are selected, analysts can further nteractvely explore the data. User flow s the major nteractve vsualzaton component n the flow vew, whch can help analyze the loyalty dynamcs for a specfc group of users selected by the brushng nteracton. To better understand the behavor of users wth dfferent loyalty levels, we studed two groups of users n the flow vew: one group of users stably stayng n a cell (layer flow) and the other swtchng from one layer to ts upper layer (branch flow). We vsually traced how both groups dstrbute across layers over tme n the engne by drawng user flows hghlghted n orange n Fg. 1 top (layer flow) and bottom (branch flow). We can clearly observe an nterestng pattern. Compared wth the user flow (top), the bottom user flow reveals that once the loyalty of users decreased (.e., the users flowng through the outflow branch), only a few users would ncrease ther loyalty agan. In contrast, the top user flow appears more unformly dstrbuted. Wth the densty map and the word cloud vew, analysts can obtan more detaled nformaton about the user behavor of the selected group of users. We vsually examned the relatonshp between satsfacton and loyalty usng the densty map. We selected two groups of users n the same layer at the same tme pont. One group s n the layer, whereas the other group s n the branch (see Fg. 1). We compared the densty maps for the two groups of users (Fg. 8). Surprsngly, the fgures show no sgnfcant dfference between the densty maps. It s not an solated case: across all layers and n all the three engnes, we were unable to dentfy any strong correlaton between loyalty and satsfacton based on the densty map. The nteracton wth the densty map enables analysts to further select users accordng to both satsfacton and loyalty. We selected a group of users wth the lowest loyalty (from the top layer n Fg. 1) and then examned the relatonshps between user loyalty and satsfacton. The densty map shows that many users were qute satsfed but had low loyalty (Regon a n Fg. 9). Our doman experts ntally speculated that the users could have just conducted navgatonal searches (.e., a user searches the name of another search engne n search engne A and then swtches to other search engnes). We brushed dfferent regons n the densty map and examned the assocated word clouds. By comparng the word cloud of Regon a (see Fg. 9 left) wth those of other regons (for example, the word cloud of Regon b n Fg. 9 rght), we found that the rato of navgatonal search n other regons s hgher than that n regon a. Ths pattern can be found consstently from the Satsfacton Loyalty (a) Satsfacton Loyalty (b) Fg. 8: Densty maps (a) and (b) show the relatonshp between loyalty (horzontal axs) and satsfacton (vertcal axs) of the selected group of users n the branch and the layer n Fg. 1, respectvely.

8 gay dead credt ebay davd hurrcane contnental cemetery oppurtontes burger fsh Modfcaton babel places Johansson Clams bkn borgnne affar leer gruvlok Lmerck job gps manson account south jet vermont amercan call lc tx star lyrcs emla markng jagger youtube Scarlett Loan brave carolna vntage bukkake Pa vdeo nagara mck cheap cragslst ernest cellular clams lode mutual request wrapped jewelry sprngp recpes translate free Non-Proft marble IFLY Obama YAHOO water club GOOGLE beach show ttans bller motor oem cdm search make destn florda cabns blue rver Scarlett asbestos holmes move forecast atlanta vole andrew leopold stephane anmal country tckets rr'com melt www'facebook'com car jet tx de Funny mount communty palm Skt letters bkn shachar tennessee www'google'com sheers crushed oho wheel algnment machne ebay san fsh odessap webcam hacohen jobs ert brthday college dvorce wndowbuy medcal center cragslst FOR cheap kate current bass battery leader harvey dutes free Schobnger 1740 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. 12, DECEMBER 2014 healthy orlando weed baby death kate mddleton python steakhouse kdms magazne engneb mystery alexander cover sck confederate facebook jentzsch axtell deposts blankets mortgage magc chldren spllng arlnes engnec shades kourtney scott kller sleepng menu hateskuwtk grey questons ffty woodforest move teeth rottng ks hands movng outback Help bowe boxcar calculator study rumors Satsfacton,'H H'O Regon A H'N H'F H'P Regon B H'H, P I F Y N J Loyalty engne A --> engne C engne A --> engne B mal bowe paralegal hates kuwtk sale akron n'y' facebook death kourtney alexander magaznemddletonmystery engnec gay jagger affar youtube manager jentzschcovergoogle'comengneb ramrez games rottng enema scott mck lots kate teeth PH,P davd sara Fg. 9: Densty map shows the relatons between loyalty and satsfacton for a user group who are less loyal to search engne A. The word clouds on the left and rght show the keywords trggerng engne swtchng for the users n Regon A and Regon B of the densty map, respectvely engne A 1~7 8~15 engne B 16~25 selected user group 26~ engne C Fg. 10: The flow vews for all the three search engnes. The wdth of the customer ndcates search engne C has the most number of users. The user flow shows the dstrbuton of the selected user group n engne C. From the swtchng hstograms, we can see that the total rato of the customers who left C to other search engnes s much lower than that n engne A and B. top layer (representng the least loyal user) n A. However, ths pattern cannot be found n any other search engnes. The experts suggested some possble reasons. Frst, our satsfacton measure determnes that a navgatonal search s an unsatsfed query because users do not stay n the search engnes for more than 30 seconds. Thus, the users were treated as unsatsfed. Second, t was possble that A s set as the default search engne n the wdely-used web browser and the users usng the browser would perform more navgatonal search from A to others. Satsfacton Loyalty (a) (b) (c) Fg. 11: Densty maps (a), (b), and (c) show the relaton between loyalty and satsfacton for user groups n the same layers at the same tme pont on the engne A, B, and C, respectvely Tracng Customer Loyalty n Multple Engnes The second case study demonstrated the use of our vsualzaton technques n comparatve analyss for multple search engnes (Fg. 10). The overall heghts n the three flow vews drectly ndcate the popularty of the three search engnes among the actve users: more users employ engne C than the other two engnes. The number of users of engne A s slghtly larger than engne B. From the swtchng hstograms for the three search engnes, we can clearly see that there s a dfferent pattern for engne C compared wth engne A and B: the total rato of the users who left C to other search engnes s much lower than that n engne A and B. By selectng a group of users, we can smultaneously see the user flows n all engnes. Fg. 10 shows the user flow of the user group selected from engne C. A few selected users also used the other two search engnes (A and B) at the same tme, but wth very dfferent loyalty dstrbutons: the users only exst n the frst layer of engne B, whereas they exst n all layers except the bottom one of engne A We then examned the relatonshp between loyalty and satsfacton for the three engnes. By selectng the users n the same layers at the same tme pont n three engnes, we obtaned the correspondng three densty maps shown n Fg. 11. We compared the three densty maps and found that despte beng much more popular and havng the largest number of users, search engne C does not exhbt a compettve advantage over ts compettors n terms of user satsfacton. It s an unexpected pattern. Obvously, the users n search engne C are less satsfed compared wth A and B. The doman experts ponted out two possble reasons: 1) users who use search engne C dd more long tal/ rare queres. It s often dffcult to get good results from the search engne [32]; 2) the satsfacton metrc should be mproved. For example, n many cases users can acqure the nformaton they want on the result page wthout clckng the lnks of the results. However, n ths case, the correspondng queres are classfed as unsatsfed queres. 6.3 Intervew wth Doman Experts We conducted n-depth ntervews wth fve experts to evaluate the usablty of the system, ncludng four program managers (PMs) and a software development engneer (SDE) from the search department of a company. The PMs are partcularly nterested n analyzng user behavor patterns from search logs. They use XSystem daly to analyze the search logs. The SDE gathers feedback about XSystem from the users and mantans the system. Each ntervew lasted for hours. It started wth a few questons to dentfy ther background, followed by a tutoral to show the system features. We then asked them to freely explore a few data sets wthn the system. We fnally asked them some post-study questons to collect ther feedbacks and suggestons. We denote the partcpants as PM1, PM2, PM3, PM4, and SDE. Overall system usablty. The system was receved very well by all partcpants. PM4, the most senor PM, pro-actvely contacted us and requested to try our system when she learned about t from other PMs. She commented The system has a great potental and s powerful for fndng and analyzng user swtchng patterns.. Both PM2 and PM3 commented that the system s capable of helpng them evaluate the effectveness of a newly added feature of the search engne. All PMs were keen on usng the comparatve vsualzaton feature provded by the system. SDE emphaszed that the system would be valuable for the users of XSystem that s heavly used by a few hundreds of project managers, appled scentsts, and other analysts n the company.

9 SHI ET AL.: LOYALTRACKER: VISUALIZING LOYALTY DYNAMICS IN SEARCH ENGINES 1741 Vsual desgn and nteractons. All partcpants were mpressed by the vsual desgn and the supported nteractons. They especally lke the narratve nature by LoyalTracker that can help reveal dynamc user loyalty varaton vsually and ntutvely. PM2 felt very excted about the vsual desgn and commented I can clearly see the macro overall trend of user loyalty varaton as well as the mcro trend of user flowng patterns n one vew. They accepted the concept of multscale exploraton very well and agreed that the swtchng hstogram s ntutve to understand and useful for showng swtchng behavor. The partcpants apprecated the nteractons supported by the system. They acknowledged the usefulness of the flterng and brushng nteractons. PM2 sad the user nteractons would greatly facltate my analyss tasks. PM2 and PM4 commented that user flow enables them to easly connect the users who share a smlar loyalty pattern and trace ther loyalty varaton across multple search engnes. The partcpants also acknowledged the usefulness of the densty map and the word cloud. PM1 and PM2 commented that the word cloud can provde them a quck overvew of the dstrbuton of the words that trgger engne swtchng behavor. PM4 lked the densty map and commented that t allows me to quckly see the relatonshp between satsfacton and loyalty for a group of users. All partcpants agreed on the usefulness of the lnked vsualzatons. Suggestons. The partcpants provde valuable feedback about the system. All PMs except for PM3 suggested that we should support more data flterng operatons, such as the flterng based on dfferent entry ponts (such as tool bars or homepages). All PMs also hghlghted the potental of the system to support other n-depth analyss. PM2 partcularly commented that The vsualzaton s good at enablng qualtatve analyss. It would be desred that quanttatve analyss can also be supported. The partcpants also had some concerns. PM1 and PM4 had concerns about the ntutveness of the system. Whle t was easy for them to understand each ndvdual vew, they felt dffcult to lnk them together. Nevertheless, they both agreed that after a short tme tranng they could get used to the lnked system. 6.4 Dscusson The search engne logs exhbt rch and valuable user nformaton, whch allow us to acqure a better understandng of user loyalty analyss and derve a set of consderate desgn prncples. Although we manly demonstrate our vsualzaton technques usng search engne logs, the technques could be easly adapted to other problems, such as user engagement on e-learnng courses, whch share smlar data characterstcs and task requrements. We employ a force-drected layout algorthm based on dfferent effectveness and aesthetcs crtera to generate a layout. The layout algorthm works well when most branches do not cross more than two layers. It may fal to create an effectve layout n some extreme cases n whch there are frequent and dramatc exchanges of quanttes across the dstant layers. In ths case, there would be a sgnfcant amount of clutter caused by crossngs between the branches and the layers. The vsual desgn based on color bars [30] would be helpful n these scenaros. Nevertheless, we beleve that our desgn works for many scenaros of customer loyalty analyss as the loyalty degree of a customer wll not dramatcally change n most cases [27]. From the feedback from the expert revew, the flow vew layout s easy to be understood. One typcal beneft of t s to allow an analyst to nteractvely specfy a farly complex query to fnd a group of users based on the user flow. However, the brushng nteractons across multple lnked vews may cause some learnng costs, as we dscussed n Secton 6.3. In the case study, we vsually compared three search engnes usng our system. It can naturally support more search engnes by addng more flow vews. But due to the lmted screen space and capablty of users for comparson tasks [47], the comparson for more than four engnes are not recommended. In LoyalTracker, we offer one densty map vew and one word cloud vew. When conductng comparatve analyss, t poses a challenge for analysts to vsually compare multple word clouds or multple densty maps. However, the smooth and responsve user nteracton can largely help crcumvent ths problem. Analyst can quckly brush and see the results mmedately. The case studes and expert revews also confrm the usablty and effectveness of current nterface and nteracton desgn. In the flow vew, we use dfferent colors to encode the categores of loyalty. Accordng to the prevous study [18] and the suggeston from our experts, we classfy the users nto four groups. The flow vew allows more than four categores. However, as people can effcently dstngush only a dozen colors [39], the number of the loyalty categores should be less than twelve. The user log data we used also has some lmtatons: Uncertanty. Users are recognzed by the ID stored n the cooke of a web browser n the data. Thus, when users change to another web browser or clean the cookes, we cannot track them any more. Furthermore, the user log data assumes one web browser n one computer used by only one person, whch s not always the case. Although all the logs are stored anonymously, t s feasble to nfer f two users are the same or not usng some data felds such as IP address or the clckng behavors, whch could ntroduce uncertanty to the data. Some vsual hnts can be dsplayed to keep user aware of the uncertanty. Scalablty. The data processng part cannot be done real-tme. Nevertheless, after that LoyalTracker can acheve nteractve performance. From the case study, we found that explanng some patterns needs more orgnal raw data from the data center. Due to the large scale of the data and the lmted computng resource we have, n the current stage we cannot retreve the raw data n real tme. However, ths problem can be solved n the future by applyng n more computng resource. 7 CONCLUSION AND FUTURE WORK In ths paper, we systematcally study the effectve vsualzaton of customer loyalty and swtchng behavor from massve data sets n busness ntellgence. We derve a set of desgn prncples to address the most mportant questons rased by the doman experts of loyalty analyss. Guded by the prncples, we propose a vsual analytcs system. The system conssts of three vews: a flow vew, a densty map vew, and a word cloud vew. We desgn a new vsualzaton technque n the flow vew, based on a flow metaphor to nteractvely reveal the evolvng patterns of customer loyalty and defecton. The other two vews: a densty map vew and a word cloud vew enable n-depth analyss. Case studes and the ntervew wth doman experts demonstrated the usefulness of the flow vew for showng the overall loyalty trend along the tme. The flow vew plays a prmary role n dscoverng the patterns we just dscussed. It enables analysts to quckly and ntutvely construct a farly complex queres, such that they can perform detaled analyss and confrm the fndngs n the densty map and word cloud. In the future, we plan to release a web based verson of LoyalTracker to the search technology department of the company wth a larger number of potental users. We are also planng to mprove the nteracton desgn n order to reduce the learnng curve and make the system easer to use. For example, to reduce the complexty of the brushng nteracton, more vsual hnts wll be used to mark analysts nteractons (.e., unon operatons and ntersecton operatons) so that the brushng results can be easly understood. Also, when analysts explore multple flow vews, synced zoom & pan operatons wll be offered. Moreover, we plan to apply our technques to other smlar problems. For example, we can use them to track and analyze the dynamcs of user engagement n e-learnng courses from web log data. Thus, our technques can be used by a large group of audence, whch enable us to get rcher feedback and suggestons. We wll also further evaluate our technques usng a formal user study. Vsualzaton of the co-evoluton pattern of loyalty and satsfacton s another potental future research drecton. ACKNOWLEDGMENTS The authors wsh to thank Avz group n Inra, France, for ther knd help on revsng the paper and the anonymous revewers for ther valuable comments. Ths research was supported n part by HK RGC GRF , the Natonal Basc Research Program of Chna (973 Program) under Grant No. 2014CB340304, the Natonal Natural Scence Foundaton of Chna , and FDYT LYM11113.

10 1742 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. 12, DECEMBER 2014 REFERENCES [1] E. W. Anderson and V. Mttal. Strengthenng the satsfacton-proft chan. Journal of Servce Research, 3(2): , [2] L. Byron and M. Wattenberg. Stacked graphs geometry & aesthetcs. IEEE Transactons on Vsualzaton and Computer Graphcs, 14(6): , [3] S. K. Card, P. Proll, M. Van Der Wege, J. B. Morrson, R. W. Reeder, P. K. Schraedley, and J. Boshart. Informaton scent as a drver of web behavor graphs: results of a protocol analyss method for web usablty. In Proceedngs of the SIGCHI Conference on Human Factors n Computng Systems, pages , [4] E. H. Ch. Improvng web usablty through vsualzaton. IEEE Internet Computng, 6(2):64 71, [5] W. Cu, S. Lu, L. Tan, C. Sh, Y. Song, Z. J. Gao, X. Tong, and H. Qu. Textflow: towards better understandng of evolvng topcs n text. IEEE Transactons on Vsualzaton and Computer Graphcs, 17(12): , [6] M. Dörk, D. M. Gruen, C. Wllamson, and M. S. T. Carpendale. A vsual backchannel for large-scale events. IEEE Transactons on Vsualzaton and Computer Graphcs, 16(6): , [7] G. Dupret, V. Murdock, and B. Pwowarsk. Web search engne evaluaton usng clckthrough data and a user model. In WWW workshop Query Log Analyss: Socal and Technologcal Challenges, number Aprl. [8] M. Gray, A. N. Badre, and M. J. Guzdal. Vsualzng usablty log data. In Proceedngs of the IEEE Symposum on Informaton Vsualzaton, pages 93 98, [9] Q. Guo, R. W. Whte, Y. Zhang, B. Anderson, and S. T. Dumas. Why searchers swtch: understandng and predctng engne swtchng ratonales. In Proceedngs of the ACM SIGIR conference on Research and development n nformaton retreval, pages , [10] M. Guzdal, P. Santos, A. Badre, S. Hudson, and M. Gray. Analyzng and vsualzng log fles: A computatonal scence of usablty. Techncal report, Georga Insttute of Technology, [11] M. Hagen, M. Potthast, B. Sten, and C. Bräutgam. Query segmentaton revsted. In Proceedngs of the 20th nternatonal conference on World wde web - WWW 11, page 97, New York, USA, Mar [12] A. Hassan, X. Sh, N. Craswell, and B. Ramsey. Beyond Clcks: Query Reformulaton as a Predctor of Search Satsfacton. In Proceedngs of the 22nd ACM nternatonal conference on Conference on nformaton & knowledge management, pages , New York,USA, [13] S. Havre, E. Hetzler, P. Whtney, and L. Nowell. Themerver: Vsualzng thematc changes n large document collectons. IEEE Transactons on Vsualzaton and Computer Graphcs, 8(1):9 20, [14] V. Hu, M. Stone, J. Pedersen, and R. W. Whte. Effects of search success on search engne re-use. In Proceedngs of the 20th ACM nternatonal conference on Informaton and knowledge management, page 1841, New York, New York, USA, Oct ACM Press. [15] J. Huang and E. N. Efthmads. Analyzng and evaluatng query reformulaton strateges n web search logs. In Proceedng of the 18th ACM conference on Informaton and knowledge management, page 77, New York, New York, USA, Nov ACM Press. [16] Y.-F. Juan and C.-C. Chang. An analyss of search engne swtchng behavor usng clck streams. In Proceedngs of the Frst Internatonal Workshop on Internet and Network Economcs, pages , [17] S. M. Keaveney and M. Parthasarathy. Customer swtchng behavor n onlne servces: An exploratory study of the role of selected atttudnal, behavoral, and demographc factors. Journal of the Academy of Marketng Scence, 29(4): , [18] P. Kotler and K. Keller. Marketng Management. Prentce Hall, 14th edton, [19] H. Lam, D. Russell, D. Tang, and T. Munzner. Sesson vewer: Vsual exploratory analyss of web sesson logs. In IEEE Symposum on Vsual Analytcs Scence and Technology, pages , [20] J. Lee, M. Podlaseck, E. Schonberg, and R. Hoch. Vsualzaton and analyss of clckstream data of onlne stores for understandng web merchandsng. Data Mnng and Knowledge Dscovery, 5(1/2):59 84, [21] S. Lu, W. Cu, Y. Wu, and M. Lu. A survey on nformaton vsualzaton: recent advances and challenges. The Vsual Computer, pages 1 21, [22] S. Lu, M. X. Zhou, S. Pan, W. Qan, W. Ca, and X. Lan. Interactve, topc-based vsual text summarzaton and analyss. In Proceedngs of the ACM Conference on Informaton and Knowledge Management, pages , [23] T. Mukhopadhyay, U. Rajan, and R. Telang. Competton between nternet search engnes. In Proceedngs of the Annual Hawa Internatonal Conference on System Scences, [24] R. Munroe. A hstory of the unted states congress partsan and deologcal makeup. [25] D. Phan, L. Xao, R. B. Yeh, P. Hanrahan, and T. Wnograd. Flow map layout. In Proceedngs of IEEE Symposum on Informaton Vsualzaton, pages , [26] J. Ptkow and K. A. Bharat. Webvz: A tool for world-wde web access log analyss. In Proceedngs of the Internatonal World Wde Web Conference, pages , [27] F. F. Rechheld and P. Schefter. E-Loyalty: your secret weapon on the web. Harvard Busness Revew, 78(4): , [28] F. F. Rechheld and J. W. Earl Sasser. Zero defectons: qualty comes to servces. Harvard Busness Revew, 68(5): , [29] Z. Shen, J. We, N. Sundaresan, and K.-L. Ma. Vsual analyss of massve web sesson data. In Proceedngs of the IEEE Symposum on Large Data Analyss and Vsualzaton, pages 65 72, [30] C. Sh, W. Cu, S. Lu, P. Xu, W. Chen, and H. Qu. RankExplorer: Vsualzaton of rankng changes n large tme seres data. IEEE Transactons on Vsualzaton and Computer Graphcs, 18(12): , [31] L. Sh, F. We, S. Lu, L. Tan, X. Lan, and M. X. Zhou. Understandng text corpora wth multple facets. In Proceedngs of the IEEE Symposum on Vsual Analytcs Scence and Technology, pages , [32] Y. Song and L.-w. He. Optmal rare query suggeston wth mplct user feedback. In Proceedngs of the 19th Internatonal Conference on World Wde Web, WWW 10, pages , New York, NY, USA, ACM. [33] Y. Song, X. Sh, and X. Fu. Evaluatng and predctng user engagement change wth degraded search relevance. In Proceedngs of the Internatonal World Wde Web Conference (To appear), [34] G.-D. Sun, Y.-C. Wu, R.-H. Lang, and S.-X. Lu. A survey of vsual analytcs technques and applcatons: State-of-the-art research and future challenges. Computer Scence and Technology, 28(5): , [35] Y. Tanahash and K.-L. Ma. Desgn consderatons for optmzng storylne vsualzatons. IEEE Transactons on Vsualzaton and Computer Graphcs, 18(12): , [36] H. Tsuj and R. Saga. Vsualzed preference transton network based on recency and frequency. IEMS, 10(4): , [37] R. van den Berg, F. W. Cornelssen, and J. B. T. M. Roerdnk. A crowdng model of vsual clutter. Journal of Vson, 9(4):Artcle 24, [38] F. B. Végas, M. Wattenberg, and J. Fenberg. Partcpatory vsualzaton wth wordle. IEEE Transactons on Vsualzaton and Computer Graphcs, 15(6): , [39] C. Ware. Informaton Vsualzaton, Thrd Edton: Percepton for Desgn (Interactve Technologes). Morgan Kaufmann, 3rd edton, [40] S. J. Waterson, J. I. Hong, T. Sohn, J. A. Landay, J. Heer, and T. Matthews. What dd they do? understandng clckstreams wth the WebQult vsualzaton system. In Proceedngs of the Workng Conference on Advanced Vsual Interfaces, pages , [41] J. We, Z. Shen, N. Sundaresan, and K.-L. Ma. Vsual cluster exploraton of web clckstream data. In Proceedngs of the IEEE Symposum on Vsual Analytcs Scence and Technology, pages 3 12, [42] R. W. Whte and S. T. Dumas. Characterzng and predctng search engne swtchng behavor. In Proceedngs of the ACM conference on Informaton and knowledge management, pages 87 96, [43] R. W. Whte, A. Kapoor, and S. T. Dumas. Modelng long-term search engne usage. In Proceedngs of the nternatonal conference on User Modelng, Adaptaton, and Personalzaton, pages 28 39, [44] R. W. Whte, M. Rchardson, M. Blenko, and A. P. Heath. Enhancng web search by promotng multple search engne use. In Proceedngs of the ACM SIGIR conference on Research and development n nformaton retreval, pages 43 50, [45] K. Wongsuphasawat and D. Gotz. Explorng flow, factors, and outcomes of temporal event sequences wth the outflow vsualzaton. IEEE Transactons on Vsualzaton and Computer Graphcs, 18(12): , [46] Z. Yang and R. T. Peterson. Customer perceved value, satsfacton, and loyalty: The role of swtchng costs. Psychology & Marketng, 21(10): , [47] S. Yants. Multelement vsual trackng: attenton and perceptual organzaton. Cogntve psychology, 24(3): , July [48] J. Zhang, C. Chen, M. S. Vogeley, D. Pan, A. Thakar, and J. Raddck. SDSS Log Vewer: vsual exploratory analyss of large-volume sql log data. In Proceedngs of the Vsualzaton and Data Analyss, 2012.