The Architecture of a Churn Prediction System Based on Stream Mining

Transcription

1 The Archiecure of a Churn Predicion Sysem Based on Sream Mining Borja Balle a, Bernardino Casas a, Alex Caarineu a, Ricard Gavaldà a, David Manzano-Macho b a Universia Poliècnica de Caalunya - BarcelonaTech. {balle,bcasas,caarineu,gavalda}@lsi.upc.edu b Ericsson Spain. david.manzano.macho@ericsson.com Absrac. Churning is he movemen of cusomers from a company o anoher. For any company, being able o predic wih some ime which of heir cusomers will churn is essenial o ake acions in order o reain hem, and for his reason mos secors inves subsanial effor in echniques for (semi)auomaically predicing churning, and daa mining and machine learning are among he echniques successfully used o his effec. In his paper we describe a prooype for churn predicion using sream mining mehods, which offer he addiional promise of deecing new paerns of churn in real-ime sreams of high-speed daa, and adaping quickly o a changing realiy. The prooype is implemened on op of he MOA (Massive Online Analysis) framework for sream mining. The applicaion implici in he prooype is he elecommunicaion operaor (mobile phone) secor. A shorer version of his paper, omiing Secion 5, was presened a CCIA 13 (hp://mon.uvic.ca/ccia2013/en/). Keywords. Daa sream mining, churn predicion, Hoeffding rees, machine learning, MOA 1. Inroducion Cusomer churning is he movemen of cusomers from a company o anoher. For any company, being able o predic which of heir cusomers will churn before hey acually leave is essenial o ake acions in order o reain hem. For his reason mos secors inves subsanial effor in echniques for (semi)auomaically predicing churning as accuraely and as early as possible. Tradiional daa mining / machine learning echniques are ofen applied for building predicive models ha assign each cusomer a probabiliy ha s/he churns in one or more prespecified periods of ime. However, hese echniques have a number of limiaions. Firs, hey end o be bach-based: all he daa o be used for he modeling mus be available upfron for he model consrucion; many of hem canno work incremenally, i.e., incorporae ino he model informaion arrived afer i has been buil; he model consrucion is compuaionally cosly, and ofen needs human inervenion for e.g. parameer opimizaion, daa

2 selecion, or model evaluaion; and, mos imporanly, mehods assume ha he source of daa is saionary, ha is, does no change in naure over ime. The laer assumpion is paenly false in many scenarios: cusomers change heir behavior over ime in reacion o marke condiions, produc or price changes, or sociological phenomena. Someimes change is gradual, bu abrup, overnigh change can also occur. The problem is paricularly acue in secors such as he elecommunicaions marke (say, companies o which cusomers subscribe for mobile phone service), which are affeced by high cusomer churn rae. In his seing, models buil someimes in he pas can quickly or gradually ge ou of sync wih he curren cusomer paerns, resuling in subopimal churn predicion rae and herefore revenue loss. An obvious soluion is o rebuild he models periodically, bu his ypically eiher requires human analys ime (which is slow and expensive) or auomaizing decisions such as when and on which amoun daa o rerain new models, which may be hard o do in absrac. Daa sream mining echniques have emerged in he las decade or so o ranspor he benefis of daa mining o such scenarios: hey inend o provide e.g. model inducion echniques ha work on sreams of daa ha may poenially never end, ha can process housands or millions of insances per second boh for raining and for predicion, and ha can mainain hese models updaed as he saisical or logical naure of he daa change over ime. In his paper we describe he archiecure and a proof-of-concep implemenaion of a sysem for churn predicion based on sream mining, wih mobile phone subscriber churning as he background scenario. The use of sream mining mehods offer he promise of deecing new paerns of churn in real-ime sreams of high-speed daa, and adaping quickly o changing realiies. The core sream mining plaform used is MOA (Massive Online Analysis) [2], and in paricular we use is implemenaion of a varian of decision rees (Hoeffding rees) ha can be buil and mainained efficienly and incremenally from sreams. Unforunaely, as usual in his field for e.g. privacy reasons, we could no use real daa for our experimens. We developed a synheic daa generaor based on descripions of real daa available in he lieraure, which may be of ineres in iself. The main conclusion from our proof-of-concep is ha he echnology exiss o perform real-ime, high-hroughpu predicion on sreams of iems conaining informaion similar o ha conained in churn predicion lieraure. On furher work we will concenrae on several issues concerning scalabiliy o inernaional scale daa volumes. The paper is srucured as follows: Secion 2 discusses in deail he problem of churning, paricularly in he elecommunicaions marke and relaed previous work. Secion 3 describes he requiremens for a churn predicion sysem in a high-speed, highly volaile scenario such as mobile phone cusomer behavior analysis. Secion 4 describes he archiecure of he proposed soluion and is main elemens, as well as some deails of he implemenaion. Secion 5 presens an example of he workflow of evens processed by he sysem. Secion 6 describes he generaor of synheic daa used o es he sysem. Secion 7 describes qualiaively he resuls of he prooype. Finally, Secion 8 summarizes he lessons

3 learned during he implemenaion and esing, he main challenges o be solved for a large-scale, fully scalable soluion, and oher fuure work. 2. Preliminaries 2.1. The Problem of Churning in he elecommunicaion marke Like in every business secor, operaors are always looking o ge a beer undersanding abou heir cusomers preferences and heir saisfacion o offer hem beer producs and services. Despie of he mauriy of his marke and due o subscriber growh down and revenues fla, churn managemen has become ino one of he mos pressing problems faced by operaors. There are many reasons ha may affec a subscriber on deciding o churn. Some of hem may be: In conras o pos-paid cusomers, prepaid cusomers are no bound by a conrac. The cenral problem concerning prepaid cusomers is ha he acual churn dae in mos cases is difficul o assess. Cusomer loyaly is direcly relaed o he cusomer service and service experience. Lack of connecion capabiliies or qualiy in places where he cusomer requires service can cause cusomers o abandon heir curren service provider in favor of ohers wih broader reach or a more robus nework. Besides, a slow response o cusomer complains or billing errors are sure pahs o a cusomer relaions disaser. Finally oher facors such as he pricing, lack of feaures (cusomers may swich carriers for feaures no provided by heir curren providers) or coverage, new echnology inroduced by compeiors (for example, high-speed daa) or he fac ha new compeiors ener he marke, are also reason ha has heir influence on he churn rae. Churn predicion modeling echniques aemp o undersand he precise cusomer behaviors and aribues which signal he risk of cusomer churn. The accuracy of he echnique used is obviously criical o he success of any proacive reenion effors. In order o have successful loyaly programs operaors need o analyze heir cusomers based on several parameers such as spending and usage of each service and produc as well as heir behavior and raffic paerns. All hese decisions should be made upon he analysis of he daa relaed o cusomer. Time is criical, which means ha online approaches are gaining more momenum o suppor operaional decisions in almos real ime. By means of online analysis, an operaor can launch and execue quickly some acions wih aim of reaining some cusomers, and measure he impac of he campaign in real ime which allows adaping hem insanly as required Previous Work There is subsanial published lieraure on churn predicion echniques, alhough probably mos of he experiences have remained unpublished in he company. We menion only a few represenaive ones of hose using daa mining, machine learning, or, more in general, ha build models from informaion gahered in

4 he pas; see [8] for a good survey. References [1,11,5,10] use mehods such as decision rees, suppor vecor machines, geneic algorihms, and mulilayer neural neworks for churn predicion in various conexs, from finanial-service providers o landline phones and mobile phones; [6] carries ou an inensive and sae-of-he ar comparison of his and oher mehods. [9] discusses he problem of evaluaing predicors in his conex. [12] discusses he use of social nework informaion (and cusomer social circle) o help in churn predicion. As menioned in he inroducion, all hese works and hose ha we are aware of use he bach paradigm: a model is buil offline by he daa analys and hen used online o make predicions. Some of he works do menion he possibiliy of reraining he model periodically. By conras, we aim a echniques ha allow building he models online oo and keeping hem accurae as he cusomers change heir behavior. We omi he many oher papers discuss he feaures acually used in pracice and he feaure engineering process, as well as he business implicaions of churn deecion and possible cusomer reenion sraegies once hey have been idenified as possible churners. For background on sream mining, see for example he books [3,4]. 3. Requiremens The elecommunicaion marke is highly and unpredicably dynamic. The abiliy for immediaely deecing such a rend on even a small segmen of subscribers may be essenial o reaining many more leaving in he near fuure. However, he reasons why subscribers churn may change suddenly. For many (and hard o define a priori) reasons, subscribers ha were no considering leaving heir company may suddenly consider anoher company very compelling and decide o leave overnigh. A scenario like his demands he applicaion of online analyics, o deec and reac in imely manner o he changes in he expeced behavior of operaors cusomers. Oher requiremens are: Accuracy. High recall (all churners are flagged as such) and relaively high precision (no many non-churners are flagged) are boh imporan. Performance. Time is criical. A righ acion aken ou of is proper window ime is useless. Deec and reac o any relevan change deeced hrough he analysis of he incoming daa flows is a compeiive advanage. Flexibiliy. Daa sources may change and any churning sysem has o have he abiliy o be adaped ino he new scenario. Scalabiliy. Churn predicion echnology needs o handle difficul conexs, in which here are big daa flows relaed o cusomers aciviy, wih realime requiremens and prone o changes. Abiliy o segmen cusomers and incorporae analys s exising knowledge. Cusomer profiling demonsraes he direc business benefi gained from analyics. Tradiional parameers include cusomer ype, spending, subscripion ype and preferred services.

5 Figure 1. Archiecure and Design of a Plaform for Adapive, Real-ime Churn Predicion 4. Archiecure 4.1. General Descripion In his paper, we describe he archiecure proposed for our prooype. I is depiced in Figure 1, and is main componens are described nex Inegraor The sysem processes as inpu a number of sreams wih diverse informaion. For example, a sream of call records, a sream wih billing acions by he company and bill paymens by he subscribers, conens from social neworking services such as Twier, ec. The Inegraor module inegraes all hese sreams, generaing a logically unique sream of evens. We disinguish several ype of evens, including a leas a subscriber joining he company, calls and SMSes, complains, bills emied and bills paid, wees by a subscriber, and churn evens (e.g., a user explicily has lef he company, or for a prepaid user, i has been declared as a churner according o he company s crierion) Daa Manager The Daa Manager module manages he cusomer informaion daabase and he Pending Predicions queue. The cusomer informaion daabase conains basic informaion abou our subscribers (age, address, ype of conrac, ec.) as well as highly dynamic informaion (e.g. las numbers called, mos frequenly numbers called). The Pending Predicions queue conains all predicions ha are awaiing for confirmaion or

6 refusal in he fuure (ha is, wheher he subscriber o which he predicion refers o has churned or no wihin a specified ime frame). These wo srucures, he cusomer informaion daabase and he Pending predicion queue, can easily become he wo main bolenecks in he sysem if no carefully implemened, boh for ime and for memory usage. If hey do no fi in RAM, a wrie-opimized disk-residen daabase will be required Record Generaor The Record Generaor receives he sream of Evens generaed by he inegraor and uses i for wo purposes. Firs, i updaes he cusomer informaion daabase according o each Even. Second, i generaes one or more Records from each Even using informaion from he cusomer informaion daabase. Thus, i creaes a sream of Records passed downsream. A Record is a vecor of feaures, he firs of which is a subscriber idenifier, and he res conains all he informaion abou ha subscriber sae ha is considered relevan for predicion. Many of hem canno be direcly derived from he Even, bu are aggregaions of informaion abou he cusomer precompued in he daabase. One of he feaures (say, he las one) indicaes if he subscriber has churned so far: i will be rue when he Even originaing he record was a churning one. The prooype currenly use his se of feaures for predicion, which figure among he mos widely repored as useful in he lieraure: Age, sex, income range Conrac ype (mobile or landline, pre-paid or pos-paid) Average call duraion during las monh Number of calls las week and las monh Increase of decrease in number of calls in he las 2 weeks and he las 2 monhs % of calls by/o his subscriber where he caller/recipien belongs o anoher company # of complains in he las 2 monhs, and % of hese ha were resolved saisfacorily Average bill value Increase or decrease of value las 2 bills We made he following opimizaion for efficiency. Every even gives rise o a leas one record, wih he excepion ha every day only one call by or from a subscriber generaes a record and a predicion. Tha is, if a subscriber ges or receives 20 calls in a day, all of hem will be used o updae his/her saisics in he cusomer daabase, bu only he firs one will generae a record and a predicion. This inroduces a delay of (a mos) one day in flagging his cusomer as churner, bu reduces a lo of overhead Record Processor The Record Processor is he hear of he sysem: i builds, mainains, and applies he predicive models. I herefore conains he daa mining or machine learning algorihms ha make predicion possible.

7 When a record no indicaing churn arrives, i passes he record hrough he curren model and makes a churn predicion for i. The record wih is predicion is sored he Pending Predicions queue, waiing for fuure confirmaion. When a record indicaing churn arrives, records for ha subscriber are searched in he Pending Predicions queue and, if found, passed o he model rainer as posiive insances of churning. Expired records in he Pending Predicions queue (corresponding o subscribers ha did no churn wihin a specified ime) are passed o he model rainer as negaive insances of churn. All records (describing curren saes of subscribers) are passed a clusering submodule o build subscriber profiles. We have used boh 1) clusering mehods available in MOA and 2) he spli induced by he Hoeffding ree branches o define cusomer segmens; hey may give alernae segmenaions of poenial use for analysis. Addiionally, a background process periodically scans cusomers for which no Even has occurred and injecs a special record indicaing no aciviy, so ha 1) a predicion for he cusomer is generaed from ime o ime (in case e.g. inaciviy may indicae churning propension) and 2) he sysem is also rained o predic well on periods of cusomer inaciviy Reporing and Inerface o oher Sysems The Record Processor module hus produces predicions, saisics and profiles of he prediced churners. The prediced churners id s and heir curren profiles are passed o he user inerface or oher pars of he cusomer managemen sysem so ha adequae acions can be assessed and aken. The subscriber profiles provide informaion for human analyss o build undersandable porrais of churners and causes for heir churning. Moreover i allows o focus he reenion acion effors, such as calling wih a promoion, o subses of he subscribers wih propensiy o churning, even before hey are flagged as churners by he sysem Implemenaion We implemened a prooype of his archiecure using he Java language. The predicion core of he sysem uses MOA (Massive Online Analysis), a plaform relaed o he popular WEKA machine learning package, which includes a collecion of machine learning algorihms (classificaion, regression, and clusering) and ools for algorihm evaluaion, visualizaion, synheic daa generaion, and sream managemen. We used in paricular MOA s Hoeffding Adapive Tree classifier, which is a variaion of he CVFDT mehod proposed in a seminal paper on sream mining by Hulen, Spencer, and Domingos [7]. A Hoeffding ree is an incremenal, anyime decision ree inducion algorihm ha is capable of learning from massive daa sreams. The Adapive version in [7] and implemened in MOA is able o adap o changes of he daa over ime, updaing and revising he srucure and conens of he ree o keep i accurae.

8 5. Example Workflow The following illusraes by a few examples he process workflow. Suppose ha Alice, currenly a company cusomer, calls Bob, who is currenly subscribed o anoher company. The (exernal) call managemen sysem passes he idenifiers (e.g. numbers) of Alice and Bob ogeher wih he informaion deemed relevan (say, sar ime, duraion, approximae locaion of Alice and Bob if available) o he Inegraor module, which generaes a Call Even and passes i o he Record generaor. The Record generaor queries is cusomer daabase, deermines ha Alice is a cusomer bu Bob is no, so i 1) updaes Alice s informaion in he daabase recording ha she paricipaed in his call and 2) generaes a record wih Alice as idenifier, as described nex. The record generaed has Alice s idenifier, and consiss of a vecor of feaures describing as accuraely as possible he sae of Alice a his poin in ime as recorded in he daabase. The record is passed o he Record processor. As i encodes a call, he Record processor will inpu i o he curren predicion model, who will oupu a predicion abou Alice s probabiliy of churning in some period of reference such as a monh; his is called he predicion expiry dae. The record, ogeher wih he predicion, is placed in he PendingPredicion queue, as i has o be verified in he fuure, and also passed o he Loyaly managemen module. The laer decides according o curren rules if he cusomer meris some reenion acion, and which one. The process is similar for mos oher ypes for evens generaed by Alice (SMS, bill paymen, complain, wee, ec., bu no hose indicaing cusomer churn): One record associaed o Alice is generaed, for which a predicion is generaed. The pair (record,predicion) is sored in he PendingPredicion queue and used by he Loyaly Managemen module o possibly generae acions aimed a reaining Alice. Concurrenly wih processing hese incoming records he Record Processor moniors he PendingQueue for expired predicions. These records are labeled nochurn, meaning ha a cusomer did no churn wihin a prescribed period, and passed o he model updaer, which will use o confirm, updae, or revise he curren model (in he usually called raining model). For example, a pending predicion creaed on April 16h indicaing ha Alice may churn wih probabiliy 60% wihin one monh, will be removed from he queue and processed as below by May 16h, because i is known a ha ime ha Alice has indeed no churned in one monh. When cusomer Alice is deermined o have churned (eiher by explici acion on her par or by e.g. lack of any aciviy in prepaid conracs), he sysem should receive a Churn even. The Record generaor will generae a record wih Alice s idenifier and conen Churn and he churning dae and pass i o he Record processor. The Record processor rerieves any exising records associaed o Alice in he PendingPredicion queue (which should be non-expired, as by he previous paragraph), and labels hem wih he Churn predicion, meaning ha he righ predicion a he ime he record was produced should have been Churn. This record is also passed o he model updaer. When eiher a predicion abou Alice expires or a record indicaing ha Alice has Churned, he Record Processor noifies he Loyaly managemen module so

9 ha his informaion can be aken ino accoun (by he module s inernal rules) when acions have o be proposed o cusomers wih a profile similar o Alice s. As a hypoheical example of wha model updae means, suppose ha a he ime in which Alice s call arrives, one of he rules used by he sysem o predic churn is cusomers who are female, aged 20-35, wih wo or more complains abou service in he las monh, paying an average monhly bill > 200, and who have used roaming in he las 6 monhs have a probabiliy of churning in he nex monh of 80%. The sysem addiionally keeps rack ha i has recenly seen 1000 such cusomers, of which 800 have indeed churned wihin one monh (hence he 80% probabiliy). If Alice fis ino his profile, he sysem will predic ha she will churn in he nex monh wihin 80%. Afer one monh, if she has indeed churned, he sysem will updae is saisics o 801/1001, so he rule will be updaed o... wih 80.1% probabiliy, and oherwise o 800/1001. More ineresingly, he sysem may realize ha for cusomers in his profile living in urban areas, he probabiliy is in fac higher (say, 90%), and lower hose living in rural areas is in fac lower (say, 70%). Then he rule above would be spli ino wo rules, he firs of which would be: cusomers who are female, aged 20-35, wih wo or more complains abou service in he las monh, paying an average monhly bill > 200, who have used roaming in he las 6 monhs, and live in urban areas have a probabiliy of churning in he nex monh of 90%. And he second would apply o cusomers living in rural areas and have a probabiliy of 70%. This descripion, inended for comprehension, does no necessarily reflec any of he known sream mining echniques ha could be used in an implemenaion of he invenion. 6. The Synheic Daa Generaor In order o es he proposed sysem, we developed a simulaor able o generae realisic synheic daa similar o wha an operaor ges from heir cusomers. Basically, he simulaion is a probabilisic dynamical sysem conaining a predeermined number of cusomers. The sysem ries o reproduce he major ypes of ineracions beween cusomers and providers and heir dynamics. For example, cusomers can place calls o oher cusomers, eiher having he same provider or a differen one. Cusomers are periodically billed for heir calls, and big bills may make hem angry. Angry cusomers someimes complain o heir provider and ask for cheaper raes, or maybe churn wihou furher noice. In paricular, each user is modeled in he sysem using five parameers. Three of hem are socio-economic facors which are known o providers: gender, age, and income. The oher wo are indices of how communicaive (C) and impulsive (I) he user is; hese are hidden parameers unknown o he provider. Furhermore, hese indices evolve over ime depending on he mood of he user, which is influenced by several facors described below. A sar users are generaed by sampling hese parameers from a fixed probabiliy disribuion. In paricular, gender, age and income follow a disribuion rained real daa. The oher indices are sampled from a hand-crafed disribuion, and in paricular are no independen of he socioeconomic facors. As a resul, he simulaion is populaed wih users ha will behave differenly from one anoher, and whose profile is loosely correlaed wih he feaures known a priori by he provider. This scheme is depiced in Figure 2.

10 Gender Age Income C I Mood Call Complain Figure 2. Facors affecing mood and behavior The acions of each user are governed by a dynamic markovian model whose curren sae deermines he user s mood, which is in one of four saes: {happy,neural,angry,churn}. The dynamics of his model are as follows: 1. Time beween sae changes is larger for smaller values of I 2. The more ime spen in angry sae, he higher probabiliy of churning 3. A high bill (w.r.. he subscriber income) or an unresolved complain makes you more angry (moves you one sae owards angry or, if you are already in angry, makes your churning probabiliy higher), wih a probabiliy ha depends on I 4. A complain resolved ok makes you go back owards happy, wih a probabiliy ha depends on I This inernal mood sae, which is unknown o he provider, affecs he behavior of he user in muliple ways. 1. A user only complains if Angry (and he longer he ime in Angry, he more s/he complains. 2. A user only churns if Angry (and his becomes more likely he longer ime s/he s been Angry ) 3. The longer ime in Angry, he less s/he calls. 4. The longer ime in Happy, he more s/he calls. 5. When s/he goes back o neural, he rae of calls per day goes back slowly owards he defaul value for he user. 6. Boh duraion and number of calls depend on he hidden parameer C. This scheme is depiced in Figure 3.

11 p n,n p n,h Neural p n,a p h,h Happy p h,n p a.n Angry p a,a p a,c Churn 7. Resuls and Scalabiliy Figure 3. Facors affecing mood and behavior We obain good levels of recall and precision, roughly o he poin ha he randomness ha we placed in he random generaor allows. Tha is, we can predic which users will churn wih an accuracy ha is close o he probabiliy wih which (randomly) decide o churn or no given heir inernal saes. In paricular, if we happen o make he subscribers absoluely deerminisic, we ge resuls close o 100%. Of course, he absolue goodness of hese figures does no mean much, oher han how difficul or easy o predic we made our synheic daa. The poin is ha he sysem is able o correcly remember and pu o use he informaion in he even sream for one paricular purpose, ha of churn predicion. We also checked ha Hoeffding rees are exremely good a adaping o changes. Via he prooype GUI we can vary during he execuion parameers such as prices of our company and he compeiion, frequency of complain calls and % of hose resolved saisfacorily, average number of calls per subscriber, ec. which affec our subscribers churn rae. We verified ha afer a change, predicion accuracy falls because he predicor ges ou of sync, bu afer a few housand calls, he new behaviors are capured by he ree and accuracy rises o almos opimal levels again. On a commodiy PC, he sysem processes abou 10,000 records per second. Average memory consumpion is abou 40Mbyes for each 1,000 subscribers wih more han realisic levels of average aciviies (40 calls day, 2% daily churn rae, ec.). Thus, here is ample room for upscaling using higher-end machines. For deploymen by large operaors, wih possibly many million subscribers, i is clear ha scaling ou by disribued processing would be necessary. Addiionally, he cusomer base would be geographically disribued over he plane, so

12 communicaion laencies among daaceners and raffic spliing and rouing mus be aken ino accoun. Finally, he emergence of new echnologies and services, as well as company culure, will undoubedly pu addiional consrains on he processing. From a daa mining poin of view, echniques for disribued model building will have o be incorporaed. In fac, building several models a geographically disinc locaion may be advanageous o capure differen cusomer paerns a differen zones. Since he models hemselves are compac, hey could possibly be exchanged among machines and sies and be used cooperaively (e.g., wih ensemble mehods) for beer accuracy. 8. Conclusions and Exensions We have hopefully shown ha sream mining echnology may help cusomer churn predicion on high-volume sreams originaing from cusomer aciviy. The main difference wih exiing, bach-oriened, daa mining approaches o he problem is he abiliy of hese echnologies for reacing and adaping fas o changes in cusomer behavior, wihou human inervenion, which may have a direc impac on revenue and image for companies. Alhough he sysem is a prooype far from being deployable, we have shown ha even on a single low-end machine we can deal wih quie high daa speeds and gracefully handle all he churn predicion process, including user segmenaion and connecing wih he cusomer relaion managemen subsysem. Furher work and addiional research includes esing he sysem wih real subscriber daa colleced from live neworks and combine addiional daa sources from ouside operaors boundaries. Acknowledgemens This work was suppored by a collaboraion agreemen beween Ericsson Research and U. Poliècnica de Caalunya. Research a UPC was also parially suppored by he BASMATI MICINN projec (TIN C04-03). We hank Alber Bife and Germán Blanco, for heir help a various sages. References [1] C. Archaux, H. Laanaya, A. Marin, and A. Khenchaf. An svm based churn deecor in prepaid mobile elephony. In Inl. Conf. on Informaion and Communicaion Technologies: from Theory o Applicaions (ICTTA), Damascus, Syria, April [2] Alber Bife, Geoff Holmes, Richard Kirkby, and Bernhard Pfahringer. MOA: Massive online analysis. J. Mach. Learn. Res., 11: , Augus [3] J. Gama. Knowledge Discovery from Daa Sreams. Daa Mining and Knowledge Discovery. Chapman & Hall/CRC, [4] J. Gama and M.M. Gaber. Learning from Daa Sreams: Processing Techniques in Sensor Neworks. New generaion compuing. Springer, [5] B. Q. Huang, T. M. Kechadi, B. Buckley, G. Kiernan, E. Keogh, and T. Rashid. A new feaure se wih new window echniques for cusomer churn predicion in land-line elecommunicaions. Exper Sys. Appl., 37(5): , May 2010.

13 [6] Bing Quan Huang, Mohand Tahar Kechadi, and Brian Buckley. Cusomer churn predicion in elecommunicaions. Exper Sys. Appl., 39(1): , [7] Geoff Hulen, Laurie Spencer, and Pedro Domingos. Mining ime-changing daa sreams. In Proc ACM SIGKDD Inl. Conf. on Knowledge Discovery and Daa Mining, pages , [8] Sahand KhakAbi, Mohammad R. Gholamian, and Moreza Namvar. Daa mining applicaions in cusomer churn managemen. In Proceedings of he 2010 Inernaional Conference on Inelligen Sysems, Modelling and Simulaion, ISMS 10, pages , Washingon, DC, USA, IEEE Compuer Sociey. [9] Sco A. Neslin, Sunil Gupa, Wagner Kamakura, Junxiang Lu, and Charloe H. Mason. Defecion Deecion: Measuring and Undersanding he Predicive Accuracy of Cusomer Churn Models. Journal of Markeing Research, 43(2): , May [10] P.C. Pendharkar. Geneic algorihm based neural nework approaches for predicing churn in cellular wireless nework services. Exper Sys. Appl., 36(3): , April [11] Ania Prinzie and Dirk Van den Poel. Incorporaing sequenial informaion ino radiional classificaion models by using an elemen/posiion-sensiive SAM. Decis. Suppor Sys., 42(2): , November [12] Yossi Richer, Elad Yom-Tov, and Noam Slonim. Predicing cusomer churn in mobile neworks hrough analysis of social groups. In SIAM Inl. Conf. on Daa Mining (SDM), pages SIAM, 2010.