An Inductive Fuzzy Classification Approach applied to Individual Marketing

Transcription

1 An Inductve Fuzzy Classfcaton Approach appled to Indvdual Marketng Mchael Kaufmann, Andreas Meer Abstract A data mnng methodology for an nductve fuzzy classfcaton s ntroduced. The nducton step s based on dervng fuzzy restrctons from data whose membershp functons are nferred from normalzed lkelhood ratos of target class membershp. A case study s presented where ths data mnng process s appled n predctve analytcs for an ndvdual marketng campagn n the onlne channel of a fnancal servce provder. Index Terms Data Mnng, Fuzzy Classfcaton, Fuzzy Logc, Predcton, Indvdual Marketng, Fuzzy Target Groups I. INTRODUCTION AND MOTIVATION Fuzzy Logc proposed by L. A. Zadeh s based on ntutve reasonng and takes nto account human subectvty and mprecson. Unlke classcal data mnng technques such as cluster or regresson analyss, fuzzy logc enables the use of classfcaton va fuzzy propostons comnbed by fuzzy logc operators. Fuzzy classfcaton creates more subtle and smooth dstnctons between equvalence classes than crsp classfcaton. [] and [2] presented the mplementaton of a classfcaton query language n order to flter data usng lngustc terms. They proposed the nvestgaton of a desgn methodology for the applcaton of fuzzy classfcaton n data mnng. In ths paper, a data mnng methodology usng nductve fuzzy classfcaton s ntroduced n order to address ths ssue. [3] emphaszes the nductve aspect of data mnng, namely learnng, and suggests a motvaton: Predcton. Ths explans how pattern models are to be dscovered and for what they can be used. If appled correctly, data models not only descrbe the past, but also help to forecast the future. In ths sense, successful data mnng s a practcal demonstraton of nductve logc: Learnng generalzatons from specfc examples. Hence, ths paper presents a methodology for the probablstc nducton of a multvarate fuzzy classfcaton based on lkelhood ratos. M. Kaufmann (correspondng author) s PhD-Student at the Research Center for Fuzzy Marketng Methods ( at the Unversty of Frbourg Swtzerland (e-mal: Mchael.Kaufmann@unfr.ch). A. Meer s Professor of Informaton Systems Resarch. He s the head of the Research Center for Fuzzy Marketng Methods at the Unversty of Frbourg Swtzerland. The paper s structured as follows: Secton II ntroduces nductve fuzzy classfcaton theoretcally. Secton III proposes a pragmatc methodology for an nductve fuzzy classfcaton. Secton IV presents a case study of a database marketng campagn where the target group has been selected usng the proposed method. Secton V summarzes the results and shows ssues for further research. II. THEORY OF INDUCTIVE FUZZY CLASSIFICATION Ths secton summarzes the theoretcal background of classfcaton, nducton and fuzzy restrctons n order to develop a theory of Inductve Fuzzy Classfcaton. A. Classfcaton Classfcaton s the process of groupng elements satsfyng the same characterstc constrants nto a set. In a class logc, a class C {e P(e)} s defned as a set C of elements e satsfyng predcate P. In the multvarate case, P refers to multple element attrbutes as a formula combned by logcal operators. B. Inductve Classfcaton Inductve classfcaton, a form of supervsed learnng ([3], [4]), s the process of learnng from examples to decde whether an element e belongs to a gven class y based on ts attrbutes. A tranng data set d s a (n+) m matrx wth n columns,, n, and a column Y ndcatng the class membershp. The columns, {,, n} are called dependent varables or attrbutes, and Y s called the target varable or the class label. The rows represent data elements e, {,, m}. The matrx content conssts of attrbute values x representng the value of the -th attrbute for the -th element, and of labels y representng the class membershp for the -th element. In case of a bnary classfcaton, for each row ndex the label y s equal to f and only f the element e s n class y. The learnng process nduces a model M from the tranng set d based on the dstrbuton of the target varable on the dependent varables, mappng from the Cartesan product of the dependent varable s domans nto the set {0,}. The model can be used to predct the class membershp of data elements. For an element e d n a new dataset d wth unknown class membershp, a predcton Y M(x,, x n ) can be computed by applyng the model to the element s attrbute values. When elements are grouped nto an

2 2 equvalence class y {e Y }, ths knd of classfcaton s nductve because the classfcaton constrants predcate has been learned from the data. C. Fuzzy Classfcaton Fuzzy classfcaton s the process of groupng elements nto a fuzzy set whose membershp functon s defned by the truth value of a fuzzy constrant predcate. A fuzzy class C s a fuzzy set [5] defned by a fuzzy proposton [6] e s R, where e s an element of a unverse of dscourse U and R s a fuzzy restrcton of U. The degree of membershp C (e) v(e s R) s defned by the truth value v of the correspondng fuzzy proposton.in the multvarate case, P s a fuzzy restrcton of U defned by a combnaton of fuzzy restrctons of attrbute domans combned by fuzzy logc operators. D. Inductve Fuzzy Classfcaton Consequently, Inductve Fuzzy Classfcaton (IFC) s the process of groupng elements nto a fuzzy set whose membershp functon s derved by supervsed learnng from data. In ths approach, the predctve model s represented by a multvarate fuzzy classfcaton. The fuzzy membershp functons of each attrbute are nduced from data usng a normalzaton of the lkelhood rato functon. The multvarate aggregaton of the unvarate fuzzy attrbute doman restrctons provdes the predcton of the classfer, whch can be defuzzfed usng an alpha-cut. The nducton step s not learnng of fuzzy rules themselves, as t s n many cases (e.g. [7], [8]) but nducng membershp functons to fuzzy restrctons of multple attrbute domans. The focus s not on the combnaton of fuzzfed nputs, but on the defnton of multvarate fuzzy set membershps for nducton, as presented n [9] and [0]. In fact, by usng lkelhood ratos for nducton, ths s a Bayes-lke approach, whose usefulness has been shown by []. III. A METHODOLOGY FOR INDUCTIVE FUZZY CLASSIFICATION As proposed by [], a desgn methodology s needed n order to apply fuzzy classfcaton to data mnng. In the followng secton, a conceptual framework for nductve fuzzy classfcaton s proposed. The am s to derve a multvarate fuzzy class nductvely n order to obtan a predctve classfcaton. The resultng classfer s able to perform a fuzzy classfcaton whch s nductve by learnng the fuzzy constrant predcate P from the lkelhood of class membershp. The resultng fuzzy class s based on a formula of fuzzy propostons on the dependent varables, whose truth values and correspondng fuzzy set membershp functons are derved from normalzed lkelhood ratos. Fgure : Inductve fuzzy classfcaton process As shown n Fgure, the process for dervng a fuzzy class by nducton conssts of preparng the data, selectng the relevant attrbutes, nducng membershp functons to the fuzzy restrctons on the attrbute domans from the tranng set, transformng the attrbute values nto unvarate fuzzy set membershp values, classfyng data elements by aggregatng the fuzzfed attrbutes nto a multvarate fuzzy classfcaton of data elements, and an evaluaton of the predctve performance of the resultng classfer. A. Inductve Fuzzy Classfcaton Process Overvew The dea of the process s to develop a well performng classfer whch accurately predcts the membershp of data elements e n the target class y. The classfer wll perform a fuzzy classfcaton by assgnng data elements to a fuzzy class y havng the followng property: The hgher the degree of membershp of a data element n the fuzzy class y, the greater the lkelhood of class membershp n the crsp target class y. In order to accomplsh ths, a tranng set s prepared form source data, and the relevant attrbutes are selected usng an nterestngness measure. For every attrbute, a fuzzy restrcton y on ts doman s defned based on the prncple of maxmum lkelhood of target class membershp. Each y s nduced from the data such that the degree of membershp of an attrbute value x n y s proportonal to the condtonal relatve frequency of Y. Then, n the unvarate classfcaton step, each relevant varable s fuzzfed by transformng t nto a degree of membershp n ts assocated fuzzy restrcton y. The multvarate fuzzy classfcaton step conssts of aggregatng the fuzzfed attrbutes nto one multvarate fuzzy class of data elements y. Ths fuzzy equvalence class then corresponds to an Inductve Fuzzy Classfcaton whch can be used for predcton. The last step of the process s model evaluaton by analyzng the predcton performance of the classfer. Ths s done by comparng the forecasts wth the real class membershps n a test set. In the followng subsectons, every step of the IFC process s descrbed n detal.

3 3 B. Data Preparaton In order to analyze the data, a tranng set s composed by combnng data from varous sources nto a sngle coherent matrx. All possbly relevant element attrbutes are merged nto one table. The class label for the target varable has to be defned, calculated, and added to the tranng set. The class label s restrcted to a bnary varable where ndcates membershp n the target class. C. Attrbute Selecton Only a few varables n the tranng set are actually relevant for predctng the class label. In order to select the relevant attrbutes, a varety of measures s avalable, for example mutual nformaton or ch squared ndependence. The resultng lst s a descendng rankng of dependent varables by ther relevance. Only the best n varables should be taken nto account, where the n-th best varable should stll show a sgnfcant dependence wth the target varable. D. Inducton of Membershp Functons Intutvely, the am s to assgn to every data element a membershp degree n the fuzzy data element class y. In order to accomplsh ths, y s derved as an aggregaton of fuzzy restrctons: For every attrbute, a fuzzy restrcton y s defned on ts doman whch represents the lkelhood of Y. The hgher the lkelhood of Y s n relaton to Y0, the greater should be the degree of membershp of a value x n y. The lkelhood of class membershp s transformed nto a membershp functon usng the followng method: For each value x dom( ), the fuzzy restrcton y s defned as the (sampled) condtonal probablty for Y, dvded by the sum of the condtonal probabltes of x for Y and Y0 respectvely. Thus, the nduced membershp degree of x n y s defned by x Y ) ( x) : () y x Y ) + x Y 0) Accordng to the defnton of lkelhood L(y x)x y) and lkelhood rato LR y (x)l(y x)/l(not(y) x), equaton () corresponds to a normalzaton of the lkelhood rato (NLR): y ( x) : + + LR ( x) Y x Y 0) L(0 x) + x Y ) L( x) : NLR ( x) x Y ) x Y ) + x Y 0) Y For categorcal varables, the calculaton of the membershp functon s straght forward: The degree of membershp for each value s defned by the correspondng NLR. However, for contnuous varables, the membershp functon could be contnuous too. In that case, a soluton s to calculate the NLR for quantles of dom( ), and to approxmate a contnuous functon. For examples of membershp functon nducton for categorcal or contnuous attrbutes, see the case study n secton IV. E. Unvarate Fuzzy Classfcaton of Attrbute Values Once the membershp functons have been nduced as descrbed n the prevous secton, the relevant attrbutes are fuzzfed. In order to transform the attrbutes nto degrees of membershp, each varable s transformed by applyng the membershp functon nduced by formula (). Ths step smply transforms every value x nto the membershp degree of the value n the correspondng fuzzy restrcton y. F. Multvarate Fuzzy Classfcaton of Data Elements In order to obtan a fuzzy membershp degree n the fuzzfed target class y for data elements e, multple attrbute membershp degrees are aggregated usng fuzzy logc operators. For smplcty, assume the followng equaton: y ( e ) y ( x ) That s, for every fuzzy restrcton on an attrbute doman there s a correspondng fuzzy set of data elements wth the same name. Consequently, the multvarate fuzzy class y s defned as an aggregaton usng a functon F : e ) : F( ( e ), K, ( e )) y' ( y There are dfferent possbltes for multvarate fuzzy aggregaton. In lterature, the focus s often on generatng a fuzzy rule base by searchng a functon as a combnaton of fuzzy logc operators. A smpler canddate for the aggregaton functon s the gamma operator [2] whch s composed of a multplcaton of an algebrac dsuncton and an algebrac conuncton weghted by an exponent parameter γ [0,]: Γ γ ( y : ( e ), K, ( e )) U y : ( γ ( e ) I y ( e ) ( x )) y yn γ γ ( yn ( x )) Ths aggregaton operator compensates between conuncton and dsuncton to a degree specfed by the γ-parameter. Thus, by combnng the nductvely fuzzfed attrbutes nto a multvarate fuzzy class of data elements defned by the membershp functon y :U [0,], a multvarate model s obtaned from the tranng set. Ths model can be used for Inductve Fuzzy Classfcaton of unlabeled data for predcton. Usng an alpha cut for an α [0,] leads to a bnary classfer. y γ (3)

4 4 Table : Frequences, condtonal probabltes and NLRs for the categorcal attrbute customer segment. : customer segment Y Y0 Y) Y0) NLR Y() Bass ' ' SMB 249 2' Gold 2'666 54' Premum 5'432 0' Total 29' '339 In the followng sectons, the applcaton of the data mnng process steps presented n secton III to the target group selecton s descrbed. Fgure 2: Analytcs appled to ndvdual marketng n the onlne channel G. Model Evaluaton In order to evaluate the predctve performance, the classfer s appled to a hold-out test set, and the predctons are compared wth the actual class label. There are dfferent metrcs to evaluate the predctve performance of the nduced model, for example the mutual nformaton between the predctons Y and the class labels Y, or alternatvely ther correlaton. Usng such a measure, the model parameters γ [0,] and α [0,] can be optmzed to yeld a maxmal predctve performance. IV. CASE STUDY: MEMBERSHIP FUNCTION INDUCTION FOR FUZZY TARGET GROUPS PostFnance Inc. ( s a proftable busness unt of Swss Post. Its actvtes contrbute sgnfcantly to the fnancal servces market n Swtzerland. PostFnance s an analytc enterprse feedng busness processes wth nformaton ganed from predctve analytcs. The marketng process uses data mnng scores on product affnty for ndvdual customers. Target groups for onlne marketng campagns are selected usng nductve classfcaton on customer databases, as llustrated by Fgure 2. For every customer, the ndvdual advertsement message s mapped accordng to the target group selected by predctve scorng. Inductve Fuzzy Classfcaton was appled n an onlne marketng campagn of PostFnance promotng nvestment funds. The am of the nductve target group selecton was to forecast the customers wth an enhanced probablty of buyng nvestments funds. The resultng fuzzy classfer yelded a fuzzy target group for an ndvdual marketng campagn, where every customer s part of the target group only to a certan degree. A. Data Preparaton In order to prepare a test set for model nducton, a sample customer data set was selected from the customer data warehouse. Ths data set contaned an anonymous customer number assocated wth customer specfc attrbutes. As target varable, the class label was set to f the customer had nvestment funds n hs product portfolo, and to 0 else. B. Attrbute Selecton Mutual nformaton of the dependent varables wth the target varable was chosen as a useful rankng method. The followng attrbutes were selected as relevant: Customer segment (0.047 bt) Number of products (0.036 bt) Overall balance (0.035 bt) Loyalty (0.02 bt) Customer group (0.06 bt) Balance on prvate account (0.04 bt) Age (0.03 bt) C. Inducton of Membershp Functons Usng the method presented n secton III, for each of the relevant attrbutes the fuzzy restrcton correspondng to the lkelhood of havng nvestment funds was nduced. In the followng secton, the nducton processes for a categorcal and a contnuous attrbute are descrbed n detal. As frst example, n the doman of the categorcal attrbute customer segment ( ), there are four values Bass, SMB, Gold and Premum. For customers who have not yet bought nvestment funds, the am s to defne a degree of membershp n a fuzzy restrcton on the customer segment doman n order to classfy them for ther lkelhood to buy that product n the future. The frequences and condtonal probabltes are presented n Table. The frst column ndcates the customer segment. Column Y contans the number of customers of each segment who have bought nvestment funds. Column Y0 contans the number of customers of each segment who have not bought that product.

5 5 Fuzzy Restrcton NLR Bass SMB Gold Premum Customer Segment Fgure 3: Membershp functon resultng from nducton for the categorcal attrbute customer segment Fuzzy Restrcton NLR F(x) 0.7/(+EP( *LN(x+))) '000 00'000 50'000 Overall Balance Fgure 4: Membershp functon resultng from nducton for the contnuous attrbute overall balance Accordng to formula (), the membershp degree of segment Bass n the fuzzy restrcton y was nduced as follows: y (" Bass") NLR " Bass" Y ) " Bass" Y ) + " Bass" Y 0) Y (" Bass") The other degrees of the membershp were nduced analogcally. The resultng values are shown by Table n column NLR. The correspondng membershp functon s llustrated by Fgure 3. As second example, for the contnuous attrbute overall balance the membershp functon was nduced n the followng way: Frst, the NLR for decles of the attrbute s doman was calculated analogcally to the prevous example, represented by grey squares n Fgure 4. Then, a functon A / ( + exb C ln(x + )))+D was ftted by optmzng the parameters A to D usng the method of [3]. The resultng membershp functon for the customer overall balance s shown as a lne n Fgure 4. D. Unvarate Fuzzy Classfcaton of Attrbute Values Every relevant attrbute of the orgnal data set was transformed to a fuzzy membershp degree usng SQL (Structured Query Language) drectly n the database. Table 2: Optmzaton of the gamma parameter for multvarate fuzzy aggregaton I(Y;Y') I(Y;Y') LR(Y') Gamma Gamma Gamma Categorcal varables were transformed usng a case dfferentaton statement. For example, the attrbute customer segment was transformed usng the followng SQL command: select case when customer_segment 'Bass' then 0.3 when customer_segment 'SMB' then 0.52 when customer_segment 'Gold' then 0.74 when customer_segment 'Premum' then 0.86 end as fuzzy_customer_segment from ads_funds Contnuous varables were transformed usng a functon expresson. For example, the attrbute overall balance was fuzzfed usng the followng SQL expresson: select 0.7/(+EP(7.-0.8*LN(overall_balance + )))+0.09 as fuzzy_overall_balance from ads_funds E. Multvarate Fuzzy Classfcaton of Data Elements The ndvdual fuzzy attrbute doman restrctons were then aggregated to a multvarate fuzzy class of customers. Ths was done usng an SQL statement mplementng the gamma operator defned n equaton (3). In order to defne the gamma parameter, dfferent performance measures were calculated. As shown n Table 2, a gamma of, correspondng to a algebrac dsuncton or full compensaton, was most successful. Thus, the multvarate fuzzy classfcaton was performed usng the followng SQL statement: select (-fuzzy_number_of_products) * (-fuzzy_customer_segment) * (-fuzzy_overall_balance) * (-fuzzy_loyalty) * (-fuzzy_age) * (-fuzzy_balance_on_prvate_account) * (-fuzzy_customer_group) ) as multvarate_fuzzy_classfcaton from ads_funds As a result, a fuzzy class of customers was calculated, whose degree of membershp ndcates the product affnty for nvestment funds and represents a product affnty score. Ths can be used n ndvdual marketng for defnng target groups for dfferent products.

6 6 Table 3: Resultng product sellng ratos per target group Test group Y Y0 Sold products : Fuzzy classfcaton % 2: Crsp classfcaton % 3: Random selecton % F. Model Evaluaton In order to test the resultng fuzzy classfer, a plot marketng campagn was performed usng the resultng fuzzy target group. A target group of 5000 customers wth the hghest membershp degree was selected from the fuzzy class usng an alpha cut (test group ). As a comparson, 5000 other customers were selected usng a crsp classfcaton (test group 2), usng the followng crsp constrants: Customer segment n {Gold, Premum, SMB} Customer_group 50 Plus Loyalty > 4 Number_of_products > Age between 35 and 75 Balance on prvate account > 3000 Overall balance > Thrd, 5000 customers were selected randomly (test group 3). To each of those 5000 customers, an onlne advertsement for nvestment funds was shown. After the marketng campagn, the product sellng rato wthn three months was measured. The results are shown n Table 3. The product sellng rato for the target group selected by nductve fuzzy classfcaton was the most effectve. V. CONCLUSION A methodology for an nductve fuzzy classfcaton was ntroduced. The nducton process conssts of generatng fuzzy restrctons on attrbute domans from normalzed lkelhood ratos. An ndvdual marketng campagn usng fuzzy nductve target group selecton showed that fuzzy classfcaton led to a hgher product sellng rato than crsp classfcaton or random selecton. In the case study, Inductve Fuzzy Classfcaton s predctons of product affnty were more accurate than those of the crsp classfcaton rules on exactly the same attrbutes. The concluson s that n comparson to crsp classfcaton, fuzzy classfcaton has an advantageous feature that leads to better predctve performance. Further research could encompass the followng ssues: Reference mplementaton of an IFC data mnng tool supportng and automatng the IFC process steps. An nterpreter for an nductve fuzzy classfcaton language (IFCL) s n development by a Master s Student Proect [4] Evaluaton of the IFC data mnng methodology usng dfferent benchmark data sets n comparson to conventonal predcton technques. Evaluaton of the mechansm how fuzzy classfcaton provdes more accurate predcton results than crsp classfcaton. Development of a conceptual framework for busness applcatons of fuzzy predctve analytcs n the context of nformaton systems research. ACKNOWLEDGMENT The authors thank Davd Wyder from PostFnance for supportng research wth a plot marketng campagn. The case study was conducted by the FMsquare reserach center for fuzzy marketng methods ( nvestgatng busness applcatons of fuzzy logc such as fuzzy data warehouses, fuzzy predctve analytcs or fuzzy customer performance measurement. REFERENCES [] A. Meer, G. Schndler, N. Werro. Fuzzy classfcaton on relatonal databases. In Galndo M. (Ed.): Handbook of Research on Fuzzy Informaton Processng n Databases. Volume II. Informaton Scence Reference, pp , [2] Ncolas Werro. Fuzzy classfcaton for onlne customers. PhD Thess, Unversty of Frbourg, [3] Ian H. Wtten and Ebe Frank. Data mnng, practcal machne learnng tools and technques, second edton. Morgan Kaufmann Publshers, [4] Eyke Hüllermeer. Fuzzy methods n machne learnng and data mnng: Status and prospects. Fuzzy Sets and Systems 56(3), , [5] L. A. Zadeh. Fuzzy sets. Informaton and Control, 8, , 965. [6] L. A. Zadeh. Calculus of fuzzy restrctons. In: L.A.Zadeh et al., Eds, Fuzzy sets and ther applcatons to cogntve and decson processes. NewYork: Academc Press, 975. [7] W. Danhu, T.S. Dllon, E.J. Chang. A data mnng approach for fuzzy classfcaton rule generaton. IFSA World Congress and 20th NAFIPS Internatonal Conference, 200. Jont 9th, pp vol.5, July 200. [8] Y. Hu, R. Chen and G. Tzeng. Fndng fuzzy classfcaton rules usng data mnng technques. Pattern Recogn. Lett. 24, -3, , [9] C.J. Km and B.D. Russell. Automatc generaton of membershp functon and fuzzy rule usng nductve reasonng. Thrd Internatonal Conference on Industral Fuzzy Control and Intellgent Systems, pp , 993. [0] T. Hong and S. Wang. Determnng approprate membershp functons to smplfy fuzzy nducton. Intell. Data Anal. 4, (Jan. 2000), [] S. Roychowdhury. Bayes-lke Classfer wth Fuzzy Lkelhood. Fuzzy Systems, 2006 IEEE Internatonal Conference on, pp , [2] H.-J. Zmmermann. Fuzzy set theory and ts applcatons. London: Kluwer Academc Publshers, 992. [3] E.G. John. Smplfed curve fttng usng spreadsheet add-ns. Int. J. Engng Ed. 4, , 998. [4] Phlppe Mayer. Implementaton of an nductve fuzzy classfcaton language nterpreter. Master s Dssertaton, Unversty of Frbourg, Swtzerland (n Progress).