Modeling Brand Choice using Boosted and Stacked Neural Networks

Transcription

1 Modelng Brand Choce usng Boosted and Stacked Neural Networks Rob Potharst, Mchel van Rthoven and Mchel van Wezel Erasus Unversty Rotterda Faculty of Econocs, Econoetrc Insttute PO Box 738, 3000 DR, Rotterda, The Netherlands Econoetrc Insttute Report EI Abstract The brand choce proble n arketng has recently been addressed wth ethods fro coputatonal ntellgence such as neural networks. Another class of ethods fro coputatonal ntellgence, the so-called enseble ethods such as boostng and stackng have never been appled to the brand choce proble, as far as we know. Enseble ethods generate a nuber of odels for the sae proble usng any base ethod and cobne the outcoes of these dfferent odels. It s well known that n any cases the predctve perforance of enseble ethods sgnfcantly exceeds the predctve perforance of the ther base ethods. In ths report we use boostng and stackng of neural networks and apply ths to a scanner dataset that s a benchark dataset n the arketng lterature. Usng these ethods, we fnd a sgnfcant proveent n predctve perforance on ths dataset.. Introducton A classcal topc n arketng s odelng brand choce. Ths aounts to settng up a predctve odel for a stuaton where a consuer or household, to purchase a specfc product avalable n k brands, chooses one of these brands, gven a nuber of household characterstcs (such as ncoe), product factors (such as prce) and stuatonal factors (such as whether or not the product s on dsplay at purchase te). In the past, nuerous dfferent odels have been proposed for brand choce probles. The ost well known are the condtonal and ultnoal logt odels (Franses & Paap, 200; McFadden, 973). Durng the last decade, ethods fro coputatonal ntellgence such as neural networks have been proposed as an alternatve to these classcal odels (Hruschka, 993; West et.al., 997). A recent contrbuton to the neural networks for brand choce lterature s the artcle (Vrooen, Franses & van Nerop, 2004) n whch neural networks are used to odel a two-stage brand choce process: frst a household chooses a so-called consderaton set (.e. a subset of the avalable brands whch are ost nterestng for the

2 consuer at hand), and next the household selects a brand fro ths consderaton set (Roberts & Lattn, 997). Another lne of research whch becae very popular durng the last decade both n the statstcs and n the coputatonal ntellgence county, s the use of enseble ethods such as boostng, baggng and stackng (Haste et al., 200; Schwenk & Bengo, 2000; Tbshran, Fredan & Haste, 2000). These ethods work by buldng not one odel for a partcular proble, but a whole seres (enseble) of odels. These odels are subsequently cobned to gve the fnal odel that s to be worked wth. The an advantage of these enseble technques s the soetes spectacular ncrease n predctve perforance that can be acheved. In arketng, enseble ethods are proposed n a forthcong paper by two of the authors of ths chapter (van Wezel & Potharst, 2004). Stacked neural networks for custoer choce odelng was also appled n (Hu & Tsoukalas, 2003). In ths chapter we wll explan soe of these enseble ethods (especally boostng and stackng) and use the by cobnng the results of a seres of neural networks for a specfc brand choce proble. All ethods explaned wll be deonstrated on an exstng set of scanner data whch has been extensvely analysed n the arketng lterature: the A.C. Nelsen household scanner panel data on purchases of lqud detergents n a Soux Falls, South Dakota, arket. Ths dataset contans 3055 purchases concernng 400 households of sx dfferent brands of lqud detergent: Tde, Wsk, Eraplus, Surf, Solo and All. Suarzng, ths chapter contans - a bref descrpton of classcal odels for brand choce - a detaled descrpton of how neural networks wth one hdden layer ay be used to odel a brand choce proble, ncludng - a dscusson of the features to be selected as explanng characterstcs - an explanaton of how the concept of a consderaton set can be odeled usng a specfc knd of hdden layer for the neural network - an exposton on the enseble ethods of boostng and stackng, a ppled to the neural network odels consdered above - deonstratons of all ethods descrbed usng freely avalable real lfe scanner data. 2. Modelng brand choce: classcal odels A classcal proble n arketng s the so-called brand choce proble: tryng to odel the purchase behavor of a consuer gven a nuber of explanatory varables. At purchase occason a consuer s faced wth a choce between brands, 2,, of a product he s gong to buy. Hs fnal choce Y (whch ust be one of the brands,, ) depends on three knds of varables. The frst knd of varable depends on the consuer or the purchase occason, but not on the brand, for nstance consuer ncoe; we wll denote such a varable by a captal X, so for nstance the ncoe of the consuer that purchases on occason s denoted by X. The second knd of varable depends only on

3 brand, not on purchase te and consuer. We wll denote such a varable by a captal Z, so for nstance the brand awareness of brand s Z. The thrd group of varables depends both on consuer/purchase occason and on brand. These varables are denoted by captal W, so for nstance the prce of the product of brand that the consuer has to pay on purchase occason s W. Many odels have been proposed for the brand choce proble n the arketng lterature. In ths secton four representatve odels wll be descrbed. We start wth the ultnoal logt odel. Accordng to ths odel, Pr[Y = X ], the probablty that on occason a consuer chooses brand, gven the value of the expanatory varable X, has the followng hypotheszed functonal for: for =,, we have exp( α + β X ) Pr[ Y = X ] = and + exp( α + β X = ) Pr[ Y = X ] = + exp( α + β =. X ) Note that, whatever the values of the paraeters a and ß and whatever the value of X, the su of these probabltes over all brands equals one, as t should be. Note further that only varables of type X play a role n ths type of odel, varables of type W or Z are excluded. When purchase data s gven, one ay estate the paraeters a and ß of ths odel by axu lkelhood ethods. The second knd of odel we wll enton s the condtonal logt odel, orgnally proposed by McFadden(973). For ths odel the probablty that brand s chosen equals, for =,, Pr[ Y = W,..., W exp( α + βw) ] =. = exp( α + βw) Note agan that the su over all probabltes equals one, and n ths type of odel we have only varables of type W; here varables of type X and Z are excluded. Note also that paraeter ß does not have an ndex: the senstvty for prce-changes s supposed to be equal for all brands. Ths constrant ay be relaxed usng the odel dscussed next. Agan, the paraeters ay be estated usng axu lkelhood. The thrd odel s the general logt odel n whch all three varable types (X, W and Z) ay play a role. For ths odel the probablty that brand s chosen equals, for =,, exp( α + β X + γ W + δz ) Pr[ Y = X, W,..., W, Z,..., Z ] =. = exp( α β X γ W δz ) Note that not only all three varable types are ncorporated n ths odel, but also varable W has a brand-specfc coeffcent, whch relaxes the entoned constrant of the condtonal logt odel.

4 3. Modelng brand choce: neural network odels In addton to the statstcal odels layed out n the prevous secton, brand choce can also be odeled usng neural networks. Ths s a technque that was put forward at the end of the eghtes n the achne learnng county and ts popularty soon becae very hgh n very dverse felds, fro lngustcs to engneerng, fro arketng to edcne. For the brand choce proble neural networks have been proposed by several authors (Hruschka, 993; Dasgupta et al., 994; West et al., 997; Hu et al., 999; Hruschka et al., 2002; Vrooen et al., 2004). The neural network ethodology s clearly explaned n the frst chapter of the ntroductory book (Sth & Gupta, 2002). Ths work also contans a nuber of other applcatons of neural network ethodology to arketng probles, such as (Potharst et al., 2002). We wll ntroduce neural network odelng for the brand choce proble on the bass of a recently proposed neural network odel that akes use of so-called consderaton sets (Vrooen et al., 2004). Let us frst revew the theory on consderatons sets. To ths purpose, the process of choosng a product of a partcular brand s vewed as consstng of two stages. In the frst stage, the consuer reduces the set of avalable brands to a saller subset: ths subset s the consderaton set, that wll exclusvely be consdered when the consuer akes hs fnal choce. In the second stage the consuer pcks hs fnal choce fro the brands that resde n the consderaton set (Roberts & Lattn, 997). (Vrooen et. al., 2004) ake use of a neural network wth one hdden layer to odel ths two stage process. Ther odel can be vsualzed as follows: constant Z p constant W q X CS FC Z p' W q' As can be seen fro ths graph the network conssts of three layers of nodes; roughly spoken, there s an nput layer, followed by a hdden layer n whch the consderaton set (CS) s odeled, and an output layer that odels the probablty of the fnal choce for a brand (FC). Three types of nput varables are used: household characterstcs (X ) lke sze of household or ncoe level, brand characterstcs (Z p ) lke prce, prooton and advertsng, and fnally choce- and brandspecfc characterstcs (W q ) lke the observed prce at the purchase occason. Let the nuber of X-type varables be I, the nuber of Z- type varables P and the nuber of W-type varables Q. For each of the brands there s a sgodal hdden node CS, that deternes the probablty that brand s n the consderaton set, as follows. For =,,

5 CS = G( α I P 0 + αx + = p= β p Z p ) where the coeffcents a 0, a and ß p are paraeters that ust be estated fro the data and G s the logstc (or sgodal) functon G( x) =. x + e Next, n the second part of the network odel, the probablty of the fnal choce (FC ) s deterned usng the consderaton set, as follows. For =,, FC = exp( γ = 0 exp( γ + 0 k= + γ k k= CS γ k k + CS k Q q q= Q + δ q= W δ q q ) W q, ) where agan the coeffcents? 0,? k and d q are paraeters that ust be estated fro the data. The fnal outcoe of the neural network s the brand that gets the largest probablty FC. In (Vrooen et al., 2004) all these paraeters are estated (or, n the usual do the network s traned) by usng the back-propagaton algorth, whch s based on gradent descent. 4. The data set: scanner data for sx lqud detergent brands The data set we use to test our ethods s the data set also used by (Vrooen et al., 2004) whch s descrbed by (Chntagunta and Prasad, 998). Ths data set s freely avalable on the nternet. It conssts of scanner data on 3055 purchases of a lqud detergent of sx possble brands: Tde, Wsk, Eraplus, Surf, Solo and All. These purchases concern 400 dfferent households. For each purchase, the values of four dfferent household-specfc varables (X) are avalable (I = 4), furtherore there are four brand-specfc varables (Z p, so P = 4) and three extra varables (W q, so Q = 3) Specfcally, we have the followng set of varables: X = the total volue the household purchased on the prevous purchase occason X 2 = the total expendture of the household on non-detergents X 3 = the sze of the household X 4 = the nter-purchase te Z = the prce of brand n cents per ounce Z 2 = or 0 accordng to whether brand was on feature prooton or not Z 3 = or 0 accordng to whether brand was on dsplay prooton or not Z 4 = an ndcaton of how recently brand was purchased (between 0 and ) W = or 0 accordng to whether the household bought brand on the prevous purchase occason, or not

6 W 2 = or 0 accordng to whether brand was the brand ost purchased over all prevous purchase occasons, or not W 3 = the fracton of recent purchases of brand by the household dvded by the total of all recent purchases of any detergent by the household For a coplete descrpton of these varables, see (Vrooen et al., 2004). Because of ncopleteness of soe of the records we elnated 798 of the, so we have = 2257 coplete records avalable. All non-bnary varables were scaled to the [0, ]- nterval n order to ake all scales coparable. 5. Enseble ethods: baggng, boostng and stackng In recent years there has been a growng nterest n the datanng and statstcs countes n so-called enseble ethods. These ethods, also known as cottee ethods or opnon pools, work by cobnng dfferent ndvdual odels (also called base odels) for the the sae proble. Great advantage of these ethods s the gan n predctve perforance that s often acheved by applyng the. We llustrate ths arguent wth the followng exaple, based on a credt scorng dataset whch s freely avalable on the nternet. The proble whch s addressed s to predct at the te of applcaton for a loan whether the loan would ever be repad or not. As can be seen fro the graph below the error rate of such predctons drops fro 32% when only one odel s used to 25% when a cobnaton of 20 odels s used to predct the outcoe Percentage correctly predcted Nuber of odels cobned

7 Three of the ost well-known enseble ethods are baggng, boostng and stackng, see (Haste et al., 200). Baggng s shorthand for 'bootstrap aggregatng' and t works roughly as follows: fro the dataset we draw a rando saple wth replaceent and buld a odel usng only the data fro ths saple; a second odel s bult usng only the data fro a second rando saple drawn fro the orgnal dataset, and so on. In ths anner we arrve at a nuber of base odels whch are subsequently cobned by castng votes. For nstance, for an enseble of 0 odels, f for a certan nput vector, 5 of the odels predct brand 3, 2 odels predct brand and the reanng three odels predct brand 2, the cobned odel predcts brand 3 (the aorty of the votes). Snce we wll not use ths ethod n ths chapter we wll not outlne the exact algorth for ths ethod. The second ethod we consder s boostng. The general dea of boostng s to create a sequence of odels, where each odel s traned on a reweghted verson of the orgnal dataset. Each tranng exaple n the dataset s assgned a weght and these weghts are dynacally adusted: when the odel n a certan teraton of the algorth akes an error n the classfcaton of the n-th tranng pattern, the weght assocated wth ths pattern s ncreased. Ths causes the odel of the next teraton to focus on the patterns that were sclassfed earler. Contnung ths way, an enseble of odels s created. The fnal odel s a weghted aorty vote of all the odels fro the enseble. The orgnal boostng algorth s called AdaBoost (Freund & Schapre, 996), and we wll now gve an exact descrpton of ths algorth. The orgnal verson of ths algorth s eant for two-class probles, where the classes are called and +. Snce n our case we wll not work wth only two brands but wth any nuber () of brands, we wll have to adapt ths algorth later to our needs. Let us assue we have a dataset consstng of N data pars (x, y ), where x s a vector of nput values and y the correspondng class value (ether + or ). In the descrpton of the algorth we wll use the followng notaton: w s the weght of the -th data par, M s the nuber of boostng teratons (so the enseble wll consst of M odels),?(a) s an ndcator functon for boolean arguents A, whch equals f A s true and 0 f A s false. The functon sgn(x) = f x 0 and f x < 0. Note further that F(x) can be any odel based on the data, whether t be a statstcal odel, a neural network or a decson tree. Here s the AdaBoost algorth:. Intalze the boostng weghts: for =,, N w = N 2. For = to M perfor each of the followng: (a) Tran the base-odel F (x) on the dataset wth weghts {w : =,, N} (b) Copute Of the sae sze as the orgnal dataset!

8 err = N w = χ ( y F ( x )) (c) Copute err α = log( ) () err (d) Redefne the weghts w : for =,, N w = w exp( α χ( y F ( x ))) (e) Noralze the weghts w : for =,, N w w = N w k = 3. Output the fnal cobned odel: k M O( x) = sgn( α F ( x)) (2) = Let us take a look at ths algorth soewhat ore closely to see how t works. Frst of all, n step 2(a), t s supposed that we are able to tran a odel on a dataset contanng observatons (purchases!) that each have a dfferent weght. Ths can be pleented n several ways. We used the so-called resaplng ethod whch wll be explaned below. In step 2(b) the error rate err of the odel bult n the prevous step s calculated. Ths s the n-saple tranng error rate, that allows for dfferent weghts n the dataset. Usng ths error rate, n step 2(c) the a coeffcent s calculated. Provded the error rate < 0.5 (whch t should be f the odel does better than rando guessng), ths coeffcent wll be a postve nuber that ncreases f the error rate decreases. Ths a coeffcent wll be used n step 3 to wegh the votes fro dfferent odels: the votes of odels wth lower error rates get a hgher weght than those fro odels that perfor less well. In step 2(d) the weghts of the saples n the dataset are updated: for each saple that s classfed ncorrectly ts weght s ncreased wth the sae factor, that nvolves the a coeffcent. The weghts of correctly classfed saples rean unchanged. In step 2(e) the weghts are noralzed (they should su to one). Fnally, n step 3 the whole enseble of odels developed s cobned usng the weghted votng schee as descrbed. The thrd enseble ethod we consder s stackng. Ths ter s usually referred to when there are two levels of learnng nvolved: on the frst level several odels are traned on the dataset. These ay be odels of dfferent types. Next, the odels are cobned not va a fxed votng schee, but by learnng an optal cobnaton ethod fro the data. Ths s the second level of learnng nvolved. Of course, such a stackng schee can be pleented n any dfferent ways. We wll use a sple stackng schee that s bult on top of our boostng procedure: nstead of cobnng the odels fro the boostng enseble drectly va Step 3, we wll learn an optal sequence of a coeffcents fro the data usng the followng algorth:

9 . Use the sequence of a coeffcents that s constructed by the boostng algorth as startng values 2. For teraton k = to K perfor the followng steps: (a) for observaton = to N do. deterne the output O(x ) usng equaton (2). f O(x ) y ncrease all coeffcents a that belong to a odel that gves the correct predcton y wth a constant factor c: a = c*a. (b) deterne the error rate of the new cobned odel (2) on the valdaton set; f t s lower than the best error rate so far, store ths sequence of a 's. 3. Output the fnal odel (2) wth the optal sequence of a 's. Here K s the nuber of tranng teratons, and c s the constant update factor. (We wll use K = 50 and c =.0.) The valdaton set s a dataset that s copletely ndependent of our tranng set. How we construct such a valdaton set wll be explaned n secton Coparng perforance Now that we have ntroduced the ethods we are gong to use and the brand choce dataset we wll apply these ethods to, we are n a poston to knot thngs together. What we would lke s ake a coparson of the perforance on the lqud detergent sales dataset of the ethods we have ntroduced. Partcularly, we would lke to see the perforance of Vrooen s odel n coparson wth boostng and stackng. As to perforance we are especally nterested n the predctve perforance of the dfferent odels (snce the cla s that t can be proved usng enseble ethods). So, n ths secton we wll be concerned wth the predctve perforance of the three ethods we want to test. In order to apply the boostng algorth to the brand choce dataset there are a nuber of decsons to be ade and probles to be solved. Frst of all, t has to be decded what knd of odels we take as our base odels, to serve as ebers of the enseble. Snce we want to ake a coparson wth Vrooen's odel, t would be a good dea to use hs odel as our base odel. Next, t should be decded what ethod we wll use buld a odel on a dataset wth weghted nstances. We decded to use the so-called resaplng ethod: we draw N tes a rando nstance fro the dataset (wth replaceent!) gvng each nstance a weght w. Thus, we get a dataset of the sae sze as the orgnal dataset, whch contans however ultple copes of soe nstances wth large weghts whle soe nstances wth sall weghts ght have dsappeared fro ths new dataset. Wth ths new dataset we buld a odel, whch s thus based on a weghted dataset. One dsadvantage of ths ethod s that the generated odel has a stochastc nature, snce t s based on a rando saple fro the dataset. Another dsadvantage s that soe records ust dsappear fro soe of these randoly chosen datasets. However, ths loss s fully copensated by the fact that we don't need to adapt the orgnal odel buldng ethod to the case of a dataset wth weghted nstances.

10 The ost portant adaptaton we had to ake regards the fact that we want our ethods to work for any brands, not ust for two brands. If we call the brands we consder,,, t follows that for the nstance pars (x, y ) n our dataset we have y {,, }. Also, for each x, F (x) (the output of the neural network) ust be an nteger n the range,,. Wth ths stuaton n nd, t s not hard to see that the generalzaton of equaton (2) to the stuaton of brands should be replaced by the followng: M O( x) = arg ax α χ( F ( x) = ) (3) = Ths equaton expresses the procedure to cast a weghted vote for the brands aong the M odels (wth weghts a ) and to pck the brand that got the largest nuber of (weghted) votes. Another nor adaptaton also had to be devsed, regardng the stuaton of nstead of two brands. In equaton (), for the stuaton of two brands, we have seen that a postve coeffcent s delvered provded the error rate err does not exceed 0.5. In the case of two brands ths s an acceptable state of affars, snce rando guessng between two brands results n an error rate of 0.5. So the only requreent that a odel should eet, would be to be better than rando guessng. However, n the stuaton of brands, rando guessng results n an error rate of (-)/ snce there are - possbltes for an ncorrect guess. Thus, f we set a coparable requreent to the forula for calculatng the a coeffcents as n the case of two brands, we should deand that a be postve as long as the error rate err does not exceed (-)/. Ths s the case, f we defne a accordng to the followng equaton: err α = log( ( )) (4) err The other portant property, naely that a ncreases when the error rate err decreases reans true wth ths defnton and t s equvalent to () n case = 2. For these reasons, we used (4) to replace () n our boostng procedure. Snce we now have a coplete descrpton of the ethods we used on the detergent dataset, we wll now descrbe the experents that we perfored and ther results. In order to get a far vew of the perforance of a odel, t should be tested on a copletely ndependent test set. By the sae token, a valdaton set, that s used to get an optal sequence of a coeffcents usng the stackng algorth, should be copletely ndependent of both the tranng set and the test set. We used the followng procedure to arrve at copletely ndependent tranng, valdaton and test sets and an assocated experental cycle:. Splt the 400 households randoly nto a three groups, one of sze 200 (tranng households, TR), and two groups each of sze 00 (valdaton households and test households, VA and TE). 2. Tranng set = all purchases n the orgnal dataset of 2257 records of the households fro TR; valdaton set = all purchases of the households fro VA; test set = all purchases of the households fro TE.

11 3. Usng the backpropagaton algorth on the tranng set a Vrooen odel was created, and the accuracy of ths odel on the tranng set, the valdaton set and the test set was deterned. 4. Usng the boostng algorth on the tranng set a whole enseble of Vrooen odels + a cobned odel was bult; the accuracy of the cobned odel was deterned for the tranng set, the valdaton set and the test set. 5. Usng the stackng algorth, startng fro the enseble of step 4 and akng use of the valdaton set, an optal sequence of a coeffcents was traned for a new cobned odel; for ths odel the accuracy on tranng, valdaton and test set was deterned. Ths coplete cycle was repeated ten tes. The resultng ean accuraces over these ten runs, together wth ther standard devatons are dsplayed n the followng table: Vrooen Boostng Stackng tranng accuracy 80.7 ± ± ±.9 valdaton accuracy 76.4 ± ± ± 2.6 test accuracy 75.9 ± ± ± 2.5 We conclude fro ths table that the use of boostng and stackng results n a clear ncrease n perforance, especally on valdaton data and copletely ndependent test data. A stacked odel predcts on average 3.2% better on unseen data than a Vrooen odel. Actually, both boostng and stackng perfor better than Vrooen on the test set n all of our ten runs. Wth boostng the ncrease vares fro 0.6% to 4.5% and wth stackng fro 0.7% tot 5.5%. So ndeed, the expected ncrease n predctve perforance s confred on the detergent dataset. Another rearkable outcoe of these experents s that n our experents the Vrooen odel does better than reported n (Vrooen et al., 2003) on the sae dataset: they report an accuracy of 73.9% on an ndependent test set, whereas we get 75.9% on average. One reason for ths dfference ght be that we use three extra varables n the second stage of the odel (the W q varables) whle they use only one extra varable, naely prce. Another dfference s that we repeat the whole experent ten tes, whereas they reported only one experent. Ther rando splt of the dataset nto tranng, valdaton and test set ght ust have been an unlucky one. By repeatng the experent ten tes we get a far dea of the stablty of our results on ths dataset. 7. Trends and conclusons In ths chapter we showed how soe new ethods fro the feld of coputatonal ntellgence (enseble ethods such as boostng and stackng) could be used for a tradtonal arketng proble lke brand choce. We found that ndeed the predctve perforance of the odels based on the enseble technques proved even copared to the ost sophstcated exstng odel that we found n the lterature. Although predctve perforance s one aspect that canddate odels should be udged wth, there are ore

12 aspects that should be taken nto consderaton. One aspect that has not been consdered here s the nterpretablty of the generated odel. Snce odels bult by enseble ethods consst of a cobnaton of dfferent (soetes any!) base odels the coplexty of the fnal odel s usually hgh, akng t dffcult to nterprete. Ths s one of the thees that should be taken nto account by future serous work nto ths drecton. References Chntagunta, P.K. & Prasad, A.R. (998). An eprcal nvestgaton of the dynac McFadden odel of purchase tng and brand choce: plcatons for arket structure. ournal of Busness and Econoc Statstcs, 6, 2-. Dasgupta, C.G., Dspensa, G.S., Ghose, S. (994). Coparng the predctve perforance of a neural network odel wth soe tradtonal arket response odels. Internatonal ournal of Forecastng, 0, Franses, P.H. & Paap, R. (200). Quanttatve Models n Marketng Research. Cabrdge, UK: Cabrdge Unversty Press. Freund, Y. & Schapre, R. (996). Experents wth a new boostng algorth. In Proceedngs of the 3 th Internatonal Conference on Machne Learnng(48-56). San Francsco, CA, Morgan Kaufann Publshers. Haste, T., Tbshran, R. & Fredan,. (200). The eleents of statstcal learnng. New York, Sprnger Verlag. Hruschka, H. (993). Deternng arket response functons by neural network odelng: a coparson to econoetrc technques. European ournal of Operatonal Research, 66, Hruschka, H., Fettes, W., Probst, M. & Mes, C. (2002). A flexble brand choce odel based on neural net ethodology. A coparson to the lnear utlty ultnoal logt odel and ts latent class extenson. OR Spectru, 24, Hu, M.Y., Shanker, M., Hung, M.S., (999). Estaton of posteror probabltes of consuer stuatonal choces wth neural network classfers. Internatonal ournal of Research n Marketng, 6, Hu, M.Y. & Tsoukalas, C. (2003). Explanng consuer choce through neural networks: The stacked generalzaton approach. European ournal of Operatonal Research, 46,

13 McFadden, D. (973). Condtonal logt analyss of qualtatve choce behavor. In P. Zarebka (Ed.), Fronters n Econoetrcs (05-42). New York: Acadec Press. Potharst, R., Kayak, U. & Pls, W. (2002). Neural networks for target selecton n drect arketng. In Sth, K. & Gupta,. (Eds.), Neural Networks n Busness: Technques and Applcatons (89-0). Hershey PA, Idea Group Publshng. Roberts,.H. & Lattn,.M. (997). Consderaton: Revew of research and prospects for future nsghts. ournal of Marketng Research, 34, Schwenk, H. & Bengo, Y. (2000). Boostng neural networks. Neural Coputaton, 2, Sth, K. & Gupta,. (Eds.) (2002), Neural Networks n Busness: Technques and Applcatons. Hershey PA, Idea Group Publshng. Tbshran, R., Fredan,. & Haste, T. (2000). Addtve logstc regresson: a statstcal vew of boostng. Annals of Statstcs, 28, Vrooen, B., Franses, P.H. & van Nerop, E., (2004). Modelng consderaton sets and brand choce usng artfcal neural networks. European ournal of Operatonal Research, 54, West, P.M., Brockett, P.L. & Golden, L.L. (997). A coparatve analyss of neural networks and statstcal ethods for predctng consuer choce. Marketng Scence, 6 (4), Wezel, M. van, & Potharst, R. (2004) Iproved custoer choce predctons usng enseble ethods. Techncal Report fro