Planning for Marketing Campaigns

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Planning for Marketing Campaigns"

Transcription

1 Plannng for Marketng Campagns Qang Yang and Hong Cheng Department of Computer Scence Hong Kong Unversty of Scence and Technology Clearwater Bay, Kowloon, Hong Kong, Chna (qyang, Abstract In busness marketng, corporatons and nsttutons are nterested n executng a sequence of marketng actons to affect a group of customers. For example, a fnancal nsttuton may derve marketng strateges for turnng ther reluctant customers nto actve ones and a telecommuncatons company may plan actons to stop ther valuable customers from leavng. These marketng plans are amed at convertng groups of customers from an undesrable class to a desrable one. In ths paper, we formulate ths group marketng-plan generaton problem as a plannng problem. We desgn a novel search algorthm to fnd a cost-effectve and hghly probable plan for swtchng a group of customers from ther ntal states to some more desrable fnal states. We explore the tradeoff among tme, space and qualty of computaton n ths plannng framework. We demonstrate the effectveness of the methods through emprcal results. Introducton Marketng campagn plannng n busness marketng can be consdered as a process of plannng n whch the obectve s to convert groups of customers from one class to another, more proftable class. In busness marketng lterature (Dbb et. al 1996), plannng for marketng campagns corresponds to developng acton plans by takng nto account the customer segmentaton, marketng obectves and budgetary constrants nto consderaton. In the marketng practce today, t s a common practce to desgn acton plans by a human experts through focus group studes (Bank Marketng Assocaton 1989). The plannng process tself s both long and laborous. Marketng plannng can be dvded nto two types. Drect marketng or one-to-one marketng plans are amed at generatng plans to target ndvdual customers. Ths s an expensve process that s only appled to valuable customers. Drect marketng usually assumes that the next acton to be performed can be decded based on observaton of a customer s current state. In ths paper we consder a second knd of marketng plans called segmentaton-marketng plans. These plans are amed at Copyrght 22, Amercan Assocaton for Artfcal Intellgence ( All rghts reserved. marketng to multple groups of customers rather than a sngle customer, where a sequence of actons s executed on a segment of chosen customers untl they are completed. Essentally, the plannng process can be consdered as buldng a statstcal model based on past data and usng the model to formulate a plan to be executed on a group of customers. These marketng actons can hardly be formulated as tradtonal classcal plannng representatons. Often marketng plans are expected to be effectve on only a subset of the customers n the target group, and the plannng s currently done by hand through varous marketng studes such as focus groups. A marketng plan thus generated wll be appled to a chosen subset of customers to be effectve. For example, a cell-phone company may decde to reduce the monthly fee for a subgroup of ts customers who are both hghly valuable and lkely to leave the company for ts compettors. To llustrate, we consder the followng scenaro. Suppose that a company s nterested n marketng to a group of 1, customers n the fnancal market to promote a specal loan sgnup. We start wth a customerloan database wth hstorcal customer nformaton on past loan-marketng results n Table 1. Suppose that we are nterested n buldng a 3-step plan to market to the selected group of customers n the new customer lst. There are many canddate plans to consder n order to move as many customers as possble from non-sgnup status to a sgnup one. The sgnup status corresponds to a postve class that we would lke to move the customers to, and the nonsgnng up status corresponds to the ntal state of our customers. Our plan wll choose not only low-cost actons, but also hghly successful actons from the past experence. For example, a canddate plan mght be: Step 1: Send mals; Step 2: Call home #; Step 3: Offer low nterest rate Ths example also ntroduced a number of nterestng aspects for the plannng problem. Frst, not all people n the group of 1, customers should be consdered as canddates for the converson. Some people should not be consdered as part of marketng campagn because they are too costly or nearly mpossble to convert. To dentfy a group of applcable customers, certan data mnng

2 algorthms for customer segmentaton can be appled. Second, the group marketng problem s to use the same plan for dfferent customers n the ntended customer group, nstead of a dfferent acton plan for each dfferent customer. Ths makes the group marketng dfferent from the drect-marketng problem that some authors have consdered n the data mnng lterature (Domngos and Rchardson 21; Pednault et. al 22; Lng and L 1998; Yang and Cheng 22). For group marketng, we don t have the luxury of observng ntermedate states durng a plan executon n order to decde what to do next. Instead, we must buld an N-step plan ahead of tme, and evaluate the plan accordng to cross-valdaton from the hstorcal records. Thrd, for the customers n the group to be marketed to, there are potentally many possble actons that we can provde. Each acton comes wth an nherent cost assocated wth t. In addton, an acton s not guaranteed to produce ts ntended result. For example, t may be more costly to call a customer at hs home than to send mal. However, sendng a mal to a customer may have less effect than callng a customer at home. Nether malng nor callng can guarantee that all customers contacted wll be converted to sgnup status afterwards. Addtonally, callng a customer may have an adverse effect of annoyng the customer more than necessary. Fnally, t s dffcult to formulate ths problem as a classcal plannng problem, because the precondtons and effects of actons are only mplct n the database, rather than gven ahead of tme by experts n a crsp logcal formulaton. We formulate the above problem as a probablstc plannng problem, where the key ssue s to look for good plans for convertng customer groups. Our approach s to frst dentfy a state space and assgn the potental customers to ntal states. We classfy the customer states nto groups belongng to desrable or undesrable classes. Our obectve becomes one to convert customers from undesrable class to desrable one. We propose an algorthm called MPlan as a soluton to ths plannng problem usng the hstorcal database as an AND-OR tree search problem. The resultng plan wll provde a bass for the fnal marketng plans. Our am s to choose hghutlty actons to be ncluded n our plan where the noton of utlty s ntroduced to ncrease the probablty of success whle reducng costs. Ths research devates from the tradtonal plannng applcatons and formulaton of classcal plannng n several aspects. Compared to classcal plannng, n group marketng we cannot guarantee wth certanty the result of actons; each acton may result n an ntal group to be splt nto several subgroups, each landng n a potental state followng a probablty dstrbuton. Ths dstrbuton can be learned from hstorcal plan traces obtaned before. A second dfference from classcal plannng s that there s no easy way to formulate the actons n terms of relatons and logc formulas needed fro precondtons and effects. The only observable fact before and after an acton s executon s from the hstorcal databases, whch records the customer status n varous attrbutes. By applyng a statstcal classfer to the attrbutes, we could learn a customer s potental standng n terms of desrable versus undesrable classes. The problem s also dfferent from the tradtonal MDP approach (Sutton and Barto 1998) to solvng the probablstc plannng problem. In MDP, t s assumed that at all tme durng a plan s executon, the ntermedate states can be known ether completely or partally. The problem s that of fndng a polcy n whch to drect an agent s acton no matter where the agent s observed to land. The MDP formulaton s more sutable for drect marketng (Lng and L 1998), whch s geared towards fndng a plan for each ndvdual such that an acton s chosen based on the agent s observed resultng state. However, n group marketng, t s often the case that we have no such opportunty to obtan the ntermedate states for a group of customers n the mddle of plan executon. Instead, we have to fnd a sngle plan for a group of smlar customers and to execute ths plan to completon. Thus, ths aspect where a sequence of actons s bult and executed s more akn to classcal plannng. The problem s also dfferent from the probablstc plannng framework of (Draper et. al 1994), whch consdered modelng each acton n a probablstc verson of the pre-condtons and post-condtons. The problem there s stll to consder how to buld a plan from a sngle ntal state to a sngle goal state. In contrast, n our problem the actons logcal representatons are not avalable; all that we can observe are acton labels and ther assocaton wth states. In addton, we consder customer groups that are scattered nto multple ntal states. The goal state (Keeney and Raffa 1976) s not clearly defnable n our case ether, because the postve class n general defnes the potental goal state sets. In data mnng area, a related area s cost-senstve learnng and decson makng (Domngos 1999; Elkan, 21) n the machne learnng communty. However, sgnfcant dfferences reman. Cost-senstve methods try to mnmze the cost of a sngle decson. However, n many applcatons, sequences of decsons n the form of plans are needed. Marketng-Plannng Problem Formulaton We now consder how to formulate the marketng problem more formally as a plannng problem. We frst consder how to buld a state space from a gven set of customer records. As n any machne learnng and data mnng schemes, the nput customer records consst of a set of attrbutes for each customer, along wth a class attrbute that descrbes the customer status. A customer s attrbute may be hs age, ncome, gender, credt wth the bank, and so on. The class attrbute may be Appled, whch s a Boolean ndcatng whether the customer has appled and s approved for loan. As wth any real customer databases, the number of attrbutes may be extremely large; for the KDDCUP-98 data (Blake and Merz 1998), there are a total

3 of 481 attrbutes to descrbe each customer. Of the many attrbutes, some may be removed when constructng a state. For convenence, we refer to ths database table as the Customer Table. Table 1 s an example of Customer Table. Table 1. An example customer-loan database. The last attrbute s the class attrbute. Customer Salar y Cars Mortga ge Table 2. An example Marketng-log database. Loan Sgnup? John 8K 3 None Y Mary 4K 1 3K Y Steve 4K 1 None N S# A# S# A# S# A# S# S1 A1 S2 A2 S3 A3 S4 S A S1 A1 S2 A4 S5 S2 A2 S3 A4 S4 A4 S7 S A S3 A4 S4 A4 S7 A second source of nput s the prevous marketngrecord database. Ths s a database that descrbes how the prevous marketng actons have changed each customer s attrbutes as a result of the actons executon. For example, after a customer receves a promotonal mal, the customer s response to the marketng acton s obtaned and recorded. As a result of the malng, the acton count for the customer n ths marketng campagn s ncremented by one, and the customer may have decded to respond by fllng out a general nformaton form and malng t back to the bank. The status of the customer at any nstant of tme s referred to as a state, and state may change as a result of executng an acton. Thus, the hstorcal marketng-record database conssts of state-acton sequences, one for each partcpatng customer. Ths sequence database wll serve as the tranng data for our planner. For convenence, ths hstorcal marketng database table s referred to as the Marketng-log Table. Table 2 s an example of Marketnglog Table. Gven the Customer table and the Marketng-log table, our frst task s to formulate the problem as a plannng problem. In partcular, we wsh to fnd a method to map the customer records n the customer table nto states usng a statstcal classfer. Ths task n tself s not trval because t maps a large attrbute space nto a more concse space. The problem s more complcated when there are mssng values n the database. In ths paper, we wll not delve nto ths ssue, because t nvolves the ssues of data cleanng and data mnng (Han and Chamber 1998). After the state space s obtaned, we wll use a second classfer to classfy the states nto ether desrable or undesrable states based on the tranng data provded n the Customer table. Classfcaton algorthms such as decson tree or Naïve Bayes are possble choces as long as the classfcaton error rate s low enough. Next, the state-acton sequences n the Marketng-log table wll be used for obtanng acton defntons n a state space, such that each acton s represented as a probablstc mappng from a state to a set of states. To make the representaton more realstc, we wll also consder the cost of executng each acton. To summarze, from the two tables we can obtan the followng nformaton: f s ( r ) = s maps a customer record r to a state s. Ths functon s known as the customer-state mappng functon; p c ( s ) s a probablty functon that returns the probablty that state s s n a desrable class. We call ths classfer the state-classfcaton functon; p( sk s, a ) returns the transton probablty that, after executng an acton a n state s, one ends up n state s k. Once the customer records are converted to states and the state transton through actons are learned from the Marketng-log table, the state space can be formulated as an AND-OR graph. In ths graph, there are two types of nodes. A state node represents a state. From each state node, an acton lnks the state node to an outcome node, whch represents the outcome of performng the acton from the state. An outcome node then splts nto multple state nodes accordng to the probablty dstrbuton gven by the p ( sk s, a ) functon. Ths graph essentally s an AND-OR graph, where each state s an OR node, wth the actons that can be performed on the node formng the OR-branches. Each outcome node s an AND node, where the dfferent arcs connectng the outcome node to the state nodes are the AND edges. A fgure llustratng the scenaro s shown n Fgure 1. Gven a set of customers for whom the marketng plan s desgned, we use a customer-state mappng functon to convert the customer records to a set of ntal states whch these customers belong to ntally n the state space. Note that because of the potentally large number of customers nvolved, there could be a set of ntal states correspondng to the customers, nstead of a sngle ntal state as n classcal plannng. These ntal states provde an ntal segmentaton of the customers. In ths settng, we can gve a defnton of the marketng-plan plannng problem. Gven a set of ntal customers, our goal s to fnd a sequence of actons for each ntal state that converts as many of the customers n that state from the undesrable class to the desrable one whle ncurrng mnmal costs. The plan must satsfy some constrants, n one of the followng forms: length constrant: the number of actons must be at most N;

4 probablty constrant: the expected probablty of beng n a desrable class of all termnal states a plan leads to must be at least Success_Threshold. S3 A2 O3 S1 O1 A1 S2 Fgure1. An example of AND-OR graph. Not all customers n the gven set of customers are convertble to the desrable class. In ths case, we also want to dentfy a subset of customers who can be converted wthn the constrant. The marketng plan generaton problem can be consdered n several forms. A varaton of the problem s to fnd a unform plan for all dfferent customer segments, regardless of whch ntal states they start from, so that as many customers as possble are converted to the desrable class under the length and probablty constrant. Ths formulaton corresponds to the need for corporatons to market to an entre group of customers wth the same actons for consstency and cost-cuttng. In ths paper, we focus on the frst problem where we can have dfferent plans for dfferent segments of customers. Marketng-Campagn Plannng Algorthm Algorthm Overvew A maor dffculty n solvng the marketng-plannng problem stems from the fact that there are potentally many states and many connectons between states. Ths potentally large space can be reduced sgnfcantly by observng that the states and ther connectons are not all equal; some states and acton sequences n ths state-space are more sgnfcant than others because they are more frequently traveled by traces n the Marketng-log table. Ths observaton allows us to use an approach n whch we explot plannng by abstracton. S S7 S4 S5 S6 A2 O4 A2 O2 S8 In partcular, sgnfcant state-acton sequences n the state space can be dscovered through a frequent strngmnng algorthm. We start by defnng a mnmumsupport threshold for fndng the frequent state-acton sequences. Support represents the number of occurrences of a state-acton sequence from the Marketng-log table. More formally, let count(seq) be the number of tmes sequence seq appears n the database for all customers. Then the support for sequence seq s defned as sup( seq ) = count( seq), (1) Then, a strng-mnng algorthm based on movng wndows wll mne the Marketng-log table database to produce state-acton subsequences whose support s no less than a user-defned mnmum-support value. For connecton purpose, we only retaned substrngs both begnnng and endng wth states, n the form of < s, a, s+ 1, a+ 1,..., sn >. Once the frequent sequences are found, we pece together the segments of paths correspondng to the sequences to buld an abstract AND-OR graph n whch we wll search for plans. If < s, a1, s2 > and < s 2, a3, s4 > are two segments found by the strngmnng algorthm, then s, a, s, a s > s a new path < 1 2 3, 4 n the AND-OR graph. Snce each component of the AND-OR graph s guaranteed to be frequent, the AND-OR graph s a hghly concse and representatve state space. Suppose that we wsh to fnd a marketng plan startng from a state s, we consder all acton sequences n the AND- OR graph that start from s satsfyng the length or probablty constrant. We used a functon f ( s, = g( + h( s, to estmate how good a plan s. Let s be an ntal state and p be a plan. Let g( be a functon that sums up the cost of each acton n the plan. Let h ( s, be a heurstc functon estmatng how promsng the plan s for transferrng customers ntally belongng to state s. h( s, s a knd of utlty estmaton of the plan. Ths functon can be determned by users n dfferent specfc applcatons. In our work, we estmated h( s, n the followng manner. We start from an ntal state and follow a plan that leads to several termnal states s, s + 1, s,, s For each of these termnal states, we estmate the state-classfcaton probablty P ( + s ). Each state has a probablty of 1 P ( + s ) to belong to a negatve class. The state requres at least one further acton to proceed to transfer the 1 P ( + s ) who reman negatve, the cost of whch s at least the mnmum of the costs of all actons. We compute heurstc estmaton for termnal states where the plan leads. For an ntermedate state leadng to several states, an expected estmaton s calculated from the heurstc estmaton of ts successve states weghted by the transton probablty p ( s k s, a ). The process starts from termnal states and propagates back to the root, untl reachng the ntal state. Fnally, we obtan the estmaton of h( s, for the ntal state s under the plan p.

5 Based on the above heurstc estmaton methods, we can perform a best-frst search n the space of plans untl the termnaton condton s met. The termnaton condtons are determned by the probablty or the length constrants n the problem doman. Search for Plans usng MPlan In the AND-OR graph, we carry out a procedure MPlan Search to perform a best-frst search for plans. We mantan a prorty queue Q by startng wth a sngle-acton plan. Plans are sorted n the prorty queue n terms of the evaluaton functon f ( s,. In each teraton of the algorthm, we select the plan wth mnmum value of f ( s, from the queue. We then estmate how promsng the plan s. That s, we compute the expected state-classfcaton probablty E ( + s ) from back to front n a smlar way as wth h( s, calculaton, startng wth the P ( + s ) of all termnal states the plan leads to and propagatng back to front, weghted by the transton probablty p ( s k s, a ). We compute E ( + s), the expected value of the state-classfcaton probablty of all termnal states. If ths expected value exceeds a predefned threshold Success_Threshold,.e. the probablty constrant, we consder the plan to be good enough and the search process termnates. Otherwse, one more acton s attached to ths plan and the new plans are nserted nto the prorty queue. E ( + s ) s the expected state-classfcaton probablty estmatng how effectve a plan s at transferrng customers from state s. Its calculaton can be defned n the followng recursve way: E ( + s ) = p( sk s, a)* E( + sk ) ; f s s a nontermnal state; or E + s ) = P( + s ) f s s a termnal state. (2) ( We defne Success_Threshold as a lower bound on E ( + s ). We conduct the above search procedure for all ntal states, fndng one plan for each. It s possble that n some AND-OR graphs, we cannot fnd a plan whose E ( + s ) exceeds the Success_Threshold, ether because the AND-OR graph s not good enough or because the Success_Threshold s too hgh. To address ths, we defne a parameter Max_Step whch defnes the maxmum length of a plan,.e. the length constrant. We wll dscard a canddate plan whch s longer than the Max_Step and E ( + s ) value less than the Success_Threshold. Table 3 s the pseudo code of the MPlan Search algorthm. Consder an example of MPlan Search algorthm usng the AND-OR graph n Fgure 1. Suppose that we are lookng for a plan for customers startng at state s. Suppose we have a fnte set of actons and the mnmum cost among these actons s denoted by MnC. Step 1. We nserted two sngle-step plans <A1> and <A2> nto Q wth the evaluaton functon as follows: Table 3. The MPlan Search Algorthm 1. Insert all possble one-acton plans nto Q. 2. Whle (Q not empty) { 3. Get a plan wth mnmum value of f ( s, from Q. 4. Calculate E ( + s) of ths plan. 5. If ( E ( + s) >= Success_Threshold) Return Plan; 6. If (length(plan) > Max_Ste Dscard Plan; 7. Else 7.1 Expand plan by appendng an acton. 7.2 Calculate f ( s, for the new plans and nsert nto Q. 8 } end whle 9 Return plan not found ; f(s, A1)=Cost(A1) + P(S1 S, A1) * (1-P(+ S1)) * MnC + P(S2 S, A1) * (1-P(+ S2)) * MnC f(s, A2)=Cost(A2) + P(S7 S, A2) * (1-P(+ S7)) * MnC + P(S8 S, A2) * (1-P(+ S8)) * MnC Step 2. Suppose <A1> s the plan wth mnmum f ( s, value n Q. Therefore, <A1> s deleted from Q and examned to see whether t s a qualfed plan. E ( + s) =P(S1 S, A1) * P(+ S1) + P(S2 S, A1) * P(+ S2) If E ( + s) s less than Success_Threshold, then <A1> s not a good plan. Thus, actons A1 and A2 are appended to the end of plan <A1> to form two new plans <A1A1> and <A1A2>. These two plans are then nserted nto Q. Because there s no path <A1A1> n the AND-OR graph, we dscard the canddate <A1A1> from Q. The f ( s, value of <A1A2> s: f(s, A1A2)=Cost(A1A2) + P(S1 S, A1) * (P(S3 S1, A2) * (1-P(+ S3)) * MnC + P(S4 S1, A2) * (1-P(+ S4)) * MnC) + P(S2 S, A1) * (P(S5 S2, A2) * (1-P(+ S5)) * MnC + P(S6 S2, A2) * (1-P(+ S6)) * MnC)) Step 3. Now we have plans <A2>, <A1A2> n Q. Suppose <A1A2> s the plan wth mnmum f ( s,. Therefore, <A1A2> s deleted from Q to see whether t s a promsng plan. E ( + s) = P(S1 S, A1) * (P(S3 S1, A2) * P(+ S3) + P(S4 S1, A2) * P(+ S4)) + P(S2 S, A1) * (P(S5 S2, A2) * P(+ S5) + P(S6 S2, A2) * P(+ S6)) If ( s) good plan wth mnmum cost from Q. E + >= Success_Threshold, then <A1A2> s a

6 Step 4. Return the plan <A1A2>. Stop. Analyss The Marketng-Plan algorthm has two maor components n terms of tme complexty one s state space abstracton by strng mnng algorthm; the other s Mplan, the bestfrst algorthm. In the strng-mnng algorthm, we fnd frequent strngs whch satsfy the predefned mnmum support threshold. Suppose that there are N sequences n the Marketng-log table. The average length of the sequences s K. We scan the sequences wth a fnte wndow of sze W. We need to fnd all the frequent strngs wth length less than or equal to W. For each sequence wth an average length K, tme complexty s O ( K( W + 1) W ( W + 1) / 2). If W << K, then t s O (K). For totally N sequences, t s O (NK). 2 If W s comparable to K, then t s O ( NK ). In the MPlan Search Algorthm, the number of teratons s bounded by the parameter Max_Step. Suppose the number of dfferent actons s A. In the worst case when the algorthm exts wth No plan found, the number of Max _ Step teratons s O ( A ). However, n an average case the plan should complete faster than the worst case. The tme complexty for a sngle teraton s determned by the sze of the state space. In general, t takes more tme to calculate f ( s, and E ( + s ) n a complex state space than n a smple one because the planner has more states and paths to explore. Note that although our proposed state space abstracton method usng strng mnng does not reduce the number of teratons of MPlan algorthm, t saves a lot of tme n explorng n the state space n each teraton because many statstcally trval paths are dscarded before the search process. Expermental Setup Although we are able to obtan Customer data, t has been dffcult to obtan Marketng-log data from real world. To test our deas, we used a smulator to generate the customer-log data accordng to some customer dstrbutons we can specfy. The Customer data are used for tranng a classfer for state-classfcaton functon p c ( s ). The Marketng-log data are used n two ways: (1) Frequent state-acton subsequences are mned from the Marketnglog data to construct a hghly concse and representatve AND-OR graph; (2) Models of transton probablty p( sk s, a ) are estmated from the statstcs of the Marketng-log data. Data set We used the IBM Synthetc Generator ( to generate a Customer dataset wth two classes and nne attrbutes. The postve class has 3, records representng successful customers and negatve has 7, representng unsuccessful ones. Those 7, negatve records are treated as startng ponts for Marketng-log data generaton. We carred out the state abstracton and mappng by feature selecton, only keepng four attrbutes out of nne. Those four attrbutes were converted from contnuous range to dscrete values. The state space has 4 dstnct states. A classfer s traned usng the C4.5 decson tree algorthm (Qunlan 1993) on the Customer dataset. The classfer wll be used later to decde on the class of a state. We generated the Marketng-log data usng another smulator. Each of the 7, negatve records s treated as an ntally faled customer. A trace s then generated for the customer, transformng the customer through ntermedate states to a fnal state. We also defned four types of actons, each of whch has a cost and mpacts on attrbute transtons. We can llustrate the defnton of an acton s mpact on attrbute transtons through an example M 13 = In ths matrx, M 13 s a matrx representng the mpact of acton A 1 on Attrbute three. The matrx s n by n f attrbute has n dfferent values. Suppose attrbute 3 has three dstnct values, 1, 2. The frst row n the matrx means f attrbute 3 takes value, after acton A 1, Attrbute three wll take on the frst value wth 8% probablty, value 1 wth 15% probablty and value 2 wth 5% probablty. After an acton s taken, attrbutes for a customer wll change probablstcally accordng to the defnton of mpact of actons. The Marketng-log data generaton algorthm s shown n Table 4. In ths procedure, we defne termnal* as follows: once a customer changes from the negatve class to the postve class, the state for the frst tme he s classfed as postve s a termnal state. If a customer receved actons for a predefned number of tmes, say, 2 tmes and stll remaned negatve, the 2 th state s the termnal state. Usng ths method, we generated 7, traces for the 7, faled records. Fgure 2 llustrates the dstrbuton of dfferent-length traces. Table 4. The Marketng-log data generaton algorthm. Input: A set of ntally faled states S and a set of actons A wth state-transton matrces. Output: Sequences of trace data preservng temporal order, n the form of s a, s, a,..., s > Algorthm: For each ntally faled state s whle s s not a termnal* state randomly select an acton a; generate the next state s accordng to the mpact of acton a on attrbutes; end whle end for <, n.

7 In Fgure 2, the horzontal axs represents the number of actons n a trace. The vertcal axs represents the number of n-actons traces. For example, the frst dot represents that there are about 18, traces that has only one acton before success. We can see that long traces wth more than 6 actons are very rare Fgure2. Dstrbuton of number of traces as a functon of plan length. Test Crtera We evaluated the qualty of the plans va smulaton. Agan, we used the IBM Synthetc Generator to generate 1, customer records that correspond to the faled class. Our goal s to fnd marketng plans to convert them to a successful class. Ths testng process corresponds to the testng phase for a traned model n machne learnng. In ths smulaton, f there s a plan sutable for convertng a customer record, a sequence of actons s carred out on that record. The plan wll then change the customer record probablstcally accordng to mpact of actons on attrbutes. At the end, the classfer s used to decde whether the changed record has turned nto a successful one. We defne a number of quanttes to measure the success of the test results. Let { s1, s2,... sn} be a set of termnal states n a current plan from an ntal state. We wsh to estmate the success probablty of the plan. We defne a quantty -- the expected success probablty E ( + s ) recursvely as equaton (2). As mentoned before, a user-defned Success_Threshold s used as a lower bound on E ( + s ) ; we consder a plan s found successfully for an ntal state only when E ( + s) s no less than the Success_Threshold. We defne the Max_Step as a length constrant, whereby plans longer than ths lmt wll not be examned. Let N be the number of customers who are to be converted through the marketng plans. Those are the customers who belong to the negatve class ntally. Let PlanSet be the set of plans that are found by the MPLan planner for these N customers. Then the Transton Rate s defned as the proporton of people who are transformed to the successful class after the applcaton of the plan. Let M be the number of customers among the N people who belong to the successful class after the plans are appled. Then M Transton Rate =, (3) N Fnally, let L be the number of ntal segments whch corresponds to L ntal states. Let K be the number of ntal states among the L where a marketng plan s successfully found wthn the stated lmts. Then the Plan Rate s defned as: K PlanRate =, (4) L We also measure the CPU tme used to search for the plan. Ths s denoted as Tme. Evaluaton Results Fgure 3 (a) llustrates the Transton Rate, Plan Rate of a plan as a functon of Success_Threshold. Success_Threshold corresponds to a threshold on the expected probablty that the termnal states are consdered belongng to the postve class. Ths parameter determnes how easy t s to fnd a successful plan. When Success_Threshold s low, many states are consdered postve, and thus plans can be easly and quckly found for most of the ntal states n the graph. Thus we can observe from Fgures 3 (a) and (b) that searchng Tme s low and Plan Rate s hgh wth low Success_Threshold. However, because Success_Threshold s low, the plans found don t guarantee hgh probablty of success. So Transton Rate s also low at frst. As Success_Threshold ncreases, so does the Transton Rate and searchng Tme. When Success_Threshold s too hgh, no plan can be found for some ntal states. Therefore, both of the Plan Rate and Transton Rate decrease. The Tme s much hgher because the searchng process doesn t termnate untl all the plans expanded longer than Max_Step. Fgures 4 and 5 llustrate the Transton Rate, Plan Rate and searchng Tme of a plan as a functon of mnmum support MnSupport. MnSupport =N means a sequence has to appear at least N tmes n Marketng-log data to be frequent. An AND-OR graph wth mnmum support MnSupport=1 corresponds to a state space constructed wthout undergong frequent strng mnng procedure. Generally speakng, MnSupport determnes the frequent state-acton sequences mned from Marketng-log data and thus the sze of the AND-OR graph the search space for a plan. When MnSupport s low, the search space s very large and complex; searchng wll then take a lot of tme to complete. As MnSupport gets larger, the search space becomes more and more compressed, and search tme wll be shorter. When MnSupport becomes extremely hgh, the search space may lose many mportant states and transtons, makng plan searchng harder agan. However, the search effcency also depends on other parameters. Success_Threshold s another mportant factor. When Success_Threshold s low, approxmately the same Plan Rate and Transton Rate can be found no matter how large the state space s, as shown n Fgure 4. When Success_Threshold s hgher, plan searchng becomes more dffcult. In a large search space, searchng for a soluton plan takes a lot more tme. In an extremely concse state space, no plans can be found because many states and

8 transtons are dscarded by state space abstracton, as shown n Fgure 5. Notce that n Fgure 5(a), Transton Rate reaches a maxmum for a state space when MnSupport=1. Ths means that wth approprate frequent strng mnng, plans found n the resulted state space won t lose n terms of effectveness compared wth orgnal state space wthout strng mnng; the feature of strng mnng even provdes better performance. It also saves tme n search space constructon and plan searchng process, as shown n Fgure 5(b). Fgures 6(a) (b) llustrates the Transton Rate, Plan Rate and searchng Tme of a plan as a functon of Max_Step. As we can see from the fgure, ncreasng the length of Max_Step has lttle effect on Transton Rate and Plan Rate. Ths s due to the fact that once a group of customers are converted to a postve class boundary; any further actons wll not mprove ther chance of beng postve. However, Tme ncreases greatly because searchng process often contnues untl all possble plans wthn the length of Max_Step are explored PlanR(%) TransR(%) Fgure 4(a). Plan Rate, Transton Rate vs. MnSupport. Success_Threshold=.5. Max_Step = Tme(s) PlanR(%) TransR(%) Fgure 4(b). CPU Tme vs. MnSupport. Success_Threshold=.5. Max_Step = Fgure 3(a). Plan Rate, Transton Rate vs. Success_Threshold. MnSupport = 1, Max_Step = PlanR(%) TransR(%) Tme(s).4 Fgure 3(b). CPU Tme vs. Success_Threshold. MnSupport = 1, Max_Step = Fgure 5(a). Plan Rate, Transton Rate vs. MnSupport. Success_Threshold=.2. Max_Step = Tme(s) Fgure 5(b). CPU Tme vs. MnSupport. Success_Threshold=.2. Max_Step =5.

9 PlanR(%) TransR(%) Acknowledgments We thank Professors Charles Lng, D.Y. Yeung and Nevn Zhang for ther valuable comments on ths work. Ths work s supported by Hong Kong Research Grant Commttee and SSRI grants Fgure 6(a). Plan Rate, Transton Rate vs. Max_Step. MnSupport=1, Success_Threshold= Tme(s) Fgure 6(b). CPU Tme vs. Max_Step. MnSupport=1, Success_Threshold=.5. Summary of the Test Results In ths test we used sequence mnng as a flter before state space constructon. We observe that usng frequent strng mnng can ndeed save a lot of searchng tme whle at the same tme, provde much more desrable plans n terms of ther performance. Increasng the Success_Threshold can brng about much more promsng plans. Increasng Max_Step does not brng any sgnfcant mprovement n plan searchng when ths parameter s over a certan threshold. Conclusons and Future Work In ths paper, we explored plannng n the marketng plannng doman, where the problem formulaton s sgnfcantly dfferent from the classcal or probablstc plannng stuatons. Our approach combnes both data mnng and plannng n order to buld an abstracton space n whch the plans are obtaned. The plans are no longer transformng an agent s state from one ntal state to a goal state; nstead, n our stuaton we formulate plans that transform groups of customers from a set of ntal states to postve class states. Ths formulaton has many realstc applcatons n the real world, well beyond marketng plannng. In the future, we wsh to consder dfferent varatons of the problem n marketng and other related domans. We wsh to obtan some more realstc data from the customer relatonshp management doman and buld a realstc system for marketng plannng. References S. Dbb, L. Smkn, and J. Bradley The Marketng Plannng Workbook. Routledge. Bank Marketng Assocaton, Buldng a Fnancal Servces Plan: Workng Plans for Product and Segment Marketng. Bank Marketng Assocaton, Fnancal Sourcebooks, Napervlle, Illnos. C. L. Blake and C.J. Merz UCI Repostory of machne learnng databases Irvne, CA: Unversty of Calforna, Department of Informaton and Computer Scence. P. Domngos, MetaCost: A general method for makng classfers cost senstve. In Proceedngs of the Ffth Internatonal Conference on Knowledge Dscovery and Data Mnng, pages ACM Press. P. Domngos and M. Rchardson, 21. Mnng the Network Value of Customers. In Proceedngs of the Seventh Internatonal Conference on Knowledge Dscovery and Data Mnng, pages San Francsco, CA: ACM Press. D. Draper, S. Hanks, and D. Weld, Probablstc plannng wth nformaton gatherng and contngent executon. In Proceedngs of the Second Internatonal Conference on A.I. Plannng Systems. C. Elkan, 21. The foundatons of cost-senstve learnng. In Proceedngs of the Seventeenth Internatonal Jont Conference on Artfcal Intellgence, pages R. L. Keeney and H. Raffa, Decsons wth Multple Obectves: Preferences and Value Trade-offs. Wley, New York. C. X. Lng and C. L Data mnng for drect marketng: Problems and solutons. In Proceedngs of the 4 th Internatonal Conference on Knowledge Dscovery and Data Mnng (KDD 98), pages 73-79, New York. E. Pednault, N. Abe and B. Zadrozny. 22. Sequental Cost-Senstve Decson Makng wth Renforcement Learnng. In Proceedngs of the 8 th ACM Internatonal Conference on Knowledge Dscovery and Data Mnng (KDD 2), pages Edmonton, Canada.

10 J. R. Qunlan C4.5: Programs for Machne Learnng. Morgan Kaufmann Publshers, Inc., San Mateo, CA. R. Sutton and A. Barto, Renforcement Learnng: An Introducton. MIT Press, Cambrdge, MA. Q. Yang and H. Cheng, 22. Mnng Case Bases for Acton Recommendaton. In Proceedngs of 22 IEEE Internatonal Conference on Data Mnng (ICDM 2). Maebash, Japan.

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis The Development of Web Log Mnng Based on Improve-K-Means Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method. Introduction. 0/1 Knapsack Problem The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

More information

Communication Networks II Contents

Communication Networks II Contents 8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

More information

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network 700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School

More information

An Alternative Way to Measure Private Equity Performance

An Alternative Way to Measure Private Equity Performance An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate

More information

IMPACT ANALYSIS OF A CELLULAR PHONE

IMPACT ANALYSIS OF A CELLULAR PHONE 4 th ASA & μeta Internatonal Conference IMPACT AALYSIS OF A CELLULAR PHOE We Lu, 2 Hongy L Bejng FEAonlne Engneerng Co.,Ltd. Bejng, Chna ABSTRACT Drop test smulaton plays an mportant role n nvestgatng

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

An Interest-Oriented Network Evolution Mechanism for Online Communities

An Interest-Oriented Network Evolution Mechanism for Online Communities An Interest-Orented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne

More information

Study on CET4 Marks in China s Graded English Teaching

Study on CET4 Marks in China s Graded English Teaching Study on CET4 Marks n Chna s Graded Englsh Teachng CHE We College of Foregn Studes, Shandong Insttute of Busness and Technology, P.R.Chna, 264005 Abstract: Ths paper deploys Logt model, and decomposes

More information

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

More information

Credit Limit Optimization (CLO) for Credit Cards

Credit Limit Optimization (CLO) for Credit Cards Credt Lmt Optmzaton (CLO) for Credt Cards Vay S. Desa CSCC IX, Ednburgh September 8, 2005 Copyrght 2003, SAS Insttute Inc. All rghts reserved. SAS Propretary Agenda Background Tradtonal approaches to credt

More information

Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications

Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications CMSC828G Prncples of Data Mnng Lecture #9 Today s Readng: HMS, chapter 9 Today s Lecture: Descrptve Modelng Clusterng Algorthms Descrptve Models model presents the man features of the data, a global summary

More information

Can Auto Liability Insurance Purchases Signal Risk Attitude?

Can Auto Liability Insurance Purchases Signal Risk Attitude? Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159-164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? Chu-Shu L Department of Internatonal Busness, Asa Unversty, Tawan Sheng-Chang

More information

1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP)

1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP) 6.3 / -- Communcaton Networks II (Görg) SS20 -- www.comnets.un-bremen.de Communcaton Networks II Contents. Fundamentals of probablty theory 2. Emergence of communcaton traffc 3. Stochastc & Markovan Processes

More information

Lecture 2: Single Layer Perceptrons Kevin Swingler

Lecture 2: Single Layer Perceptrons Kevin Swingler Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCulloch-Ptts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses

More information

On the Optimal Control of a Cascade of Hydro-Electric Power Stations

On the Optimal Control of a Cascade of Hydro-Electric Power Stations On the Optmal Control of a Cascade of Hydro-Electrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;

More information

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

Extending Probabilistic Dynamic Epistemic Logic

Extending Probabilistic Dynamic Epistemic Logic Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set

More information

Mining Multiple Large Data Sources

Mining Multiple Large Data Sources The Internatonal Arab Journal of Informaton Technology, Vol. 7, No. 3, July 2 24 Mnng Multple Large Data Sources Anmesh Adhkar, Pralhad Ramachandrarao 2, Bhanu Prasad 3, and Jhml Adhkar 4 Department of

More information

Vision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION

Vision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION Vson Mouse Saurabh Sarkar a* a Unversty of Cncnnat, Cncnnat, USA ABSTRACT The report dscusses a vson based approach towards trackng of eyes and fngers. The report descrbes the process of locatng the possble

More information

ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING

ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING Matthew J. Lberatore, Department of Management and Operatons, Vllanova Unversty, Vllanova, PA 19085, 610-519-4390,

More information

Nonlinear data mapping by neural networks

Nonlinear data mapping by neural networks Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal

More information

Conversion between the vector and raster data structures using Fuzzy Geographical Entities

Conversion between the vector and raster data structures using Fuzzy Geographical Entities Converson between the vector and raster data structures usng Fuzzy Geographcal Enttes Cdála Fonte Department of Mathematcs Faculty of Scences and Technology Unversty of Combra, Apartado 38, 3 454 Combra,

More information

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

8 Algorithm for Binary Searching in Trees

8 Algorithm for Binary Searching in Trees 8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

More information

Financial Mathemetics

Financial Mathemetics Fnancal Mathemetcs 15 Mathematcs Grade 12 Teacher Gude Fnancal Maths Seres Overvew In ths seres we am to show how Mathematcs can be used to support personal fnancal decsons. In ths seres we jon Tebogo,

More information

A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION. Michael E. Kuhl Radhamés A. Tolentino-Peña

A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION. Michael E. Kuhl Radhamés A. Tolentino-Peña Proceedngs of the 2008 Wnter Smulaton Conference S. J. Mason, R. R. Hll, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler eds. A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION

More information

Software project management with GAs

Software project management with GAs Informaton Scences 177 (27) 238 241 www.elsever.com/locate/ns Software project management wth GAs Enrque Alba *, J. Francsco Chcano Unversty of Málaga, Grupo GISUM, Departamento de Lenguajes y Cencas de

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

Control Charts for Means (Simulation)

Control Charts for Means (Simulation) Chapter 290 Control Charts for Means (Smulaton) Introducton Ths procedure allows you to study the run length dstrbuton of Shewhart (Xbar), Cusum, FIR Cusum, and EWMA process control charts for means usng

More information

Robust Design of Public Storage Warehouses. Yeming (Yale) Gong EMLYON Business School

Robust Design of Public Storage Warehouses. Yeming (Yale) Gong EMLYON Business School Robust Desgn of Publc Storage Warehouses Yemng (Yale) Gong EMLYON Busness School Rene de Koster Rotterdam school of management, Erasmus Unversty Abstract We apply robust optmzaton and revenue management

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy 4.02 Quz Solutons Fall 2004 Multple-Choce Questons (30/00 ponts) Please, crcle the correct answer for each of the followng 0 multple-choce questons. For each queston, only one of the answers s correct.

More information

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

Data Broadcast on a Multi-System Heterogeneous Overlayed Wireless Network *

Data Broadcast on a Multi-System Heterogeneous Overlayed Wireless Network * JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 24, 819-840 (2008) Data Broadcast on a Mult-System Heterogeneous Overlayed Wreless Network * Department of Computer Scence Natonal Chao Tung Unversty Hsnchu,

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

Traffic State Estimation in the Traffic Management Center of Berlin

Traffic State Estimation in the Traffic Management Center of Berlin Traffc State Estmaton n the Traffc Management Center of Berln Authors: Peter Vortsch, PTV AG, Stumpfstrasse, D-763 Karlsruhe, Germany phone ++49/72/965/35, emal peter.vortsch@ptv.de Peter Möhl, PTV AG,

More information

Lecture 3: Force of Interest, Real Interest Rate, Annuity

Lecture 3: Force of Interest, Real Interest Rate, Annuity Lecture 3: Force of Interest, Real Interest Rate, Annuty Goals: Study contnuous compoundng and force of nterest Dscuss real nterest rate Learn annuty-mmedate, and ts present value Study annuty-due, and

More information

Realistic Image Synthesis

Realistic Image Synthesis Realstc Image Synthess - Combned Samplng and Path Tracng - Phlpp Slusallek Karol Myszkowsk Vncent Pegoraro Overvew: Today Combned Samplng (Multple Importance Samplng) Renderng and Measurng Equaton Random

More information

Using Series to Analyze Financial Situations: Present Value

Using Series to Analyze Financial Situations: Present Value 2.8 Usng Seres to Analyze Fnancal Stuatons: Present Value In the prevous secton, you learned how to calculate the amount, or future value, of an ordnary smple annuty. The amount s the sum of the accumulated

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.

More information

Bayesian Network Based Causal Relationship Identification and Funding Success Prediction in P2P Lending

Bayesian Network Based Causal Relationship Identification and Funding Success Prediction in P2P Lending Proceedngs of 2012 4th Internatonal Conference on Machne Learnng and Computng IPCSIT vol. 25 (2012) (2012) IACSIT Press, Sngapore Bayesan Network Based Causal Relatonshp Identfcaton and Fundng Success

More information

Section 5.4 Annuities, Present Value, and Amortization

Section 5.4 Annuities, Present Value, and Amortization Secton 5.4 Annutes, Present Value, and Amortzaton Present Value In Secton 5.2, we saw that the present value of A dollars at nterest rate per perod for n perods s the amount that must be deposted today

More information

Single and multiple stage classifiers implementing logistic discrimination

Single and multiple stage classifiers implementing logistic discrimination Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul - PUCRS Av. Ipranga,

More information

Performance Analysis and Coding Strategy of ECOC SVMs

Performance Analysis and Coding Strategy of ECOC SVMs Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.67-76 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School

More information

Effective Network Defense Strategies against Malicious Attacks with Various Defense Mechanisms under Quality of Service Constraints

Effective Network Defense Strategies against Malicious Attacks with Various Defense Mechanisms under Quality of Service Constraints Effectve Network Defense Strateges aganst Malcous Attacks wth Varous Defense Mechansms under Qualty of Servce Constrants Frank Yeong-Sung Ln Department of Informaton Natonal Tawan Unversty Tape, Tawan,

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

An Evaluation of the Extended Logistic, Simple Logistic, and Gompertz Models for Forecasting Short Lifecycle Products and Services

An Evaluation of the Extended Logistic, Simple Logistic, and Gompertz Models for Forecasting Short Lifecycle Products and Services An Evaluaton of the Extended Logstc, Smple Logstc, and Gompertz Models for Forecastng Short Lfecycle Products and Servces Charles V. Trappey a,1, Hsn-yng Wu b a Professor (Management Scence), Natonal Chao

More information

Business Process Improvement using Multi-objective Optimisation K. Vergidis 1, A. Tiwari 1 and B. Majeed 2

Business Process Improvement using Multi-objective Optimisation K. Vergidis 1, A. Tiwari 1 and B. Majeed 2 Busness Process Improvement usng Mult-objectve Optmsaton K. Vergds 1, A. Twar 1 and B. Majeed 2 1 Manufacturng Department, School of Industral and Manufacturng Scence, Cranfeld Unversty, Cranfeld, MK43

More information

A DATA MINING APPLICATION IN A STUDENT DATABASE

A DATA MINING APPLICATION IN A STUDENT DATABASE JOURNAL OF AERONAUTICS AND SPACE TECHNOLOGIES JULY 005 VOLUME NUMBER (53-57) A DATA MINING APPLICATION IN A STUDENT DATABASE Şenol Zafer ERDOĞAN Maltepe Ünversty Faculty of Engneerng Büyükbakkalköy-Istanbul

More information

Lecture 18: Clustering & classification

Lecture 18: Clustering & classification O CPS260/BGT204. Algorthms n Computatonal Bology October 30, 2003 Lecturer: Pana K. Agarwal Lecture 8: Clusterng & classfcaton Scrbe: Daun Hou Open Problem In HomeWor 2, problem 5 has an open problem whch

More information

Intra-year Cash Flow Patterns: A Simple Solution for an Unnecessary Appraisal Error

Intra-year Cash Flow Patterns: A Simple Solution for an Unnecessary Appraisal Error Intra-year Cash Flow Patterns: A Smple Soluton for an Unnecessary Apprasal Error By C. Donald Wggns (Professor of Accountng and Fnance, the Unversty of North Florda), B. Perry Woodsde (Assocate Professor

More information

1. Math 210 Finite Mathematics

1. Math 210 Finite Mathematics 1. ath 210 Fnte athematcs Chapter 5.2 and 5.3 Annutes ortgages Amortzaton Professor Rchard Blecksmth Dept. of athematcal Scences Northern Illnos Unversty ath 210 Webste: http://math.nu.edu/courses/math210

More information

Multiple-Period Attribution: Residuals and Compounding

Multiple-Period Attribution: Residuals and Compounding Multple-Perod Attrbuton: Resduals and Compoundng Our revewer gave these authors full marks for dealng wth an ssue that performance measurers and vendors often regard as propretary nformaton. In 1994, Dens

More information

Number of Levels Cumulative Annual operating Income per year construction costs costs ($) ($) ($) 1 600,000 35,000 100,000 2 2,200,000 60,000 350,000

Number of Levels Cumulative Annual operating Income per year construction costs costs ($) ($) ($) 1 600,000 35,000 100,000 2 2,200,000 60,000 350,000 Problem Set 5 Solutons 1 MIT s consderng buldng a new car park near Kendall Square. o unversty funds are avalable (overhead rates are under pressure and the new faclty would have to pay for tself from

More information

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and POLYSA: A Polynomal Algorthm for Non-bnary Constrant Satsfacton Problems wth and Mguel A. Saldo, Federco Barber Dpto. Sstemas Informátcos y Computacón Unversdad Poltécnca de Valenca, Camno de Vera s/n

More information

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts Power-of-wo Polces for Sngle- Warehouse Mult-Retaler Inventory Systems wth Order Frequency Dscounts José A. Ventura Pennsylvana State Unversty (USA) Yale. Herer echnon Israel Insttute of echnology (Israel)

More information

CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol

CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK Sample Stablty Protocol Background The Cholesterol Reference Method Laboratory Network (CRMLN) developed certfcaton protocols for total cholesterol, HDL

More information

1.1 The University may award Higher Doctorate degrees as specified from time-to-time in UPR AS11 1.

1.1 The University may award Higher Doctorate degrees as specified from time-to-time in UPR AS11 1. HIGHER DOCTORATE DEGREES SUMMARY OF PRINCIPAL CHANGES General changes None Secton 3.2 Refer to text (Amendments to verson 03.0, UPR AS02 are shown n talcs.) 1 INTRODUCTION 1.1 The Unversty may award Hgher

More information

Calculation of Sampling Weights

Calculation of Sampling Weights Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a two-stage stratfed cluster desgn. 1 The frst stage conssted of a sample

More information

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there

More information

When do data mining results violate privacy? Individual Privacy: Protect the record

When do data mining results violate privacy? Individual Privacy: Protect the record When do data mnng results volate prvacy? Chrs Clfton March 17, 2004 Ths s jont work wth Jashun Jn and Murat Kantarcıoğlu Indvdual Prvacy: Protect the record Indvdual tem n database must not be dsclosed

More information

Project Networks With Mixed-Time Constraints

Project Networks With Mixed-Time Constraints Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa

More information

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression Novel Methodology of Workng Captal Management for Large Publc Constructons by Usng Fuzzy S-curve Regresson Cheng-Wu Chen, Morrs H. L. Wang and Tng-Ya Hseh Department of Cvl Engneerng, Natonal Central Unversty,

More information

A GENERAL APPROACH FOR SECURITY MONITORING AND PREVENTIVE CONTROL OF NETWORKS WITH LARGE WIND POWER PRODUCTION

A GENERAL APPROACH FOR SECURITY MONITORING AND PREVENTIVE CONTROL OF NETWORKS WITH LARGE WIND POWER PRODUCTION A GENERAL APPROACH FOR SECURITY MONITORING AND PREVENTIVE CONTROL OF NETWORKS WITH LARGE WIND POWER PRODUCTION Helena Vasconcelos INESC Porto hvasconcelos@nescportopt J N Fdalgo INESC Porto and FEUP jfdalgo@nescportopt

More information

Staff Paper. Farm Savings Accounts: Examining Income Variability, Eligibility, and Benefits. Brent Gloy, Eddy LaDue, and Charles Cuykendall

Staff Paper. Farm Savings Accounts: Examining Income Variability, Eligibility, and Benefits. Brent Gloy, Eddy LaDue, and Charles Cuykendall SP 2005-02 August 2005 Staff Paper Department of Appled Economcs and Management Cornell Unversty, Ithaca, New York 14853-7801 USA Farm Savngs Accounts: Examnng Income Varablty, Elgblty, and Benefts Brent

More information

DEFINING %COMPLETE IN MICROSOFT PROJECT

DEFINING %COMPLETE IN MICROSOFT PROJECT CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMI-SP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,

More information

Fault tolerance in cloud technologies presented as a service

Fault tolerance in cloud technologies presented as a service Internatonal Scentfc Conference Computer Scence 2015 Pavel Dzhunev, PhD student Fault tolerance n cloud technologes presented as a servce INTRODUCTION Improvements n technques for vrtualzaton and performance

More information

Thursday, December 10, 2009 Noon - 1:50 pm Faraday 143

Thursday, December 10, 2009 Noon - 1:50 pm Faraday 143 1. ath 210 Fnte athematcs Chapter 5.2 and 4.3 Annutes ortgages Amortzaton Professor Rchard Blecksmth Dept. of athematcal Scences Northern Illnos Unversty ath 210 Webste: http://math.nu.edu/courses/math210

More information

FREQUENCY OF OCCURRENCE OF CERTAIN CHEMICAL CLASSES OF GSR FROM VARIOUS AMMUNITION TYPES

FREQUENCY OF OCCURRENCE OF CERTAIN CHEMICAL CLASSES OF GSR FROM VARIOUS AMMUNITION TYPES FREQUENCY OF OCCURRENCE OF CERTAIN CHEMICAL CLASSES OF GSR FROM VARIOUS AMMUNITION TYPES Zuzanna BRO EK-MUCHA, Grzegorz ZADORA, 2 Insttute of Forensc Research, Cracow, Poland 2 Faculty of Chemstry, Jagellonan

More information

Formulating & Solving Integer Problems Chapter 11 289

Formulating & Solving Integer Problems Chapter 11 289 Formulatng & Solvng Integer Problems Chapter 11 289 The Optonal Stop TSP If we drop the requrement that every stop must be vsted, we then get the optonal stop TSP. Ths mght correspond to a ob sequencng

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

Inter-Ing 2007. INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, 15-16 November 2007.

Inter-Ing 2007. INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, 15-16 November 2007. Inter-Ing 2007 INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, 15-16 November 2007. UNCERTAINTY REGION SIMULATION FOR A SERIAL ROBOT STRUCTURE MARIUS SEBASTIAN

More information

Brigid Mullany, Ph.D University of North Carolina, Charlotte

Brigid Mullany, Ph.D University of North Carolina, Charlotte Evaluaton And Comparson Of The Dfferent Standards Used To Defne The Postonal Accuracy And Repeatablty Of Numercally Controlled Machnng Center Axes Brgd Mullany, Ph.D Unversty of North Carolna, Charlotte

More information

THE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION

THE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION Internatonal Journal of Electronc Busness Management, Vol. 3, No. 4, pp. 30-30 (2005) 30 THE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION Yu-Mn Chang *, Yu-Cheh

More information

To manage leave, meeting institutional requirements and treating individual staff members fairly and consistently.

To manage leave, meeting institutional requirements and treating individual staff members fairly and consistently. Corporate Polces & Procedures Human Resources - Document CPP216 Leave Management Frst Produced: Current Verson: Past Revsons: Revew Cycle: Apples From: 09/09/09 26/10/12 09/09/09 3 years Immedately Authorsaton:

More information

Performance attribution for multi-layered investment decisions

Performance attribution for multi-layered investment decisions Performance attrbuton for mult-layered nvestment decsons 880 Thrd Avenue 7th Floor Ne Yor, NY 10022 212.866.9200 t 212.866.9201 f qsnvestors.com Inna Oounova Head of Strategc Asset Allocaton Portfolo Management

More information

Portfolio Loss Distribution

Portfolio Loss Distribution Portfolo Loss Dstrbuton Rsky assets n loan ortfolo hghly llqud assets hold-to-maturty n the bank s balance sheet Outstandngs The orton of the bank asset that has already been extended to borrowers. Commtment

More information

Implementation of Deutsch's Algorithm Using Mathcad

Implementation of Deutsch's Algorithm Using Mathcad Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages - n "Machnes, Logc and Quantum Physcs"

More information

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPP-ATBD-ClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller

More information

Simple Interest Loans (Section 5.1) :

Simple Interest Loans (Section 5.1) : Chapter 5 Fnance The frst part of ths revew wll explan the dfferent nterest and nvestment equatons you learned n secton 5.1 through 5.4 of your textbook and go through several examples. The second part

More information

ECE544NA Final Project: Robust Machine Learning Hardware via Classifier Ensemble

ECE544NA Final Project: Robust Machine Learning Hardware via Classifier Ensemble 1 ECE544NA Fnal Project: Robust Machne Learnng Hardware va Classfer Ensemble Sa Zhang, szhang12@llnos.edu Dept. of Electr. & Comput. Eng., Unv. of Illnos at Urbana-Champagn, Urbana, IL, USA Abstract In

More information

1. Measuring association using correlation and regression

1. Measuring association using correlation and regression How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a

More information

Understanding the Impact of Marketing Actions in Traditional Channels on the Internet: Evidence from a Large Scale Field Experiment

Understanding the Impact of Marketing Actions in Traditional Channels on the Internet: Evidence from a Large Scale Field Experiment A research and educaton ntatve at the MT Sloan School of Management Understandng the mpact of Marketng Actons n Tradtonal Channels on the nternet: Evdence from a Large Scale Feld Experment Paper 216 Erc

More information

Efficient Project Portfolio as a tool for Enterprise Risk Management

Efficient Project Portfolio as a tool for Enterprise Risk Management Effcent Proect Portfolo as a tool for Enterprse Rsk Management Valentn O. Nkonov Ural State Techncal Unversty Growth Traectory Consultng Company January 5, 27 Effcent Proect Portfolo as a tool for Enterprse

More information

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence 1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh

More information

An Enhanced Super-Resolution System with Improved Image Registration, Automatic Image Selection, and Image Enhancement

An Enhanced Super-Resolution System with Improved Image Registration, Automatic Image Selection, and Image Enhancement An Enhanced Super-Resoluton System wth Improved Image Regstraton, Automatc Image Selecton, and Image Enhancement Yu-Chuan Kuo ( ), Chen-Yu Chen ( ), and Chou-Shann Fuh ( ) Department of Computer Scence

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

An MILP model for planning of batch plants operating in a campaign-mode

An MILP model for planning of batch plants operating in a campaign-mode An MILP model for plannng of batch plants operatng n a campagn-mode Yanna Fumero Insttuto de Desarrollo y Dseño CONICET UTN yfumero@santafe-concet.gov.ar Gabrela Corsano Insttuto de Desarrollo y Dseño

More information

Customer Lifetime Value Modeling and Its Use for Customer Retention Planning

Customer Lifetime Value Modeling and Its Use for Customer Retention Planning Customer Lfetme Value Modelng and Its Use for Customer Retenton Plannng Saharon Rosset Enat Neumann Ur Eck Nurt Vatnk Yzhak Idan Amdocs Ltd. 8 Hapnna St. Ra anana 43, Israel {saharonr, enatn, ureck, nurtv,

More information

Construction Rules for Morningstar Canada Target Dividend Index SM

Construction Rules for Morningstar Canada Target Dividend Index SM Constructon Rules for Mornngstar Canada Target Dvdend Index SM Mornngstar Methodology Paper October 2014 Verson 1.2 2014 Mornngstar, Inc. All rghts reserved. The nformaton n ths document s the property

More information

Ants Can Schedule Software Projects

Ants Can Schedule Software Projects Ants Can Schedule Software Proects Broderck Crawford 1,2, Rcardo Soto 1,3, Frankln Johnson 4, and Erc Monfroy 5 1 Pontfca Unversdad Católca de Valparaíso, Chle FrstName.Name@ucv.cl 2 Unversdad Fns Terrae,

More information

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6 PAR TESTS If a WEIGHT varable s specfed, t s used to replcate a case as many tmes as ndcated by the weght value rounded to the nearest nteger. If the workspace requrements are exceeded and samplng has

More information

Allocating Time and Resources in Project Management Under Uncertainty

Allocating Time and Resources in Project Management Under Uncertainty Proceedngs of the 36th Hawa Internatonal Conference on System Scences - 23 Allocatng Tme and Resources n Project Management Under Uncertanty Mark A. Turnqust School of Cvl and Envronmental Eng. Cornell

More information

Estimating the Development Effort of Web Projects in Chile

Estimating the Development Effort of Web Projects in Chile Estmatng the Development Effort of Web Projects n Chle Sergo F. Ochoa Computer Scences Department Unversty of Chle (56 2) 678-4364 sochoa@dcc.uchle.cl M. Cecla Bastarrca Computer Scences Department Unversty

More information

Improved Mining of Software Complexity Data on Evolutionary Filtered Training Sets

Improved Mining of Software Complexity Data on Evolutionary Filtered Training Sets Improved Mnng of Software Complexty Data on Evolutonary Fltered Tranng Sets VILI PODGORELEC Insttute of Informatcs, FERI Unversty of Marbor Smetanova ulca 17, SI-2000 Marbor SLOVENIA vl.podgorelec@un-mb.s

More information