New Approaches to Support Vector Ordinal Regression

Size: px
Start display at page:

Download "New Approaches to Support Vector Ordinal Regression"

Transcription

1 New Approaches to Support Vector Ordnal Regresson We Chu Gatsby Computatonal Neuroscence Unt, Unversty College London, London, WCN 3AR, UK S. Sathya Keerth Yahoo! Research Labs, 0 S. DeLacey Avenue, Pasadena, CA-905, USA Abstract In ths paper, we propose two new support vector approaches for ordnal regresson, whch optmze multple thresholds to defne parallel dscrmnant hyperplanes for the ordnal scales. Both approaches guarantee that the thresholds are properly ordered at the optmal soluton. The sze of these optmzaton problems s lnear n the number of tranng samples. The SMO algorthm s adapted for the resultng optmzaton problems; t s extremely easy to mplement and scales effcently as a quadratc functon of the number of examples. The results of numercal experments on benchmark datasets verfy the usefulness of these approaches.. Introducton We consder the supervsed learnng problem of predctng varables of ordnal scale, a settng that brdges metrc regresson and classfcaton, and referred to as rankng learnng or ordnal regresson. Ordnal regresson arses frequently n socal scence and nformaton retreval where human preferences play a maor role. The tranng samples are labelled by a set of ranks, whch exhbts an orderng among the dfferent categores. In contrast to metrc regresson problems, these ranks are of fnte types and the metrc dstances between the ranks are not defned. These ranks are also dfferent from the labels of multple classes n classfcaton problems due to the exstence of the orderng nformaton. There are several approaches to tackle ordnal regresson problems n the doman of machne learnng. The nave dea s to transform the ordnal scales nto numerc values, and then solve the problem as a stan- Appearng n Proceedngs of the st Internatonal Conference on Machne Learnng, Bonn, Germany, 005. Copyrght 005 by the author(s/owner(s. dard regresson problem. Kramer et al. (00 nvestgated the use of a regresson tree learner n ths way. A problem wth ths approach s that there mght be no prncpled way of devsng an approprate mappng functon snce the true metrc dstances between the ordnal scales are unknown n most of the tasks. Another dea s to decompose the orgnal ordnal regresson problem nto a set of bnary classfcaton tasks. Frank and Hall (00 converted an ordnal regresson problem nto nested bnary classfcaton problems that encode the orderng of the orgnal ranks and then organzed the results of these bnary classfers n some ad hoc way for predcton. It s also possble to formulate the orgnal problem as a large augmented bnary classfcaton problem. Har-Peled et al. (00 proposed a constrant classfcaton approach that provdes a unfed framework for solvng rankng and mult-classfcaton problems. Herbrch et al. (000 appled the prncple of Structural Rsk Mnmzaton (Vapnk, 995 to ordnal regresson leadng to a new dstrbuton-ndependent learnng algorthm based on a loss functon between pars of ranks. The man dffculty wth these two algorthms (Har-Peled et al., 00; Herbrch et al., 000 s that the problem sze of these formulatons s a quadratc functon of the tranng data sze. As for sequental learnng, Crammer and Snger (00 proposed a proceptronbased onlne algorthm for rank predcton, known as the PRank algorthm. Shashua and Levn (003 generalzed the support vector formulaton for ordnal regresson by fndng r thresholds that dvde the real lne nto r consecutve ntervals for the r ordered categores. However there s a problem wth ther approach: the ordnal nequaltes on the thresholds, b b... b r, are not ncluded n ther formulaton. Ths omsson may result n dsordered thresholds at the soluton on some unfortunate cases (see secton 4. for an example. In ths paper, we propose two new approaches for support vector ordnal regresson. The frst one takes only the adacent ranks nto account n determnng

2 New Approaches to Support Vector Ordnal Regresson the thresholds, exactly as Shashua and Levn (003 proposed, but we ntroduce explct constrants n the problem formulaton that enforce the nequaltes on the thresholds. The second approach s entrely new; t consders the tranng samples from all the ranks to determne each threshold. Interestngly, we show that, n ths second approach, the ordnal nequalty constrants on the thresholds are automatcally satsfed at the optmal soluton though there are no explct constrants on these thresholds. For both approaches the sze of the optmzaton problems s lnear n the number of tranng samples. We show that the popular SMO algorthm (Platt, 999; Keerth et al., 00 for SVMs can be easly adapted for the two approaches. The resultng algorthms scale effcently; emprcal analyss shows that the cost s roughly a quadratc functon of the problem sze. Usng several benchmark datasets we demonstrate that the generalzaton capabltes of the two approaches are much better than that of the nave approach of dong standard regresson on the ordnal labels. The paper s organzed as follows. In secton we present the frst approach wth explct nequalty constrants on the thresholds, derve the optmalty condtons for the dual problem, and adapt the SMO algorthm for the soluton. In secton 3 we present the second approach wth mplct constrants. In secton 4 we do an emprcally study to show the scalng propertes of the two algorthms and ther generalzaton performance. We conclude n secton 5. Notatons Throughout ths paper we wll use x to denote the nput vector of the ordnal regresson problem and φ(x to denote the feature vector n a hgh dmensonal reproducng kernel Hlbert space (RKHS related to x by transformaton. All computatons wll be done usng the reproducng kernel functon only, whch s defned as K(x, x = φ(x φ(x ( where denotes nner product n the RKHS. Wthout loss of generalty, we consder an ordnal regresson problem wth r ordered categores and denote these categores as consecutve ntegers Y = {,,...,r} to keep the known orderng nformaton. In the -th category, where Y, the number of tranng samples s denoted as n,andthe -th tranng sample s denoted as x where x Rd. The total number of tranng samples r = n s denoted as n. b, =,...,r denotethe(r thresholds.. Explct Constrants on Thresholds As a powerful computatonal tool for supervsed learnng, support vector machnes (SVMs map the nput vectors nto feature vectors n a hgh dmensonal y= ξ + y= y=3 ξ b - b b + ξ + ξ b b - b + f(x = w. φ(x Fgure. An llustraton of the defnton of slack varables ξ and ξ for the thresholds. The samples from dfferent ranks, represented as crcles flled wth dfferent patterns, are mapped by w φ(x onto the axs of functon value. Note that a sample from rank + could be counted twce for errors f t s sandwched by b + andb +where b + <b +, and the samples from rank +, etc. never gve contrbutons to the threshold b. RKHS (Vapnk, 995; Schölkopf & Smola, 00, where a lnear machne s constructed by mnmzng a regularzed functonal. For bnary classfcaton (a specal case of ordnal regresson wth r =, SVMs fnd an optmal drecton that maps the feature vectors nto functon values on the real lne, and a sngle optmzed threshold s used to dvde the real lne nto two regons for the two classes respectvely. In the settng of ordnal regresson, the support vector formulaton could attempt to fnd an optmal mappng drecton w, andr thresholds, whch defne r parallel dscrmnant hyperplanes for the r ranks accordngly. For each threshold b, Shashua and Levn (003 suggested consderng the samples from the two adacent categores, and +, for emprcal errors (see Fgure for an llustraton. More exactly, each sample n the -th category should have a functon value that s less than the lower margn b, otherwse w φ(x (b s the error (denoted as ξ ; smlarly, each sample from the ( +-th category should have a functon value that s greater than the upper margn b +, otherwse (b + w φ(x + s the error (denoted as ξ +. Shashua and Levn (003 generalzed the prmal problem of SVMs to ordnal regresson as follows: r mn w,b,ξ,ξ w w + C = ( n + n ξ + ξ + = = subect to w φ(x b +ξ, ξ 0, for =,...,n ; w φ(x + b + ξ +, ξ + 0, for =,...,n + ; where runs over,...,r andc>0. ( (3 The superscrpt n ξ + denotes that the error s assocated wth a sample n the adacent upper category of the -th threshold.

3 New Approaches to Support Vector Ordnal Regresson A problem wth the above formulaton s that the natural ordnal nequaltes on the thresholds,.e., b b... b r cannot be guaranteed to hold at the soluton. To tackle ths problem, we explctly nclude the followng constrants n (3: b b, for =,...,r. (4.. Prmal and Dual Problems By ntroducng two auxlary varables b 0 = and b r =+, the modfed prmal problem n ( (4 can be equvalently wrtten as follows: mn w,b,ξ,ξ w w + C r = subect to n = ( ξ + ξ w φ(x b +ξ, ξ 0,, ; w φ(x b + ξ, ξ 0,, ; b b,. (5 (6 The dual problem can be derved by standard Lagrangan technques. Let α 0, γ 0, α 0, γ 0andµ 0 be the Lagrangan multplers for the nequaltes n (6. The Lagrangan for the prmal problem s: L e = w w + C r ( n = = ξ + ξ r n = = α ( +ξ w φ(x + b r n = ( +ξ + w φ(x b = α r = γ ξ r = γ ξ r = µ (b b. (7 The KKT condtons for the prmal problem requre the followng to hold: L e b L e w = w r = L e ξ L e ξ = n = n = ( α α φ(x = 0; (8 = C α γ =0,, ; (9 = C α γ (α + µ n + =0,, ; (0 ( α + + µ + =0,. = Note that the dummy varables assocated wth b 0 and b r,.e. µ, µ r, α s and α r s, are always zero. The condtons (9 and (0 gve rse to the constrants 0 α C and 0 α C respectvely. Let us now apply Wolfe dualty theory to the prmal problem. By ntroducng the KKT condtons (8 (0 nto the Lagrangan (7 and applyng the kernel trck (, the dual problem becomes a maxmzaton problem nvolvng the Lagrangan multplers α, α and µ: max (α +α (α α (α α K(x,x,,, ( subect to 0 α C,,, 0 α + C,,, n = α + µ = n ( + = α + + µ +,, µ 0,, where runs over,...,r. Leavng the dummy varables out of account, the sze of the optmzaton problem s n n n r (α and α plusr (forµ. The dual problem ( ( s a convex quadratc programmng problem. Once the α, α and µ are obtaned by solvng ths problem, w s obtaned from (8. The determnaton of the b s wll be addressed n the next secton. The dscrmnant functon value for a new nput vector x s f(x = w x =, (α α K(x,x. (3 The predctve ordnal decson functon s gven by arg mn { : f(x <b }... Optmalty Condtons for the Dual To derve proper stoppng condtons for algorthms that solve the dual problem and also determne the thresholds b s, t s mportant to wrte down the optmalty condtons for the dual. Though the resultng condtons that are derved below look a bt clumsy because of the notatons, the deas behnd them are very much smlar to those for the bnary SVM classfer case. The Lagrangan for the dual can be wrtten down as follows:,, (α α (α α K(x,x + r = β ( n = α n + = α + + µ µ + L d =, (η α + η π (C α α, (π (C α + r = λ µ, (α + α where the Lagrangan multplers η, η, π, π and λ are non-negatve, whle β can take any value. The KKT condtons assocated wth β can be gven as follows: L d α = f(x η + π + β =0,π 0, η 0,π (C α =0,η α =0, for =,...,n ; = f(x + η + + π + β =0, L d α + π + η + 0,η + 0,π + (C α + =0, for =,...,n + ; α + =0, (4 where f(x s defned as n (3, whle the KKT condtons assocated wth the µ are β β λ =0,λ µ =0,λ 0, (5

4 New Approaches to Support Vector Ordnal Regresson where =,...,r. The condtons n (4 can be re-grouped nto the followng sx cases: case : α =0 f(x + β case : 0 <α <C f(x +=β case 3 : α = C f(x + β case 4 : α + =0 f(x + β case 5 : 0 <α + <C f(x + =β case 6 : α + = C f(x + β We can classfy any varable nto one of the followng sx sets: I 0a = { {,...,n } :0<α <C} I 0b = { {,...,n+ } :0<α + <C} I = { {,...,n+ } : α + =0} I = { {,...,n } : α =0} I 3 = { {,...,n } : α = C} I 4 = { {,...,n+ } : α + = C} Let us denote I 0 = I 0a I 0b, I up = I 0 I I 3 and I low = I 0 I I 4. We further defne F up(β onthe set Iup as { Fup(β f(x = + f I 0a I 3 f(x + f I 0b I and Flow (β on the set I low as F low(β = { f(x + f I 0a I f(x + f I 0b I 4 Then the condtons can be smplfed as β Fup(β Iup and β Flow (β I low, whch can be compactly wrtten as: b low β b up (6 where b up = mn{fup(β : Iup} and b low = max{flow : I low }. The KKT condtons n (5 ndcate that the condton, β β always holds, and that β = β f µ > 0. To merge the condtons (5 and (6, let us defne B low = max{bk low : k =,...,} and B up = mn{b k up : k =,...,r }, where =,...,r. The overall optmalty condtons can be smply wrtten as B low β Bup where { B+ B low = low f µ + > 0 B low otherwse and { B Bup up f µ = > 0 B up otherwse. Table. The basc framework of the SMO algorthm for support vector ordnal regresson usng explct threshold constrants. SMO start at a vald pont, α, α and µ, that satsfy (, fnd the current B up and B low Loop do. determne the actve threshold J. optmze the par of actve varables and the set µ a 3. compute B up and B low at the new pont whle the optmalty condton (7 has not been satsfed Ext return α, α and b We ntroduce a tolerance parameter τ > 0, usually 0.00, to defne approxmate optmalty condtons. The overall stoppng condton becomes max{b low B up : =,...,r } τ. (7 From the condtons n (4 and (3, t s easy to see the close relatonshp between the b s n the prmal problem and the multplers β s. In partcular, at the optmal soluton, β and b are dentcal. Thus b can be taken to be any value from the nterval, [B low,b up]. We can resolve any non-unqueness by smply takng b = (B low + B up. Note that the KKT condtons n (5, comng from the addtve constrants n (4 we ntroduced n Shashua and Levn s formulaton, enforce B low B low and B up Bup at the soluton, whch guarantee that the thresholds specfed n these feasble regons wll satsfy the nequalty constrants b b ; wthout the constrants n (4, the thresholds mght be dsordered at the soluton!.3. SMO Algorthm In ths secton we adapt the SMO algorthm (Platt, 999; Keerth et al., 00 for the soluton of ( (. The key dea of SMO conssts of startng wth a vald ntal pont and optmzng only one par of varables at a tme whle fxng all the other varables. The suboptmzaton problem of the two actve varables can be solved analytcally. Table presents an outlne of the SMO mplementaton for our optmzaton problem. In order to determne the par of actve varables to optmze, we select the actve threshold frst. The ndex of the actve threshold s defned as J = arg max { : B low B up >τ}. Let us assume that Blow J and BJ up are actually defned by b o low and bu up respectvely, and that the two multplers assocated wth b o low and bu up are α o and α u. The par of multplers (α o,α u s optmzed from the current pont (αo old new pont, (αo new,αu new.,α old u toreachthe It s possble that o u. In ths case, named as cross update, more than one equalty constrant n ( s nvolved n the optmzaton that may update the

5 New Approaches to Support Vector Ordnal Regresson varable set µ a = {µ mn{o,u}+,...,µ max{o,u} }, a subset of µ. In the case of o = u, named as standard update, only one equalty constrant s nvolved and the varables of µ are keep ntact,.e. µ a =. These suboptmzaton problems can be solved analytcally, and the detaled formulas for updatng can be found n our longer techncal report (Chu & Keerth, Implct Constrants on Thresholds In ths secton we present a new approach to support vector ordnal regresson. Instead of consderng only the emprcal errors from the samples of adacent categores to determne a threshold, we allow the samples n all the categores to contrbute errors for each threshold. A very nce property of ths approach s that the ordnal nequaltes on the thresholds are satsfed automatcally at the optmal soluton n spte of the fact that such constrants on the thresholds are not explctly ncluded n the new formulaton. Fgure explans the new defnton of slack varables ξ and ξ. For a threshold b, the functon values of all the samples from all the lower categores, should be less than the lower margn b ; f that does not hold, then ξ k = w φ(xk (b s taken as the error assocated wth the sample x k for b, where k. Smlarly, the functon values of all the samples from the upper categores should be greater than the upper margn b + ; otherwse ξ k =(b + w φ(x k s the error assocated wth the sample x k for b, where k>. Here, the subscrpt k denotes that the slack varable s assocated wth the -th nput sample n the k-th category; the superscrpt denotes that the slack varable s assocated wth the lower categores of b ; and the superscrpt denotes that the slack varable s assocated wth the upper categores of b. 3.. Prmal Problem By takng all the errors assocated wth all r thresholds nto account, the prmal problem can be defned as follows: r mn w,b,ξ,ξ w w + C = ( n k ξ k + k= = r n k k=+ = ξ k (8 subect to w φ(x k b +ξ k, ξ k 0, for k =,..., and =,...,n k ; w φ(x k b + ξ k, ξ k 0, (9 for k = +,...,r and =,...,n k ; where runs over,...,r. Note that there are r nequalty constrants for each sample x k (one for each threshold. y= ξ y= y=3 ξ ξ 3 b - b b + ξ 3 ξ b - b b + f(x = w. φ(x Fgure. An llustraton on the new defnton of slack varables ξ and ξ that mposes mplct constrants on the thresholds. All the samples are mapped by w φ(x onto the axs of functon values. Note the term ξ3 n ths graph. To prove the nequaltes on the thresholds at the optmal soluton, let us consder the stuaton where w s fxed and only the b s are optmzed. Note that the ξ k and ξ k are automatcally determned once the b are gven. To elmnate these varables, let us defne, for k r, (b ={ {,...,n k } : w φ(x k b }, I up k (b ={ {,...,nk } : w φ(x k b }. It s easy to see that b s optmal ff t mnmzes the functon e (b = k= Ik low(b( w φ(xk b + + r k=+ I up (b( w φ(xk + b + (0 k Let B denote the set of all mnmzers of e (b. By convexty, B s a closed nterval. Gven two ntervals B =[c,d ]andb =[c,d ], we say B B f c c and d d. I low k Lemma. B B B r Proof. The rght sde dervatve of e wth respect to b s g (b = (b + r (b ( k= Ilow k k=+ Iup k Take any one and consder B =[c,d ]andb+ = [c +,d + ]. Suppose c >c +. Defne b = c and b + = c +. Snce b + s strctly to the left of the nterval B that mnmzes e, we have g (b + < 0. Snce b + s a mnmzer of e + we also have g + (b + 0. Thus we have g +(b + g (b + > 0; also, by ( we get 0 <g + (b + g (b + = I+(b low + I up + (b + whch s mpossble. In a smlar way, d >d + s also not possble. Ths proves the lemma. If the optmal b are all unque, then Lemma mples that the b satsfy the natural ordnal orderng. Even when one or more b s are non-unque, Lemma says that there exst choces for the b that obey If, n the prmal problem, we regularze the b also (.e., nclude the extra cost term b / then the b are guaranteed to be unque. Lemma stll holds n ths case.

6 New Approaches to Support Vector Ordnal Regresson the natural orderng. The fact that the order preservaton comes about automatcally s nterestng and non-trval, whch dffers from the PRank algorthm (Crammer & Snger, 00 where the order preservaton on the thresholds s easly brought n va ther update rule. It s also worth notng that Lemma holds even for an extended problem formulaton that allows the use of dfferent costs (dfferent C values for dfferent msclassfcatons (class k msclassfed as class can have a C k. In applcatons such as collaboratve flterng such a problem formulaton can be very approprate; for example, an A rated move that s msrated as C may need to be penalzed much more than f a B rated move s msrated as C. Shashua and Levn s formulaton and ts extenson gven n secton of ths paper do not precsely support such a dfferental cost structure. Ths s another good reason n support of the mplct problem formulaton of the current secton. 3.. Dual Problem Let α k 0, γ k 0, α k 0andγ k 0 be the Lagrangan multplers for the nequaltes n (9. Usng deas parallel to those n secton. we can show that the dual of (8 (9 s the followng maxmzaton problem that nvolves only the multplers α and α : max α,α r =k α k k, k, subect to n k α k = k= = ( k α k r ( k α k α k = =k = ( k α k + r α k = =k K(x k,x k + k, r n k k=+ = α k 0 α k C and k 0 α k C and k>. ( (3 The dual problem ( (3 s a convex quadratc programmng problem. The sze of the optmzaton problem s (r n where n = r k= nk s the total number of tranng samples. The dscrmnant functon value for a new nput vector x s f(x = w x = ( k α k r α k K(x k,x. k, = =k The predctve ordnal decson functon s gven by arg mn{ : f(x <b }. The deas for adaptng SMO to ( (3 are smlar to those n secton.3. The resultng suboptmzaton problem s analogous to the case of standard update n secton.3 where only one of the equalty constrants from (3 s nvolved. Full detals of the dervaton of the dual problem as well as the SMO algorthm have been skpped for lack of space. These detals are gven n our longer techncal report (Chu & Keerth, Numercal Experments We have mplemented the two SMO algorthms for the ordnal regresson formulatons wth explct constrants (EXC and mplct constrants (IMC, 3 along wth the algorthm of Shashua and Levn (003 for comparson purpose. The functon cachng technque and the double-loop scheme proposed by Keerth et al. (00 have been ncorporated n the mplementaton for effcency. We begn ths secton wth a smple dataset to llustrate the typcal behavor of the three algorthms, and then emprcally study the scalng propertes of our algorthms. Then we compare the generalzaton performance of our algorthms aganst standard support vector regresson on eght benchmark datasets for ordnal regresson. The followng Gaussan kernel was used n these experments: ( K(x, x = exp κ d ς= (x ς x ς (4 where x ς denotes the ς-th element of the nput vector x. The tolerance parameter τ was set to 0.00 for all the algorthms. We have utlzed two evaluaton metrcs whch quantfy the accuracy of predcted ordnal scales {ŷ,...,ŷ t } wth respect to true targets {y,...,y t }: amean absolute error s the average devaton of the predcton from the true target, t.e. t = ŷ y, n whch we treat the ordnal scales as consecutve ntegers; bmean zero-one error s smply the fracton of ncorrect predctons. 4.. Gradng Dataset The gradng dataset was used n chapter 4 of Johnson and Albert (999 as an example of the ordnal regresson problem. 4 There are 30 samples of students score. The sat-math score and grade n prerequste probablty course of these students are used as nput features, and ther fnal grades are taken as the targets. In our experments, the sx students wth fnal grade A or E were not used, and the feature assocated wth the grade n prerequste probablty course was treated as a contnuous varable though t had an ordnal scale. In Fgure 3 we present the soluton obtaned by the 3 The source code (wrtten n ANSI C of our mplementaton of the two algorthms can be found at chuwe/svor.htm. 4 The gradng dataset s avalable at

7 New Approaches to Support Vector Ordnal Regresson Grade n probablty course Shashua and Levn s formulaton wth explct constrants wth mplct constrants b =0.0 b = 0.5 (a (b (c 5 5 Grade n probablty course 4 3 b =b = 0.07 Grade n probablty course b 4 = 0.5 b = CPU tme n seconds 5 ordnal scales n the target 0 4 mplct constrants explct constrants support vector regresson slope.3 slope.8 0 slope.43 CPU tme n seconds ordnal scales n the target mplct constrants explct constrants support vector regresson slope.3 slope.33 slope Sat math score Sat math score Sat math score Fgure 3. The tranng results of the three algorthms usng a Gaussan kernel on the gradng dataset. The dscrmnant functon values are presented as contour graphs ndexed by the two thresholds. The crcles denote the students wth grade D, the dots denote grade C, and the squares denote grade B. three algorthms usng the Gaussan kernel (4 wth κ =0.5 and the regularzaton factor value of C =. In ths partcular settng, the soluton to Shashua and Levn (003 s formulaton has dsordered thresholds b <b as shown n Fgure 3 (left plot; the formulaton wth explct constrants corrects ths dsorder and yelds equal values for the two thresholds as shown n Fgure 3 (mddle plot. 4.. Scalng In ths experment, we emprcally studed how the two SMO algorthms scale wth respect to tranng data sze and the number of ordnal scales n the target. The Calforna Housng dataset was used n the scalng experments. 5 Twenty-eght tranng datasets wth szes rangng from 00 to 5,000 were generated by random selecton from the orgnal dataset. The contnuous target varable of the Calforna Housng data was dscretzed to ordnal scale by usng 5 or 0 equalfrequency bns. The standard support vector regresson (SVR was used as a baselne, n whch the ordnal targets were treated as contnuous values and ɛ = 0.. These datasets were traned by the two algorthms usng a Gaussan kernel wth κ = and a regularzaton factor value of C = 00. Fgure 4 gves plots of the computatonal costs of the three algorthms as functons of the problem sze, for the two cases of 5 and 0 target bns. Our algorthms scale well wth scalng exponents between.3 and.33, whle the scalng exponent of SVR s about.40 n ths case. Ths nearquadratc property n scalng comes from the sparseness property of SVMs,.e., non-support vectors affect the computatonal cost only mldly. The EXC and IMC algorthms cost more than the SVR approach due to the larger problem sze. For large szes, the cost of EXC s only about x tmes that of SVR. As expected, we also notced that the computatonal cost of IMC s dependent on r, the number of ordnal scales n the 5 The Calforna Housng dataset can be found at Tranng data sze Tranng data sze Fgure 4. Plots of CPU tme versus tranng data sze on log log scale, ndexed by the estmated slopes respectvely. We used the Gaussan kernel wth κ =andthe regularzaton factor value of C = 00 n the experment. target. The cost for 0 ranks s observed to be roughly 5 tmes that for 5 ranks, whereas the cost of EXC s nearly the same for the two cases. These observatons are consstent wth the sze of the optmzaton problems. The problem sze of IMC s (r n (whch s heavly nfluenced by r whle the problem sze of EXC s about n + r (whch largely depends on n only snce we usually have n r. Ths factor of effcency can be a key advantage for the EXC formulaton Benchmark datasets Next, we compared the generalzaton performance of the two approaches aganst the nave approach of usng standard support vector regresson (SVR and the method (SLA of Shashua and Levn (003. We collected eght benchmark datasets that were used for metrc regresson problems. 6 For each dataset, the target values were dscretzed nto ten ordnal quanttes usng equal-frequency bnnng. We randomly parttoned each dataset nto tranng/test splts as specfed n Table. The parttonng was repeated 0 tmes ndependently. The nput vectors were normalzed to zero mean and unt varance, coordnate-wse. The Gaussan kernel (4 was used for all the algorthms. 5-fold cross valdaton was used to determne the optmal values of model parameters (the Gaussan kernel parameter κ and the regularzaton factor C nvolved n the problem formulatons, and the test error was obtaned usng the optmal model parameters for each formulaton. The ntal search was done on a 7 7 coarse grd lnearly spaced n the regon {(log 0 C, log 0 κ 3 log 0 C 3, 3 log 0 κ 3}, followed by a fne search on a 9 9 unform grd lnearly spaced by 0. n the (log 0 C, log 0 κ space. The ordnal targets were treated as contnuous values n standard SVR, and the predctons for test cases were rounded to the nearest ordnal scale. The nsenstve zone parameter, ɛ of SVR was fxed at 0.. The test results of the four algorthms are recorded n Table. It s very clear that the generalzaton capabltes 6 These regresson datasets are avalable at ltorgo/regresson/datasets.html.

8 New Approaches to Support Vector Ordnal Regresson Table. Test results of the four algorthms usng a Gaussan kernel. The targets of these benchmark datasets were dscretzed by 0 equal-frequency bns. The results are the averages over 0 trals, along wth the standard devaton. d denotes the nput dmenson and tranng/test denotes the partton sze. We use bold face to ndcate the lowest average value among the results of the four algorthms. The symbols are used to ndcate the cases sgnfcantly worse than the wnnng entry; A p-value threshold of 0.0 n Wlcoxon rank sum test was used to decde ths. Partton Mean zero-one error Mean absolute error Dataset d tranng/test SVR SLA EXC IMC SVR SLA EXC IMC Pyrmdnes 7 50/ ± ± ± ± ± ± ± ±0.04 Machnecpu 6 50/ ± ± ± ± ±0.4.00± ± ±0.5 Boston 3 300/ ± ± ± ± ± ± ± ±0.049 Abalone 8 000/ ± ± ± ± ± ± ±0.0.36±0.03 Bank / ± ± ± ± ± ±0.0.5± ±0.0 Computer 4000/ ± ± ± ± ± ± ± ±0.008 Calforna / ± ± ± ± ± ± ± ±0.005 Census / ± ± ± ± ± ± ± ±0.007 of the three ordnal regresson algorthms are better than that of the approach of SVR. The performance of Shashua and Levn s method s smlar to our EXC approach, as expected, snce the two formulatons are pretty much the same. Our ordnal algorthms are comparable on the mean zero-one error, but the results also show the IMC algorthm yelds much more stable results on mean absolute error than the EXC algorthm. 7 From the vew of the formulatons, EXC only consders the extremely worst samples between successve ranks, whereas IMC takes all the samples nto account. Thus the outlers may affect the results of EXC sgnfcantly, whle the results of IMC are relatvely more stable n both valdaton and test. 5. Concluson In ths paper we proposed two new approaches to support vector ordnal regresson that determne r parallel dscrmnant hyperplanes for the r ranks by usng r thresholds. The ordnal nequalty constrants on the thresholds are mposed explctly n the frst approach and mplctly n the second one. The problem sze of the two approaches s lnear n the number of tranng samples. We also desgned SMO algorthms that scale only about quadratcally wth the problem sze. The results of numercal experments verfed that the generalzaton capabltes of these approaches are much better than the nave approach of applyng standard regresson. Acknowledgments A part of the work was carred out at IPAM of UCLA. WC was supported by the Natonal Insttutes of Health and ts Natonal Insttute of General Medcal Scences dvson 7 As ponted out by a revewer, ξ + ξ + n ( of EXC s an upper bound on the zero-one error of the -th example, whle, n (8 of IMC, k= ξ k + r k=+ ξ k s an upper bound on the absolute error. Note that, n all the examples we use consecutve ntegers to represent the ordnal scales. under Grant Number P0 GM6308. References Chu, W., & Keerth, S. S. (005. New approaches to support vector ordnal regresson (Techncal Report. Yahoo! Research Labs. Crammer, K., & Snger, Y. (00. Prankng wth rankng. Advances n Neural Informaton Processng Systems 4 (pp Cambrdge, MA: MIT Press. Frank, E., & Hall, M. (00. A smple approach to ordnal classfcaton. Proceedngs of the European Conference on Machne Learnng (pp Har-Peled, S., Roth, D., & Zmak, D. (00. Constrant classfcaton: A new approach to multclass classfcaton and rankng. Advances n Neural Informaton Processng Systems 5. Herbrch, R., Graepel, T., & Obermayer, K. (000. Large margn rank boundares for ordnal regresson. Advances n Large Margn Classfers (pp MIT Press. Johnson, V. E., & Albert, J. H. (999. Ordnal data modelng (statstcs for socal scence and publc polcy. Sprnger-Verlag. Keerth, S. S., Shevade, S. K., Bhattacharyya, C., & Murthy, K. R. K. (00. Improvements to Platt s SMO algorthm for SVM classfer desgn. Neural Computaton, 3, Kramer, S., Wdmer, G., Pfahrnger, B., & DeGroeve, M. (00. Predcton of ordnal classes usng regresson trees. Fundamenta Informatcae, 47, 3. Platt, J. C. (999. Fast tranng of support vector machnes usng sequental mnmal optmzaton. Advances n Kernel Methods - Support Vector Learnng (pp MIT Press. Schölkopf,B.,&Smola,A.J.(00. Learnng wth kernels. The MIT Press. Shashua, A., & Levn, A. (003. Rankng wth large margn prncple: two approaches. Advances n Neural Informaton Processng Systems 5 (pp Vapnk, V. N. (995. The nature of statstcal learnng theory. New York: Sprnger-Verlag.

Support Vector Machines

Support Vector Machines Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

Financial market forecasting using a two-step kernel learning method for the support vector regression

Financial market forecasting using a two-step kernel learning method for the support vector regression Ann Oper Res (2010) 174: 103 120 DOI 10.1007/s10479-008-0357-7 Fnancal market forecastng usng a two-step kernel learnng method for the support vector regresson L Wang J Zhu Publshed onlne: 28 May 2008

More information

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPP-ATBD-ClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

SVM Tutorial: Classification, Regression, and Ranking

SVM Tutorial: Classification, Regression, and Ranking SVM Tutoral: Classfcaton, Regresson, and Rankng Hwanjo Yu and Sungchul Km 1 Introducton Support Vector Machnes(SVMs) have been extensvely researched n the data mnng and machne learnng communtes for the

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

Lecture 2: Single Layer Perceptrons Kevin Swingler

Lecture 2: Single Layer Perceptrons Kevin Swingler Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCulloch-Ptts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses

More information

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence 1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh

More information

1 Example 1: Axis-aligned rectangles

1 Example 1: Axis-aligned rectangles COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton

More information

Learning to Classify Ordinal Data: The Data Replication Method

Learning to Classify Ordinal Data: The Data Replication Method Journal of Machne Learnng Research 8 (7) 393-49 Submtted /6; Revsed 9/6; Publshed 7/7 Learnng to Classfy Ordnal Data: The Data Replcaton Method Jame S. Cardoso INESC Porto, Faculdade de Engenhara, Unversdade

More information

Project Networks With Mixed-Time Constraints

Project Networks With Mixed-Time Constraints Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa

More information

L10: Linear discriminants analysis

L10: Linear discriminants analysis L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss

More information

An MILP model for planning of batch plants operating in a campaign-mode

An MILP model for planning of batch plants operating in a campaign-mode An MILP model for plannng of batch plants operatng n a campagn-mode Yanna Fumero Insttuto de Desarrollo y Dseño CONICET UTN yfumero@santafe-concet.gov.ar Gabrela Corsano Insttuto de Desarrollo y Dseño

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

Extending Probabilistic Dynamic Epistemic Logic

Extending Probabilistic Dynamic Epistemic Logic Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set

More information

Performance Analysis and Coding Strategy of ECOC SVMs

Performance Analysis and Coding Strategy of ECOC SVMs Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.67-76 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School

More information

Support Vector Machine Model for Currency Crisis Discrimination. Arindam Chaudhuri 1. Abstract

Support Vector Machine Model for Currency Crisis Discrimination. Arindam Chaudhuri 1. Abstract Support Vector Machne Model for Currency Crss Dscrmnaton Arndam Chaudhur Abstract Support Vector Machne (SVM) s powerful classfcaton technque based on the dea of structural rsk mnmzaton. Use of kernel

More information

Single and multiple stage classifiers implementing logistic discrimination

Single and multiple stage classifiers implementing logistic discrimination Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul - PUCRS Av. Ipranga,

More information

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there

More information

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts Power-of-wo Polces for Sngle- Warehouse Mult-Retaler Inventory Systems wth Order Frequency Dscounts José A. Ventura Pennsylvana State Unversty (USA) Yale. Herer echnon Israel Insttute of echnology (Israel)

More information

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of

More information

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis The Development of Web Log Mnng Based on Improve-K-Means Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.

More information

Calculation of Sampling Weights

Calculation of Sampling Weights Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a two-stage stratfed cluster desgn. 1 The frst stage conssted of a sample

More information

greatest common divisor

greatest common divisor 4. GCD 1 The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no

More information

Can Auto Liability Insurance Purchases Signal Risk Attitude?

Can Auto Liability Insurance Purchases Signal Risk Attitude? Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159-164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? Chu-Shu L Department of Internatonal Busness, Asa Unversty, Tawan Sheng-Chang

More information

An Interest-Oriented Network Evolution Mechanism for Online Communities

An Interest-Oriented Network Evolution Mechanism for Online Communities An Interest-Orented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne

More information

Least 1-Norm SVMs: a New SVM Variant between Standard and LS-SVMs

Least 1-Norm SVMs: a New SVM Variant between Standard and LS-SVMs ESANN proceedngs, European Smposum on Artfcal Neural Networks - Computatonal Intellgence and Machne Learnng. Bruges (Belgum), 8-3 Aprl, d-sde publ., ISBN -9337--. Least -Norm SVMs: a New SVM Varant between

More information

8 Algorithm for Binary Searching in Trees

8 Algorithm for Binary Searching in Trees 8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

More information

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and POLYSA: A Polynomal Algorthm for Non-bnary Constrant Satsfacton Problems wth and Mguel A. Saldo, Federco Barber Dpto. Sstemas Informátcos y Computacón Unversdad Poltécnca de Valenca, Camno de Vera s/n

More information

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression Novel Methodology of Workng Captal Management for Large Publc Constructons by Usng Fuzzy S-curve Regresson Cheng-Wu Chen, Morrs H. L. Wang and Tng-Ya Hseh Department of Cvl Engneerng, Natonal Central Unversty,

More information

The Analysis of Outliers in Statistical Data

The Analysis of Outliers in Statistical Data THALES Project No. xxxx The Analyss of Outlers n Statstcal Data Research Team Chrysses Caron, Assocate Professor (P.I.) Vaslk Karot, Doctoral canddate Polychrons Economou, Chrstna Perrakou, Postgraduate

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION. Michael E. Kuhl Radhamés A. Tolentino-Peña

A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION. Michael E. Kuhl Radhamés A. Tolentino-Peña Proceedngs of the 2008 Wnter Smulaton Conference S. J. Mason, R. R. Hll, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler eds. A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION

More information

v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

v a 1 b 1 i, a 2 b 2 i,..., a n b n i. SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

More information

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

More information

Fisher Markets and Convex Programs

Fisher Markets and Convex Programs Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

More information

A study on the ability of Support Vector Regression and Neural Networks to Forecast Basic Time Series Patterns

A study on the ability of Support Vector Regression and Neural Networks to Forecast Basic Time Series Patterns A study on the ablty of Support Vector Regresson and Neural Networks to Forecast Basc Tme Seres Patterns Sven F. Crone, Jose Guajardo 2, and Rchard Weber 2 Lancaster Unversty, Department of Management

More information

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

More information

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

More information

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that

More information

On the Optimal Control of a Cascade of Hydro-Electric Power Stations

On the Optimal Control of a Cascade of Hydro-Electric Power Stations On the Optmal Control of a Cascade of Hydro-Electrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;

More information

Realistic Image Synthesis

Realistic Image Synthesis Realstc Image Synthess - Combned Samplng and Path Tracng - Phlpp Slusallek Karol Myszkowsk Vncent Pegoraro Overvew: Today Combned Samplng (Multple Importance Samplng) Renderng and Measurng Equaton Random

More information

Loop Parallelization

Loop Parallelization - - Loop Parallelzaton C-52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I-,J]+B[I-,J-] ED FOR ED FOR analyze

More information

Logistic Regression. Steve Kroon

Logistic Regression. Steve Kroon Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro

More information

DEFINING %COMPLETE IN MICROSOFT PROJECT

DEFINING %COMPLETE IN MICROSOFT PROJECT CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMI-SP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,

More information

Brigid Mullany, Ph.D University of North Carolina, Charlotte

Brigid Mullany, Ph.D University of North Carolina, Charlotte Evaluaton And Comparson Of The Dfferent Standards Used To Defne The Postonal Accuracy And Repeatablty Of Numercally Controlled Machnng Center Axes Brgd Mullany, Ph.D Unversty of North Carolna, Charlotte

More information

CHAPTER 14 MORE ABOUT REGRESSION

CHAPTER 14 MORE ABOUT REGRESSION CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp

More information

A Probabilistic Theory of Coherence

A Probabilistic Theory of Coherence A Probablstc Theory of Coherence BRANDEN FITELSON. The Coherence Measure C Let E be a set of n propostons E,..., E n. We seek a probablstc measure C(E) of the degree of coherence of E. Intutvely, we want

More information

The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method. Introduction. 0/1 Knapsack Problem The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

More information

Bayesian Network Based Causal Relationship Identification and Funding Success Prediction in P2P Lending

Bayesian Network Based Causal Relationship Identification and Funding Success Prediction in P2P Lending Proceedngs of 2012 4th Internatonal Conference on Machne Learnng and Computng IPCSIT vol. 25 (2012) (2012) IACSIT Press, Sngapore Bayesan Network Based Causal Relatonshp Identfcaton and Fundng Success

More information

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60 BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

More information

ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING

ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING Matthew J. Lberatore, Department of Management and Operatons, Vllanova Unversty, Vllanova, PA 19085, 610-519-4390,

More information

Joint Scheduling of Processing and Shuffle Phases in MapReduce Systems

Joint Scheduling of Processing and Shuffle Phases in MapReduce Systems Jont Schedulng of Processng and Shuffle Phases n MapReduce Systems Fangfe Chen, Mural Kodalam, T. V. Lakshman Department of Computer Scence and Engneerng, The Penn State Unversty Bell Laboratores, Alcatel-Lucent

More information

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001. Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.

More information

Multiclass sparse logistic regression for classification of multiple cancer types using gene expression data

Multiclass sparse logistic regression for classification of multiple cancer types using gene expression data Computatonal Statstcs & Data Analyss 51 (26) 1643 1655 www.elsever.com/locate/csda Multclass sparse logstc regresson for classfcaton of multple cancer types usng gene expresson data Yongda Km a,, Sunghoon

More information

"Research Note" APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES *

Research Note APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES * Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789-794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC

More information

Support vector domain description

Support vector domain description Pattern Recognton Letters 20 (1999) 1191±1199 www.elsever.nl/locate/patrec Support vector doman descrpton Davd M.J. Tax *,1, Robert P.W. Dun Pattern Recognton Group, Faculty of Appled Scence, Delft Unversty

More information

Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications

Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications CMSC828G Prncples of Data Mnng Lecture #9 Today s Readng: HMS, chapter 9 Today s Lecture: Descrptve Modelng Clusterng Algorthms Descrptve Models model presents the man features of the data, a global summary

More information

We are now ready to answer the question: What are the possible cardinalities for finite fields?

We are now ready to answer the question: What are the possible cardinalities for finite fields? Chapter 3 Fnte felds We have seen, n the prevous chapters, some examples of fnte felds. For example, the resdue class rng Z/pZ (when p s a prme) forms a feld wth p elements whch may be dentfed wth the

More information

Credit Limit Optimization (CLO) for Credit Cards

Credit Limit Optimization (CLO) for Credit Cards Credt Lmt Optmzaton (CLO) for Credt Cards Vay S. Desa CSCC IX, Ednburgh September 8, 2005 Copyrght 2003, SAS Insttute Inc. All rghts reserved. SAS Propretary Agenda Background Tradtonal approaches to credt

More information

OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL. Thomas S. Ferguson and C. Zachary Gilstein UCLA and Bell Communications May 1985, revised 2004

OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL. Thomas S. Ferguson and C. Zachary Gilstein UCLA and Bell Communications May 1985, revised 2004 OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL Thomas S. Ferguson and C. Zachary Glsten UCLA and Bell Communcatons May 985, revsed 2004 Abstract. Optmal nvestment polces for maxmzng the expected

More information

Sketching Sampled Data Streams

Sketching Sampled Data Streams Sketchng Sampled Data Streams Florn Rusu, Aln Dobra CISE Department Unversty of Florda Ganesvlle, FL, USA frusu@cse.ufl.edu adobra@cse.ufl.edu Abstract Samplng s used as a unversal method to reduce the

More information

The Geometry of Online Packing Linear Programs

The Geometry of Online Packing Linear Programs The Geometry of Onlne Packng Lnear Programs Marco Molnaro R. Rav Abstract We consder packng lnear programs wth m rows where all constrant coeffcents are n the unt nterval. In the onlne model, we know the

More information

IMPACT ANALYSIS OF A CELLULAR PHONE

IMPACT ANALYSIS OF A CELLULAR PHONE 4 th ASA & μeta Internatonal Conference IMPACT AALYSIS OF A CELLULAR PHOE We Lu, 2 Hongy L Bejng FEAonlne Engneerng Co.,Ltd. Bejng, Chna ABSTRACT Drop test smulaton plays an mportant role n nvestgatng

More information

Dynamic Resource Allocation and Power Management in Virtualized Data Centers

Dynamic Resource Allocation and Power Management in Virtualized Data Centers Dynamc Resource Allocaton and Power Management n Vrtualzed Data Centers Rahul Urgaonkar, Ulas C. Kozat, Ken Igarash, Mchael J. Neely urgaonka@usc.edu, {kozat, garash}@docomolabs-usa.com, mjneely@usc.edu

More information

The Application of Fractional Brownian Motion in Option Pricing

The Application of Fractional Brownian Motion in Option Pricing Vol. 0, No. (05), pp. 73-8 http://dx.do.org/0.457/jmue.05.0..6 The Applcaton of Fractonal Brownan Moton n Opton Prcng Qng-xn Zhou School of Basc Scence,arbn Unversty of Commerce,arbn zhouqngxn98@6.com

More information

NON-CONSTANT SUM RED-AND-BLACK GAMES WITH BET-DEPENDENT WIN PROBABILITY FUNCTION LAURA PONTIGGIA, University of the Sciences in Philadelphia

NON-CONSTANT SUM RED-AND-BLACK GAMES WITH BET-DEPENDENT WIN PROBABILITY FUNCTION LAURA PONTIGGIA, University of the Sciences in Philadelphia To appear n Journal o Appled Probablty June 2007 O-COSTAT SUM RED-AD-BLACK GAMES WITH BET-DEPEDET WI PROBABILITY FUCTIO LAURA POTIGGIA, Unversty o the Scences n Phladelpha Abstract In ths paper we nvestgate

More information

2008/8. An integrated model for warehouse and inventory planning. Géraldine Strack and Yves Pochet

2008/8. An integrated model for warehouse and inventory planning. Géraldine Strack and Yves Pochet 2008/8 An ntegrated model for warehouse and nventory plannng Géraldne Strack and Yves Pochet CORE Voe du Roman Pays 34 B-1348 Louvan-la-Neuve, Belgum. Tel (32 10) 47 43 04 Fax (32 10) 47 43 01 E-mal: corestat-lbrary@uclouvan.be

More information

Course outline. Financial Time Series Analysis. Overview. Data analysis. Predictive signal. Trading strategy

Course outline. Financial Time Series Analysis. Overview. Data analysis. Predictive signal. Trading strategy Fnancal Tme Seres Analyss Patrck McSharry patrck@mcsharry.net www.mcsharry.net Trnty Term 2014 Mathematcal Insttute Unversty of Oxford Course outlne 1. Data analyss, probablty, correlatons, vsualsaton

More information

+ + + - - This circuit than can be reduced to a planar circuit

+ + + - - This circuit than can be reduced to a planar circuit MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to

More information

Learning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing

Learning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing Internatonal Journal of Machne Learnng and Computng, Vol. 4, No. 3, June 04 Learnng from Large Dstrbuted Data: A Scalng Down Samplng Scheme for Effcent Data Processng Che Ngufor and Janusz Wojtusak part

More information

A Lyapunov Optimization Approach to Repeated Stochastic Games

A Lyapunov Optimization Approach to Repeated Stochastic Games PROC. ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING, OCT. 2013 1 A Lyapunov Optmzaton Approach to Repeated Stochastc Games Mchael J. Neely Unversty of Southern Calforna http://www-bcf.usc.edu/

More information

1. Math 210 Finite Mathematics

1. Math 210 Finite Mathematics 1. ath 210 Fnte athematcs Chapter 5.2 and 5.3 Annutes ortgages Amortzaton Professor Rchard Blecksmth Dept. of athematcal Scences Northern Illnos Unversty ath 210 Webste: http://math.nu.edu/courses/math210

More information

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek HE DISRIBUION OF LOAN PORFOLIO VALUE * Oldrch Alfons Vascek he amount of captal necessary to support a portfolo of debt securtes depends on the probablty dstrbuton of the portfolo loss. Consder a portfolo

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

Research Article Integrated Model of Multiple Kernel Learning and Differential Evolution for EUR/USD Trading

Research Article Integrated Model of Multiple Kernel Learning and Differential Evolution for EUR/USD Trading Hndaw Publshng Corporaton e Scentfc World Journal, Artcle ID 914641, 12 pages http://dx.do.org/10.1155/2014/914641 Research Artcle Integrated Model of Multple Kernel Learnng and Dfferental Evoluton for

More information

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM BARRIOT Jean-Perre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jean-perre.barrot@cnes.fr 1/Introducton The

More information

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network 700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School

More information

A machine vision approach for detecting and inspecting circular parts

A machine vision approach for detecting and inspecting circular parts A machne vson approach for detectng and nspectng crcular parts Du-Mng Tsa Machne Vson Lab. Department of Industral Engneerng and Management Yuan-Ze Unversty, Chung-L, Tawan, R.O.C. E-mal: edmtsa@saturn.yzu.edu.tw

More information

An efficient constraint handling methodology for multi-objective evolutionary algorithms

An efficient constraint handling methodology for multi-objective evolutionary algorithms Rev. Fac. Ing. Unv. Antoqua N. 49. pp. 141-150. Septembre, 009 An effcent constrant handlng methodology for mult-objectve evolutonary algorthms Una metodología efcente para manejo de restrccones en algortmos

More information

Alternate Approximation of Concave Cost Functions for

Alternate Approximation of Concave Cost Functions for Alternate Approxmaton of Concave Cost Functons for Process Desgn and Supply Chan Optmzaton Problems Dego C. Cafaro * and Ignaco E. Grossmann INTEC (UNL CONICET), Güemes 3450, 3000 Santa Fe, ARGENTINA Department

More information

Thursday, December 10, 2009 Noon - 1:50 pm Faraday 143

Thursday, December 10, 2009 Noon - 1:50 pm Faraday 143 1. ath 210 Fnte athematcs Chapter 5.2 and 4.3 Annutes ortgages Amortzaton Professor Rchard Blecksmth Dept. of athematcal Scences Northern Illnos Unversty ath 210 Webste: http://math.nu.edu/courses/math210

More information

Joe Pimbley, unpublished, 2005. Yield Curve Calculations

Joe Pimbley, unpublished, 2005. Yield Curve Calculations Joe Pmbley, unpublshed, 005. Yeld Curve Calculatons Background: Everythng s dscount factors Yeld curve calculatons nclude valuaton of forward rate agreements (FRAs), swaps, nterest rate optons, and forward

More information

) of the Cell class is created containing information about events associated with the cell. Events are added to the Cell instance

) of the Cell class is created containing information about events associated with the cell. Events are added to the Cell instance Calbraton Method Instances of the Cell class (one nstance for each FMS cell) contan ADC raw data and methods assocated wth each partcular FMS cell. The calbraton method ncludes event selecton (Class Cell

More information

Lecture 5,6 Linear Methods for Classification. Summary

Lecture 5,6 Linear Methods for Classification. Summary Lecture 5,6 Lnear Methods for Classfcaton Rce ELEC 697 Farnaz Koushanfar Fall 2006 Summary Bayes Classfers Lnear Classfers Lnear regresson of an ndcator matrx Lnear dscrmnant analyss (LDA) Logstc regresson

More information

An Efficient and Simplified Model for Forecasting using SRM

An Efficient and Simplified Model for Forecasting using SRM HAFIZ MUHAMMAD SHAHZAD ASIF*, MUHAMMAD FAISAL HAYAT*, AND TAUQIR AHMAD* RECEIVED ON 15.04.013 ACCEPTED ON 09.01.014 ABSTRACT Learnng form contnuous fnancal systems play a vtal role n enterprse operatons.

More information

Heuristic Static Load-Balancing Algorithm Applied to CESM

Heuristic Static Load-Balancing Algorithm Applied to CESM Heurstc Statc Load-Balancng Algorthm Appled to CESM 1 Yur Alexeev, 1 Sher Mckelson, 1 Sven Leyffer, 1 Robert Jacob, 2 Anthony Crag 1 Argonne Natonal Laboratory, 9700 S. Cass Avenue, Argonne, IL 60439,

More information

Vision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION

Vision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION Vson Mouse Saurabh Sarkar a* a Unversty of Cncnnat, Cncnnat, USA ABSTRACT The report dscusses a vson based approach towards trackng of eyes and fngers. The report descrbes the process of locatng the possble

More information

Implementation of Deutsch's Algorithm Using Mathcad

Implementation of Deutsch's Algorithm Using Mathcad Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages - n "Machnes, Logc and Quantum Physcs"

More information

Lecture 3: Force of Interest, Real Interest Rate, Annuity

Lecture 3: Force of Interest, Real Interest Rate, Annuity Lecture 3: Force of Interest, Real Interest Rate, Annuty Goals: Study contnuous compoundng and force of nterest Dscuss real nterest rate Learn annuty-mmedate, and ts present value Study annuty-due, and

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

Learning from Multiple Outlooks

Learning from Multiple Outlooks Learnng from Multple Outlooks Maayan Harel Department of Electrcal Engneerng, Technon, Hafa, Israel She Mannor Department of Electrcal Engneerng, Technon, Hafa, Israel maayanga@tx.technon.ac.l she@ee.technon.ac.l

More information