Support Vector Machines

Size: px
Start display at page:

Download "Support Vector Machines"

Transcription

1 Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes. 1 Prelmnares Our task s to predct whether a test sample belongs to one of two classes. We receve tranng examples of the form: {x, y }, = 1,..., n and x R d, y { 1, +1}. We call {x } the co-varates or nput vectors and {y } the response varables or labels. We consder a very smple example where the data are n fact lnearly separable:.e. I can draw a straght lne f(x) = w T x b such that all cases wth y = 1 fall on one sde and have f(x ) < 0 and cases wth y = +1 fall on the other and have f(x ) > 0. Gven that we have acheved that, we could classfy new test cases accordng to the rule y test = sgn(x test ). However, typcally there are nfntely many such hyper-planes obtaned by small perturbatons of a gven soluton. How do we choose between all these hyper-planes whch the solve the separaton problem for our tranng data, but may have dfferent performance on the newly arrvng test cases. For nstance, we could choose to put the lne very close to members of one partcular class, say y = 1. Intutvely, when test cases arrve we wll not make many mstakes on cases that should be classfed wth y = +1, but we wll make very easly mstakes on the cases wth y = 1 (for nstance, magne that a new batch of test cases arrves whch are small perturbatons of the tranng data). A sensble thng thus seems to choose the separaton lne as far away from both y = 1 and y = +1 tranng cases as we can,.e. rght n the mddle. Geometrcally, the vector w s drected orthogonal to the lne defned by w T x = b. Ths can be understood as follows. Frst take b = 0. Now t s clear that all vectors, x, wth vanshng nner product wth w satsfy ths equaton,.e. all vectors orthogonal to w satsfy ths equaton. Now translate the hyperplane away from the orgn over a vector a. The equaton for the plane now becomes: (x a) T w = 0,.e. we fnd that for the offset b = a T w, whch s the projecton of a onto to the vector w. Wthout loss of generalty we may thus choose a perpendcular to the plane, n whch case the length a = b / w represents the shortest, orthogonal dstance between the orgn and the hyperplane. We now defne 2 more hyperplanes parallel to the separatng hyperplane. They represent that planes that cut through the closest tranng examples on ether sde. We wll call them

2 support hyper-planes n the followng, because the data-vectors they contan support the plane. We defne the dstance between the these hyperplanes and the separatng hyperplane to be d + and d respectvely. The margn, γ, s defned to be d + + d. Our goal s now to fnd a the separatng hyperplane so that the margn s largest, whle the separatng hyperplane s equdstant from both. We can wrte the followng equatons for the support hyperplanes: w T x = b + δ (1) w T x = b δ (2) We now note that we have over-parameterzed the problem: f we scale w, b and δ by a constant factor α, the equatons for x are stll satsfed. To remove ths ambguty we wll requre that δ = 1, ths sets the scale of the problem,.e. f we measure dstance n mllmeters or meters. We can now also compute the values for d + = ( b+1 b )/ w = 1/ w (ths s only true f b / ( 1, 0) snce the orgn doesn t fall n between the hyperplanes n that case. If b ( 1, 0) you should use d + = ( b b )/ w = 1/ w ). Hence the margn s equal to twce that value: γ = 2/ w. Wth the above defnton of the support planes we can wrte down the followng constrant that any soluton must satsfy, w T x b 1 y = 1 (3) w T x b +1 y = +1 (4) or n one equaton, y (w T x b) 1 0 (5) We now formulate the prmal problem of the SVM: 1 mnmze 2 w 2 subject to y (w T x b) 1 0 (6) Thus, we maxmze the margn, subject to the constrants that all tranng cases fall on ether sde of the support hyper-planes. The data-cases that le on the hyperplane are called support vectors, snce they support the hyper-planes and hence determne the soluton to the problem. The prmal problem can be solved by a quadratc program. However, t s not ready to be kernelsed, because ts dependence s not only on nner products between data-vectors. Hence, we transform to the dual formulaton by frst wrtng the problem usng a Lagrangan, L(w, b, α) = 1 N [ 2 w 2 α y (w T x b) 1 ] (7) =1 The soluton that mnmzes the prmal problem subject to the constrants s gven by mn w max α L(w, α),.e. a saddle pont problem. When the orgnal objectve-functon s convex, (and only then), we can nterchange the mnmzaton and maxmzaton. Dong that, we fnd that we can fnd the condton on w that must hold at the saddle pont we are solvng for. Ths s done by takng dervatves wrt w and b and solvng, w α y x = 0 w = α y x (8) α y = 0 (9)

3 Insertng ths back nto the Lagrangan we obtan what s known as the dual problem, N maxmze L D = α 1 α α j y y j x T x j 2 =1 j subject to α y = 0 (10) α 0 (11) The dual formulaton of the problem s also a quadratc program, but note that the number of varables, α n ths problem s equal to the number of data-cases, N. The crucal pont s however, that ths problem only depends on x through the nner product x T x j. Ths s readly kernelsed through the substtuton x T x j k(x, x j ). Ths s a recurrent theme: the dual problem lends tself to kernelsaton, whle the prmal problem dd not. The theory of dualty guarantees that for convex problems, the dual problem wll be concave, and moreover, that the unque soluton of the prmal problem corresponds tot the unque soluton of the dual problem. In fact, we have: L P (w ) = L D (α ),.e. the dualty-gap s zero. Next we turn to the condtons that must necessarly hold at the saddle pont and thus the soluton of the problem. These are called the KKT condtons (whch stands for Karush- Kuhn-Tucker). These condtons are necessary n general, and suffcent for convex optmzaton problems. They can be derved from the prmal problem by settng the dervatves wrt to w to zero. Also, the constrants themselves are part of these condtons and we need that for nequalty constrants the Lagrange multplers are non-negatve. Fnally, an mportant constrant called complementary slackness needs to be satsfed, w L P = 0 w α y x = 0 (12) b L P = 0 α y = 0 (13) constrant - 1 y (w T x b) 1 0 (14) multpler condton α 0 (15) complementary slackness α [ y (w T x b) 1 ] = 0 (16) It s the last equaton whch may be somewhat surprsng. It states that ether the nequalty constrant s satsfed, but not saturated: y (w T x b) 1 > 0 n whch case α for that data-case must be zero, or the nequalty constrant s saturated y (w T x b) 1 = 0, n whch case α can be any value α 0. Inequalty constrants whch are saturated are sad to be actve, whle unsaturated constrants are nactve. One could magne the process of searchng for a soluton as a ball whch runs down the prmary objectve functon usng gradent descent. At some pont, t wll ht a wall whch s the constrant and although the dervatve s stll pontng partally towards the wall, the constrants prohbts the ball to go on. Ths s an actve constrant because the ball s glued to that wall. When a fnal soluton s reached, we could remove some constrants, wthout changng the soluton, these are nactve constrants. One could thnk of the term w L P as the force actng on the ball. We see from the frst equaton above that only the forces wth α 0 exsert a force on the ball that balances wth the force from the curved quadratc surface w. The tranng cases wth α > 0, representng actve constrants on the poston of the support hyperplane are called support vectors. These are the vectors that are stuated n the support hyperplane and they determne the soluton. Typcally, there are only few of them, whch people call a sparse soluton (most α s vansh).

4 What we are really nterested n s the functon f( ) whch can be used to classfy future test cases, f(x) = w T x b = α y x T x b (17) As an applcaton of the KKT condtons we derve a soluton for b by usng the complementary slackness condton, b = j α j y j x T j x y a support vector (18) where we used y 2 = 1. So, usng any support vector one can determne b, but for numercal stablty t s better to average over all of them (although they should obvously be consstent). The most mportant concluson s agan that ths functon f( ) can thus be expressed solely n terms of nner products x T x whch we can replace wth kernel matrces k(x, x j ) to move to hgh dmensonal non-lnear spaces. Moreover, snce α s typcally very sparse, we don t need to evaluate many kernel entres n order to predct the class of the new nput x. 2 The Non-Separable case Obvously, not all datasets are lnearly separable, and so we need to change the formalsm to account for that. Clearly, the problem les n the constrants, whch cannot always be satsfed. So, let s relax those constrants by ntroducng slack varables, ξ, w T x b 1 + ξ y = 1 (19) w T x b +1 ξ y = +1 (20) ξ 0 (21) The varables, ξ allow for volatons of the constrant. We should penalze the objectve functon for these volatons, otherwse the above constrants become vod (smply always pck ξ very large). Penalty functons of the form C( ξ ) k wll lead to convex optmzaton problems for postve ntegers k. For k = 1, 2 t s stll a quadratc program (QP). In the followng we wll choose k = 1. C controls the tradeoff between the penalty and margn. To be on the wrong sde of the separatng hyperplane, a data-case would need ξ > 1. Hence, the sum ξ could be nterpreted as measure of how bad the volatons are and s an upper bound on the number of volatons. The new prmal problem thus becomes, mnmze L P = 1 2 w 2 + C ξ leadng to the Lagrangan, subject to y (w T x b) 1 + ξ 0 (22) ξ 0 (23) L(w, b, ξ, α, µ) = 1 2 w 2 +C N [ ξ α y (w T ] N x b) 1 + ξ µ ξ (24) =1 =1

5 from whch we derve the KKT condtons, 1. w L P = 0 w α y x = 0 (25) 2. b L P = 0 α y = 0 (26) 3. ξ L P = 0 C α µ = 0 (27) 4.constrant-1 y (w T x b) 1 + ξ 0 (28) 5.constrant-2 ξ 0 (29) 6.multpler condton-1 α 0 (30) 7.multpler condton-2 µ 0 (31) 8.complementary slackness-1 [ α y (w T ] x b) 1 + ξ = 0 (32) 9.complementary slackness-1 µ ξ = 0 (33) (34) From here we can deduce the followng facts. If we assume that ξ > 0, then µ = 0 (9), hence α = C (1) and thus ξ = 1 y (x T w b) (8). Also, when ξ = 0 we have µ > 0 (9) and hence α < C. If n addton to ξ = 0 we also have that y (w T x b) 1 = 0, then α > 0 (8). Otherwse, f y (w T x b) 1 > 0 then α = 0. In summary, as before for ponts not on the support plane and on the correct sde we have ξ = α = 0 (all constrants nactve). On the support plane, we stll have ξ = 0, but now α > 0. Fnally, for data-cases on the wrong sde of the support hyperplane the α max-out to α = C and the ξ balance the volaton of the constrant such that y (w T x b) 1 + ξ = 0. Geometrcally, we can calculate the gap between support hyperplane and the volatng datacase to be ξ / w. Ths can be seen because the plane defned by y (w T x b) 1+ξ = 0 s parallel to the support plane at a dstance 1 + y b ξ / w from the orgn. Snce the support plane s at a dstance 1 + y b / w the result follows. Fnally, we need to convert to the dual problem to solve t effcently and to kernelse t. Agan, we use the KKT equatons to get rd of w, b and ξ, N maxmze L D = α 1 α α j y y j x T x j 2 =1 j subject to α y = 0 (35) 0 α C (36) Surprsngly, ths s almost the same QP s before, but wth an extra constrant on the multplers α whch now lve n a box. Ths constrant s derved from the fact that α = C µ and µ 0. We also note that t only depends on nner products x T x j whch are ready to be kernelsed.

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2) MATH 16T Exam 1 : Part I (In-Class) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total

More information

v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

v a 1 b 1 i, a 2 b 2 i,..., a n b n i. SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

More information

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

More information

Fisher Markets and Convex Programs

Fisher Markets and Convex Programs Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

More information

Lecture 2: Single Layer Perceptrons Kevin Swingler

Lecture 2: Single Layer Perceptrons Kevin Swingler Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCulloch-Ptts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses

More information

New Approaches to Support Vector Ordinal Regression

New Approaches to Support Vector Ordinal Regression New Approaches to Support Vector Ordnal Regresson We Chu chuwe@gatsby.ucl.ac.uk Gatsby Computatonal Neuroscence Unt, Unversty College London, London, WCN 3AR, UK S. Sathya Keerth selvarak@yahoo-nc.com

More information

Least Squares Fitting of Data

Least Squares Fitting of Data Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2016. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng

More information

1 Example 1: Axis-aligned rectangles

1 Example 1: Axis-aligned rectangles COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

SVM Tutorial: Classification, Regression, and Ranking

SVM Tutorial: Classification, Regression, and Ranking SVM Tutoral: Classfcaton, Regresson, and Rankng Hwanjo Yu and Sungchul Km 1 Introducton Support Vector Machnes(SVMs) have been extensvely researched n the data mnng and machne learnng communtes for the

More information

21 Vectors: The Cross Product & Torque

21 Vectors: The Cross Product & Torque 21 Vectors: The Cross Product & Torque Do not use our left hand when applng ether the rght-hand rule for the cross product of two vectors dscussed n ths chapter or the rght-hand rule for somethng curl

More information

Production. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem.

Production. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem. Producer Theory Producton ASSUMPTION 2.1 Propertes of the Producton Set The producton set Y satsfes the followng propertes 1. Y s non-empty If Y s empty, we have nothng to talk about 2. Y s closed A set

More information

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.

More information

Support Vector Machine Model for Currency Crisis Discrimination. Arindam Chaudhuri 1. Abstract

Support Vector Machine Model for Currency Crisis Discrimination. Arindam Chaudhuri 1. Abstract Support Vector Machne Model for Currency Crss Dscrmnaton Arndam Chaudhur Abstract Support Vector Machne (SVM) s powerful classfcaton technque based on the dea of structural rsk mnmzaton. Use of kernel

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

Logistic Regression. Steve Kroon

Logistic Regression. Steve Kroon Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

A Probabilistic Theory of Coherence

A Probabilistic Theory of Coherence A Probablstc Theory of Coherence BRANDEN FITELSON. The Coherence Measure C Let E be a set of n propostons E,..., E n. We seek a probablstc measure C(E) of the degree of coherence of E. Intutvely, we want

More information

Loop Parallelization

Loop Parallelization - - Loop Parallelzaton C-52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I-,J]+B[I-,J-] ED FOR ED FOR analyze

More information

L10: Linear discriminants analysis

L10: Linear discriminants analysis L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss

More information

Chapter 4 ECONOMIC DISPATCH AND UNIT COMMITMENT

Chapter 4 ECONOMIC DISPATCH AND UNIT COMMITMENT Chapter 4 ECOOMIC DISATCH AD UIT COMMITMET ITRODUCTIO A power system has several power plants. Each power plant has several generatng unts. At any pont of tme, the total load n the system s met by the

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

Generalizing the degree sequence problem

Generalizing the degree sequence problem Mddlebury College March 2009 Arzona State Unversty Dscrete Mathematcs Semnar The degree sequence problem Problem: Gven an nteger sequence d = (d 1,...,d n ) determne f there exsts a graph G wth d as ts

More information

where the coordinates are related to those in the old frame as follows.

where the coordinates are related to those in the old frame as follows. Chapter 2 - Cartesan Vectors and Tensors: Ther Algebra Defnton of a vector Examples of vectors Scalar multplcaton Addton of vectors coplanar vectors Unt vectors A bass of non-coplanar vectors Scalar product

More information

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching) Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton

More information

The Mathematical Derivation of Least Squares

The Mathematical Derivation of Least Squares Pscholog 885 Prof. Federco The Mathematcal Dervaton of Least Squares Back when the powers that e forced ou to learn matr algera and calculus, I et ou all asked ourself the age-old queston: When the hell

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

8 Algorithm for Binary Searching in Trees

8 Algorithm for Binary Searching in Trees 8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

More information

Formulating & Solving Integer Problems Chapter 11 289

Formulating & Solving Integer Problems Chapter 11 289 Formulatng & Solvng Integer Problems Chapter 11 289 The Optonal Stop TSP If we drop the requrement that every stop must be vsted, we then get the optonal stop TSP. Ths mght correspond to a ob sequencng

More information

The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method. Introduction. 0/1 Knapsack Problem The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

7.5. Present Value of an Annuity. Investigate

7.5. Present Value of an Annuity. Investigate 7.5 Present Value of an Annuty Owen and Anna are approachng retrement and are puttng ther fnances n order. They have worked hard and nvested ther earnngs so that they now have a large amount of money on

More information

On the Solution of Indefinite Systems Arising in Nonlinear Optimization

On the Solution of Indefinite Systems Arising in Nonlinear Optimization On the Soluton of Indefnte Systems Arsng n Nonlnear Optmzaton Slva Bonettn, Valera Ruggero and Federca Tnt Dpartmento d Matematca, Unverstà d Ferrara Abstract We consder the applcaton of the precondtoned

More information

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of

More information

PERRON FROBENIUS THEOREM

PERRON FROBENIUS THEOREM PERRON FROBENIUS THEOREM R. CLARK ROBINSON Defnton. A n n matrx M wth real entres m, s called a stochastc matrx provded () all the entres m satsfy 0 m, () each of the columns sum to one, m = for all, ()

More information

OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL. Thomas S. Ferguson and C. Zachary Gilstein UCLA and Bell Communications May 1985, revised 2004

OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL. Thomas S. Ferguson and C. Zachary Gilstein UCLA and Bell Communications May 1985, revised 2004 OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL Thomas S. Ferguson and C. Zachary Glsten UCLA and Bell Communcatons May 985, revsed 2004 Abstract. Optmal nvestment polces for maxmzng the expected

More information

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy 4.02 Quz Solutons Fall 2004 Multple-Choce Questons (30/00 ponts) Please, crcle the correct answer for each of the followng 0 multple-choce questons. For each queston, only one of the answers s correct.

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

Simple Interest Loans (Section 5.1) :

Simple Interest Loans (Section 5.1) : Chapter 5 Fnance The frst part of ths revew wll explan the dfferent nterest and nvestment equatons you learned n secton 5.1 through 5.4 of your textbook and go through several examples. The second part

More information

Solution: Let i = 10% and d = 5%. By definition, the respective forces of interest on funds A and B are. i 1 + it. S A (t) = d (1 dt) 2 1. = d 1 dt.

Solution: Let i = 10% and d = 5%. By definition, the respective forces of interest on funds A and B are. i 1 + it. S A (t) = d (1 dt) 2 1. = d 1 dt. Chapter 9 Revew problems 9.1 Interest rate measurement Example 9.1. Fund A accumulates at a smple nterest rate of 10%. Fund B accumulates at a smple dscount rate of 5%. Fnd the pont n tme at whch the forces

More information

Support vector domain description

Support vector domain description Pattern Recognton Letters 20 (1999) 1191±1199 www.elsever.nl/locate/patrec Support vector doman descrpton Davd M.J. Tax *,1, Robert P.W. Dun Pattern Recognton Group, Faculty of Appled Scence, Delft Unversty

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

Period and Deadline Selection for Schedulability in Real-Time Systems

Period and Deadline Selection for Schedulability in Real-Time Systems Perod and Deadlne Selecton for Schedulablty n Real-Tme Systems Thdapat Chantem, Xaofeng Wang, M.D. Lemmon, and X. Sharon Hu Department of Computer Scence and Engneerng, Department of Electrcal Engneerng

More information

Performance Analysis and Coding Strategy of ECOC SVMs

Performance Analysis and Coding Strategy of ECOC SVMs Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.67-76 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School

More information

Energies of Network Nastsemble

Energies of Network Nastsemble Supplementary materal: Assessng the relevance of node features for network structure Gnestra Bancon, 1 Paolo Pn,, 3 and Matteo Marsl 1 1 The Abdus Salam Internatonal Center for Theoretcal Physcs, Strada

More information

+ + + - - This circuit than can be reduced to a planar circuit

+ + + - - This circuit than can be reduced to a planar circuit MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to

More information

Financial market forecasting using a two-step kernel learning method for the support vector regression

Financial market forecasting using a two-step kernel learning method for the support vector regression Ann Oper Res (2010) 174: 103 120 DOI 10.1007/s10479-008-0357-7 Fnancal market forecastng usng a two-step kernel learnng method for the support vector regresson L Wang J Zhu Publshed onlne: 28 May 2008

More information

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and POLYSA: A Polynomal Algorthm for Non-bnary Constrant Satsfacton Problems wth and Mguel A. Saldo, Federco Barber Dpto. Sstemas Informátcos y Computacón Unversdad Poltécnca de Valenca, Camno de Vera s/n

More information

Level Annuities with Payments Less Frequent than Each Interest Period

Level Annuities with Payments Less Frequent than Each Interest Period Level Annutes wth Payments Less Frequent than Each Interest Perod 1 Annuty-mmedate 2 Annuty-due Level Annutes wth Payments Less Frequent than Each Interest Perod 1 Annuty-mmedate 2 Annuty-due Symoblc approach

More information

Ring structure of splines on triangulations

Ring structure of splines on triangulations www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon

More information

An Alternative Way to Measure Private Equity Performance

An Alternative Way to Measure Private Equity Performance An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate

More information

General Auction Mechanism for Search Advertising

General Auction Mechanism for Search Advertising General Aucton Mechansm for Search Advertsng Gagan Aggarwal S. Muthukrshnan Dávd Pál Martn Pál Keywords game theory, onlne auctons, stable matchngs ABSTRACT Internet search advertsng s often sold by an

More information

The Full-Wave Rectifier

The Full-Wave Rectifier 9/3/2005 The Full Wae ectfer.doc /0 The Full-Wae ectfer Consder the followng juncton dode crcut: s (t) Power Lne s (t) 2 Note that we are usng a transformer n ths crcut. The job of ths transformer s to

More information

Learning to Classify Ordinal Data: The Data Replication Method

Learning to Classify Ordinal Data: The Data Replication Method Journal of Machne Learnng Research 8 (7) 393-49 Submtted /6; Revsed 9/6; Publshed 7/7 Learnng to Classfy Ordnal Data: The Data Replcaton Method Jame S. Cardoso INESC Porto, Faculdade de Engenhara, Unversdade

More information

Downlink Power Allocation for Multi-class. Wireless Systems

Downlink Power Allocation for Multi-class. Wireless Systems Downlnk Power Allocaton for Mult-class 1 Wreless Systems Jang-Won Lee, Rav R. Mazumdar, and Ness B. Shroff School of Electrcal and Computer Engneerng Purdue Unversty West Lafayette, IN 47907, USA {lee46,

More information

Mooring Pattern Optimization using Genetic Algorithms

Mooring Pattern Optimization using Genetic Algorithms 6th World Congresses of Structural and Multdscplnary Optmzaton Ro de Janero, 30 May - 03 June 005, Brazl Moorng Pattern Optmzaton usng Genetc Algorthms Alonso J. Juvnao Carbono, Ivan F. M. Menezes Luz

More information

How To Calculate The Accountng Perod Of Nequalty

How To Calculate The Accountng Perod Of Nequalty Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.

More information

arxiv:1311.2444v1 [cs.dc] 11 Nov 2013

arxiv:1311.2444v1 [cs.dc] 11 Nov 2013 FLEXIBLE PARALLEL ALGORITHMS FOR BIG DATA OPTIMIZATION Francsco Facchne 1, Smone Sagratella 1, Gesualdo Scutar 2 1 Dpt. of Computer, Control, and Management Eng., Unversty of Rome La Sapenza", Roma, Italy.

More information

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

More information

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM BARRIOT Jean-Perre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jean-perre.barrot@cnes.fr 1/Introducton The

More information

We are now ready to answer the question: What are the possible cardinalities for finite fields?

We are now ready to answer the question: What are the possible cardinalities for finite fields? Chapter 3 Fnte felds We have seen, n the prevous chapters, some examples of fnte felds. For example, the resdue class rng Z/pZ (when p s a prme) forms a feld wth p elements whch may be dentfed wth the

More information

"Research Note" APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES *

Research Note APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES * Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789-794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC

More information

A Lyapunov Optimization Approach to Repeated Stochastic Games

A Lyapunov Optimization Approach to Repeated Stochastic Games PROC. ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING, OCT. 2013 1 A Lyapunov Optmzaton Approach to Repeated Stochastc Games Mchael J. Neely Unversty of Southern Calforna http://www-bcf.usc.edu/

More information

Least 1-Norm SVMs: a New SVM Variant between Standard and LS-SVMs

Least 1-Norm SVMs: a New SVM Variant between Standard and LS-SVMs ESANN proceedngs, European Smposum on Artfcal Neural Networks - Computatonal Intellgence and Machne Learnng. Bruges (Belgum), 8-3 Aprl, d-sde publ., ISBN -9337--. Least -Norm SVMs: a New SVM Varant between

More information

An MILP model for planning of batch plants operating in a campaign-mode

An MILP model for planning of batch plants operating in a campaign-mode An MILP model for plannng of batch plants operatng n a campagn-mode Yanna Fumero Insttuto de Desarrollo y Dseño CONICET UTN yfumero@santafe-concet.gov.ar Gabrela Corsano Insttuto de Desarrollo y Dseño

More information

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts Power-of-wo Polces for Sngle- Warehouse Mult-Retaler Inventory Systems wth Order Frequency Dscounts José A. Ventura Pennsylvana State Unversty (USA) Yale. Herer echnon Israel Insttute of echnology (Israel)

More information

Quantization Effects in Digital Filters

Quantization Effects in Digital Filters Quantzaton Effects n Dgtal Flters Dstrbuton of Truncaton Errors In two's complement representaton an exact number would have nfntely many bts (n general). When we lmt the number of bts to some fnte value

More information

Equlbra Exst and Trade S effcent proportionally

Equlbra Exst and Trade S effcent proportionally On Compettve Nonlnear Prcng Andrea Attar Thomas Marott Franços Salané February 27, 2013 Abstract A buyer of a dvsble good faces several dentcal sellers. The buyer s preferences are her prvate nformaton,

More information

Research Article Enhanced Two-Step Method via Relaxed Order of α-satisfactory Degrees for Fuzzy Multiobjective Optimization

Research Article Enhanced Two-Step Method via Relaxed Order of α-satisfactory Degrees for Fuzzy Multiobjective Optimization Hndaw Publshng Corporaton Mathematcal Problems n Engneerng Artcle ID 867836 pages http://dxdoorg/055/204/867836 Research Artcle Enhanced Two-Step Method va Relaxed Order of α-satsfactory Degrees for Fuzzy

More information

Problem Set 3. a) We are asked how people will react, if the interest rate i on bonds is negative.

Problem Set 3. a) We are asked how people will react, if the interest rate i on bonds is negative. Queston roblem Set 3 a) We are asked how people wll react, f the nterest rate on bonds s negatve. When

More information

Series Solutions of ODEs 2 the Frobenius method. The basic idea of the Frobenius method is to look for solutions of the form 3

Series Solutions of ODEs 2 the Frobenius method. The basic idea of the Frobenius method is to look for solutions of the form 3 Royal Holloway Unversty of London Department of Physs Seres Solutons of ODEs the Frobenus method Introduton to the Methodology The smple seres expanson method works for dfferental equatons whose solutons

More information

An efficient constraint handling methodology for multi-objective evolutionary algorithms

An efficient constraint handling methodology for multi-objective evolutionary algorithms Rev. Fac. Ing. Unv. Antoqua N. 49. pp. 141-150. Septembre, 009 An effcent constrant handlng methodology for mult-objectve evolutonary algorthms Una metodología efcente para manejo de restrccones en algortmos

More information

SPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:

SPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background: SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and

More information

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence 1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh

More information

On the Optimal Control of a Cascade of Hydro-Electric Power Stations

On the Optimal Control of a Cascade of Hydro-Electric Power Stations On the Optmal Control of a Cascade of Hydro-Electrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;

More information

Addendum to: Importing Skill-Biased Technology

Addendum to: Importing Skill-Biased Technology Addendum to: Importng Skll-Based Technology Arel Bursten UCLA and NBER Javer Cravno UCLA August 202 Jonathan Vogel Columba and NBER Abstract Ths Addendum derves the results dscussed n secton 3.3 of our

More information

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression Novel Methodology of Workng Captal Management for Large Publc Constructons by Usng Fuzzy S-curve Regresson Cheng-Wu Chen, Morrs H. L. Wang and Tng-Ya Hseh Department of Cvl Engneerng, Natonal Central Unversty,

More information

2008/8. An integrated model for warehouse and inventory planning. Géraldine Strack and Yves Pochet

2008/8. An integrated model for warehouse and inventory planning. Géraldine Strack and Yves Pochet 2008/8 An ntegrated model for warehouse and nventory plannng Géraldne Strack and Yves Pochet CORE Voe du Roman Pays 34 B-1348 Louvan-la-Neuve, Belgum. Tel (32 10) 47 43 04 Fax (32 10) 47 43 01 E-mal: corestat-lbrary@uclouvan.be

More information

Enabling P2P One-view Multi-party Video Conferencing

Enabling P2P One-view Multi-party Video Conferencing Enablng P2P One-vew Mult-party Vdeo Conferencng Yongxang Zhao, Yong Lu, Changja Chen, and JanYn Zhang Abstract Mult-Party Vdeo Conferencng (MPVC) facltates realtme group nteracton between users. Whle P2P

More information

Optimal Bidding Strategies for Generation Companies in a Day-Ahead Electricity Market with Risk Management Taken into Account

Optimal Bidding Strategies for Generation Companies in a Day-Ahead Electricity Market with Risk Management Taken into Account Amercan J. of Engneerng and Appled Scences (): 8-6, 009 ISSN 94-700 009 Scence Publcatons Optmal Bddng Strateges for Generaton Companes n a Day-Ahead Electrcty Market wth Rsk Management Taken nto Account

More information

Embedding lattices in the Kleene degrees

Embedding lattices in the Kleene degrees F U N D A M E N T A MATHEMATICAE 62 (999) Embeddng lattces n the Kleene degrees by Hsato M u r a k (Nagoya) Abstract. Under ZFC+CH, we prove that some lattces whose cardnaltes do not exceed ℵ can be embedded

More information

A machine vision approach for detecting and inspecting circular parts

A machine vision approach for detecting and inspecting circular parts A machne vson approach for detectng and nspectng crcular parts Du-Mng Tsa Machne Vson Lab. Department of Industral Engneerng and Management Yuan-Ze Unversty, Chung-L, Tawan, R.O.C. E-mal: edmtsa@saturn.yzu.edu.tw

More information

Using Series to Analyze Financial Situations: Present Value

Using Series to Analyze Financial Situations: Present Value 2.8 Usng Seres to Analyze Fnancal Stuatons: Present Value In the prevous secton, you learned how to calculate the amount, or future value, of an ordnary smple annuty. The amount s the sum of the accumulated

More information

Faraday's Law of Induction

Faraday's Law of Induction Introducton Faraday's Law o Inducton In ths lab, you wll study Faraday's Law o nducton usng a wand wth col whch swngs through a magnetc eld. You wll also examne converson o mechanc energy nto electrc energy

More information

NON-CONSTANT SUM RED-AND-BLACK GAMES WITH BET-DEPENDENT WIN PROBABILITY FUNCTION LAURA PONTIGGIA, University of the Sciences in Philadelphia

NON-CONSTANT SUM RED-AND-BLACK GAMES WITH BET-DEPENDENT WIN PROBABILITY FUNCTION LAURA PONTIGGIA, University of the Sciences in Philadelphia To appear n Journal o Appled Probablty June 2007 O-COSTAT SUM RED-AD-BLACK GAMES WITH BET-DEPEDET WI PROBABILITY FUCTIO LAURA POTIGGIA, Unversty o the Scences n Phladelpha Abstract In ths paper we nvestgate

More information

Lecture 3: Force of Interest, Real Interest Rate, Annuity

Lecture 3: Force of Interest, Real Interest Rate, Annuity Lecture 3: Force of Interest, Real Interest Rate, Annuty Goals: Study contnuous compoundng and force of nterest Dscuss real nterest rate Learn annuty-mmedate, and ts present value Study annuty-due, and

More information

Heuristic Static Load-Balancing Algorithm Applied to CESM

Heuristic Static Load-Balancing Algorithm Applied to CESM Heurstc Statc Load-Balancng Algorthm Appled to CESM 1 Yur Alexeev, 1 Sher Mckelson, 1 Sven Leyffer, 1 Robert Jacob, 2 Anthony Crag 1 Argonne Natonal Laboratory, 9700 S. Cass Avenue, Argonne, IL 60439,

More information

ONE of the most crucial problems that every image

ONE of the most crucial problems that every image IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 10, OCTOBER 2014 4413 Maxmum Margn Projecton Subspace Learnng for Vsual Data Analyss Symeon Nktds, Anastasos Tefas, Member, IEEE, and Ioanns Ptas, Fellow,

More information

Goals Rotational quantities as vectors. Math: Cross Product. Angular momentum

Goals Rotational quantities as vectors. Math: Cross Product. Angular momentum Physcs 106 Week 5 Torque and Angular Momentum as Vectors SJ 7thEd.: Chap 11.2 to 3 Rotatonal quanttes as vectors Cross product Torque expressed as a vector Angular momentum defned Angular momentum as a

More information

Online Auctions in IaaS Clouds: Welfare and Profit Maximization with Server Costs

Online Auctions in IaaS Clouds: Welfare and Profit Maximization with Server Costs Onlne Auctons n IaaS Clouds: Welfare and roft Maxmzaton wth Server Costs aox Zhang Dept. of Computer Scence The Unvety of Hong Kong xxzhang@cs.hku.hk Zongpeng L Dept. of Computer Scence Unvety of Calgary

More information

Project Networks With Mixed-Time Constraints

Project Networks With Mixed-Time Constraints Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa

More information

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers Foundatons and Trends R n Machne Learnng Vol. 3, No. 1 (2010) 1 122 c 2011 S. Boyd, N. Parkh, E. Chu, B. Peleato and J. Ecksten DOI: 10.1561/2200000016 Dstrbuted Optmzaton and Statstcal Learnng va the

More information

Finite Math Chapter 10: Study Guide and Solution to Problems

Finite Math Chapter 10: Study Guide and Solution to Problems Fnte Math Chapter 10: Study Gude and Soluton to Problems Basc Formulas and Concepts 10.1 Interest Basc Concepts Interest A fee a bank pays you for money you depost nto a savngs account. Prncpal P The amount

More information

Compiling for Parallelism & Locality. Dependence Testing in General. Algorithms for Solving the Dependence Problem. Dependence Testing

Compiling for Parallelism & Locality. Dependence Testing in General. Algorithms for Solving the Dependence Problem. Dependence Testing Complng for Parallelsm & Localty Dependence Testng n General Assgnments Deadlne for proect 4 extended to Dec 1 Last tme Data dependences and loops Today Fnsh data dependence analyss for loops General code

More information

On Robust Network Planning

On Robust Network Planning On Robust Network Plannng Al Tzghadam School of Electrcal and Computer Engneerng Unversty of Toronto, Toronto, Canada Emal: al.tzghadam@utoronto.ca Alberto Leon-Garca School of Electrcal and Computer Engneerng

More information