Support Vector Machines

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Support Vector Machines"

Transcription

1 Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada Abstract Ths s a note to explan support vector machnes. 1 Prelmnares Our task s to predct whether a test sample belongs to one of two classes. We receve tranng examples of the form: {x, y }, = 1,..., n and x R d, y { 1, +1}. We call {x } the co-varates or nput vectors and {y } the response varables or labels. We consder a very smple example where the data are n fact lnearly separable:.e. I can draw a straght lne f(x) = w T x b such that all cases wth y = 1 fall on one sde and have f(x ) < 0 and cases wth y = +1 fall on the other and have f(x ) > 0. Gven that we have acheved that, we could classfy new test cases accordng to the rule y test = sgn(x test ). However, typcally there are nfntely many such hyper-planes obtaned by small perturbatons of a gven soluton. How do we choose between all these hyper-planes whch the solve the separaton problem for our tranng data, but may have dfferent performance on the newly arrvng test cases. For nstance, we could choose to put the lne very close to members of one partcular class, say y = 1. Intutvely, when test cases arrve we wll not make many mstakes on cases that should be classfed wth y = +1, but we wll make very easly mstakes on the cases wth y = 1 (for nstance, magne that a new batch of test cases arrves whch are small perturbatons of the tranng data). A sensble thng thus seems to choose the separaton lne as far away from both y = 1 and y = +1 tranng cases as we can,.e. rght n the mddle. Geometrcally, the vector w s drected orthogonal to the lne defned by w T x = b. Ths can be understood as follows. Frst take b = 0. Now t s clear that all vectors, x, wth vanshng nner product wth w satsfy ths equaton,.e. all vectors orthogonal to w satsfy ths equaton. Now translate the hyperplane away from the orgn over a vector a. The equaton for the plane now becomes: (x a) T w = 0,.e. we fnd that for the offset b = a T w, whch s the projecton of a onto to the vector w. Wthout loss of generalty we may thus choose a perpendcular to the plane, n whch case the length a = b / w represents the shortest, orthogonal dstance between the orgn and the hyperplane. We now defne 2 more hyperplanes parallel to the separatng hyperplane. They represent that planes that cut through the closest tranng examples on ether sde. We wll call them

2 support hyper-planes n the followng, because the data-vectors they contan support the plane. We defne the dstance between the these hyperplanes and the separatng hyperplane to be d + and d respectvely. The margn, γ, s defned to be d + + d. Our goal s now to fnd a the separatng hyperplane so that the margn s largest, whle the separatng hyperplane s equdstant from both. We can wrte the followng equatons for the support hyperplanes: w T x = b + δ (1) w T x = b δ (2) We now note that we have over-parameterzed the problem: f we scale w, b and δ by a constant factor α, the equatons for x are stll satsfed. To remove ths ambguty we wll requre that δ = 1, ths sets the scale of the problem,.e. f we measure dstance n mllmeters or meters. We can now also compute the values for d + = ( b+1 b )/ w = 1/ w (ths s only true f b / ( 1, 0) snce the orgn doesn t fall n between the hyperplanes n that case. If b ( 1, 0) you should use d + = ( b b )/ w = 1/ w ). Hence the margn s equal to twce that value: γ = 2/ w. Wth the above defnton of the support planes we can wrte down the followng constrant that any soluton must satsfy, w T x b 1 y = 1 (3) w T x b +1 y = +1 (4) or n one equaton, y (w T x b) 1 0 (5) We now formulate the prmal problem of the SVM: 1 mnmze 2 w 2 subject to y (w T x b) 1 0 (6) Thus, we maxmze the margn, subject to the constrants that all tranng cases fall on ether sde of the support hyper-planes. The data-cases that le on the hyperplane are called support vectors, snce they support the hyper-planes and hence determne the soluton to the problem. The prmal problem can be solved by a quadratc program. However, t s not ready to be kernelsed, because ts dependence s not only on nner products between data-vectors. Hence, we transform to the dual formulaton by frst wrtng the problem usng a Lagrangan, L(w, b, α) = 1 N [ 2 w 2 α y (w T x b) 1 ] (7) =1 The soluton that mnmzes the prmal problem subject to the constrants s gven by mn w max α L(w, α),.e. a saddle pont problem. When the orgnal objectve-functon s convex, (and only then), we can nterchange the mnmzaton and maxmzaton. Dong that, we fnd that we can fnd the condton on w that must hold at the saddle pont we are solvng for. Ths s done by takng dervatves wrt w and b and solvng, w α y x = 0 w = α y x (8) α y = 0 (9)

3 Insertng ths back nto the Lagrangan we obtan what s known as the dual problem, N maxmze L D = α 1 α α j y y j x T x j 2 =1 j subject to α y = 0 (10) α 0 (11) The dual formulaton of the problem s also a quadratc program, but note that the number of varables, α n ths problem s equal to the number of data-cases, N. The crucal pont s however, that ths problem only depends on x through the nner product x T x j. Ths s readly kernelsed through the substtuton x T x j k(x, x j ). Ths s a recurrent theme: the dual problem lends tself to kernelsaton, whle the prmal problem dd not. The theory of dualty guarantees that for convex problems, the dual problem wll be concave, and moreover, that the unque soluton of the prmal problem corresponds tot the unque soluton of the dual problem. In fact, we have: L P (w ) = L D (α ),.e. the dualty-gap s zero. Next we turn to the condtons that must necessarly hold at the saddle pont and thus the soluton of the problem. These are called the KKT condtons (whch stands for Karush- Kuhn-Tucker). These condtons are necessary n general, and suffcent for convex optmzaton problems. They can be derved from the prmal problem by settng the dervatves wrt to w to zero. Also, the constrants themselves are part of these condtons and we need that for nequalty constrants the Lagrange multplers are non-negatve. Fnally, an mportant constrant called complementary slackness needs to be satsfed, w L P = 0 w α y x = 0 (12) b L P = 0 α y = 0 (13) constrant - 1 y (w T x b) 1 0 (14) multpler condton α 0 (15) complementary slackness α [ y (w T x b) 1 ] = 0 (16) It s the last equaton whch may be somewhat surprsng. It states that ether the nequalty constrant s satsfed, but not saturated: y (w T x b) 1 > 0 n whch case α for that data-case must be zero, or the nequalty constrant s saturated y (w T x b) 1 = 0, n whch case α can be any value α 0. Inequalty constrants whch are saturated are sad to be actve, whle unsaturated constrants are nactve. One could magne the process of searchng for a soluton as a ball whch runs down the prmary objectve functon usng gradent descent. At some pont, t wll ht a wall whch s the constrant and although the dervatve s stll pontng partally towards the wall, the constrants prohbts the ball to go on. Ths s an actve constrant because the ball s glued to that wall. When a fnal soluton s reached, we could remove some constrants, wthout changng the soluton, these are nactve constrants. One could thnk of the term w L P as the force actng on the ball. We see from the frst equaton above that only the forces wth α 0 exsert a force on the ball that balances wth the force from the curved quadratc surface w. The tranng cases wth α > 0, representng actve constrants on the poston of the support hyperplane are called support vectors. These are the vectors that are stuated n the support hyperplane and they determne the soluton. Typcally, there are only few of them, whch people call a sparse soluton (most α s vansh).

4 What we are really nterested n s the functon f( ) whch can be used to classfy future test cases, f(x) = w T x b = α y x T x b (17) As an applcaton of the KKT condtons we derve a soluton for b by usng the complementary slackness condton, b = j α j y j x T j x y a support vector (18) where we used y 2 = 1. So, usng any support vector one can determne b, but for numercal stablty t s better to average over all of them (although they should obvously be consstent). The most mportant concluson s agan that ths functon f( ) can thus be expressed solely n terms of nner products x T x whch we can replace wth kernel matrces k(x, x j ) to move to hgh dmensonal non-lnear spaces. Moreover, snce α s typcally very sparse, we don t need to evaluate many kernel entres n order to predct the class of the new nput x. 2 The Non-Separable case Obvously, not all datasets are lnearly separable, and so we need to change the formalsm to account for that. Clearly, the problem les n the constrants, whch cannot always be satsfed. So, let s relax those constrants by ntroducng slack varables, ξ, w T x b 1 + ξ y = 1 (19) w T x b +1 ξ y = +1 (20) ξ 0 (21) The varables, ξ allow for volatons of the constrant. We should penalze the objectve functon for these volatons, otherwse the above constrants become vod (smply always pck ξ very large). Penalty functons of the form C( ξ ) k wll lead to convex optmzaton problems for postve ntegers k. For k = 1, 2 t s stll a quadratc program (QP). In the followng we wll choose k = 1. C controls the tradeoff between the penalty and margn. To be on the wrong sde of the separatng hyperplane, a data-case would need ξ > 1. Hence, the sum ξ could be nterpreted as measure of how bad the volatons are and s an upper bound on the number of volatons. The new prmal problem thus becomes, mnmze L P = 1 2 w 2 + C ξ leadng to the Lagrangan, subject to y (w T x b) 1 + ξ 0 (22) ξ 0 (23) L(w, b, ξ, α, µ) = 1 2 w 2 +C N [ ξ α y (w T ] N x b) 1 + ξ µ ξ (24) =1 =1

5 from whch we derve the KKT condtons, 1. w L P = 0 w α y x = 0 (25) 2. b L P = 0 α y = 0 (26) 3. ξ L P = 0 C α µ = 0 (27) 4.constrant-1 y (w T x b) 1 + ξ 0 (28) 5.constrant-2 ξ 0 (29) 6.multpler condton-1 α 0 (30) 7.multpler condton-2 µ 0 (31) 8.complementary slackness-1 [ α y (w T ] x b) 1 + ξ = 0 (32) 9.complementary slackness-1 µ ξ = 0 (33) (34) From here we can deduce the followng facts. If we assume that ξ > 0, then µ = 0 (9), hence α = C (1) and thus ξ = 1 y (x T w b) (8). Also, when ξ = 0 we have µ > 0 (9) and hence α < C. If n addton to ξ = 0 we also have that y (w T x b) 1 = 0, then α > 0 (8). Otherwse, f y (w T x b) 1 > 0 then α = 0. In summary, as before for ponts not on the support plane and on the correct sde we have ξ = α = 0 (all constrants nactve). On the support plane, we stll have ξ = 0, but now α > 0. Fnally, for data-cases on the wrong sde of the support hyperplane the α max-out to α = C and the ξ balance the volaton of the constrant such that y (w T x b) 1 + ξ = 0. Geometrcally, we can calculate the gap between support hyperplane and the volatng datacase to be ξ / w. Ths can be seen because the plane defned by y (w T x b) 1+ξ = 0 s parallel to the support plane at a dstance 1 + y b ξ / w from the orgn. Snce the support plane s at a dstance 1 + y b / w the result follows. Fnally, we need to convert to the dual problem to solve t effcently and to kernelse t. Agan, we use the KKT equatons to get rd of w, b and ξ, N maxmze L D = α 1 α α j y y j x T x j 2 =1 j subject to α y = 0 (35) 0 α C (36) Surprsngly, ths s almost the same QP s before, but wth an extra constrant on the multplers α whch now lve n a box. Ths constrant s derved from the fact that α = C µ and µ 0. We also note that t only depends on nner products x T x j whch are ready to be kernelsed.

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2) MATH 16T Exam 1 : Part I (In-Class) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total

More information

Moment of a force about a point and about an axis

Moment of a force about a point and about an axis 3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should

More information

v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

v a 1 b 1 i, a 2 b 2 i,..., a n b n i. SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

More information

Solution of Algebraic and Transcendental Equations

Solution of Algebraic and Transcendental Equations CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton

More information

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

More information

The example below solves a system in the unknowns α and β:

The example below solves a system in the unknowns α and β: The Fnd Functon The functon Fnd returns a soluton to a system of equatons gven by a solve block. You can use Fnd to solve a lnear system, as wth lsolve, or to solve nonlnear systems. The example below

More information

Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2

Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2 18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest

More information

Fisher Markets and Convex Programs

Fisher Markets and Convex Programs Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

More information

1 Example 1: Axis-aligned rectangles

1 Example 1: Axis-aligned rectangles COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton

More information

Least Squares Fitting of Data

Least Squares Fitting of Data Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2016. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng

More information

New Approaches to Support Vector Ordinal Regression

New Approaches to Support Vector Ordinal Regression New Approaches to Support Vector Ordnal Regresson We Chu chuwe@gatsby.ucl.ac.uk Gatsby Computatonal Neuroscence Unt, Unversty College London, London, WCN 3AR, UK S. Sathya Keerth selvarak@yahoo-nc.com

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

A Computer Technique for Solving LP Problems with Bounded Variables

A Computer Technique for Solving LP Problems with Bounded Variables Dhaka Unv. J. Sc. 60(2): 163-168, 2012 (July) A Computer Technque for Solvng LP Problems wth Bounded Varables S. M. Atqur Rahman Chowdhury * and Sanwar Uddn Ahmad Department of Mathematcs; Unversty of

More information

Lecture 2: Single Layer Perceptrons Kevin Swingler

Lecture 2: Single Layer Perceptrons Kevin Swingler Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCulloch-Ptts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

Solution : (a) FALSE. Let C be a binary one-error correcting code of length 9. Then it follows from the Sphere packing bound that.

Solution : (a) FALSE. Let C be a binary one-error correcting code of length 9. Then it follows from the Sphere packing bound that. MATH 29T Exam : Part I Solutons. TRUE/FALSE? Prove your answer! (a) (5 pts) There exsts a bnary one-error correctng code of length 9 wth 52 codewords. (b) (5 pts) There exsts a ternary one-error correctng

More information

SVM Tutorial: Classification, Regression, and Ranking

SVM Tutorial: Classification, Regression, and Ranking SVM Tutoral: Classfcaton, Regresson, and Rankng Hwanjo Yu and Sungchul Km 1 Introducton Support Vector Machnes(SVMs) have been extensvely researched n the data mnng and machne learnng communtes for the

More information

21 Vectors: The Cross Product & Torque

21 Vectors: The Cross Product & Torque 21 Vectors: The Cross Product & Torque Do not use our left hand when applng ether the rght-hand rule for the cross product of two vectors dscussed n ths chapter or the rght-hand rule for somethng curl

More information

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

Production. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem.

Production. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem. Producer Theory Producton ASSUMPTION 2.1 Propertes of the Producton Set The producton set Y satsfes the followng propertes 1. Y s non-empty If Y s empty, we have nothng to talk about 2. Y s closed A set

More information

Support Vector Machine Model for Currency Crisis Discrimination. Arindam Chaudhuri 1. Abstract

Support Vector Machine Model for Currency Crisis Discrimination. Arindam Chaudhuri 1. Abstract Support Vector Machne Model for Currency Crss Dscrmnaton Arndam Chaudhur Abstract Support Vector Machne (SVM) s powerful classfcaton technque based on the dea of structural rsk mnmzaton. Use of kernel

More information

The eigenvalue derivatives of linear damped systems

The eigenvalue derivatives of linear damped systems Control and Cybernetcs vol. 32 (2003) No. 4 The egenvalue dervatves of lnear damped systems by Yeong-Jeu Sun Department of Electrcal Engneerng I-Shou Unversty Kaohsung, Tawan 840, R.O.C e-mal: yjsun@su.edu.tw

More information

Logistic Regression. Steve Kroon

Logistic Regression. Steve Kroon Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro

More information

PLANAR GRAPHS. Plane graph (or embedded graph) A graph that is drawn on the plane without edge crossing, is called a Plane graph

PLANAR GRAPHS. Plane graph (or embedded graph) A graph that is drawn on the plane without edge crossing, is called a Plane graph PLANAR GRAPHS Basc defntons Isomorphc graphs Two graphs G(V,E) and G2(V2,E2) are somorphc f there s a one-to-one correspondence F of ther vertces such that the followng holds: - u,v V, uv E, => F(u)F(v)

More information

Solutions to the exam in SF2862, June 2009

Solutions to the exam in SF2862, June 2009 Solutons to the exam n SF86, June 009 Exercse 1. Ths s a determnstc perodc-revew nventory model. Let n = the number of consdered wees,.e. n = 4 n ths exercse, and r = the demand at wee,.e. r 1 = r = r

More information

A Probabilistic Theory of Coherence

A Probabilistic Theory of Coherence A Probablstc Theory of Coherence BRANDEN FITELSON. The Coherence Measure C Let E be a set of n propostons E,..., E n. We seek a probablstc measure C(E) of the degree of coherence of E. Intutvely, we want

More information

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2 EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a non-zero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :

More information

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

More information

b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei

b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei Mathematcal Propertes of the Least Squares Regresson The least squares regresson lne obeys certan mathematcal propertes whch are useful to know n practce. The followng propertes can be establshed algebracally:

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

Introduction: Analysis of Electronic Circuits

Introduction: Analysis of Electronic Circuits /30/008 ntroducton / ntroducton: Analyss of Electronc Crcuts Readng Assgnment: KVL and KCL text from EECS Just lke EECS, the majorty of problems (hw and exam) n EECS 3 wll be crcut analyss problems. Thus,

More information

Ping Pong Fun - Video Analysis Project

Ping Pong Fun - Video Analysis Project Png Pong Fun - Vdeo Analyss Project Objectve In ths experment we are gong to nvestgate the projectle moton of png pong balls usng Verner s Logger Pro Software. Does the object travel n a straght lne? What

More information

QUANTUM MECHANICS, BRAS AND KETS

QUANTUM MECHANICS, BRAS AND KETS PH575 SPRING QUANTUM MECHANICS, BRAS AND KETS The followng summares the man relatons and defntons from quantum mechancs that we wll be usng. State of a phscal sstem: The state of a phscal sstem s represented

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

Generalizing the degree sequence problem

Generalizing the degree sequence problem Mddlebury College March 2009 Arzona State Unversty Dscrete Mathematcs Semnar The degree sequence problem Problem: Gven an nteger sequence d = (d 1,...,d n ) determne f there exsts a graph G wth d as ts

More information

Chapter 4 ECONOMIC DISPATCH AND UNIT COMMITMENT

Chapter 4 ECONOMIC DISPATCH AND UNIT COMMITMENT Chapter 4 ECOOMIC DISATCH AD UIT COMMITMET ITRODUCTIO A power system has several power plants. Each power plant has several generatng unts. At any pont of tme, the total load n the system s met by the

More information

Optimal portfolios using Linear Programming models

Optimal portfolios using Linear Programming models Optmal portfolos usng Lnear Programmng models Chrstos Papahrstodoulou Mälardalen Unversty, Västerås, Sweden Abstract The classcal Quadratc Programmng formulaton of the well known portfolo selecton problem,

More information

Loop Parallelization

Loop Parallelization - - Loop Parallelzaton C-52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I-,J]+B[I-,J-] ED FOR ED FOR analyze

More information

greatest common divisor

greatest common divisor 4. GCD 1 The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no

More information

L10: Linear discriminants analysis

L10: Linear discriminants analysis L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss

More information

The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets

The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets . The Magnetc Feld Concepts and Prncples Movng Charges All charged partcles create electrc felds, and these felds can be detected by other charged partcles resultng n electrc force. However, a completely

More information

State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness

State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness Schroednger equaton Basc postulates of quantum mechancs. Operators: Hermtan operators, commutators State functon: egenfunctons of hermtan operators-> normalzaton, orthogonalty completeness egenvalues and

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

where the coordinates are related to those in the old frame as follows.

where the coordinates are related to those in the old frame as follows. Chapter 2 - Cartesan Vectors and Tensors: Ther Algebra Defnton of a vector Examples of vectors Scalar multplcaton Addton of vectors coplanar vectors Unt vectors A bass of non-coplanar vectors Scalar product

More information

A linear recurrence sequence of composite numbers

A linear recurrence sequence of composite numbers LMS J Comput Math 15 (2012) 360 373 C 2012 Author do:101112/s1461157012001143 A lnear recurrence sequence of composte numbers Jonas Šurys Abstract We prove that for each postve nteger k n the range 2 k

More information

8 Algorithm for Binary Searching in Trees

8 Algorithm for Binary Searching in Trees 8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

More information

The Mathematical Derivation of Least Squares

The Mathematical Derivation of Least Squares Pscholog 885 Prof. Federco The Mathematcal Dervaton of Least Squares Back when the powers that e forced ou to learn matr algera and calculus, I et ou all asked ourself the age-old queston: When the hell

More information

Math 131: Homework 4 Solutions

Math 131: Homework 4 Solutions Math 3: Homework 4 Solutons Greg Parker, Wyatt Mackey, Chrstan Carrck October 6, 05 Problem (Munkres 3.) Let {A n } be a sequence of connected subspaces of X such that A n \ A n+ 6= ; for all n. Then S

More information

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching) Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton

More information

2.4 Bivariate distributions

2.4 Bivariate distributions page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together

More information

II. PROBABILITY OF AN EVENT

II. PROBABILITY OF AN EVENT II. PROBABILITY OF AN EVENT As ndcated above, probablty s a quantfcaton, or a mathematcal model, of a random experment. Ths quantfcaton s a measure of the lkelhood that a gven event wll occur when the

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method. Introduction. 0/1 Knapsack Problem The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

More information

EE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN

EE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN EE201 Crcut Theory I 2015 Sprng Dr. Yılmaz KALKAN 1. Basc Concepts (Chapter 1 of Nlsson - 3 Hrs.) Introducton, Current and Voltage, Power and Energy 2. Basc Laws (Chapter 2&3 of Nlsson - 6 Hrs.) Voltage

More information

Formulating & Solving Integer Problems Chapter 11 289

Formulating & Solving Integer Problems Chapter 11 289 Formulatng & Solvng Integer Problems Chapter 11 289 The Optonal Stop TSP If we drop the requrement that every stop must be vsted, we then get the optonal stop TSP. Ths mght correspond to a ob sequencng

More information

Graph Theory and Cayley s Formula

Graph Theory and Cayley s Formula Graph Theory and Cayley s Formula Chad Casarotto August 10, 2006 Contents 1 Introducton 1 2 Bascs and Defntons 1 Cayley s Formula 4 4 Prüfer Encodng A Forest of Trees 7 1 Introducton In ths paper, I wll

More information

INTRODUCTION. governed by a differential equation Need systematic approaches to generate FE equations

INTRODUCTION. governed by a differential equation Need systematic approaches to generate FE equations WEIGHTED RESIDUA METHOD INTRODUCTION Drect stffness method s lmted for smple D problems PMPE s lmted to potental problems FEM can be appled to many engneerng problems that are governed by a dfferental

More information

PERRON FROBENIUS THEOREM

PERRON FROBENIUS THEOREM PERRON FROBENIUS THEOREM R. CLARK ROBINSON Defnton. A n n matrx M wth real entres m, s called a stochastc matrx provded () all the entres m satsfy 0 m, () each of the columns sum to one, m = for all, ()

More information

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of

More information

Week 6 Market Failure due to Externalities

Week 6 Market Failure due to Externalities Week 6 Market Falure due to Externaltes 1. Externaltes n externalty exsts when the acton of one agent unavodably affects the welfare of another agent. The affected agent may be a consumer, gvng rse to

More information

7.5. Present Value of an Annuity. Investigate

7.5. Present Value of an Annuity. Investigate 7.5 Present Value of an Annuty Owen and Anna are approachng retrement and are puttng ther fnances n order. They have worked hard and nvested ther earnngs so that they now have a large amount of money on

More information

On the Solution of Indefinite Systems Arising in Nonlinear Optimization

On the Solution of Indefinite Systems Arising in Nonlinear Optimization On the Soluton of Indefnte Systems Arsng n Nonlnear Optmzaton Slva Bonettn, Valera Ruggero and Federca Tnt Dpartmento d Matematca, Unverstà d Ferrara Abstract We consder the applcaton of the precondtoned

More information

OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL. Thomas S. Ferguson and C. Zachary Gilstein UCLA and Bell Communications May 1985, revised 2004

OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL. Thomas S. Ferguson and C. Zachary Gilstein UCLA and Bell Communications May 1985, revised 2004 OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL Thomas S. Ferguson and C. Zachary Glsten UCLA and Bell Communcatons May 985, revsed 2004 Abstract. Optmal nvestment polces for maxmzng the expected

More information

Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect

Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect Chapter 3 Group Theory p. - 3. Compact Course: Groups Theory emark: Ths s only a bref summary of most mportant results of groups theory wth respect to the applcatons dscussed n the followng chapters. For

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier

Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier Lesson 2 Chapter Two Three Phase Uncontrolled Rectfer. Operatng prncple of three phase half wave uncontrolled rectfer The half wave uncontrolled converter s the smplest of all three phase rectfer topologes.

More information

Support vector domain description

Support vector domain description Pattern Recognton Letters 20 (1999) 1191±1199 www.elsever.nl/locate/patrec Support vector doman descrpton Davd M.J. Tax *,1, Robert P.W. Dun Pattern Recognton Group, Faculty of Appled Scence, Delft Unversty

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

z(t) = z 1 (t) + t(z 2 z 1 ) z(t) = 1 + i + t( 2 3i (1 + i)) z(t) = 1 + i + t( 3 4i); 0 t 1

z(t) = z 1 (t) + t(z 2 z 1 ) z(t) = 1 + i + t( 2 3i (1 + i)) z(t) = 1 + i + t( 3 4i); 0 t 1 (4.): ontours. Fnd an admssble parametrzaton. (a). the lne segment from z + to z 3. z(t) z (t) + t(z z ) z(t) + + t( 3 ( + )) z(t) + + t( 3 4); t (b). the crcle jz j 4 traversed once clockwse startng at

More information

Simple Interest Loans (Section 5.1) :

Simple Interest Loans (Section 5.1) : Chapter 5 Fnance The frst part of ths revew wll explan the dfferent nterest and nvestment equatons you learned n secton 5.1 through 5.4 of your textbook and go through several examples. The second part

More information

Solution: Let i = 10% and d = 5%. By definition, the respective forces of interest on funds A and B are. i 1 + it. S A (t) = d (1 dt) 2 1. = d 1 dt.

Solution: Let i = 10% and d = 5%. By definition, the respective forces of interest on funds A and B are. i 1 + it. S A (t) = d (1 dt) 2 1. = d 1 dt. Chapter 9 Revew problems 9.1 Interest rate measurement Example 9.1. Fund A accumulates at a smple nterest rate of 10%. Fund B accumulates at a smple dscount rate of 5%. Fnd the pont n tme at whch the forces

More information

+ + + - - This circuit than can be reduced to a planar circuit

+ + + - - This circuit than can be reduced to a planar circuit MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to

More information

Performance Analysis and Coding Strategy of ECOC SVMs

Performance Analysis and Coding Strategy of ECOC SVMs Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.67-76 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School

More information

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60 BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

More information

SCALAR A physical quantity that is completely characterized by a real number (or by its numerical value) is called a scalar. In other words, a scalar

SCALAR A physical quantity that is completely characterized by a real number (or by its numerical value) is called a scalar. In other words, a scalar SCALAR A phscal quantt that s completel charactered b a real number (or b ts numercal value) s called a scalar. In other words, a scalar possesses onl a magntude. Mass, denst, volume, temperature, tme,

More information

Multivariate EWMA Control Chart

Multivariate EWMA Control Chart Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant

More information

Level Annuities with Payments Less Frequent than Each Interest Period

Level Annuities with Payments Less Frequent than Each Interest Period Level Annutes wth Payments Less Frequent than Each Interest Perod 1 Annuty-mmedate 2 Annuty-due Level Annutes wth Payments Less Frequent than Each Interest Perod 1 Annuty-mmedate 2 Annuty-due Symoblc approach

More information

FORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER

FORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER FORCED CONVECION HEA RANSFER IN A DOUBLE PIPE HEA EXCHANGER Dr. J. Mchael Doster Department of Nuclear Engneerng Box 7909 North Carolna State Unversty Ralegh, NC 27695-7909 Introducton he convectve heat

More information

Financial market forecasting using a two-step kernel learning method for the support vector regression

Financial market forecasting using a two-step kernel learning method for the support vector regression Ann Oper Res (2010) 174: 103 120 DOI 10.1007/s10479-008-0357-7 Fnancal market forecastng usng a two-step kernel learnng method for the support vector regresson L Wang J Zhu Publshed onlne: 28 May 2008

More information

EXPLORATION 2.5A Exploring the motion diagram of a dropped object

EXPLORATION 2.5A Exploring the motion diagram of a dropped object -5 Acceleraton Let s turn now to moton that s not at constant elocty. An example s the moton of an object you release from rest from some dstance aboe the floor. EXPLORATION.5A Explorng the moton dagram

More information

Supplementary material: Assessing the relevance of node features for network structure

Supplementary material: Assessing the relevance of node features for network structure Supplementary materal: Assessng the relevance of node features for network structure Gnestra Bancon, 1 Paolo Pn,, 3 and Matteo Marsl 1 1 The Abdus Salam Internatonal Center for Theoretcal Physcs, Strada

More information

Communication Networks II Contents

Communication Networks II Contents 8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

More information

MANY machine learning and pattern recognition applications

MANY machine learning and pattern recognition applications 1 Trace Rato Problem Revsted Yangqng Ja, Fepng Ne, and Changshu Zhang Abstract Dmensonalty reducton s an mportant ssue n many machne learnng and pattern recognton applcatons, and the trace rato problem

More information

Downlink Power Allocation for Multi-class. Wireless Systems

Downlink Power Allocation for Multi-class. Wireless Systems Downlnk Power Allocaton for Mult-class 1 Wreless Systems Jang-Won Lee, Rav R. Mazumdar, and Ness B. Shroff School of Electrcal and Computer Engneerng Purdue Unversty West Lafayette, IN 47907, USA {lee46,

More information

Period and Deadline Selection for Schedulability in Real-Time Systems

Period and Deadline Selection for Schedulability in Real-Time Systems Perod and Deadlne Selecton for Schedulablty n Real-Tme Systems Thdapat Chantem, Xaofeng Wang, M.D. Lemmon, and X. Sharon Hu Department of Computer Scence and Engneerng, Department of Electrcal Engneerng

More information

Semiconductor sensors of temperature

Semiconductor sensors of temperature Semconductor sensors of temperature he measurement objectve 1. Identfy the unknown bead type thermstor. Desgn the crcutry for lnearzaton of ts transfer curve.. Fnd the dependence of forward voltage drop

More information

Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006

Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006 Aryabhata s Root Extracton Methods Abhshek Parakh Lousana State Unversty Aug 1 st 1 Introducton Ths artcle presents an analyss of the root extracton algorthms of Aryabhata gven n hs book Āryabhatīya [1,

More information

Properties of American Derivative Securities

Properties of American Derivative Securities Capter 6 Propertes of Amercan Dervatve Securtes 6.1 Te propertes Defnton 6.1 An Amercan dervatve securty s a sequence of non-negatve random varables fg k g n k= suc tat eac G k s F k -measurable. Te owner

More information

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy 4.02 Quz Solutons Fall 2004 Multple-Choce Questons (30/00 ponts) Please, crcle the correct answer for each of the followng 0 multple-choce questons. For each queston, only one of the answers s correct.

More information

VLSI Technology Dr. Nandita Dasgupta Department of Electrical Engineering Indian Institute of Technology, Madras

VLSI Technology Dr. Nandita Dasgupta Department of Electrical Engineering Indian Institute of Technology, Madras VLI Technology Dr. Nandta Dasgupta Department of Electrcal Engneerng Indan Insttute of Technology, Madras Lecture - 11 Oxdaton I netcs of Oxdaton o, the unt process step that we are gong to dscuss today

More information

Ring structure of splines on triangulations

Ring structure of splines on triangulations www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon

More information

Linear Regression, Regularization Bias-Variance Tradeoff

Linear Regression, Regularization Bias-Variance Tradeoff HTF: Ch3, 7 B: Ch3 Lnear Regresson, Regularzaton Bas-Varance Tradeoff Thanks to C Guestrn, T Detterch, R Parr, N Ray 1 Outlne Lnear Regresson MLE = Least Squares! Bass functons Evaluatng Predctors Tranng

More information

U.C. Berkeley CS270: Algorithms Lecture 4 Professor Vazirani and Professor Rao Jan 27,2011 Lecturer: Umesh Vazirani Last revised February 10, 2012

U.C. Berkeley CS270: Algorithms Lecture 4 Professor Vazirani and Professor Rao Jan 27,2011 Lecturer: Umesh Vazirani Last revised February 10, 2012 U.C. Berkeley CS270: Algorthms Lecture 4 Professor Vazran and Professor Rao Jan 27,2011 Lecturer: Umesh Vazran Last revsed February 10, 2012 Lecture 4 1 The multplcatve weghts update method The multplcatve

More information

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and

POLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and POLYSA: A Polynomal Algorthm for Non-bnary Constrant Satsfacton Problems wth and Mguel A. Saldo, Federco Barber Dpto. Sstemas Informátcos y Computacón Unversdad Poltécnca de Valenca, Camno de Vera s/n

More information

Yves Genin, Yurii Nesterov, Paul Van Dooren. CESAME, Universite Catholique de Louvain. B^atiment Euler, Avenue G. Lema^tre 4-6

Yves Genin, Yurii Nesterov, Paul Van Dooren. CESAME, Universite Catholique de Louvain. B^atiment Euler, Avenue G. Lema^tre 4-6 Submtted to ECC 99 as a regular paper n Lnear Systems Postve transfer functons and convex optmzaton 1 Yves Genn, Yur Nesterov, Paul Van Dooren CESAME, Unverste Catholque de Louvan B^atment Euler, Avenue

More information