# Support Vector Machines

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada Abstract Ths s a note to explan support vector machnes. 1 Prelmnares Our task s to predct whether a test sample belongs to one of two classes. We receve tranng examples of the form: {x, y }, = 1,..., n and x R d, y { 1, +1}. We call {x } the co-varates or nput vectors and {y } the response varables or labels. We consder a very smple example where the data are n fact lnearly separable:.e. I can draw a straght lne f(x) = w T x b such that all cases wth y = 1 fall on one sde and have f(x ) < 0 and cases wth y = +1 fall on the other and have f(x ) > 0. Gven that we have acheved that, we could classfy new test cases accordng to the rule y test = sgn(x test ). However, typcally there are nfntely many such hyper-planes obtaned by small perturbatons of a gven soluton. How do we choose between all these hyper-planes whch the solve the separaton problem for our tranng data, but may have dfferent performance on the newly arrvng test cases. For nstance, we could choose to put the lne very close to members of one partcular class, say y = 1. Intutvely, when test cases arrve we wll not make many mstakes on cases that should be classfed wth y = +1, but we wll make very easly mstakes on the cases wth y = 1 (for nstance, magne that a new batch of test cases arrves whch are small perturbatons of the tranng data). A sensble thng thus seems to choose the separaton lne as far away from both y = 1 and y = +1 tranng cases as we can,.e. rght n the mddle. Geometrcally, the vector w s drected orthogonal to the lne defned by w T x = b. Ths can be understood as follows. Frst take b = 0. Now t s clear that all vectors, x, wth vanshng nner product wth w satsfy ths equaton,.e. all vectors orthogonal to w satsfy ths equaton. Now translate the hyperplane away from the orgn over a vector a. The equaton for the plane now becomes: (x a) T w = 0,.e. we fnd that for the offset b = a T w, whch s the projecton of a onto to the vector w. Wthout loss of generalty we may thus choose a perpendcular to the plane, n whch case the length a = b / w represents the shortest, orthogonal dstance between the orgn and the hyperplane. We now defne 2 more hyperplanes parallel to the separatng hyperplane. They represent that planes that cut through the closest tranng examples on ether sde. We wll call them

2 support hyper-planes n the followng, because the data-vectors they contan support the plane. We defne the dstance between the these hyperplanes and the separatng hyperplane to be d + and d respectvely. The margn, γ, s defned to be d + + d. Our goal s now to fnd a the separatng hyperplane so that the margn s largest, whle the separatng hyperplane s equdstant from both. We can wrte the followng equatons for the support hyperplanes: w T x = b + δ (1) w T x = b δ (2) We now note that we have over-parameterzed the problem: f we scale w, b and δ by a constant factor α, the equatons for x are stll satsfed. To remove ths ambguty we wll requre that δ = 1, ths sets the scale of the problem,.e. f we measure dstance n mllmeters or meters. We can now also compute the values for d + = ( b+1 b )/ w = 1/ w (ths s only true f b / ( 1, 0) snce the orgn doesn t fall n between the hyperplanes n that case. If b ( 1, 0) you should use d + = ( b b )/ w = 1/ w ). Hence the margn s equal to twce that value: γ = 2/ w. Wth the above defnton of the support planes we can wrte down the followng constrant that any soluton must satsfy, w T x b 1 y = 1 (3) w T x b +1 y = +1 (4) or n one equaton, y (w T x b) 1 0 (5) We now formulate the prmal problem of the SVM: 1 mnmze 2 w 2 subject to y (w T x b) 1 0 (6) Thus, we maxmze the margn, subject to the constrants that all tranng cases fall on ether sde of the support hyper-planes. The data-cases that le on the hyperplane are called support vectors, snce they support the hyper-planes and hence determne the soluton to the problem. The prmal problem can be solved by a quadratc program. However, t s not ready to be kernelsed, because ts dependence s not only on nner products between data-vectors. Hence, we transform to the dual formulaton by frst wrtng the problem usng a Lagrangan, L(w, b, α) = 1 N [ 2 w 2 α y (w T x b) 1 ] (7) =1 The soluton that mnmzes the prmal problem subject to the constrants s gven by mn w max α L(w, α),.e. a saddle pont problem. When the orgnal objectve-functon s convex, (and only then), we can nterchange the mnmzaton and maxmzaton. Dong that, we fnd that we can fnd the condton on w that must hold at the saddle pont we are solvng for. Ths s done by takng dervatves wrt w and b and solvng, w α y x = 0 w = α y x (8) α y = 0 (9)

3 Insertng ths back nto the Lagrangan we obtan what s known as the dual problem, N maxmze L D = α 1 α α j y y j x T x j 2 =1 j subject to α y = 0 (10) α 0 (11) The dual formulaton of the problem s also a quadratc program, but note that the number of varables, α n ths problem s equal to the number of data-cases, N. The crucal pont s however, that ths problem only depends on x through the nner product x T x j. Ths s readly kernelsed through the substtuton x T x j k(x, x j ). Ths s a recurrent theme: the dual problem lends tself to kernelsaton, whle the prmal problem dd not. The theory of dualty guarantees that for convex problems, the dual problem wll be concave, and moreover, that the unque soluton of the prmal problem corresponds tot the unque soluton of the dual problem. In fact, we have: L P (w ) = L D (α ),.e. the dualty-gap s zero. Next we turn to the condtons that must necessarly hold at the saddle pont and thus the soluton of the problem. These are called the KKT condtons (whch stands for Karush- Kuhn-Tucker). These condtons are necessary n general, and suffcent for convex optmzaton problems. They can be derved from the prmal problem by settng the dervatves wrt to w to zero. Also, the constrants themselves are part of these condtons and we need that for nequalty constrants the Lagrange multplers are non-negatve. Fnally, an mportant constrant called complementary slackness needs to be satsfed, w L P = 0 w α y x = 0 (12) b L P = 0 α y = 0 (13) constrant - 1 y (w T x b) 1 0 (14) multpler condton α 0 (15) complementary slackness α [ y (w T x b) 1 ] = 0 (16) It s the last equaton whch may be somewhat surprsng. It states that ether the nequalty constrant s satsfed, but not saturated: y (w T x b) 1 > 0 n whch case α for that data-case must be zero, or the nequalty constrant s saturated y (w T x b) 1 = 0, n whch case α can be any value α 0. Inequalty constrants whch are saturated are sad to be actve, whle unsaturated constrants are nactve. One could magne the process of searchng for a soluton as a ball whch runs down the prmary objectve functon usng gradent descent. At some pont, t wll ht a wall whch s the constrant and although the dervatve s stll pontng partally towards the wall, the constrants prohbts the ball to go on. Ths s an actve constrant because the ball s glued to that wall. When a fnal soluton s reached, we could remove some constrants, wthout changng the soluton, these are nactve constrants. One could thnk of the term w L P as the force actng on the ball. We see from the frst equaton above that only the forces wth α 0 exsert a force on the ball that balances wth the force from the curved quadratc surface w. The tranng cases wth α > 0, representng actve constrants on the poston of the support hyperplane are called support vectors. These are the vectors that are stuated n the support hyperplane and they determne the soluton. Typcally, there are only few of them, whch people call a sparse soluton (most α s vansh).

4 What we are really nterested n s the functon f( ) whch can be used to classfy future test cases, f(x) = w T x b = α y x T x b (17) As an applcaton of the KKT condtons we derve a soluton for b by usng the complementary slackness condton, b = j α j y j x T j x y a support vector (18) where we used y 2 = 1. So, usng any support vector one can determne b, but for numercal stablty t s better to average over all of them (although they should obvously be consstent). The most mportant concluson s agan that ths functon f( ) can thus be expressed solely n terms of nner products x T x whch we can replace wth kernel matrces k(x, x j ) to move to hgh dmensonal non-lnear spaces. Moreover, snce α s typcally very sparse, we don t need to evaluate many kernel entres n order to predct the class of the new nput x. 2 The Non-Separable case Obvously, not all datasets are lnearly separable, and so we need to change the formalsm to account for that. Clearly, the problem les n the constrants, whch cannot always be satsfed. So, let s relax those constrants by ntroducng slack varables, ξ, w T x b 1 + ξ y = 1 (19) w T x b +1 ξ y = +1 (20) ξ 0 (21) The varables, ξ allow for volatons of the constrant. We should penalze the objectve functon for these volatons, otherwse the above constrants become vod (smply always pck ξ very large). Penalty functons of the form C( ξ ) k wll lead to convex optmzaton problems for postve ntegers k. For k = 1, 2 t s stll a quadratc program (QP). In the followng we wll choose k = 1. C controls the tradeoff between the penalty and margn. To be on the wrong sde of the separatng hyperplane, a data-case would need ξ > 1. Hence, the sum ξ could be nterpreted as measure of how bad the volatons are and s an upper bound on the number of volatons. The new prmal problem thus becomes, mnmze L P = 1 2 w 2 + C ξ leadng to the Lagrangan, subject to y (w T x b) 1 + ξ 0 (22) ξ 0 (23) L(w, b, ξ, α, µ) = 1 2 w 2 +C N [ ξ α y (w T ] N x b) 1 + ξ µ ξ (24) =1 =1

5 from whch we derve the KKT condtons, 1. w L P = 0 w α y x = 0 (25) 2. b L P = 0 α y = 0 (26) 3. ξ L P = 0 C α µ = 0 (27) 4.constrant-1 y (w T x b) 1 + ξ 0 (28) 5.constrant-2 ξ 0 (29) 6.multpler condton-1 α 0 (30) 7.multpler condton-2 µ 0 (31) 8.complementary slackness-1 [ α y (w T ] x b) 1 + ξ = 0 (32) 9.complementary slackness-1 µ ξ = 0 (33) (34) From here we can deduce the followng facts. If we assume that ξ > 0, then µ = 0 (9), hence α = C (1) and thus ξ = 1 y (x T w b) (8). Also, when ξ = 0 we have µ > 0 (9) and hence α < C. If n addton to ξ = 0 we also have that y (w T x b) 1 = 0, then α > 0 (8). Otherwse, f y (w T x b) 1 > 0 then α = 0. In summary, as before for ponts not on the support plane and on the correct sde we have ξ = α = 0 (all constrants nactve). On the support plane, we stll have ξ = 0, but now α > 0. Fnally, for data-cases on the wrong sde of the support hyperplane the α max-out to α = C and the ξ balance the volaton of the constrant such that y (w T x b) 1 + ξ = 0. Geometrcally, we can calculate the gap between support hyperplane and the volatng datacase to be ξ / w. Ths can be seen because the plane defned by y (w T x b) 1+ξ = 0 s parallel to the support plane at a dstance 1 + y b ξ / w from the orgn. Snce the support plane s at a dstance 1 + y b / w the result follows. Fnally, we need to convert to the dual problem to solve t effcently and to kernelse t. Agan, we use the KKT equatons to get rd of w, b and ξ, N maxmze L D = α 1 α α j y y j x T x j 2 =1 j subject to α y = 0 (35) 0 α C (36) Surprsngly, ths s almost the same QP s before, but wth an extra constrant on the multplers α whch now lve n a box. Ths constrant s derved from the fact that α = C µ and µ 0. We also note that t only depends on nner products x T x j whch are ready to be kernelsed.

### Recurrence. 1 Definitions and main statements

Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

### BERNSTEIN POLYNOMIALS

On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

### Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

### n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)

MATH 16T Exam 1 : Part I (In-Class) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total

### Moment of a force about a point and about an axis

3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should

### v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

### Solution of Algebraic and Transcendental Equations

CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton

### Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

### The example below solves a system in the unknowns α and β:

The Fnd Functon The functon Fnd returns a soluton to a system of equatons gven by a solve block. You can use Fnd to solve a lnear system, as wth lsolve, or to solve nonlnear systems. The example below

### Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2

18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest

### Fisher Markets and Convex Programs

Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

### 1 Example 1: Axis-aligned rectangles

COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton

### Least Squares Fitting of Data

Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2016. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng

### New Approaches to Support Vector Ordinal Regression

New Approaches to Support Vector Ordnal Regresson We Chu chuwe@gatsby.ucl.ac.uk Gatsby Computatonal Neuroscence Unt, Unversty College London, London, WCN 3AR, UK S. Sathya Keerth selvarak@yahoo-nc.com

### PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

### A Computer Technique for Solving LP Problems with Bounded Variables

Dhaka Unv. J. Sc. 60(2): 163-168, 2012 (July) A Computer Technque for Solvng LP Problems wth Bounded Varables S. M. Atqur Rahman Chowdhury * and Sanwar Uddn Ahmad Department of Mathematcs; Unversty of

### Lecture 2: Single Layer Perceptrons Kevin Swingler

Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCulloch-Ptts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses

### 8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

### Solution : (a) FALSE. Let C be a binary one-error correcting code of length 9. Then it follows from the Sphere packing bound that.

MATH 29T Exam : Part I Solutons. TRUE/FALSE? Prove your answer! (a) (5 pts) There exsts a bnary one-error correctng code of length 9 wth 52 codewords. (b) (5 pts) There exsts a ternary one-error correctng

### SVM Tutorial: Classification, Regression, and Ranking

SVM Tutoral: Classfcaton, Regresson, and Rankng Hwanjo Yu and Sungchul Km 1 Introducton Support Vector Machnes(SVMs) have been extensvely researched n the data mnng and machne learnng communtes for the

### 21 Vectors: The Cross Product & Torque

21 Vectors: The Cross Product & Torque Do not use our left hand when applng ether the rght-hand rule for the cross product of two vectors dscussed n ths chapter or the rght-hand rule for somethng curl

### Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits

Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.

### Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

### Production. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem.

Producer Theory Producton ASSUMPTION 2.1 Propertes of the Producton Set The producton set Y satsfes the followng propertes 1. Y s non-empty If Y s empty, we have nothng to talk about 2. Y s closed A set

### Support Vector Machine Model for Currency Crisis Discrimination. Arindam Chaudhuri 1. Abstract

Support Vector Machne Model for Currency Crss Dscrmnaton Arndam Chaudhur Abstract Support Vector Machne (SVM) s powerful classfcaton technque based on the dea of structural rsk mnmzaton. Use of kernel

### The eigenvalue derivatives of linear damped systems

Control and Cybernetcs vol. 32 (2003) No. 4 The egenvalue dervatves of lnear damped systems by Yeong-Jeu Sun Department of Electrcal Engneerng I-Shou Unversty Kaohsung, Tawan 840, R.O.C e-mal: yjsun@su.edu.tw

### Logistic Regression. Steve Kroon

Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro

### PLANAR GRAPHS. Plane graph (or embedded graph) A graph that is drawn on the plane without edge crossing, is called a Plane graph

PLANAR GRAPHS Basc defntons Isomorphc graphs Two graphs G(V,E) and G2(V2,E2) are somorphc f there s a one-to-one correspondence F of ther vertces such that the followng holds: - u,v V, uv E, => F(u)F(v)

### Solutions to the exam in SF2862, June 2009

Solutons to the exam n SF86, June 009 Exercse 1. Ths s a determnstc perodc-revew nventory model. Let n = the number of consdered wees,.e. n = 4 n ths exercse, and r = the demand at wee,.e. r 1 = r = r

### A Probabilistic Theory of Coherence

A Probablstc Theory of Coherence BRANDEN FITELSON. The Coherence Measure C Let E be a set of n propostons E,..., E n. We seek a probablstc measure C(E) of the degree of coherence of E. Intutvely, we want

### 6. EIGENVALUES AND EIGENVECTORS 3 = 3 2

EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a non-zero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :

### A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

### b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei

Mathematcal Propertes of the Least Squares Regresson The least squares regresson lne obeys certan mathematcal propertes whch are useful to know n practce. The followng propertes can be establshed algebracally:

### 1 Approximation Algorithms

CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

### Introduction: Analysis of Electronic Circuits

/30/008 ntroducton / ntroducton: Analyss of Electronc Crcuts Readng Assgnment: KVL and KCL text from EECS Just lke EECS, the majorty of problems (hw and exam) n EECS 3 wll be crcut analyss problems. Thus,

### Ping Pong Fun - Video Analysis Project

Png Pong Fun - Vdeo Analyss Project Objectve In ths experment we are gong to nvestgate the projectle moton of png pong balls usng Verner s Logger Pro Software. Does the object travel n a straght lne? What

### QUANTUM MECHANICS, BRAS AND KETS

PH575 SPRING QUANTUM MECHANICS, BRAS AND KETS The followng summares the man relatons and defntons from quantum mechancs that we wll be usng. State of a phscal sstem: The state of a phscal sstem s represented

### J. Parallel Distrib. Comput.

J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

### Generalizing the degree sequence problem

Mddlebury College March 2009 Arzona State Unversty Dscrete Mathematcs Semnar The degree sequence problem Problem: Gven an nteger sequence d = (d 1,...,d n ) determne f there exsts a graph G wth d as ts

### Chapter 4 ECONOMIC DISPATCH AND UNIT COMMITMENT

Chapter 4 ECOOMIC DISATCH AD UIT COMMITMET ITRODUCTIO A power system has several power plants. Each power plant has several generatng unts. At any pont of tme, the total load n the system s met by the

### Optimal portfolios using Linear Programming models

Optmal portfolos usng Lnear Programmng models Chrstos Papahrstodoulou Mälardalen Unversty, Västerås, Sweden Abstract The classcal Quadratc Programmng formulaton of the well known portfolo selecton problem,

### Loop Parallelization

- - Loop Parallelzaton C-52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I-,J]+B[I-,J-] ED FOR ED FOR analyze

### greatest common divisor

4. GCD 1 The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no

### L10: Linear discriminants analysis

L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss

### The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets

. The Magnetc Feld Concepts and Prncples Movng Charges All charged partcles create electrc felds, and these felds can be detected by other charged partcles resultng n electrc force. However, a completely

### State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness

Schroednger equaton Basc postulates of quantum mechancs. Operators: Hermtan operators, commutators State functon: egenfunctons of hermtan operators-> normalzaton, orthogonalty completeness egenvalues and

### Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

### Forecasting the Direction and Strength of Stock Market Movement

Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

### where the coordinates are related to those in the old frame as follows.

Chapter 2 - Cartesan Vectors and Tensors: Ther Algebra Defnton of a vector Examples of vectors Scalar multplcaton Addton of vectors coplanar vectors Unt vectors A bass of non-coplanar vectors Scalar product

### A linear recurrence sequence of composite numbers

LMS J Comput Math 15 (2012) 360 373 C 2012 Author do:101112/s1461157012001143 A lnear recurrence sequence of composte numbers Jonas Šurys Abstract We prove that for each postve nteger k n the range 2 k

### 8 Algorithm for Binary Searching in Trees

8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

### The Mathematical Derivation of Least Squares

Pscholog 885 Prof. Federco The Mathematcal Dervaton of Least Squares Back when the powers that e forced ou to learn matr algera and calculus, I et ou all asked ourself the age-old queston: When the hell

### Math 131: Homework 4 Solutions

Math 3: Homework 4 Solutons Greg Parker, Wyatt Mackey, Chrstan Carrck October 6, 05 Problem (Munkres 3.) Let {A n } be a sequence of connected subspaces of X such that A n \ A n+ 6= ; for all n. Then S

### Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)

Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton

### 2.4 Bivariate distributions

page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together

### II. PROBABILITY OF AN EVENT

II. PROBABILITY OF AN EVENT As ndcated above, probablty s a quantfcaton, or a mathematcal model, of a random experment. Ths quantfcaton s a measure of the lkelhood that a gven event wll occur when the

### benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

### The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

### EE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN

EE201 Crcut Theory I 2015 Sprng Dr. Yılmaz KALKAN 1. Basc Concepts (Chapter 1 of Nlsson - 3 Hrs.) Introducton, Current and Voltage, Power and Energy 2. Basc Laws (Chapter 2&3 of Nlsson - 6 Hrs.) Voltage

### Formulating & Solving Integer Problems Chapter 11 289

Formulatng & Solvng Integer Problems Chapter 11 289 The Optonal Stop TSP If we drop the requrement that every stop must be vsted, we then get the optonal stop TSP. Ths mght correspond to a ob sequencng

### Graph Theory and Cayley s Formula

Graph Theory and Cayley s Formula Chad Casarotto August 10, 2006 Contents 1 Introducton 1 2 Bascs and Defntons 1 Cayley s Formula 4 4 Prüfer Encodng A Forest of Trees 7 1 Introducton In ths paper, I wll

### INTRODUCTION. governed by a differential equation Need systematic approaches to generate FE equations

WEIGHTED RESIDUA METHOD INTRODUCTION Drect stffness method s lmted for smple D problems PMPE s lmted to potental problems FEM can be appled to many engneerng problems that are governed by a dfferental

### PERRON FROBENIUS THEOREM

PERRON FROBENIUS THEOREM R. CLARK ROBINSON Defnton. A n n matrx M wth real entres m, s called a stochastc matrx provded () all the entres m satsfy 0 m, () each of the columns sum to one, m = for all, ()

### Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting

Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of

### Week 6 Market Failure due to Externalities

Week 6 Market Falure due to Externaltes 1. Externaltes n externalty exsts when the acton of one agent unavodably affects the welfare of another agent. The affected agent may be a consumer, gvng rse to

### 7.5. Present Value of an Annuity. Investigate

7.5 Present Value of an Annuty Owen and Anna are approachng retrement and are puttng ther fnances n order. They have worked hard and nvested ther earnngs so that they now have a large amount of money on

### On the Solution of Indefinite Systems Arising in Nonlinear Optimization

On the Soluton of Indefnte Systems Arsng n Nonlnear Optmzaton Slva Bonettn, Valera Ruggero and Federca Tnt Dpartmento d Matematca, Unverstà d Ferrara Abstract We consder the applcaton of the precondtoned

### OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL. Thomas S. Ferguson and C. Zachary Gilstein UCLA and Bell Communications May 1985, revised 2004

OPTIMAL INVESTMENT POLICIES FOR THE HORSE RACE MODEL Thomas S. Ferguson and C. Zachary Glsten UCLA and Bell Communcatons May 985, revsed 2004 Abstract. Optmal nvestment polces for maxmzng the expected

### Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect

Chapter 3 Group Theory p. - 3. Compact Course: Groups Theory emark: Ths s only a bref summary of most mportant results of groups theory wth respect to the applcatons dscussed n the followng chapters. For

### What is Candidate Sampling

What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

### Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier

Lesson 2 Chapter Two Three Phase Uncontrolled Rectfer. Operatng prncple of three phase half wave uncontrolled rectfer The half wave uncontrolled converter s the smplest of all three phase rectfer topologes.

### Support vector domain description

Pattern Recognton Letters 20 (1999) 1191±1199 www.elsever.nl/locate/patrec Support vector doman descrpton Davd M.J. Tax *,1, Robert P.W. Dun Pattern Recognton Group, Faculty of Appled Scence, Delft Unversty

### The OC Curve of Attribute Acceptance Plans

The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

### z(t) = z 1 (t) + t(z 2 z 1 ) z(t) = 1 + i + t( 2 3i (1 + i)) z(t) = 1 + i + t( 3 4i); 0 t 1

(4.): ontours. Fnd an admssble parametrzaton. (a). the lne segment from z + to z 3. z(t) z (t) + t(z z ) z(t) + + t( 3 ( + )) z(t) + + t( 3 4); t (b). the crcle jz j 4 traversed once clockwse startng at

### Simple Interest Loans (Section 5.1) :

Chapter 5 Fnance The frst part of ths revew wll explan the dfferent nterest and nvestment equatons you learned n secton 5.1 through 5.4 of your textbook and go through several examples. The second part

### Solution: Let i = 10% and d = 5%. By definition, the respective forces of interest on funds A and B are. i 1 + it. S A (t) = d (1 dt) 2 1. = d 1 dt.

Chapter 9 Revew problems 9.1 Interest rate measurement Example 9.1. Fund A accumulates at a smple nterest rate of 10%. Fund B accumulates at a smple dscount rate of 5%. Fnd the pont n tme at whch the forces

### + + + - - This circuit than can be reduced to a planar circuit

MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to

### Performance Analysis and Coding Strategy of ECOC SVMs

Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.67-76 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School

### x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

### SCALAR A physical quantity that is completely characterized by a real number (or by its numerical value) is called a scalar. In other words, a scalar

SCALAR A phscal quantt that s completel charactered b a real number (or b ts numercal value) s called a scalar. In other words, a scalar possesses onl a magntude. Mass, denst, volume, temperature, tme,

### Multivariate EWMA Control Chart

Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant

### Level Annuities with Payments Less Frequent than Each Interest Period

Level Annutes wth Payments Less Frequent than Each Interest Perod 1 Annuty-mmedate 2 Annuty-due Level Annutes wth Payments Less Frequent than Each Interest Perod 1 Annuty-mmedate 2 Annuty-due Symoblc approach

### FORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER

FORCED CONVECION HEA RANSFER IN A DOUBLE PIPE HEA EXCHANGER Dr. J. Mchael Doster Department of Nuclear Engneerng Box 7909 North Carolna State Unversty Ralegh, NC 27695-7909 Introducton he convectve heat

### Financial market forecasting using a two-step kernel learning method for the support vector regression

Ann Oper Res (2010) 174: 103 120 DOI 10.1007/s10479-008-0357-7 Fnancal market forecastng usng a two-step kernel learnng method for the support vector regresson L Wang J Zhu Publshed onlne: 28 May 2008

### EXPLORATION 2.5A Exploring the motion diagram of a dropped object

-5 Acceleraton Let s turn now to moton that s not at constant elocty. An example s the moton of an object you release from rest from some dstance aboe the floor. EXPLORATION.5A Explorng the moton dagram

### Supplementary material: Assessing the relevance of node features for network structure

Supplementary materal: Assessng the relevance of node features for network structure Gnestra Bancon, 1 Paolo Pn,, 3 and Matteo Marsl 1 1 The Abdus Salam Internatonal Center for Theoretcal Physcs, Strada

### Communication Networks II Contents

8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

### MANY machine learning and pattern recognition applications

1 Trace Rato Problem Revsted Yangqng Ja, Fepng Ne, and Changshu Zhang Abstract Dmensonalty reducton s an mportant ssue n many machne learnng and pattern recognton applcatons, and the trace rato problem

### Downlink Power Allocation for Multi-class. Wireless Systems

Downlnk Power Allocaton for Mult-class 1 Wreless Systems Jang-Won Lee, Rav R. Mazumdar, and Ness B. Shroff School of Electrcal and Computer Engneerng Purdue Unversty West Lafayette, IN 47907, USA {lee46,

### Period and Deadline Selection for Schedulability in Real-Time Systems

Perod and Deadlne Selecton for Schedulablty n Real-Tme Systems Thdapat Chantem, Xaofeng Wang, M.D. Lemmon, and X. Sharon Hu Department of Computer Scence and Engneerng, Department of Electrcal Engneerng

### Semiconductor sensors of temperature

Semconductor sensors of temperature he measurement objectve 1. Identfy the unknown bead type thermstor. Desgn the crcutry for lnearzaton of ts transfer curve.. Fnd the dependence of forward voltage drop

### Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006

Aryabhata s Root Extracton Methods Abhshek Parakh Lousana State Unversty Aug 1 st 1 Introducton Ths artcle presents an analyss of the root extracton algorthms of Aryabhata gven n hs book Āryabhatīya [1,

### Properties of American Derivative Securities

Capter 6 Propertes of Amercan Dervatve Securtes 6.1 Te propertes Defnton 6.1 An Amercan dervatve securty s a sequence of non-negatve random varables fg k g n k= suc tat eac G k s F k -measurable. Te owner

### Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy

4.02 Quz Solutons Fall 2004 Multple-Choce Questons (30/00 ponts) Please, crcle the correct answer for each of the followng 0 multple-choce questons. For each queston, only one of the answers s correct.

### VLSI Technology Dr. Nandita Dasgupta Department of Electrical Engineering Indian Institute of Technology, Madras

VLI Technology Dr. Nandta Dasgupta Department of Electrcal Engneerng Indan Insttute of Technology, Madras Lecture - 11 Oxdaton I netcs of Oxdaton o, the unt process step that we are gong to dscuss today

### Ring structure of splines on triangulations

www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon

### Linear Regression, Regularization Bias-Variance Tradeoff

HTF: Ch3, 7 B: Ch3 Lnear Regresson, Regularzaton Bas-Varance Tradeoff Thanks to C Guestrn, T Detterch, R Parr, N Ray 1 Outlne Lnear Regresson MLE = Least Squares! Bass functons Evaluatng Predctors Tranng

### U.C. Berkeley CS270: Algorithms Lecture 4 Professor Vazirani and Professor Rao Jan 27,2011 Lecturer: Umesh Vazirani Last revised February 10, 2012

U.C. Berkeley CS270: Algorthms Lecture 4 Professor Vazran and Professor Rao Jan 27,2011 Lecturer: Umesh Vazran Last revsed February 10, 2012 Lecture 4 1 The multplcatve weghts update method The multplcatve