L10: Linear discriminants analysis

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "L10: Linear discriminants analysis"

Transcription

1 L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna

2 Lnear dscrmnant analyss, two-classes Objectve LDA seeks to reduce dmensonalty whle preservng as much of the class dscrmnatory nformaton as possble Assume we have a set of D-dmensonal samples x (, x (, x (N, N of whch belong to class ω, and N to class ω We seek to obtan a scalar y by projectng the samples x onto a lne y = w T x Of all the possble lnes we would lke to select the one that maxmzes the separablty of the scalars x x x CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna x

3 In order to fnd a good projecton vector, we need to defne a measure of separaton The mean vector of each class n x-space and y-space s μ = N x ω x and μ = N y ω y = N x ω w T x = w T μ We could then choose the dstance between the projected means as our objectve functon J w = μ μ = w T μ μ However, the dstance between projected means s not a good measure snce t does not account for the standard devaton wthn classes x Ths axs yelds better class separablty x Ths axs has a larger dstance between means CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna

4 Fsher s soluton Fsher suggested maxmzng the dfference between the means, normalzed by a measure of the wthn-class scatter For each class we defne the scatter, an equvalent of the varance, as s = y μ y ω where the quantty s + s s called the wthn-class scatter of the projected examples The Fsher lnear dscrmnant s defned as the lnear functon w T x that maxmzes the crteron functon J w = μ μ s +s Therefore, we are lookng for a projecton where examples from the same class are projected very close to each other and, at the same tme, the projected means are as farther apart as possble x x CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna

5 To fnd the optmum w, we must express J(w) as a functon of w Frst, we defne a measure of the scatter n feature space x T S = x ω x μ x μ S + S = S W where S W s called the wthn-class scatter matrx The scatter of the projecton y can then be expressed as a functon of the scatter matrx n feature space x s = y ω y μ = w T x w T x ω μ = = x ω w T x μ x μ T w = w T S w s + s = w T S W w Smlarly, the dfference between the projected means can be expressed n terms of the means n the orgnal feature space μ μ = w T μ w T μ = w T T μ μ μ μ w = w T S B w The matrx S B s called the between-class scatter. Note that, snce S B s the outer product of two vectors, ts rank s at most one We can fnally express the Fsher crteron n terms of S W and S B as J w = wt S B w w T S W w CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna S B

6 To fnd the maxmum of J(w) we derve and equate to zero d dw J w = d dw w T S W w d wt S B w w T S B w w T S W w = 0 w T S dw B w d wt S W w dw w T S W w S B w w T S B w S W w = 0 Dvdng by w T S W w = 0 w T S W w S w T S W w Bw wt S B w S w T S W w Ww = 0 S B w JS W w = 0 S W S B w Jw = 0 Solvng the generalzed egenvalue problem (S W S B w = Jw) yelds w = arg max wt S B w w T S W w = S W μ μ Ths s know as Fsher s lnear dscrmnant (96), although t s not a dscrmnant but rather a specfc choce of drecton for the projecton of the data down to one dmenson CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna 6

7 Example Compute the LDA projecton for the followng D dataset X = {(,), (,), (,), (,6), (,)} X = {(9,0), (6,8), (9,), (8,7), (0,8)} SOLUTION (by hand) The class statstcs are S =.8..6 S = x w LDA μ =.0.6 T ; μ = T The wthn- and between-class scatter are S B = S W =.6..8 The LDA projecton s then obtaned as the soluton of the generalzed egenvalue problem S W S B v = λv S W S B λi = Or drectly by w = S W μ μ v v =.6 v v v v =.9.9 =.9.9 T.89 λ λ x = 0 λ =.6 CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna 7

8 LDA, C classes Fsher s LDA generalzes gracefully for C-class problems Instead of one projecton y, we wll now seek (C ) projectons [y, y, y C ] by means of (C ) projecton vectors w arranged by columns nto a projecton matrx W = [w w w C ]: y = w T x y = W T x Dervaton x The wthn-class scatter generalzes as S W = C = S where S = x μ x μ T x ω and μ = N x ω x And the between-class scatter becomes S B = N μ μ μ μ T C = where μ = N x x = N C = N μ Matrx S T = S B + S W s called the total scatter S W S W S B S B S B S W x CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna 8

9 Smlarly, we defne the mean vector and scatter matrces for the projected samples as μ = N μ = N C = T y ω y S W = y μ y μ y ω y y S B = C = N μ μ μ μ T From our dervaton for the two-class problem, we can wrte S W = W T S W W S B = W T S B W Recall that we are lookng for a projecton that maxmzes the rato of between-class to wthn-class scatter. Snce the projecton s no longer a scalar (t has C dmensons), we use the determnant of the scatter matrces to obtan a scalar objectve functon J W = S B S W = WT S B W W T S W W And we wll seek the projecton matrx W that maxmzes ths rato CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna 9

10 It can be shown that the optmal projecton matrx W s the one whose columns are the egenvectors correspondng to the largest egenvalues of the followng generalzed egenvalue problem NOTES W = w w w C = arg max WT S B W W T S W W S B λ S W w = 0 S B s the sum of C matrces of rank and the mean vectors are constraned by C C = μ = μ Therefore, S B wll be of rank (C ) or less Ths means that only (C ) of the egenvalues λ wll be non-zero The projectons wth maxmum class separablty nformaton are the egenvectors correspondng to the largest egenvalues of S W S B LDA can be derved as the Maxmum Lkelhood method for the case of normal class-condtonal denstes wth equal covarance matrces CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna 0

11 LDA vs. PCA Ths example llustrates the performance of PCA and LDA on an odor recognton problem Fve types of coffee beans were presented to an array of gas sensors For each coffee type, snffs were performed and the response of the gas sensor array was processed n order to obtan a 60-dmensonal feature vector Results From the D scatter plots t s clear that LDA outperforms PCA n terms of class dscrmnaton Ths s one example where the dscrmnatory nformaton s not algned wth the drecton of maxmum varance PCA Sensor response normalzed data Sulaw esy Kenya Araban Sumatra Colomba LDA axs axs 00 axs axs 0. axs axs CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna

12 Lmtatons of LDA LDA produces at most C feature projectons If the classfcaton error estmates establsh that more features are needed, some other method must be employed to provde those addtonal features LDA s a parametrc method (t assumes unmodal Gaussan lkelhoods) If the dstrbutons are sgnfcantly non-gaussan, the LDA projectons may not preserve complex structure n the data needed for classfcaton = = = = x LDA wll also fal f dscrmnatory nformaton s not n the mean but n the varance of the data P C A L D A x CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna

13 Varants of LDA Non-parametrc LDA (Fukunaga) NPLDA relaxes the unmodal Gaussan assumpton by computng S B usng local nformaton and the knn rule. As a result of ths The matrx S B s full-rank, allowng us to extract more than (C ) features The projectons are able to preserve the structure of the data more closely Orthonormal LDA (Okada and Tomta) OLDA computes projectons that maxmze the Fsher crteron and, at the same tme, are par-wse orthonormal The method used n OLDA combnes the egenvalue soluton of S W S B and the Gram- Schmdt orthonormalzaton procedure OLDA sequentally fnds axes that maxmze the Fsher crteron n the subspace orthogonal to all features already extracted OLDA s also capable of fndng more than (C ) features Generalzed LDA (Lowe) GLDA generalzes the Fsher crteron by ncorporatng a cost functon smlar to the one we used to compute the Bayes Rsk As a result, LDA can produce projects that are based by the cost functon,.e., classes wth a hgher cost C j wll be placed further apart n the low-dmensonal projecton Multlayer perceptrons (Webb and Lowe) It has been shown that the hdden layers of mult-layer perceptrons perform nonlnear dscrmnant analyss by maxmzng Tr[S B S T ], where the scatter matrces are measured at the output of the last hdden layer CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna

14 Other dmensonalty reducton methods Exploratory Projecton Pursut (Fredman and Tukey) EPP seeks an M-dmensonal (M=, typcally) lnear projecton of the data that maxmzes a measure of nterestngness Interestngness s measured as departure from multvarate normalty Ths measure s not the varance and s commonly scale-free. In most mplementatons t s also affne nvarant, so t does not depend on correlatons between features. [Rpley, 996] In other words, EPP seeks projectons that separate clusters as much as possble and keeps these clusters compact, a smlar crteron as Fsher s, but EPP does NOT use class labels Once an nterestng projecton s found, t s mportant to remove the structure t reveals to allow other nterestng vews to be found more easly Unnterestng Interestng x x x CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna x

15 Sammon s non-lnear mappng (Sammon) Ths method seeks a mappng onto an M-dmensonal space that preserves the nter-pont dstances n the orgnal N-dmensonal space Ths s accomplshed by mnmzng the followng objectve functon E d, d = j d P,P j d P,Pj d P,P j The orgnal method dd not obtan an explct mappng but only a lookup table for the elements n the tranng set Newer mplementatons based on neural networks do provde an explct mappng for test data and also consder cost functons (e.g., Neuroscale) Sammon s mappng s closely related to Mult Dmensonal Scalng (MDS), a famly of multvarate statstcal methods commonly used n the socal scences We wll revew MDS technques when we cover manfold learnng x x P j P j d(p, P j )= d(p, P j ) ",j P P x x x CSCE 666 Pattern Analyss Rcardo Guterrez-Osuna

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching) Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton

More information

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there

More information

Nonlinear data mapping by neural networks

Nonlinear data mapping by neural networks Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal

More information

v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

v a 1 b 1 i, a 2 b 2 i,..., a n b n i. SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

More information

LETTER IMAGE RECOGNITION

LETTER IMAGE RECOGNITION LETTER IMAGE RECOGNITION 1. Introducton. 1. Introducton. Objectve: desgn classfers for letter mage recognton. consder accuracy and tme n takng the decson. 20,000 samples: Startng set: mages based on 20

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

Lecture 5,6 Linear Methods for Classification. Summary

Lecture 5,6 Linear Methods for Classification. Summary Lecture 5,6 Lnear Methods for Classfcaton Rce ELEC 697 Farnaz Koushanfar Fall 2006 Summary Bayes Classfers Lnear Classfers Lnear regresson of an ndcator matrx Lnear dscrmnant analyss (LDA) Logstc regresson

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei

b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei Mathematcal Propertes of the Least Squares Regresson The least squares regresson lne obeys certan mathematcal propertes whch are useful to know n practce. The followng propertes can be establshed algebracally:

More information

Logistic Regression. Steve Kroon

Logistic Regression. Steve Kroon Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro

More information

MANY machine learning and pattern recognition applications

MANY machine learning and pattern recognition applications 1 Trace Rato Problem Revsted Yangqng Ja, Fepng Ne, and Changshu Zhang Abstract Dmensonalty reducton s an mportant ssue n many machne learnng and pattern recognton applcatons, and the trace rato problem

More information

Using Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions

Using Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions Usng Mxture Covarance Matrces to Improve Face and Facal Expresson Recogntons Carlos E. homaz, Duncan F. Glles and Raul Q. Fetosa 2 Imperal College of Scence echnology and Medcne, Department of Computng,

More information

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of

More information

Multivariate EWMA Control Chart

Multivariate EWMA Control Chart Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant

More information

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001. Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.

More information

Single and multiple stage classifiers implementing logistic discrimination

Single and multiple stage classifiers implementing logistic discrimination Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul - PUCRS Av. Ipranga,

More information

Table of Contents EQ.10...46 EQ.6...46 EQ.8...46

Table of Contents EQ.10...46 EQ.6...46 EQ.8...46 Table of Contents CHAPTER II - PATTERN RECOGNITION.... THE PATTERN RECOGNITION PROBLEM.... STATISTICAL FORMULATION OF CLASSIFIERS...6 3. CONCLUSIONS...30 UNDERSTANDING BAYES RULE...3 BAYESIAN THRESHOLD...33

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.

More information

Georey E. Hinton. University oftoronto. Email: zoubin@cs.toronto.edu. Technical Report CRG-TR-96-1. May 21, 1996 (revised Feb 27, 1997) Abstract

Georey E. Hinton. University oftoronto. Email: zoubin@cs.toronto.edu. Technical Report CRG-TR-96-1. May 21, 1996 (revised Feb 27, 1997) Abstract The EM Algorthm for Mxtures of Factor Analyzers Zoubn Ghahraman Georey E. Hnton Department of Computer Scence Unversty oftoronto 6 Kng's College Road Toronto, Canada M5S A4 Emal: zoubn@cs.toronto.edu Techncal

More information

THE TITANIC SHIPWRECK: WHO WAS

THE TITANIC SHIPWRECK: WHO WAS THE TITANIC SHIPWRECK: WHO WAS MOST LIKELY TO SURVIVE? A STATISTICAL ANALYSIS Ths paper examnes the probablty of survvng the Ttanc shpwreck usng lmted dependent varable regresson analyss. Ths appled analyss

More information

Learning from Multiple Outlooks

Learning from Multiple Outlooks Learnng from Multple Outlooks Maayan Harel Department of Electrcal Engneerng, Technon, Hafa, Israel She Mannor Department of Electrcal Engneerng, Technon, Hafa, Israel maayanga@tx.technon.ac.l she@ee.technon.ac.l

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

Using mixture covariance matrices to improve face and facial expression recognitions

Using mixture covariance matrices to improve face and facial expression recognitions Pattern Recognton Letters 24 (2003) 2159 2165 www.elsever.com/locate/patrec Usng mxture covarance matrces to mprove face and facal expresson recogntons Carlos E. Thomaz a, *, Duncan F. Glles a, Raul Q.

More information

QUANTUM MECHANICS, BRAS AND KETS

QUANTUM MECHANICS, BRAS AND KETS PH575 SPRING QUANTUM MECHANICS, BRAS AND KETS The followng summares the man relatons and defntons from quantum mechancs that we wll be usng. State of a phscal sstem: The state of a phscal sstem s represented

More information

Loop Parallelization

Loop Parallelization - - Loop Parallelzaton C-52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I-,J]+B[I-,J-] ED FOR ED FOR analyze

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

Moment of a force about a point and about an axis

Moment of a force about a point and about an axis 3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should

More information

ONE of the most crucial problems that every image

ONE of the most crucial problems that every image IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 10, OCTOBER 2014 4413 Maxmum Margn Projecton Subspace Learnng for Vsual Data Analyss Symeon Nktds, Anastasos Tefas, Member, IEEE, and Ioanns Ptas, Fellow,

More information

Naïve Bayes classifier & Evaluation framework

Naïve Bayes classifier & Evaluation framework Lecture aïve Bayes classfer & Evaluaton framework Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Generatve approach to classfcaton Idea:. Represent and learn the dstrbuton p x, y. Use t to defne probablstc

More information

Data Visualization by Pairwise Distortion Minimization

Data Visualization by Pairwise Distortion Minimization Communcatons n Statstcs, Theory and Methods 34 (6), 005 Data Vsualzaton by Parwse Dstorton Mnmzaton By Marc Sobel, and Longn Jan Lateck* Department of Statstcs and Department of Computer and Informaton

More information

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

Kernel Methods for General Pattern Analysis

Kernel Methods for General Pattern Analysis Kernel Methods for General Pattern Analyss Nello Crstann Unversty of Calforna, Davs nello@support-vector.net Overvew Kernel Methods are a new class of pattern analyss algorthms whch can operate on very

More information

Optimal portfolios using Linear Programming models

Optimal portfolios using Linear Programming models Optmal portfolos usng Lnear Programmng models Chrstos Papahrstodoulou Mälardalen Unversty, Västerås, Sweden Abstract The classcal Quadratc Programmng formulaton of the well known portfolo selecton problem,

More information

Lecture 18: Clustering & classification

Lecture 18: Clustering & classification O CPS260/BGT204. Algorthms n Computatonal Bology October 30, 2003 Lecturer: Pana K. Agarwal Lecture 8: Clusterng & classfcaton Scrbe: Daun Hou Open Problem In HomeWor 2, problem 5 has an open problem whch

More information

Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect

Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect Chapter 3 Group Theory p. - 3. Compact Course: Groups Theory emark: Ths s only a bref summary of most mportant results of groups theory wth respect to the applcatons dscussed n the followng chapters. For

More information

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

More information

Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering

Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering Out-of-Sample Extensons for LLE, Isomap, MDS, Egenmaps, and Spectral Clusterng Yoshua Bengo, Jean-Franços Paement, Pascal Vncent Olver Delalleau, Ncolas Le Roux and Mare Oumet Département d Informatque

More information

Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2

Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2 18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest

More information

Estimating the Number of Clusters in Genetics of Acute Lymphoblastic Leukemia Data

Estimating the Number of Clusters in Genetics of Acute Lymphoblastic Leukemia Data Journal of Al Azhar Unversty-Gaza (Natural Scences), 2011, 13 : 109-118 Estmatng the Number of Clusters n Genetcs of Acute Lymphoblastc Leukema Data Mahmoud K. Okasha, Khaled I.A. Almghar Department of

More information

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2 EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a non-zero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :

More information

+ + + - - This circuit than can be reduced to a planar circuit

+ + + - - This circuit than can be reduced to a planar circuit MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to

More information

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.

More information

Solution of Algebraic and Transcendental Equations

Solution of Algebraic and Transcendental Equations CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton

More information

Brigid Mullany, Ph.D University of North Carolina, Charlotte

Brigid Mullany, Ph.D University of North Carolina, Charlotte Evaluaton And Comparson Of The Dfferent Standards Used To Defne The Postonal Accuracy And Repeatablty Of Numercally Controlled Machnng Center Axes Brgd Mullany, Ph.D Unversty of North Carolna, Charlotte

More information

Bypassing Synthesis: PLS for Face Recognition with Pose, Low-Resolution and Sketch

Bypassing Synthesis: PLS for Face Recognition with Pose, Low-Resolution and Sketch Bypassng Synthess: PLS for Face Recognton wth Pose, Low-Resoluton and Setch Abhshe Sharma Insttute of Advanced Computer Scence Unversty of Maryland, USA bhoaal@umacs.umd.edu Davd W Jacobs Insttute of Advanced

More information

Statistical Methods to Develop Rating Models

Statistical Methods to Develop Rating Models Statstcal Methods to Develop Ratng Models [Evelyn Hayden and Danel Porath, Österrechsche Natonalbank and Unversty of Appled Scences at Manz] Source: The Basel II Rsk Parameters Estmaton, Valdaton, and

More information

Estimation and Robustness of Linear Mixed Models in Credibility Context

Estimation and Robustness of Linear Mixed Models in Credibility Context Estmaton and Robustness of Lnear Mxed Models n Credblty Context by Wng Kam Fung and Xao Chen Xu ABSTRACT In ths paper, lnear mxed models are employed for estmaton of structural parameters n credblty context.

More information

Learning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing

Learning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing Internatonal Journal of Machne Learnng and Computng, Vol. 4, No. 3, June 04 Learnng from Large Dstrbuted Data: A Scalng Down Samplng Scheme for Effcent Data Processng Che Ngufor and Janusz Wojtusak part

More information

Can Auto Liability Insurance Purchases Signal Risk Attitude?

Can Auto Liability Insurance Purchases Signal Risk Attitude? Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159-164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? Chu-Shu L Department of Internatonal Busness, Asa Unversty, Tawan Sheng-Chang

More information

Questions that we may have about the variables

Questions that we may have about the variables Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent

More information

Image as a Feature Vector. Eigenfaces: linear projection. Announcements

Image as a Feature Vector. Eigenfaces: linear projection. Announcements Announcements Face Recognton: Dmensonalty Reducton & Unconstraned Face Rec Bometrcs CSE 190 Lecture 14 H3 assgned, due on hursday. Last H fngerprnt recognton to be assgned next week. hursday guest lecture

More information

where the coordinates are related to those in the old frame as follows.

where the coordinates are related to those in the old frame as follows. Chapter 2 - Cartesan Vectors and Tensors: Ther Algebra Defnton of a vector Examples of vectors Scalar multplcaton Addton of vectors coplanar vectors Unt vectors A bass of non-coplanar vectors Scalar product

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

Time Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University

Time Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton

More information

The eigenvalue derivatives of linear damped systems

The eigenvalue derivatives of linear damped systems Control and Cybernetcs vol. 32 (2003) No. 4 The egenvalue dervatves of lnear damped systems by Yeong-Jeu Sun Department of Electrcal Engneerng I-Shou Unversty Kaohsung, Tawan 840, R.O.C e-mal: yjsun@su.edu.tw

More information

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

More information

General Iteration Algorithm for Classification Ratemaking

General Iteration Algorithm for Classification Ratemaking General Iteraton Algorthm for Classfcaton Ratemakng by Luyang Fu and Cheng-sheng eter Wu ABSTRACT In ths study, we propose a flexble and comprehensve teraton algorthm called general teraton algorthm (GIA)

More information

I. SCOPE, APPLICABILITY AND PARAMETERS Scope

I. SCOPE, APPLICABILITY AND PARAMETERS Scope D Executve Board Annex 9 Page A/R ethodologcal Tool alculaton of the number of sample plots for measurements wthn A/R D project actvtes (Verson 0) I. SOPE, PIABIITY AD PARAETERS Scope. Ths tool s applcable

More information

PRACTICE 1: MUTUAL FUNDS EVALUATION USING MATLAB.

PRACTICE 1: MUTUAL FUNDS EVALUATION USING MATLAB. PRACTICE 1: MUTUAL FUNDS EVALUATION USING MATLAB. INDEX 1. Load data usng the Edtor wndow and m-fle 2. Learnng to save results from the Edtor wndow. 3. Computng the Sharpe Rato 4. Obtanng the Treynor Rato

More information

Imperial College London

Imperial College London F. Fang 1, C.C. Pan 1, I.M. Navon 2, M.D. Pggott 1, G.J. Gorman 1, P.A. Allson 1 and A.J.H. Goddard 1 1 Appled Modellng and Computaton Group Department of Earth Scence and Engneerng Imperal College London,

More information

5. Simultaneous eigenstates: Consider two operators that commute: Â η = a η (13.29)

5. Simultaneous eigenstates: Consider two operators that commute:  η = a η (13.29) 5. Smultaneous egenstates: Consder two operators that commute: [ Â, ˆB ] = 0 (13.28) Let  satsfy the followng egenvalue equaton: Multplyng both sdes by ˆB  η = a η (13.29) ˆB [  η ] = ˆB [a η ] = a

More information

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network 700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School

More information

2.4 Bivariate distributions

2.4 Bivariate distributions page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together

More information

ChE 4520/5520: Mass Transport. Objective/Introduction. Outline. Gerardine G. Botte

ChE 4520/5520: Mass Transport. Objective/Introduction. Outline. Gerardine G. Botte ChE 450/550: Mass Transport Gerardne G. Botte Objectve/Introducton In prevous chapters we neglected transport lmtatons In ths chapter we wll learn how to evaluate the effect of transport lmtatons We wll

More information

Performance Analysis and Coding Strategy of ECOC SVMs

Performance Analysis and Coding Strategy of ECOC SVMs Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.67-76 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School

More information

Compiling for Parallelism & Locality. Dependence Testing in General. Algorithms for Solving the Dependence Problem. Dependence Testing

Compiling for Parallelism & Locality. Dependence Testing in General. Algorithms for Solving the Dependence Problem. Dependence Testing Complng for Parallelsm & Localty Dependence Testng n General Assgnments Deadlne for proect 4 extended to Dec 1 Last tme Data dependences and loops Today Fnsh data dependence analyss for loops General code

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60 BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

More information

Project Networks With Mixed-Time Constraints

Project Networks With Mixed-Time Constraints Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPP-ATBD-ClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller

More information

Portfolio Loss Distribution

Portfolio Loss Distribution Portfolo Loss Dstrbuton Rsky assets n loan ortfolo hghly llqud assets hold-to-maturty n the bank s balance sheet Outstandngs The orton of the bank asset that has already been extended to borrowers. Commtment

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

9.1 The Cumulative Sum Control Chart

9.1 The Cumulative Sum Control Chart Learnng Objectves 9.1 The Cumulatve Sum Control Chart 9.1.1 Basc Prncples: Cusum Control Chart for Montorng the Process Mean If s the target for the process mean, then the cumulatve sum control chart s

More information

Abstract. Clustering ensembles have emerged as a powerful method for improving both the

Abstract. Clustering ensembles have emerged as a powerful method for improving both the Clusterng Ensembles: {topchyal, Models jan, of punch}@cse.msu.edu Consensus and Weak Parttons * Alexander Topchy, Anl K. Jan, and Wllam Punch Department of Computer Scence and Engneerng, Mchgan State Unversty

More information

Fisher Markets and Convex Programs

Fisher Markets and Convex Programs Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

More information

A cooperative connectionist IDS model to identify independent anomalous SNMP situations

A cooperative connectionist IDS model to identify independent anomalous SNMP situations A cooperatve connectonst IDS model to dentfy ndependent anomalous SNMP stuatons Álvaro Herrero, Emlo Corchado, José Manuel Sáz Department of Cvl Engneerng, Unversty of Burgos, Span escorchado@ubu.es Abstract

More information

The Distribution of Eigenvalues of Covariance Matrices of Residuals in Analysis of Variance

The Distribution of Eigenvalues of Covariance Matrices of Residuals in Analysis of Variance JOURNAL OF RESEARCH of the Natonal Bureau of Standards - B. Mathem atca l Scence s Vol. 74B, No.3, July-September 1970 The Dstrbuton of Egenvalues of Covarance Matrces of Resduals n Analyss of Varance

More information

Overview. Naive Bayes Classifiers. A Sample Data Set. Frequencies and Probabilities. Connectionist and Statistical Language Processing

Overview. Naive Bayes Classifiers. A Sample Data Set. Frequencies and Probabilities. Connectionist and Statistical Language Processing Overvew Nave Bayes Classfers Connectonst and Statstcal Language Processng Frank Keller keller@col.un-sb.de Computerlngustk Unverstät des Saarlandes Sample data set wth frequences and probabltes Classfcaton

More information

Communication Networks II Contents

Communication Networks II Contents 8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

More information

Least Squares Fitting of Data

Least Squares Fitting of Data Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2016. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng

More information

An Alternative Way to Measure Private Equity Performance

An Alternative Way to Measure Private Equity Performance An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate

More information

The Analysis of Covariance. ERSH 8310 Keppel and Wickens Chapter 15

The Analysis of Covariance. ERSH 8310 Keppel and Wickens Chapter 15 The Analyss of Covarance ERSH 830 Keppel and Wckens Chapter 5 Today s Class Intal Consderatons Covarance and Lnear Regresson The Lnear Regresson Equaton TheAnalyss of Covarance Assumptons Underlyng the

More information

Interpreting Patterns and Analysis of Acute Leukemia Gene Expression Data by Multivariate Statistical Analysis

Interpreting Patterns and Analysis of Acute Leukemia Gene Expression Data by Multivariate Statistical Analysis Interpretng Patterns and Analyss of Acute Leukema Gene Expresson Data by Multvarate Statstcal Analyss ChangKyoo Yoo * and Peter A. Vanrolleghem BIOMATH, Department of Appled Mathematcs, Bometrcs and Process

More information

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.

More information

An Algorithm for Data-Driven Bandwidth Selection

An Algorithm for Data-Driven Bandwidth Selection IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 25, NO. 2, FEBRUARY 2003 An Algorthm for Data-Drven Bandwdth Selecton Dorn Comancu, Member, IEEE Abstract The analyss of a feature space

More information

Visualization of high-dimensional data with relational perspective map

Visualization of high-dimensional data with relational perspective map (2004) 3, 49 59 & 2004 Palgrave Macmllan Ltd. All rghts reserved 1473-8716 $25.00 www.palgrave-journals.com/vs Vsualzaton of hgh-dmensonal data wth relatonal perspectve map James Xnzh L 1 1 Edgehll Dr.

More information

CHAPTER 14 MORE ABOUT REGRESSION

CHAPTER 14 MORE ABOUT REGRESSION CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp

More information

SIX WAYS TO SOLVE A SIMPLE PROBLEM: FITTING A STRAIGHT LINE TO MEASUREMENT DATA

SIX WAYS TO SOLVE A SIMPLE PROBLEM: FITTING A STRAIGHT LINE TO MEASUREMENT DATA SIX WAYS TO SOLVE A SIMPLE PROBLEM: FITTING A STRAIGHT LINE TO MEASUREMENT DATA E. LAGENDIJK Department of Appled Physcs, Delft Unversty of Technology Lorentzweg 1, 68 CJ, The Netherlands E-mal: e.lagendjk@tnw.tudelft.nl

More information

Nonparametric Estimation of Asymmetric First Price Auctions: A Simplified Approach

Nonparametric Estimation of Asymmetric First Price Auctions: A Simplified Approach Nonparametrc Estmaton of Asymmetrc Frst Prce Auctons: A Smplfed Approach Bn Zhang, Kemal Guler Intellgent Enterprse Technologes Laboratory HP Laboratores Palo Alto HPL-2002-86(R.) November 23, 2004 frst

More information

An Empirical Study of Search Engine Advertising Effectiveness

An Empirical Study of Search Engine Advertising Effectiveness An Emprcal Study of Search Engne Advertsng Effectveness Sanjog Msra, Smon School of Busness Unversty of Rochester Edeal Pnker, Smon School of Busness Unversty of Rochester Alan Rmm-Kaufman, Rmm-Kaufman

More information

Lecture 10: Linear Regression Approach, Assumptions and Diagnostics

Lecture 10: Linear Regression Approach, Assumptions and Diagnostics Approach to Modelng I Lecture 1: Lnear Regresson Approach, Assumptons and Dagnostcs Sandy Eckel seckel@jhsph.edu 8 May 8 General approach for most statstcal modelng: Defne the populaton of nterest State

More information

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM BARRIOT Jean-Perre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jean-perre.barrot@cnes.fr 1/Introducton The

More information

CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol

CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK Sample Stablty Protocol Background The Cholesterol Reference Method Laboratory Network (CRMLN) developed certfcaton protocols for total cholesterol, HDL

More information

Linear Regression Analysis for STARDEX

Linear Regression Analysis for STARDEX Lnear Regresson Analss for STARDEX Malcolm Halock, Clmatc Research Unt The followng document s an overvew of lnear regresson methods for reference b members of STARDEX. Whle t ams to cover the most common

More information

Introduction to Regression

Introduction to Regression Introducton to Regresson Regresson a means of predctng a dependent varable based one or more ndependent varables. -Ths s done by fttng a lne or surface to the data ponts that mnmzes the total error. -

More information

Capital asset pricing model, arbitrage pricing theory and portfolio management

Capital asset pricing model, arbitrage pricing theory and portfolio management Captal asset prcng model, arbtrage prcng theory and portfolo management Vnod Kothar The captal asset prcng model (CAPM) s great n terms of ts understandng of rsk decomposton of rsk nto securty-specfc rsk

More information

Active Learning for Interactive Visualization

Active Learning for Interactive Visualization Actve Learnng for Interactve Vsualzaton Tomoharu Iwata Nel Houlsby Zoubn Ghahraman Unversty of Cambrdge Unversty of Cambrdge Unversty of Cambrdge Abstract Many automatc vsualzaton methods have been. However,

More information

Nonlinear Time Series Analysis in a Nutshell

Nonlinear Time Series Analysis in a Nutshell 9 Nonlnear Tme Seres Analyss n a Nutshell Ralph Gregor Andrzejak Contents 9. Introducton...25 9.2 Dynamcs and Tme Seres... 27 9.2. Nonlnear Determnstc Dynamcs Lorenz Dynamcs... 27 9.2.2 Nosy Determnstc

More information