L10: Linear discriminants analysis


 Loraine Cunningham
 2 years ago
 Views:
Transcription
1 L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss Rcardo GuterrezOsuna
2 Lnear dscrmnant analyss, twoclasses Objectve LDA seeks to reduce dmensonalty whle preservng as much of the class dscrmnatory nformaton as possble Assume we have a set of Ddmensonal samples x (, x (, x (N, N of whch belong to class ω, and N to class ω We seek to obtan a scalar y by projectng the samples x onto a lne y = w T x Of all the possble lnes we would lke to select the one that maxmzes the separablty of the scalars x x x CSCE 666 Pattern Analyss Rcardo GuterrezOsuna x
3 In order to fnd a good projecton vector, we need to defne a measure of separaton The mean vector of each class n xspace and yspace s μ = N x ω x and μ = N y ω y = N x ω w T x = w T μ We could then choose the dstance between the projected means as our objectve functon J w = μ μ = w T μ μ However, the dstance between projected means s not a good measure snce t does not account for the standard devaton wthn classes x Ths axs yelds better class separablty x Ths axs has a larger dstance between means CSCE 666 Pattern Analyss Rcardo GuterrezOsuna
4 Fsher s soluton Fsher suggested maxmzng the dfference between the means, normalzed by a measure of the wthnclass scatter For each class we defne the scatter, an equvalent of the varance, as s = y μ y ω where the quantty s + s s called the wthnclass scatter of the projected examples The Fsher lnear dscrmnant s defned as the lnear functon w T x that maxmzes the crteron functon J w = μ μ s +s Therefore, we are lookng for a projecton where examples from the same class are projected very close to each other and, at the same tme, the projected means are as farther apart as possble x x CSCE 666 Pattern Analyss Rcardo GuterrezOsuna
5 To fnd the optmum w, we must express J(w) as a functon of w Frst, we defne a measure of the scatter n feature space x T S = x ω x μ x μ S + S = S W where S W s called the wthnclass scatter matrx The scatter of the projecton y can then be expressed as a functon of the scatter matrx n feature space x s = y ω y μ = w T x w T x ω μ = = x ω w T x μ x μ T w = w T S w s + s = w T S W w Smlarly, the dfference between the projected means can be expressed n terms of the means n the orgnal feature space μ μ = w T μ w T μ = w T T μ μ μ μ w = w T S B w The matrx S B s called the betweenclass scatter. Note that, snce S B s the outer product of two vectors, ts rank s at most one We can fnally express the Fsher crteron n terms of S W and S B as J w = wt S B w w T S W w CSCE 666 Pattern Analyss Rcardo GuterrezOsuna S B
6 To fnd the maxmum of J(w) we derve and equate to zero d dw J w = d dw w T S W w d wt S B w w T S B w w T S W w = 0 w T S dw B w d wt S W w dw w T S W w S B w w T S B w S W w = 0 Dvdng by w T S W w = 0 w T S W w S w T S W w Bw wt S B w S w T S W w Ww = 0 S B w JS W w = 0 S W S B w Jw = 0 Solvng the generalzed egenvalue problem (S W S B w = Jw) yelds w = arg max wt S B w w T S W w = S W μ μ Ths s know as Fsher s lnear dscrmnant (96), although t s not a dscrmnant but rather a specfc choce of drecton for the projecton of the data down to one dmenson CSCE 666 Pattern Analyss Rcardo GuterrezOsuna 6
7 Example Compute the LDA projecton for the followng D dataset X = {(,), (,), (,), (,6), (,)} X = {(9,0), (6,8), (9,), (8,7), (0,8)} SOLUTION (by hand) The class statstcs are S =.8..6 S = x w LDA μ =.0.6 T ; μ = T The wthn and betweenclass scatter are S B = S W =.6..8 The LDA projecton s then obtaned as the soluton of the generalzed egenvalue problem S W S B v = λv S W S B λi = Or drectly by w = S W μ μ v v =.6 v v v v =.9.9 =.9.9 T.89 λ λ x = 0 λ =.6 CSCE 666 Pattern Analyss Rcardo GuterrezOsuna 7
8 LDA, C classes Fsher s LDA generalzes gracefully for Cclass problems Instead of one projecton y, we wll now seek (C ) projectons [y, y, y C ] by means of (C ) projecton vectors w arranged by columns nto a projecton matrx W = [w w w C ]: y = w T x y = W T x Dervaton x The wthnclass scatter generalzes as S W = C = S where S = x μ x μ T x ω and μ = N x ω x And the betweenclass scatter becomes S B = N μ μ μ μ T C = where μ = N x x = N C = N μ Matrx S T = S B + S W s called the total scatter S W S W S B S B S B S W x CSCE 666 Pattern Analyss Rcardo GuterrezOsuna 8
9 Smlarly, we defne the mean vector and scatter matrces for the projected samples as μ = N μ = N C = T y ω y S W = y μ y μ y ω y y S B = C = N μ μ μ μ T From our dervaton for the twoclass problem, we can wrte S W = W T S W W S B = W T S B W Recall that we are lookng for a projecton that maxmzes the rato of betweenclass to wthnclass scatter. Snce the projecton s no longer a scalar (t has C dmensons), we use the determnant of the scatter matrces to obtan a scalar objectve functon J W = S B S W = WT S B W W T S W W And we wll seek the projecton matrx W that maxmzes ths rato CSCE 666 Pattern Analyss Rcardo GuterrezOsuna 9
10 It can be shown that the optmal projecton matrx W s the one whose columns are the egenvectors correspondng to the largest egenvalues of the followng generalzed egenvalue problem NOTES W = w w w C = arg max WT S B W W T S W W S B λ S W w = 0 S B s the sum of C matrces of rank and the mean vectors are constraned by C C = μ = μ Therefore, S B wll be of rank (C ) or less Ths means that only (C ) of the egenvalues λ wll be nonzero The projectons wth maxmum class separablty nformaton are the egenvectors correspondng to the largest egenvalues of S W S B LDA can be derved as the Maxmum Lkelhood method for the case of normal classcondtonal denstes wth equal covarance matrces CSCE 666 Pattern Analyss Rcardo GuterrezOsuna 0
11 LDA vs. PCA Ths example llustrates the performance of PCA and LDA on an odor recognton problem Fve types of coffee beans were presented to an array of gas sensors For each coffee type, snffs were performed and the response of the gas sensor array was processed n order to obtan a 60dmensonal feature vector Results From the D scatter plots t s clear that LDA outperforms PCA n terms of class dscrmnaton Ths s one example where the dscrmnatory nformaton s not algned wth the drecton of maxmum varance PCA Sensor response normalzed data Sulaw esy Kenya Araban Sumatra Colomba LDA axs axs 00 axs axs 0. axs axs CSCE 666 Pattern Analyss Rcardo GuterrezOsuna
12 Lmtatons of LDA LDA produces at most C feature projectons If the classfcaton error estmates establsh that more features are needed, some other method must be employed to provde those addtonal features LDA s a parametrc method (t assumes unmodal Gaussan lkelhoods) If the dstrbutons are sgnfcantly nongaussan, the LDA projectons may not preserve complex structure n the data needed for classfcaton = = = = x LDA wll also fal f dscrmnatory nformaton s not n the mean but n the varance of the data P C A L D A x CSCE 666 Pattern Analyss Rcardo GuterrezOsuna
13 Varants of LDA Nonparametrc LDA (Fukunaga) NPLDA relaxes the unmodal Gaussan assumpton by computng S B usng local nformaton and the knn rule. As a result of ths The matrx S B s fullrank, allowng us to extract more than (C ) features The projectons are able to preserve the structure of the data more closely Orthonormal LDA (Okada and Tomta) OLDA computes projectons that maxmze the Fsher crteron and, at the same tme, are parwse orthonormal The method used n OLDA combnes the egenvalue soluton of S W S B and the Gram Schmdt orthonormalzaton procedure OLDA sequentally fnds axes that maxmze the Fsher crteron n the subspace orthogonal to all features already extracted OLDA s also capable of fndng more than (C ) features Generalzed LDA (Lowe) GLDA generalzes the Fsher crteron by ncorporatng a cost functon smlar to the one we used to compute the Bayes Rsk As a result, LDA can produce projects that are based by the cost functon,.e., classes wth a hgher cost C j wll be placed further apart n the lowdmensonal projecton Multlayer perceptrons (Webb and Lowe) It has been shown that the hdden layers of multlayer perceptrons perform nonlnear dscrmnant analyss by maxmzng Tr[S B S T ], where the scatter matrces are measured at the output of the last hdden layer CSCE 666 Pattern Analyss Rcardo GuterrezOsuna
14 Other dmensonalty reducton methods Exploratory Projecton Pursut (Fredman and Tukey) EPP seeks an Mdmensonal (M=, typcally) lnear projecton of the data that maxmzes a measure of nterestngness Interestngness s measured as departure from multvarate normalty Ths measure s not the varance and s commonly scalefree. In most mplementatons t s also affne nvarant, so t does not depend on correlatons between features. [Rpley, 996] In other words, EPP seeks projectons that separate clusters as much as possble and keeps these clusters compact, a smlar crteron as Fsher s, but EPP does NOT use class labels Once an nterestng projecton s found, t s mportant to remove the structure t reveals to allow other nterestng vews to be found more easly Unnterestng Interestng x x x CSCE 666 Pattern Analyss Rcardo GuterrezOsuna x
15 Sammon s nonlnear mappng (Sammon) Ths method seeks a mappng onto an Mdmensonal space that preserves the nterpont dstances n the orgnal Ndmensonal space Ths s accomplshed by mnmzng the followng objectve functon E d, d = j d P,P j d P,Pj d P,P j The orgnal method dd not obtan an explct mappng but only a lookup table for the elements n the tranng set Newer mplementatons based on neural networks do provde an explct mappng for test data and also consder cost functons (e.g., Neuroscale) Sammon s mappng s closely related to Mult Dmensonal Scalng (MDS), a famly of multvarate statstcal methods commonly used n the socal scences We wll revew MDS technques when we cover manfold learnng x x P j P j d(p, P j )= d(p, P j ) ",j P P x x x CSCE 666 Pattern Analyss Rcardo GuterrezOsuna
Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)
Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton
More informationCS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements
Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there
More informationNonlinear data mapping by neural networks
Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal
More informationv a 1 b 1 i, a 2 b 2 i,..., a n b n i.
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are
More informationLETTER IMAGE RECOGNITION
LETTER IMAGE RECOGNITION 1. Introducton. 1. Introducton. Objectve: desgn classfers for letter mage recognton. consder accuracy and tme n takng the decson. 20,000 samples: Startng set: mages based on 20
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract  Stock market s one of the most complcated systems
More informationLecture 5,6 Linear Methods for Classification. Summary
Lecture 5,6 Lnear Methods for Classfcaton Rce ELEC 697 Farnaz Koushanfar Fall 2006 Summary Bayes Classfers Lnear Classfers Lnear regresson of an ndcator matrx Lnear dscrmnant analyss (LDA) Logstc regresson
More information8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by
6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng
More informationb) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei
Mathematcal Propertes of the Least Squares Regresson The least squares regresson lne obeys certan mathematcal propertes whch are useful to know n practce. The followng propertes can be establshed algebracally:
More informationLogistic Regression. Steve Kroon
Logstc Regresson Steve Kroon Course notes sectons: 24.324.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro
More informationMANY machine learning and pattern recognition applications
1 Trace Rato Problem Revsted Yangqng Ja, Fepng Ne, and Changshu Zhang Abstract Dmensonalty reducton s an mportant ssue n many machne learnng and pattern recognton applcatons, and the trace rato problem
More informationUsing Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions
Usng Mxture Covarance Matrces to Improve Face and Facal Expresson Recogntons Carlos E. homaz, Duncan F. Glles and Raul Q. Fetosa 2 Imperal College of Scence echnology and Medcne, Department of Computng,
More informationCausal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting
Causal, Explanatory Forecastng Assumes causeandeffect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of
More informationMultivariate EWMA Control Chart
Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant
More informationInequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.
Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.
More informationSingle and multiple stage classifiers implementing logistic discrimination
Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul  PUCRS Av. Ipranga,
More informationTable of Contents EQ.10...46 EQ.6...46 EQ.8...46
Table of Contents CHAPTER II  PATTERN RECOGNITION.... THE PATTERN RECOGNITION PROBLEM.... STATISTICAL FORMULATION OF CLASSIFIERS...6 3. CONCLUSIONS...30 UNDERSTANDING BAYES RULE...3 BAYESIAN THRESHOLD...33
More informationSupport Vector Machines
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.
More informationGeorey E. Hinton. University oftoronto. Email: zoubin@cs.toronto.edu. Technical Report CRGTR961. May 21, 1996 (revised Feb 27, 1997) Abstract
The EM Algorthm for Mxtures of Factor Analyzers Zoubn Ghahraman Georey E. Hnton Department of Computer Scence Unversty oftoronto 6 Kng's College Road Toronto, Canada M5S A4 Emal: zoubn@cs.toronto.edu Techncal
More informationTHE TITANIC SHIPWRECK: WHO WAS
THE TITANIC SHIPWRECK: WHO WAS MOST LIKELY TO SURVIVE? A STATISTICAL ANALYSIS Ths paper examnes the probablty of survvng the Ttanc shpwreck usng lmted dependent varable regresson analyss. Ths appled analyss
More informationLearning from Multiple Outlooks
Learnng from Multple Outlooks Maayan Harel Department of Electrcal Engneerng, Technon, Hafa, Israel She Mannor Department of Electrcal Engneerng, Technon, Hafa, Israel maayanga@tx.technon.ac.l she@ee.technon.ac.l
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationUsing mixture covariance matrices to improve face and facial expression recognitions
Pattern Recognton Letters 24 (2003) 2159 2165 www.elsever.com/locate/patrec Usng mxture covarance matrces to mprove face and facal expresson recogntons Carlos E. Thomaz a, *, Duncan F. Glles a, Raul Q.
More informationQUANTUM MECHANICS, BRAS AND KETS
PH575 SPRING QUANTUM MECHANICS, BRAS AND KETS The followng summares the man relatons and defntons from quantum mechancs that we wll be usng. State of a phscal sstem: The state of a phscal sstem s represented
More informationLoop Parallelization
  Loop Parallelzaton C52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I,J]+B[I,J] ED FOR ED FOR analyze
More informationLogistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification
Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson
More informationMoment of a force about a point and about an axis
3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should
More informationONE of the most crucial problems that every image
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 23, NO. 10, OCTOBER 2014 4413 Maxmum Margn Projecton Subspace Learnng for Vsual Data Analyss Symeon Nktds, Anastasos Tefas, Member, IEEE, and Ioanns Ptas, Fellow,
More informationNaïve Bayes classifier & Evaluation framework
Lecture aïve Bayes classfer & Evaluaton framework Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Generatve approach to classfcaton Idea:. Represent and learn the dstrbuton p x, y. Use t to defne probablstc
More informationData Visualization by Pairwise Distortion Minimization
Communcatons n Statstcs, Theory and Methods 34 (6), 005 Data Vsualzaton by Parwse Dstorton Mnmzaton By Marc Sobel, and Longn Jan Lateck* Department of Statstcs and Department of Computer and Informaton
More informationCS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering
Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that
More informationJ. Parallel Distrib. Comput.
J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n
More informationKernel Methods for General Pattern Analysis
Kernel Methods for General Pattern Analyss Nello Crstann Unversty of Calforna, Davs nello@supportvector.net Overvew Kernel Methods are a new class of pattern analyss algorthms whch can operate on very
More informationOptimal portfolios using Linear Programming models
Optmal portfolos usng Lnear Programmng models Chrstos Papahrstodoulou Mälardalen Unversty, Västerås, Sweden Abstract The classcal Quadratc Programmng formulaton of the well known portfolo selecton problem,
More informationLecture 18: Clustering & classification
O CPS260/BGT204. Algorthms n Computatonal Bology October 30, 2003 Lecturer: Pana K. Agarwal Lecture 8: Clusterng & classfcaton Scrbe: Daun Hou Open Problem In HomeWor 2, problem 5 has an open problem whch
More informationChapter 3 Group Theory p. 1  Remark: This is only a brief summary of most important results of groups theory with respect
Chapter 3 Group Theory p.  3. Compact Course: Groups Theory emark: Ths s only a bref summary of most mportant results of groups theory wth respect to the applcatons dscussed n the followng chapters. For
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
More informationOutofSample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering
OutofSample Extensons for LLE, Isomap, MDS, Egenmaps, and Spectral Clusterng Yoshua Bengo, JeanFranços Paement, Pascal Vncent Olver Delalleau, Ncolas Le Roux and Mare Oumet Département d Informatque
More informationLecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationEstimating the Number of Clusters in Genetics of Acute Lymphoblastic Leukemia Data
Journal of Al Azhar UnverstyGaza (Natural Scences), 2011, 13 : 109118 Estmatng the Number of Clusters n Genetcs of Acute Lymphoblastc Leukema Data Mahmoud K. Okasha, Khaled I.A. Almghar Department of
More information6. EIGENVALUES AND EIGENVECTORS 3 = 3 2
EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a nonzero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :
More information+ + +   This circuit than can be reduced to a planar circuit
MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to
More informationThe covariance is the two variable analog to the variance. The formula for the covariance between two variables is
Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.
More informationSolution of Algebraic and Transcendental Equations
CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton
More informationBrigid Mullany, Ph.D University of North Carolina, Charlotte
Evaluaton And Comparson Of The Dfferent Standards Used To Defne The Postonal Accuracy And Repeatablty Of Numercally Controlled Machnng Center Axes Brgd Mullany, Ph.D Unversty of North Carolna, Charlotte
More informationBypassing Synthesis: PLS for Face Recognition with Pose, LowResolution and Sketch
Bypassng Synthess: PLS for Face Recognton wth Pose, LowResoluton and Setch Abhshe Sharma Insttute of Advanced Computer Scence Unversty of Maryland, USA bhoaal@umacs.umd.edu Davd W Jacobs Insttute of Advanced
More informationStatistical Methods to Develop Rating Models
Statstcal Methods to Develop Ratng Models [Evelyn Hayden and Danel Porath, Österrechsche Natonalbank and Unversty of Appled Scences at Manz] Source: The Basel II Rsk Parameters Estmaton, Valdaton, and
More informationEstimation and Robustness of Linear Mixed Models in Credibility Context
Estmaton and Robustness of Lnear Mxed Models n Credblty Context by Wng Kam Fung and Xao Chen Xu ABSTRACT In ths paper, lnear mxed models are employed for estmaton of structural parameters n credblty context.
More informationLearning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing
Internatonal Journal of Machne Learnng and Computng, Vol. 4, No. 3, June 04 Learnng from Large Dstrbuted Data: A Scalng Down Samplng Scheme for Effcent Data Processng Che Ngufor and Janusz Wojtusak part
More informationCan Auto Liability Insurance Purchases Signal Risk Attitude?
Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? ChuShu L Department of Internatonal Busness, Asa Unversty, Tawan ShengChang
More informationQuestions that we may have about the variables
Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent
More informationImage as a Feature Vector. Eigenfaces: linear projection. Announcements
Announcements Face Recognton: Dmensonalty Reducton & Unconstraned Face Rec Bometrcs CSE 190 Lecture 14 H3 assgned, due on hursday. Last H fngerprnt recognton to be assgned next week. hursday guest lecture
More informationwhere the coordinates are related to those in the old frame as follows.
Chapter 2  Cartesan Vectors and Tensors: Ther Algebra Defnton of a vector Examples of vectors Scalar multplcaton Addton of vectors coplanar vectors Unt vectors A bass of noncoplanar vectors Scalar product
More informationThe OC Curve of Attribute Acceptance Plans
The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4
More informationTime Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University
Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton
More informationThe eigenvalue derivatives of linear damped systems
Control and Cybernetcs vol. 32 (2003) No. 4 The egenvalue dervatves of lnear damped systems by YeongJeu Sun Department of Electrcal Engneerng IShou Unversty Kaohsung, Tawan 840, R.O.C emal: yjsun@su.edu.tw
More informationTHE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES
The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered
More informationGeneral Iteration Algorithm for Classification Ratemaking
General Iteraton Algorthm for Classfcaton Ratemakng by Luyang Fu and Chengsheng eter Wu ABSTRACT In ths study, we propose a flexble and comprehensve teraton algorthm called general teraton algorthm (GIA)
More informationI. SCOPE, APPLICABILITY AND PARAMETERS Scope
D Executve Board Annex 9 Page A/R ethodologcal Tool alculaton of the number of sample plots for measurements wthn A/R D project actvtes (Verson 0) I. SOPE, PIABIITY AD PARAETERS Scope. Ths tool s applcable
More informationPRACTICE 1: MUTUAL FUNDS EVALUATION USING MATLAB.
PRACTICE 1: MUTUAL FUNDS EVALUATION USING MATLAB. INDEX 1. Load data usng the Edtor wndow and mfle 2. Learnng to save results from the Edtor wndow. 3. Computng the Sharpe Rato 4. Obtanng the Treynor Rato
More informationImperial College London
F. Fang 1, C.C. Pan 1, I.M. Navon 2, M.D. Pggott 1, G.J. Gorman 1, P.A. Allson 1 and A.J.H. Goddard 1 1 Appled Modellng and Computaton Group Department of Earth Scence and Engneerng Imperal College London,
More information5. Simultaneous eigenstates: Consider two operators that commute: Â η = a η (13.29)
5. Smultaneous egenstates: Consder two operators that commute: [ Â, ˆB ] = 0 (13.28) Let Â satsfy the followng egenvalue equaton: Multplyng both sdes by ˆB Â η = a η (13.29) ˆB [ Â η ] = ˆB [a η ] = a
More informationForecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network
700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School
More information2.4 Bivariate distributions
page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together
More informationChE 4520/5520: Mass Transport. Objective/Introduction. Outline. Gerardine G. Botte
ChE 450/550: Mass Transport Gerardne G. Botte Objectve/Introducton In prevous chapters we neglected transport lmtatons In ths chapter we wll learn how to evaluate the effect of transport lmtatons We wll
More informationPerformance Analysis and Coding Strategy of ECOC SVMs
Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.6776 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School
More informationCompiling for Parallelism & Locality. Dependence Testing in General. Algorithms for Solving the Dependence Problem. Dependence Testing
Complng for Parallelsm & Localty Dependence Testng n General Assgnments Deadlne for proect 4 extended to Dec 1 Last tme Data dependences and loops Today Fnsh data dependence analyss for loops General code
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationx f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60
BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true
More informationProject Networks With MixedTime Constraints
Project Networs Wth MxedTme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa
More information1 Approximation Algorithms
CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons
More informationMAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date
Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPPATBDClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller
More informationPortfolio Loss Distribution
Portfolo Loss Dstrbuton Rsky assets n loan ortfolo hghly llqud assets holdtomaturty n the bank s balance sheet Outstandngs The orton of the bank asset that has already been extended to borrowers. Commtment
More informationbenefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
More information9.1 The Cumulative Sum Control Chart
Learnng Objectves 9.1 The Cumulatve Sum Control Chart 9.1.1 Basc Prncples: Cusum Control Chart for Montorng the Process Mean If s the target for the process mean, then the cumulatve sum control chart s
More informationAbstract. Clustering ensembles have emerged as a powerful method for improving both the
Clusterng Ensembles: {topchyal, Models jan, of punch}@cse.msu.edu Consensus and Weak Parttons * Alexander Topchy, Anl K. Jan, and Wllam Punch Department of Computer Scence and Engneerng, Mchgan State Unversty
More informationFisher Markets and Convex Programs
Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and
More informationA cooperative connectionist IDS model to identify independent anomalous SNMP situations
A cooperatve connectonst IDS model to dentfy ndependent anomalous SNMP stuatons Álvaro Herrero, Emlo Corchado, José Manuel Sáz Department of Cvl Engneerng, Unversty of Burgos, Span escorchado@ubu.es Abstract
More informationThe Distribution of Eigenvalues of Covariance Matrices of Residuals in Analysis of Variance
JOURNAL OF RESEARCH of the Natonal Bureau of Standards  B. Mathem atca l Scence s Vol. 74B, No.3, JulySeptember 1970 The Dstrbuton of Egenvalues of Covarance Matrces of Resduals n Analyss of Varance
More informationOverview. Naive Bayes Classifiers. A Sample Data Set. Frequencies and Probabilities. Connectionist and Statistical Language Processing
Overvew Nave Bayes Classfers Connectonst and Statstcal Language Processng Frank Keller keller@col.unsb.de Computerlngustk Unverstät des Saarlandes Sample data set wth frequences and probabltes Classfcaton
More informationCommunication Networks II Contents
8 / 1  Communcaton Networs II (Görg)  www.comnets.unbremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP
More informationLeast Squares Fitting of Data
Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 19982016. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng
More informationAn Alternative Way to Measure Private Equity Performance
An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate
More informationThe Analysis of Covariance. ERSH 8310 Keppel and Wickens Chapter 15
The Analyss of Covarance ERSH 830 Keppel and Wckens Chapter 5 Today s Class Intal Consderatons Covarance and Lnear Regresson The Lnear Regresson Equaton TheAnalyss of Covarance Assumptons Underlyng the
More informationInterpreting Patterns and Analysis of Acute Leukemia Gene Expression Data by Multivariate Statistical Analysis
Interpretng Patterns and Analyss of Acute Leukema Gene Expresson Data by Multvarate Statstcal Analyss ChangKyoo Yoo * and Peter A. Vanrolleghem BIOMATH, Department of Appled Mathematcs, Bometrcs and Process
More informationLinear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits
Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.
More informationAn Algorithm for DataDriven Bandwidth Selection
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 25, NO. 2, FEBRUARY 2003 An Algorthm for DataDrven Bandwdth Selecton Dorn Comancu, Member, IEEE Abstract The analyss of a feature space
More informationVisualization of highdimensional data with relational perspective map
(2004) 3, 49 59 & 2004 Palgrave Macmllan Ltd. All rghts reserved 14738716 $25.00 www.palgravejournals.com/vs Vsualzaton of hghdmensonal data wth relatonal perspectve map James Xnzh L 1 1 Edgehll Dr.
More informationCHAPTER 14 MORE ABOUT REGRESSION
CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp
More informationSIX WAYS TO SOLVE A SIMPLE PROBLEM: FITTING A STRAIGHT LINE TO MEASUREMENT DATA
SIX WAYS TO SOLVE A SIMPLE PROBLEM: FITTING A STRAIGHT LINE TO MEASUREMENT DATA E. LAGENDIJK Department of Appled Physcs, Delft Unversty of Technology Lorentzweg 1, 68 CJ, The Netherlands Emal: e.lagendjk@tnw.tudelft.nl
More informationNonparametric Estimation of Asymmetric First Price Auctions: A Simplified Approach
Nonparametrc Estmaton of Asymmetrc Frst Prce Auctons: A Smplfed Approach Bn Zhang, Kemal Guler Intellgent Enterprse Technologes Laboratory HP Laboratores Palo Alto HPL200286(R.) November 23, 2004 frst
More informationAn Empirical Study of Search Engine Advertising Effectiveness
An Emprcal Study of Search Engne Advertsng Effectveness Sanjog Msra, Smon School of Busness Unversty of Rochester Edeal Pnker, Smon School of Busness Unversty of Rochester Alan RmmKaufman, RmmKaufman
More informationLecture 10: Linear Regression Approach, Assumptions and Diagnostics
Approach to Modelng I Lecture 1: Lnear Regresson Approach, Assumptons and Dagnostcs Sandy Eckel seckel@jhsph.edu 8 May 8 General approach for most statstcal modelng: Defne the populaton of nterest State
More informationGRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 NORM
GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 NORM BARRIOT JeanPerre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jeanperre.barrot@cnes.fr 1/Introducton The
More informationCHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol
CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK Sample Stablty Protocol Background The Cholesterol Reference Method Laboratory Network (CRMLN) developed certfcaton protocols for total cholesterol, HDL
More informationLinear Regression Analysis for STARDEX
Lnear Regresson Analss for STARDEX Malcolm Halock, Clmatc Research Unt The followng document s an overvew of lnear regresson methods for reference b members of STARDEX. Whle t ams to cover the most common
More informationIntroduction to Regression
Introducton to Regresson Regresson a means of predctng a dependent varable based one or more ndependent varables. Ths s done by fttng a lne or surface to the data ponts that mnmzes the total error. 
More informationCapital asset pricing model, arbitrage pricing theory and portfolio management
Captal asset prcng model, arbtrage prcng theory and portfolo management Vnod Kothar The captal asset prcng model (CAPM) s great n terms of ts understandng of rsk decomposton of rsk nto securtyspecfc rsk
More informationActive Learning for Interactive Visualization
Actve Learnng for Interactve Vsualzaton Tomoharu Iwata Nel Houlsby Zoubn Ghahraman Unversty of Cambrdge Unversty of Cambrdge Unversty of Cambrdge Abstract Many automatc vsualzaton methods have been. However,
More informationNonlinear Time Series Analysis in a Nutshell
9 Nonlnear Tme Seres Analyss n a Nutshell Ralph Gregor Andrzejak Contents 9. Introducton...25 9.2 Dynamcs and Tme Seres... 27 9.2. Nonlnear Determnstc Dynamcs Lorenz Dynamcs... 27 9.2.2 Nosy Determnstc
More information