Using Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions


 Horace Floyd
 2 years ago
 Views:
Transcription
1 Usng Mxture Covarance Matrces to Improve Face and Facal Expresson Recogntons Carlos E. homaz, Duncan F. Glles and Raul Q. Fetosa 2 Imperal College of Scence echnology and Medcne, Department of Computng, 80 Queen s Gate, London SW7 2BZ, Unted Kngdom {cet, 2 Catholc Unversty of Ro de Janero, Department of Electrcal Engneerng, r. Marques de Sao Vcente 225, Ro de Janero , Brazl Abstract. In several pattern recognton problems, partcularly n mage recognton ones, there are often a large number of features avalable, but the number of tranng samples for each pattern s sgnfcantly less than the dmenson of the feature space. hs statement mples that the sample group covarance matrces often used n the Gaussan maxmum probablty classfer are sngular. A common soluton to ths problem s to assume that all groups have equal covarance matrces and to use as ther estmates the pooled covarance matrx calculated from the whole tranng set. hs paper uses an alternatve estmate for the sample group covarance matrces, here called the mxture covarance, gven by an approprate lnear combnaton of the sample group and pooled covarance matrces. Experments were carred out to evaluate the performance assocated wth ths estmate n two recognton applcatons: face and facal expresson. he average recognton rates obtaned by usng the mxture covarance matrces were hgher than the usual estmates. Introducton A crtcal ssue for the Gaussan maxmum probablty classfer s the nverse of the sample group covarance matrces. Snce n practce these matrces are not nown, estmates must be computed based on the observatons (patterns) avalable n a tranng set. In some applcatons, however, there are often a large number of features avalable, but the number of tranng samples for each group s lmted and sgnfcantly less than the dmenson of the feature space. hs mples that the sample group covarance matrces wll be sngular. hs problem, whch s called a small sample sze problem [5], s qute common n pattern recognton, partcularly n mage recognton where the number of features s very large. One way to overcome ths problem s to assume that all groups have equal covarance matrces and to use as ther estmates the weghtng average of each sample group covarance matrx, gven by the pooled covarance matrx calculated from the whole tranng set. hs paper uses another estmate for the sample group covarance matrces [4], here called mxture covarance matrces, gven by an approprate lnear combnaton of the sample group covarance matrx and the pooled covarance one. he mxture covarance matrces have the property of havng the same ran as the pooled estmate, whle allowng
2 a dfferent estmate for each group. hus, the mxture estmate may result n hgher accuracy. In order to evaluate ths approach, two pattern recognton applcatons were consdered: face recognton and facal expresson recognton. he evaluaton used dfferent mage databases for each applcaton and a probablstc model combnes the wellnown dmensonalty reducton technque called Prncpal Component Analyss (PCA) and the Gaussan maxmum probablty classfer to nvestgate the mxture covarance matrces on the referred recognton tass. Experments carred out show that the mxture covarance estmates attaned the best performance n both applcatons consdered. 2 Dmensonalty Reducton One of the most successful approaches to the problem of creatng a low dmensonal mage representaton s based on Prncpal Component Analyss (PCA). PCA generates a set of orthonomal bass vectors, nown as prncpal components, that mnmzes the mean square reconstructon error and descrbe major varatons n the whole tranng set consdered. he reasonng behnd applyng PCA frst for dmensonalty reducton nstead of analyzng the maxmum probablty classfer drectly on the face or facal expresson mages s based on the orgnal hghdmensonal space. As the number of tranng samples s lmted and sgnfcantly less than the number of pxels of each mage, the hghdmensonal space s ndeed sparsely represented, mang the parameter estmaton qute complcated ths behavour s called the curse of dmensonalty [8]. Furthermore, many researchers have confrmed that the PCA representaton has good generalzaton ablty especally when the dstrbutons of each class are separated by the mean dfference [,6,7,9]. 3 Maxmum Probablty Classfer he basc problem n the decsontheoretc methods for pattern recognton conssts of fndng a set of g dscrmnant functons d (x), d 2 (x),..., d g (x), where g s the number of groups or classes, wth the decson rule such that f the pdmensonal pattern vector x belongs to the class ( g), then d (x) d j (x), for all j and j g. he Bayes classfer desgned to maxmze the total probablty of correct classfcaton, where equal pror probabltes for all groups are assumed, corresponds to a set of dscrmnant functons equal to the correspondng probablty densty functons, that s, d (x)=f (x) for all classes [8]. he most common probablty densty functon appled to pattern recognton systems s based on the Gaussan multvarate dstrbuton d ( x) = f ( x µ, Σ ) = exp ( x µ ) ( ) p / 2 / 2 Σ x µ, (2π) Σ 2 () where µ and Σ are the class populaton mean vector and covarance matrx. Usually the true values of the mean and the covarance matrx are seldom nown and must be estmated from tranng samples. he mean s estmated by the usual sample mean
3 x, x = j = x µ, (2) where j s observaton j from class, and s the number of tranng observatons from class. he covarance matrx s commonly estmated by the sample group covarance matrx defned as Σ S = ( ) j=, j ( x, x )( x, x ) j j. (3) From replacng the true values of the mean and the covarance matrx n () by ther respectve estmates, the Bayes decson rule acheves optmal classfcaton accuracy only when the number of tranng samples ncreases toward nfnty [4]. In fact for p dmensonal patterns the sample covarance matrx s sngular f less than p + tranng samples from each class are avalable, that s, the sample covarance matrx can not be calculated f s less than the dmenson of the feature space. One method routnely appled to solve ths problem s to assume that all classes have equal covarance matrces, and to use as ther estmates the pooled covarance matrx. hs covarance matrx s a weghtng average of each sample group covarance matrx and, assumng that all classes have the same number of tranng observatons, s gven by S = g pooled S g =. (4) Snce more observatons are taen to calculate the pooled covarance matrx S pooled, ths one wll potentally have a hgher ran than S and wll be eventually full ran. Although the pooled estmate does provde a soluton for the algebrac problem arsng from the nsuffcent number of tranng samples n each group, assumng equal covarance for all groups may brng about dstortons n the modelng of the classfcaton problem and consequently lower accuracy. 4 Mxture Covarance Matrx he choce between the sample group covarance matrx and the pooled covarance one represents a restrctve set of estmates for the true covarance matrx. A less lmted set can be obtaned usng the mxture covarance matrx. 4. Defnton he mxture covarance matrx s a lnear combnaton between the pooled covarance matrx S pooled and the sample covarance matrx of each class S. It s gven by Smx ( w ) w S ( w ) S = pooled +. (5)
4 he mxture parameter w taes on values 0 < w and s dfferent for each class. hs parameter controls the degree of shrnage of the sample group covarance matrx estmates toward the pooled one. Each Smx matrx has the mportant property of admttng an nverse f the pooled estmate S pooled does so [2]. hs mples that f the pooled estmate s nonsngular and the mxture parameter taes on values w > 0, then the Smx wll be nonsngular. hen the remanng queston s: what s the value of the w that gves a relevant lnear mxture between the pooled and sample covarance estmates? A method that determnes an approprate value of the mxture parameter s descrbed n the next secton. 4.2 he mxture parameter Accordng to Hoffbec and Landgrebe [4], the value of the mxture parameter w can be approprately selected so that a best ft to the tranng samples s acheved. her technque s based on the leaveoneoutlelhood (L) parameter estmaton. In the L method, one sample of the class tranng set s removed and the mean and covarance matrx from the remanng samples are estmated. hen the lelhood of the excluded sample s calculated gven the prevous mean and covarance matrx estmates. hs operaton s repeated tmes and the average log lelhood s computed over all the samples. her strategy s to evaluate several dfferent values of w n the range 0 < w, and then choose w that maxmzes the average log lelhood. he mean of class wthout sample r may be computed as x \ r = x, j x, r. (6) ( ) j = he notaton \r ndcates the correspondng quantty s calculated wth the r th observaton from class removed. Followng the same dea, the sample covarance matrx and the pooled covarance matrx of class wthout sample r are S\ r = ( x, j x \ r )( x, j x \ r ) ( x, r x \ r )( x, r x \ r ) ( 2) j =, (7) g S pooled = S j S + S r r \. (8) \ g j = hen the average log lelhood of the excluded samples can be wrtten as follows: L ( w ) = ln[ f ( x, r x \ r, Smx \ r ( w ))], (9) r = where ( x x, Smx ( w )) r f s the Gaussan probablty functon defned n () wth, r \ r \ r x \ mean vector and Smx ( w ) covarance matrx defned as Smx \ r = +. (0) \ r ( w ) w S pooled ( w ) S \ r \ r
5 hs approach, f mplemented n a straghtforward way, would requre computng the nverse and determnant of the Smx\ r ( w ) for each tranng sample. As the Smx\ r ( w ) s a p by p matrx and p s typcally a large number, ths computaton would be qute expensve. Hoffbec and Landgrebe [4], usng the ShermanMorrsonWoodbury formula [3], have showed that t s possble to sgnfcantly reduce the requred computaton by wrtng the mxture covarance matrx n a form as follows: ln where d [ ( )] = [ ] f x x, Smx ( w ) ln Q ( vd), r \ r \ r, () 2 2 vd ( ) Q = ( w ) + w S + w S pooled, (2) ( 2) g( 2) 2 v = ( )( w 2) ( g ), g (3) ( x, r x ) Q ( x, r d = x ). (4) Once the mxture parameter w that maxmzes the average log lelhood s selected, the proposed covarance matrx estmate s calculated usng all the tranng samples and replaced nto the maxmum probablty classfer defned n (). 5 Experments wo experments wth two dfferent databases were performed. In the face recognton experment the ORL Face Database contanng ten mages for each of 40 ndvduals, a total of 400 mages, were used. he ohou Unversty has provded the database for the facal expresson experment. hs database s composed of 93 mages of expressons posed by nne Japanese females. Each person posed three or four examples of each sx fundamental facal expresson: anger, dsgust, fear, happness, sadness and surprse. he database has at least 29 mages for each fundamental facal expresson. For mplementaton convenence all mages were frst reszed to 64x64 pxels. he experments were carred out as follows. Frst PCA reduces the dmensonalty of the orgnal mages and secondly the Gaussan maxmum probablty classfer usng one out of the three covarance estmates (S, S pooled and Smx ) was appled. Each experment was repeated 25 tmes usng several PCA dmensons. Dstnct tranng and testng sets were randomly drawn, and the mean and standard devaton of the recognton rate were calculated. he face recognton classfcaton was computed usng for each ndvdual 5 mages to tran and 5 mages to test. In the facal expresson recognton, the tranng and test sets were respectvely composed of 20 and 9 mages. he sze of the mxture parameter (0 < w ) optmzaton range was taen to be 20, that s w = [0.05, 0.0, 0.5,, ].
6 6 Results ables and 2 present the tranng and test average recognton rates (wth standard devatons) of the face and facal expresson databases, respectvely, over the dfferent PCA dmensons. Snce only 5 mages of each ndvdual were used to form the face recognton tranng set, the results relatve to the sample group covarance estmate were lmted to 4 PCA components. able shows that n all but one experment the Smx estmate led to hgher accuracy than dd both the pooled covarance and sample group covarance matrces. In terms of how senstve the mxture covarance results were to the choce of the tranng and test sets, t s far to say that the Smx standard devatons were smlar to the other two covarance estmates. able 2 shows the results of the facal expresson recognton. For more than 20 components when the sample group covarance estmate became sngular, the mxture covarance estmate reached hgher recognton rates than the pooled covarance estmate. Agan, regardng the computed standard devatons, the Smx estmate showed to be as senstve to the choce of the tranng and test sets as the other two estmates. PCA Sgroup Spooled Smx Components ranng est ranng est ranng est (0.4) 5.6 (4.4) 73.3 (3.) 59.5 (3.0) 90. (2.) 70.8 (3.2) (.2) 88.4 (.4) 99.4 (0.5) 92.0 (.5) (0.6) 9.8 (.8) 00.0 (0.) 94.5 (.7) (0.2) 94.7 (.7) 00.0 (0.0) 95.9 (.5) (0.0) 95.4 (.5) 00.0 (0.0) 96.2 (.6) (0.0) 95.7 (.2) 00.0 (0.0) 96.4 (.5) (0.0) 95.0 (.6) 00.0 (0.0) 95.8 (.6) (0.0) 94.9 (.6) 00.0 (0.0) 95.4 (.6) able. Face Recognton Results PCA Sgroup Spooled Smx Components ranng est ranng est ranng est (4.2) 20.6 (3.9) 32.3 (3.0) 2.6 (3.8) 34.9 (3.3) 2.3 (4.) (3.6) 38.8 (5.6) 49.6 (3.9) 26.5 (6.8) 58.5 (3.7) 27.9 (5.6) (0.5) 64.3 (6.4) 69. (3.6) 44.4 (5.3) 82.9 (2.9) 49.7 (7.7) (2.6) 55.9 (7.7) 9.4 (2.8) 6.3 (7.) (2.8) 64.9 (6.9) 94.8 (2.2) 68.3 (5.) (.7) 70. (7.8) 96.8 (.3) 72.3 (6.2) (.7) 72.0 (7.4) 97.7 (.) 75.6 (5.5) (.4) 75.6 (7.) 98.3 (.) 77.2 (5.7) (.3) 78.4 (6.5) 98.6 (0.8) 79. (5.4) (.0) 79.4 (5.8) 99.2 (0.7) 8.0 (6.6) (0.9) 8.6 (6.6) 99.5 (0.6) 82.8 (6.3) (0.8) 82. (5.9) 99.6 (0.6) 83.6 (7.2) (0.6) 83.3 (5.5) 99.8 (0.4) 84.5 (6.2) able 2. Facal Expresson Recognton Results
7 7 Concluson hs paper used an estmate for the sample group covarance matrces, here called mxture covarance matrces, gven by an approprate lnear combnaton of the sample group covarance matrx and the pooled covarance one. he mxture covarance matrces have the property of havng the same ran as the pooled estmate, whle allowng a dfferent estmate for each group. Extensve experments were carred out to evaluate ths approach on two recognton tass: face recognton and facal expresson recognton. A Gaussan maxmum probablty classfer was bult usng the mxture estmate and the typcal sample group and pooled estmates. In both tass the mxture covarance estmate acheved the hghest accuracy. Regardng the senstveness to the choce of the tranng and test sets, the mxture covarance matrces presented smlar performance to the other two usual estmates. References. C. Lu and H. Wechsler, Learnng the Face Space Representaton and Recognton. In Proc. 5 th Int l Conference on Pattern Recognton, ICPR 2000, Barcelona, Span, September C.E. homaz, R.Q. Fetosa, A. Vega, SeparateGroup Covarance Estmaton wth Insuffcent Data for Object Recognton. In Proc. Ffth AllUranan Internatonal Conference, pp. 224, Urane, G.H. Golub and C.F. Van Loan, Matrx Computatons, second edton. Baltmore: Johns Hopns Unv. Press, J.P. Hoffbec and D.A. Landgrebe, "Covarance Matrx Estmaton and Classfcaton wth Lmted ranng Data", IEEE rans. Pattern Analyss and Machne Intellgence, vol. 8, no. 7, July K. Fuunaga, Introducton to Statstcal Pattern Recognton, second edton. Boston: Academc Press, M. Krby and L. Srovch, "Applcaton of the KarhunenLoeve procedure for the characterzaton of human faces", IEEE rans. Pattern Analyss and Machne Intellgence, Vol. 2, No., Jan M. ur and A. Pentland, Egenfaces for Recognton, Journal of Cogntve Neuroscence, Vol. 3, pp , R.A. Johnson R.A. and D.W. Wchern, Appled Multvarate Statstcal Analyss, by Prentce Hall, Inc., 3d. edton, W. Zhao, R. Chellappa and A. Krshnaswamy, Dscrmnant Analyss of Prncpal Components for Face Recognton, Proc. 2nd Internatonal Conference on Automatc Face and Gesture Recognton, pp , Japan, Aprl, 998.
Using mixture covariance matrices to improve face and facial expression recognitions
Pattern Recognton Letters 24 (2003) 2159 2165 www.elsever.com/locate/patrec Usng mxture covarance matrces to mprove face and facal expresson recogntons Carlos E. Thomaz a, *, Duncan F. Glles a, Raul Q.
More informationSingle and multiple stage classifiers implementing logistic discrimination
Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul  PUCRS Av. Ipranga,
More informationFace Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)
Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton
More informationL10: Linear discriminants analysis
L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationNonlinear data mapping by neural networks
Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal
More informationLETTER IMAGE RECOGNITION
LETTER IMAGE RECOGNITION 1. Introducton. 1. Introducton. Objectve: desgn classfers for letter mage recognton. consder accuracy and tme n takng the decson. 20,000 samples: Startng set: mages based on 20
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
More informationPSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 12
14 The Chsquared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed
More informationv a 1 b 1 i, a 2 b 2 i,..., a n b n i.
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are
More information8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by
6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationCommunication Networks II Contents
8 / 1  Communcaton Networs II (Görg)  www.comnets.unbremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP
More informationThe Analysis of Outliers in Statistical Data
THALES Project No. xxxx The Analyss of Outlers n Statstcal Data Research Team Chrysses Caron, Assocate Professor (P.I.) Vaslk Karot, Doctoral canddate Polychrons Economou, Chrstna Perrakou, Postgraduate
More informationLogistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification
Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson
More informationbenefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
More informationLogistic Regression. Steve Kroon
Logstc Regresson Steve Kroon Course notes sectons: 24.324.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro
More informationA Computer Technique for Solving LP Problems with Bounded Variables
Dhaka Unv. J. Sc. 60(2): 163168, 2012 (July) A Computer Technque for Solvng LP Problems wth Bounded Varables S. M. Atqur Rahman Chowdhury * and Sanwar Uddn Ahmad Department of Mathematcs; Unversty of
More informationgreatest common divisor
4. GCD 1 The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract  Stock market s one of the most complcated systems
More informationCS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering
Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that
More informationInequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.
Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.
More informationNaïve Bayes classifier & Evaluation framework
Lecture aïve Bayes classfer & Evaluaton framework Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Generatve approach to classfcaton Idea:. Represent and learn the dstrbuton p x, y. Use t to defne probablstc
More informationCS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements
Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there
More information9.1 The Cumulative Sum Control Chart
Learnng Objectves 9.1 The Cumulatve Sum Control Chart 9.1.1 Basc Prncples: Cusum Control Chart for Montorng the Process Mean If s the target for the process mean, then the cumulatve sum control chart s
More informationThe Development of Web Log Mining Based on ImproveKMeans Clustering Analysis
The Development of Web Log Mnng Based on ImproveKMeans Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.
More informationMANY machine learning and pattern recognition applications
1 Trace Rato Problem Revsted Yangqng Ja, Fepng Ne, and Changshu Zhang Abstract Dmensonalty reducton s an mportant ssue n many machne learnng and pattern recognton applcatons, and the trace rato problem
More informationEE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN
EE201 Crcut Theory I 2015 Sprng Dr. Yılmaz KALKAN 1. Basc Concepts (Chapter 1 of Nlsson  3 Hrs.) Introducton, Current and Voltage, Power and Energy 2. Basc Laws (Chapter 2&3 of Nlsson  6 Hrs.) Voltage
More informationGender Classification for RealTime Audience Analysis System
Gender Classfcaton for RealTme Audence Analyss System Vladmr Khryashchev, Lev Shmaglt, Andrey Shemyakov, Anton Lebedev Yaroslavl State Unversty Yaroslavl, Russa vhr@yandex.ru, shmaglt_lev@yahoo.com, andrey.shemakov@gmal.com,
More informationDescriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications
CMSC828G Prncples of Data Mnng Lecture #9 Today s Readng: HMS, chapter 9 Today s Lecture: Descrptve Modelng Clusterng Algorthms Descrptve Models model presents the man features of the data, a global summary
More informationSIX WAYS TO SOLVE A SIMPLE PROBLEM: FITTING A STRAIGHT LINE TO MEASUREMENT DATA
SIX WAYS TO SOLVE A SIMPLE PROBLEM: FITTING A STRAIGHT LINE TO MEASUREMENT DATA E. LAGENDIJK Department of Appled Physcs, Delft Unversty of Technology Lorentzweg 1, 68 CJ, The Netherlands Emal: e.lagendjk@tnw.tudelft.nl
More informationForecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network
700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School
More informationVision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION
Vson Mouse Saurabh Sarkar a* a Unversty of Cncnnat, Cncnnat, USA ABSTRACT The report dscusses a vson based approach towards trackng of eyes and fngers. The report descrbes the process of locatng the possble
More informationCan Auto Liability Insurance Purchases Signal Risk Attitude?
Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? ChuShu L Department of Internatonal Busness, Asa Unversty, Tawan ShengChang
More informationHYPOTHESIS TESTING OF PARAMETERS FOR ORDINARY LINEAR CIRCULAR REGRESSION
HYPOTHESIS TESTING OF PARAMETERS FOR ORDINARY LINEAR CIRCULAR REGRESSION Abdul Ghapor Hussn Centre for Foundaton Studes n Scence Unversty of Malaya 563 KUALA LUMPUR Emal: ghapor@umedumy Abstract Ths paper
More informationTime Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University
Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton
More informationOn the Optimal Control of a Cascade of HydroElectric Power Stations
On the Optmal Control of a Cascade of HydroElectrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;
More informationLearning from Multiple Outlooks
Learnng from Multple Outlooks Maayan Harel Department of Electrcal Engneerng, Technon, Hafa, Israel She Mannor Department of Electrcal Engneerng, Technon, Hafa, Israel maayanga@tx.technon.ac.l she@ee.technon.ac.l
More informationLecture 2: Absorbing states in Markov chains. Mean time to absorption. WrightFisher Model. Moran Model.
Lecture 2: Absorbng states n Markov chans. Mean tme to absorpton. WrghtFsher Model. Moran Model. Antonna Mtrofanova, NYU, department of Computer Scence December 8, 2007 Hgher Order Transton Probabltes
More informationHallucinating Multiple Occluded CCTV Face Images of Different Resolutions
In Proc. IEEE Internatonal Conference on Advanced Vdeo and Sgnal based Survellance (AVSS 05), September 2005 Hallucnatng Multple Occluded CCTV Face Images of Dfferent Resolutons Ku Ja Shaogang Gong Computer
More informationCalculation of Sampling Weights
Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a twostage stratfed cluster desgn. 1 The frst stage conssted of a sample
More informationInstitute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic
Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange
More informationThe Analysis of Covariance. ERSH 8310 Keppel and Wickens Chapter 15
The Analyss of Covarance ERSH 830 Keppel and Wckens Chapter 5 Today s Class Intal Consderatons Covarance and Lnear Regresson The Lnear Regresson Equaton TheAnalyss of Covarance Assumptons Underlyng the
More informationOcclusion Invariant Face Recognition Using Selective Lnmf Basis Images
Occluson Invarant Face Recognton Usng Selectve Lnmf Bass Images Hyun Jun Oh 1, Kyoung Mu Lee 2, Sang Uk Lee 2, and ChungHyuk Ym 3 1 SK Telecom, Korea purete5@nate.com 2 School of Electrcal Eng. and Computer
More informationLecture 18: Clustering & classification
O CPS260/BGT204. Algorthms n Computatonal Bology October 30, 2003 Lecturer: Pana K. Agarwal Lecture 8: Clusterng & classfcaton Scrbe: Daun Hou Open Problem In HomeWor 2, problem 5 has an open problem whch
More informationStudy on CET4 Marks in China s Graded English Teaching
Study on CET4 Marks n Chna s Graded Englsh Teachng CHE We College of Foregn Studes, Shandong Insttute of Busness and Technology, P.R.Chna, 264005 Abstract: Ths paper deploys Logt model, and decomposes
More informationMultivariate EWMA Control Chart
Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant
More informationRing structure of splines on triangulations
www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAMReport 201448 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon
More informationCHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol
CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK Sample Stablty Protocol Background The Cholesterol Reference Method Laboratory Network (CRMLN) developed certfcaton protocols for total cholesterol, HDL
More informationThe eigenvalue derivatives of linear damped systems
Control and Cybernetcs vol. 32 (2003) No. 4 The egenvalue dervatves of lnear damped systems by YeongJeu Sun Department of Electrcal Engneerng IShou Unversty Kaohsung, Tawan 840, R.O.C emal: yjsun@su.edu.tw
More informationIMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUAREROOT INTERVAL METHOD FOR MULTIPLE ZEROS 1
Nov Sad J. Math. Vol. 36, No. 2, 2006, 009 IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUAREROOT INTERVAL METHOD FOR MULTIPLE ZEROS Modrag S. Petkovć 2, Dušan M. Mloševć 3 Abstract. A new theorem concerned
More informationRobust Design of Public Storage Warehouses. Yeming (Yale) Gong EMLYON Business School
Robust Desgn of Publc Storage Warehouses Yemng (Yale) Gong EMLYON Busness School Rene de Koster Rotterdam school of management, Erasmus Unversty Abstract We apply robust optmzaton and revenue management
More informationQuestions that we may have about the variables
Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent
More informationState function: eigenfunctions of hermitian operators> normalization, orthogonality completeness
Schroednger equaton Basc postulates of quantum mechancs. Operators: Hermtan operators, commutators State functon: egenfunctons of hermtan operators> normalzaton, orthogonalty completeness egenvalues and
More informationFault tolerance in cloud technologies presented as a service
Internatonal Scentfc Conference Computer Scence 2015 Pavel Dzhunev, PhD student Fault tolerance n cloud technologes presented as a servce INTRODUCTION Improvements n technques for vrtualzaton and performance
More informationFORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER
FORCED CONVECION HEA RANSFER IN A DOUBLE PIPE HEA EXCHANGER Dr. J. Mchael Doster Department of Nuclear Engneerng Box 7909 North Carolna State Unversty Ralegh, NC 276957909 Introducton he convectve heat
More informationProject Networks With MixedTime Constraints
Project Networs Wth MxedTme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa
More informationThe covariance is the two variable analog to the variance. The formula for the covariance between two variables is
Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.
More informationStatistical Methods to Develop Rating Models
Statstcal Methods to Develop Ratng Models [Evelyn Hayden and Danel Porath, Österrechsche Natonalbank and Unversty of Appled Scences at Manz] Source: The Basel II Rsk Parameters Estmaton, Valdaton, and
More informationRisk Model of LongTerm Production Scheduling in Open Pit Gold Mining
Rsk Model of LongTerm Producton Schedulng n Open Pt Gold Mnng R Halatchev 1 and P Lever 2 ABSTRACT Open pt gold mnng s an mportant sector of the Australan mnng ndustry. It uses large amounts of nvestments,
More informationOn Mean Squared Error of Hierarchical Estimator
S C H E D A E I N F O R M A T I C A E VOLUME 0 0 On Mean Squared Error of Herarchcal Estmator Stans law Brodowsk Faculty of Physcs, Astronomy, and Appled Computer Scence, Jagellonan Unversty, Reymonta
More informationData Visualization by Pairwise Distortion Minimization
Communcatons n Statstcs, Theory and Methods 34 (6), 005 Data Vsualzaton by Parwse Dstorton Mnmzaton By Marc Sobel, and Longn Jan Lateck* Department of Statstcs and Department of Computer and Informaton
More informationA DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATIONBASED OPTIMIZATION. Michael E. Kuhl Radhamés A. TolentinoPeña
Proceedngs of the 2008 Wnter Smulaton Conference S. J. Mason, R. R. Hll, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler eds. A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATIONBASED OPTIMIZATION
More informationMining Feature Importance: Applying Evolutionary Algorithms within a Webbased Educational System
Mnng Feature Importance: Applyng Evolutonary Algorthms wthn a Webbased Educatonal System Behrouz MINAEIBIDGOLI 1, and Gerd KORTEMEYER 2, and Wllam F. PUNCH 1 1 Genetc Algorthms Research and Applcatons
More informationLecture 5,6 Linear Methods for Classification. Summary
Lecture 5,6 Lnear Methods for Classfcaton Rce ELEC 697 Farnaz Koushanfar Fall 2006 Summary Bayes Classfers Lnear Classfers Lnear regresson of an ndcator matrx Lnear dscrmnant analyss (LDA) Logstc regresson
More informationThe Probit Model. Alexander Spermann. SoSe 2009
The Probt Model Aleander Spermann Unversty of Freburg SoSe 009 Course outlne. Notaton and statstcal foundatons. Introducton to the Probt model 3. Applcaton 4. Coeffcents and margnal effects 5. Goodnessofft
More informationPERRON FROBENIUS THEOREM
PERRON FROBENIUS THEOREM R. CLARK ROBINSON Defnton. A n n matrx M wth real entres m, s called a stochastc matrx provded () all the entres m satsfy 0 m, () each of the columns sum to one, m = for all, ()
More informationNumber of Levels Cumulative Annual operating Income per year construction costs costs ($) ($) ($) 1 600,000 35,000 100,000 2 2,200,000 60,000 350,000
Problem Set 5 Solutons 1 MIT s consderng buldng a new car park near Kendall Square. o unversty funds are avalable (overhead rates are under pressure and the new faclty would have to pay for tself from
More informationSection 5.4 Annuities, Present Value, and Amortization
Secton 5.4 Annutes, Present Value, and Amortzaton Present Value In Secton 5.2, we saw that the present value of A dollars at nterest rate per perod for n perods s the amount that must be deposted today
More information1E6 Electrical Engineering AC Circuit Analysis and Power Lecture 12: Parallel Resonant Circuits
E6 Electrcal Engneerng A rcut Analyss and Power ecture : Parallel esonant rcuts. Introducton There are equvalent crcuts to the seres combnatons examned whch exst n parallel confguratons. The ssues surroundng
More informationTable of Contents EQ.10...46 EQ.6...46 EQ.8...46
Table of Contents CHAPTER II  PATTERN RECOGNITION.... THE PATTERN RECOGNITION PROBLEM.... STATISTICAL FORMULATION OF CLASSIFIERS...6 3. CONCLUSIONS...30 UNDERSTANDING BAYES RULE...3 BAYESIAN THRESHOLD...33
More informationSensitivity Analysis in a Generic MultiAttribute Decision Support System
Senstvty Analyss n a Generc MultAttrbute Decson Support System Sxto RíosInsua, Antono Jménez and Alfonso Mateos Department of Artfcal Intellgence, Madrd Techncal Unversty Campus de Montegancedo s/n,
More informationRiskbased Fatigue Estimate of Deep Water Risers  Course Project for EM388F: Fracture Mechanics, Spring 2008
Rskbased Fatgue Estmate of Deep Water Rsers  Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn
More informationDEFINING %COMPLETE IN MICROSOFT PROJECT
CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMISP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,
More informationFast Fuzzy Clustering of Web Page Collections
Fast Fuzzy Clusterng of Web Page Collectons Chrstan Borgelt and Andreas Nürnberger Dept. of Knowledge Processng and Language Engneerng OttovonGuerckeUnversty of Magdeburg Unverstätsplatz, D396 Magdeburg,
More informationImplementation of Deutsch's Algorithm Using Mathcad
Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages  n "Machnes, Logc and Quantum Physcs"
More informationI. SCOPE, APPLICABILITY AND PARAMETERS Scope
D Executve Board Annex 9 Page A/R ethodologcal Tool alculaton of the number of sample plots for measurements wthn A/R D project actvtes (Verson 0) I. SOPE, PIABIITY AD PARAETERS Scope. Ths tool s applcable
More informationTime Delayed Independent Component Analysis for Data Quality Monitoring
IWSSIP 117th Internatonal Conference on Systems, Sgnals and Image Processng Tme Delayed Independent Component Analyss for Data Qualty Montorng José Márco Faer Sgnal Processng Laboratory, COE/Pol Federal
More informationErrorPropagation.nb 1. Error Propagation
ErrorPropagaton.nb Error Propagaton Suppose that we make observatons of a quantty x that s subject to random fluctuatons or measurement errors. Our best estmate of the true value for ths quantty s then
More informationA FASTER EXTERNAL SORTING ALGORITHM USING NO ADDITIONAL DISK SPACE
47 A FASTER EXTERAL SORTIG ALGORITHM USIG O ADDITIOAL DISK SPACE Md. Rafqul Islam +, Mohd. oor Md. Sap ++, Md. Sumon Sarker +, Sk. Razbul Islam + + Computer Scence and Engneerng Dscplne, Khulna Unversty,
More informationBERNSTEIN POLYNOMIALS
OnLne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful
More informationPerformance Analysis and Coding Strategy of ECOC SVMs
Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.6776 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School
More informationModelling highdimensional data by mixtures of factor analyzers
Computatonal Statstcs & Data Analyss 41 (2003) 379 388 www.elsever.com/locate/csda Modellng hghdmensonal data by mxtures of factor analyzers G.J. McLachlan, D. Peel, R.W. Bean Department of Mathematcs,
More informationNaive Rule Induction for Text Classification based on Keyphrases
Nave Rule Inducton for Text Classfcaton based on Keyphrases Nktas N. Karankolas & Chrstos Skourlas Department of Informatcs, Technologcal Educatonal Insttute of Athens, Greece. Abstract In ths paper,
More informationStochastic Protocol Modeling for Anomaly Based Network Intrusion Detection
Stochastc Protocol Modelng for Anomaly Based Network Intruson Detecton Juan M. EstevezTapador, Pedro GarcaTeodoro, and Jesus E. DazVerdejo Department of Electroncs and Computer Technology Unversty of
More informationAryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006
Aryabhata s Root Extracton Methods Abhshek Parakh Lousana State Unversty Aug 1 st 1 Introducton Ths artcle presents an analyss of the root extracton algorthms of Aryabhata gven n hs book Āryabhatīya [1,
More informationDesign and Development of a Security Evaluation Platform Based on International Standards
Internatonal Journal of Informatcs Socety, VOL.5, NO.2 (203) 780 7 Desgn and Development of a Securty Evaluaton Platform Based on Internatonal Standards Yuj Takahash and Yoshm Teshgawara Graduate School
More information2.4 Bivariate distributions
page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together
More informationANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING
ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING Matthew J. Lberatore, Department of Management and Operatons, Vllanova Unversty, Vllanova, PA 19085, 6105194390,
More informationSupport vector domain description
Pattern Recognton Letters 20 (1999) 1191±1199 www.elsever.nl/locate/patrec Support vector doman descrpton Davd M.J. Tax *,1, Robert P.W. Dun Pattern Recognton Group, Faculty of Appled Scence, Delft Unversty
More informationImplementation and Evaluation of a Random Forest Machine Learning Algorithm
Implementaton and Evaluaton of a Random Forest Machne Learnng Algorthm Vachaslau Sazonau Unversty of Manchester, Oxford Road, Manchester, M13 9PL,UK sazonauv@cs.manchester.ac.uk Abstract hs work s amed
More informationA BonusMalus System as a Markov SetChain
A Bonusalus System as a arov SetChan Famly and frst name: emec algorzata Organsaton: Warsaw School of Economcs Insttute of Econometrcs Al. epodleglosc 64 2554 Warszawa Poland Telephone number: +48 6
More information"Research Note" APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES *
Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC
More informationStudy on Model of Risks Assessment of Standard Operation in Rural Power Network
Study on Model of Rsks Assessment of Standard Operaton n Rural Power Network Qngj L 1, Tao Yang 2 1 Qngj L, College of Informaton and Electrcal Engneerng, Shenyang Agrculture Unversty, Shenyang 110866,
More informationRecurrence. 1 Definitions and main statements
Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.
More information1. Measuring association using correlation and regression
How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a
More informationEstimating the Number of Clusters in Genetics of Acute Lymphoblastic Leukemia Data
Journal of Al Azhar UnverstyGaza (Natural Scences), 2011, 13 : 109118 Estmatng the Number of Clusters n Genetcs of Acute Lymphoblastc Leukema Data Mahmoud K. Okasha, Khaled I.A. Almghar Department of
More informationAn Alternative Way to Measure Private Equity Performance
An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate
More informationSupport Vector Machines
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.
More informationLearning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing
Internatonal Journal of Machne Learnng and Computng, Vol. 4, No. 3, June 04 Learnng from Large Dstrbuted Data: A Scalng Down Samplng Scheme for Effcent Data Processng Che Ngufor and Janusz Wojtusak part
More information