Georey E. Hinton. University oftoronto. Technical Report CRGTR May 21, 1996 (revised Feb 27, 1997) Abstract


 Edwina Thornton
 2 years ago
 Views:
Transcription
1 The EM Algorthm for Mxtures of Factor Analyzers Zoubn Ghahraman Georey E. Hnton Department of Computer Scence Unversty oftoronto 6 Kng's College Road Toronto, Canada M5S A4 Emal: Techncal Report CRGTR96 May 2, 996 (revsed Feb 27, 997) Abstract Factor analyss, a statstcal method for modelng the covarance structure of hgh dmensonal data usng a small number of latent varables, can be extended by allowng derent local factor models n derent regons of the nput space. Ths results n a model whch concurrently performs clusterng and dmensonalty reducton, and can be thought of as a reduced dmenson mxture of Gaussans. We present an exact Expectaton{Maxmzaton algorthm for ttng the parameters of ths mxture of factor analyzers. Introducton Clusterng and dmensonalty reducton have long been consdered two of the fundamental problems n unsupervsed learnng (Duda & Hart, 973 Chapter 6). In clusterng, the goal s to group data ponts by smlartybetween ther features. Conversely, n dmensonalty reducton, the goal s to group (or compress) features that are hghly correlated. In ths paper we present an EM learnng algorthm for a method whch combnes one of the basc forms of dmensonalty reducton factor analyss wth a basc method for clusterng the Gaussan mxture model. What results s a statstcal method whch concurrently performs clusterng and, wthn each cluster, local dmensonalty reducton. Local dmensonalty reducton presents several benets over a scheme n whch clusterng and dmensonalty reducton are performed separately. Frst, derent features may be correlated wthn derent clusters and thus the metrc for dmensonalty reducton may need to vary between derent clusters. Conversely, the metrc nduced n dmensonalty reducton may gude the process of cluster formaton.e. derent clusters may appear more separated dependng on the local metrc. Recently, there has been a great deal of research on the topc of local dmensonalty reducton, resultng n several varants on the basc concept wth successful applcatons to character and face recognton (Bregler and Omohundro, 994 Kambhatla and Leen, 994 Sung and Poggo, 994 Schwenk and Mlgram, 995 Hnton et al., 995). The algorthm used by these authors for dmensonalty reducton s prncpal components analyss (PCA).
2  z? x Fgure : The factor analyss generatve model (n vector form). PCA, unlke maxmum lkelhood factor analyss (FA), does not dene a proper densty model for the data, as the cost of codng a data pont s equal anywhere along the prncpal component subspace (.e. the densty s unnormalzed along these drectons). Furthermore, PCA s not robust to ndependent nose n the features of the data (see Hnton et al., 996, for a comparson of PCA and FA models). Hnton, Dayan, and Revow (996), also explorng an applcaton to dgt recognton, were the rst to extend mxtures of prncpal components analyzers to a mxture of factor analyzers. Ther learnng algorthm conssted of an outer loop of approxmate EM to t the mxture components, combned wth an nner loop of gradent descent to t each ndvdual factor model. In ths note we present an exact EM algorthm for mxtures of factor analyzers whch obvates the need for an outer and nner loop. Ths smples the mplementaton, reduces the number of heurstc parameters (.e. learnng rates or steps of conugate gradent descent), and can potentally result n speedups. In the next secton we present background materal on factor analyss and the EM algorthm. Ths s followed by the dervaton of the learnng algorthm for mxture of factor analyzers n secton 3. We close wth a dscusson n secton 4. 2 Factor Analyss In maxmum lkelhood factor analyss (FA), a pdmensonal realvalued data vector x s modeled usng a kdmensonal vector of realvalued factors, z, where k s generally much smaller than p (Evertt, 984). The generatve model s gven by: x =z + u () where s known as the factor loadng matrx (see Fgure ). The factors z are assumed to be N (0 I) dstrbuted (zeromean ndependent normals, wth unt varance). The p dmensonal random varable u s dstrbuted N (0 ), where s a dagonal matrx. The dagonalty of soneofthekey assumptons of factor analyss: The observed varables are ndependent gven the factors. Accordng to ths model, x s therefore dstrbuted wth zero mean and covarance 0 + and the goal of factor analyss s to nd the and that best model the covarance structure of x. The factor varables z model correlatons between the elements of x, whle the u varables account for ndependent nose n each elementofx. The k factors play the same role as the prncpal components n PCA: They are nformatve proectons of the data. Gven and, the expected value of the factors can be 2
3 computed through the lnear proecton: E(zx) =x (2) where 0 ( + 0 ), a fact that results from the ont normalty of data and factors: " #! " # " #! x P = N : (3) z 0 0 I Note that snce s dagonal, the p p matrx ( + 0 ), can be ecently nverted usng the matrx nverson lemma: ( + 0 ) = ; (I + 0 ) 0 where I s the k k dentty matrx. Furthermore, t s possble (and n fact necessary for EM) to compute the second moment of the factors, E(zz 0 x) = Var(zx)+E(zx)E(zx) 0 = I ; +xx 0 0 (4) whch provdes a measure of uncertanty n the factors, a quantty that has no analogue n PCA. The expectatons (2) and (4) form the bass of the EM algorthm for maxmum lkelhood factor analyss (see Appendx A and Rubn & Thayer, 982): Estep: Compute E(zx )ande(zz 0 x ) for each data pont x, gven and. Mstep: = nx nx x E(zx ) 0! = l= = ( nx n dag x x 0 ; E[zx ]x 0 = E(zz 0 x l )! (5) ) (6) where the dag operator sets all the odagonal elements of a matrx to zero. 3 Mxture of Factor Analyzers Assume wehave a mxture of m factor analyzers ndexed by!, = ::: m. The generatve model now obeys the followng mxture dstrbuton (see Fgure 2): P (x) = mx Z = P (xz! )P (z! )P (! )dz: (7) As n regular factor analyss, the factors are all assumed to be N (0 I) dstrbuted, therefore, P (z! )=P (z) =N (0 I): (8) 3
4 ! S S  Sw / x z Fgure 2: The mxture of factor analyss generatve model. Whereas n factor analyss the data mean was rrelevant and was subtracted before ttng the model, here we have the freedom to gve each factor analyzer a derent mean,, thereby allowng each to model the data covarance structure n a derent part of nput space, P (xz! )=N ( + z ): (9) The parameters of ths model are f( ) m = g the vector parametrzes the adaptable mxng proportons, = P (! ). The latent varables n ths model are the factors z and the mxture ndcator varable!, where w = when the data pont was generated by!. For the Estep of the EM algorthm, one needs to compute expectatons of all the nteractons of the hdden varables that appear n the log lkelhood. Fortunately, the followng statements can be easly vered, Denng E[w zx ] = E[w x ] E[z! x ] (0) E[w zz 0 x ] = E[w x ] E[zz 0! x ]: () h = E[w x ] / P (x! )= N (x ; 0 + ) (2) and usng equatons (2) and (0) we obtan E[w zx ]=h (x ; ) (3) where 0 ( + 0 ). Smlarly, usng equatons (4) and () we obtan E[w zz 0 x ]=h I ; + (x ; )(x ; ) 0 0 : (4) The EM algorthm for mxtures of factor analyzers therefore becomes: Estep: Compute h, E[zx! ]ande[zz 0 x! ] for all data ponts and mxture components. Mstep: Solve a set of lnear equatons for,, and (see Appendx B). The mxture of factor analyzers s, n essence, a reduced dmensonalty mxture of Gaussans. Each factor analyzer ts a Gaussan to a porton of the data, weghted by the posteror probabltes, h. Snce the covarance matrx for each Gaussan s speced through the lower dmensonal factor loadng matrces, the model has mkp + p, rather than mp(p + )=2, parameters dedcated to modelng covarance structure. Note that each model can also be allowed to have a separate matrx. Ths, however, changes ts nterpretaton as sensor nose. 4
5 4 Dscusson We have descrbed an EM algorthm for ttng a mxture of factor analyzers. Matlab source code for the algorthm can be obtaned from ftp://ftp.cs.toronto.edu/pub/zoubn/ mfa.tar.gz. An extenson of ths archtecture to tme seres data, n whch both the factors z and the dscrete varables! depend on ther value at a prevous tme step, s currently beng developed. One of the mportant ssues not addressed n ths note s model selecton. In ttng a mxture of factor analyzers the modeler has two free parameters to decde: The number of factor analyzers to use (m), and the number of factor n each analyzer (k). One method by whch these can be selected s crossvaldaton: several values of m and k are t to the data and the log lkelhood on a valdaton set s used to select the nal values. Greedy methods based on prunng or growng the mxture may be more ecent at the cost of some performance loss. Alternatvely, a fulledged Bayesan analyss, n whch these model parameters are ntegrated over, may also be possble. Acknowledgements We thank C. Bshop for comments on the manuscrpt. The research was funded by grants from the Canadan Natural Scence and Engneerng Research Councl and the Ontaro Informaton Technology Research Center. GEH s the NesbttBurns fellow of the Canadan Insttute for Advanced Research. A EM for Factor Analyss The expected log lkelhood for factor analyss s Q = E " log Y = c ; n 2 log ;X = c ; n 2 log ;X (2) p=2 =2 expf; 2 [x ; z] 0 [x ; z]g E 2 x0 x ; x 0 z + 2 z0 0 2 x0 x ; x 0 # z E[zx ]+ 2 tr h 0 E[zz 0 x ] where c s a constant, ndependent of the parameters, and tr s the trace operator. To reestmate the factor loadng matrx = ; X x E[zx ] 0 + X l E[zz 0 x l ]=0 obtanng E[zz 0 x l ] 0! = X x E[zx ] 0 X l 5
6 from whch we get equaton (5). We reestmate the matrx through ts nverse, = n X 2 2 x x 0 ; E[zx ] x E[zz 0 x ] 0 Substtutng equaton (5), =0: n 2 = X 2 x x 0 ; 2 E[zx ] x 0 and usng the dagonal constrant, = n dag ( X x x 0 ; E[zx ]x 0 ) : B EM for Mxture of Factor Analyzers The expected log lkelhood for mxture of factor analyss s Q = E 2 4 Y Y log (2) p=2 =2 expf; 2 [x ; ; z] 0 3 w [x ; ; z]g 5 To ontly estmate the mean and the factor loadngs t s useful to dene an augmented column vector of factors " # z ~z = and an augmented factor loadng matrx ~ =[ ]. The expected log lkelhood s then Q = E 2 Y Y 4 log = c ; n 2 log ;X (2) p=2 =2 expf; 2 [x ; ~ ~z] 0 where c s a constant. To estmate ~ weset 3 [x ; ~ w ~z]g 5 2 h x 0 x ; h x 0 ~ E[~zx! ]+ h 2 h tr ~ 0 ~ E[~z~z 0 ~ = ; X h x E[~zx! ] 0 + h ~ E[~z~z 0 x! ]=0: Ths results n a lnear equaton for reestmatng the means and factor loadngs, h = ~ X = h x E[~zx! ] 0! X l h l E[~z~z 0 x l! ]! (5) 6
7 where and We reestmate the matrx E[~z~z 0 x l! ]= " E[zx! E[~zx! ]= ] " E[zz0 x l! ] E[zx l! ] E[zx l! ] 0 through ts nverse, settng # = n 2 ; X 2 h x x 0 ; h ~ E[~zx! ]x h ~ E[~z~z 0 x! ] ~ 0 =0: Substtutng equaton (5) for ~ and usng the dagonal constrant on = n dag 8 < :X h x ; ~ Fnally, to reestmate the mxng proportons we use the denton, = P (! )= Z we obtan, 9 = E[~zx! ] x 0 : (6) P (! x)p (x) dx: Snce h = P (! x ), usng the emprcal dstrbuton of the data as an estmate of P (x) we get References = n Bregler, C. and Omohundro, S. M. (994). Surface learnng wth applcatons to lpreadng. In Cowan, J. D., Tesauro, G., and Alspector, J., edtors, Advances n Neural Informaton Processng Systems 6, pages 43{50. Morgan Kaufman Publshers, San Francsco, CA. Duda, R. O. and Hart, P. E. (973). Pattern Classcaton and Scene Analyss. Wley, New York. Evertt, B. S. (984). An Introducton to Latent Varable Models. Chapman and Hall, London. Hnton, G., Revow, M., and Dayan, P. (995). Recognzng handwrtten dgts usng mxtures of Lnear models. In Tesauro, G., Touretzky, D., and Leen, T., edtors, Advances n Neural Informaton Processng Systems 7, pages 05{022. MIT Press, Cambrdge, MA. nx = h : Hnton, G. E., Dayan, P., and Revow, M. (996). handwrtten dgts. Submtted for Publcaton. Modelng the manfolds of Images of 7
8 Kambhatla, N. and Leen, T. K. (994). Fast nonlnear dmenson reducton. In Cowan, J. D., Tesauro, G., and Alspector, J., edtors, Advances n Neural Informaton Processng Systems 6, pages 52{59. Morgan Kaufman Publshers, San Francsco, CA. Rubn, D. and Thayer, D. (982). EM algorthms for ML factor analyss. Psychometrka, 47():69{76. Schwenk, H. and Mlgram, M. (995). Transformaton nvarant autoassocaton wth applcaton to handwrtten character recognton. In Tesauro, G., Touretzky, D., and Leen, T., edtors, Advances n Neural Informaton Processng Systems 7, pages 99{998. MIT Press, Cambrdge, MA. Sung, K.K. and Poggo, T. (994). Examplebased learnng for vewbased human face detecton. MIT AI Memo 52, CBCL Paper 2. 8
Modelling highdimensional data by mixtures of factor analyzers
Computatonal Statstcs & Data Analyss 41 (2003) 379 388 www.elsever.com/locate/csda Modellng hghdmensonal data by mxtures of factor analyzers G.J. McLachlan, D. Peel, R.W. Bean Department of Mathematcs,
More informationL10: Linear discriminants analysis
L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss
More informationVision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION
Vson Mouse Saurabh Sarkar a* a Unversty of Cncnnat, Cncnnat, USA ABSTRACT The report dscusses a vson based approach towards trackng of eyes and fngers. The report descrbes the process of locatng the possble
More informationData Visualization by Pairwise Distortion Minimization
Communcatons n Statstcs, Theory and Methods 34 (6), 005 Data Vsualzaton by Parwse Dstorton Mnmzaton By Marc Sobel, and Longn Jan Lateck* Department of Statstcs and Department of Computer and Informaton
More informationbenefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
More informationClustering Gene Expression Data. (Slides thanks to Dr. Mark Craven)
Clusterng Gene Epresson Data Sldes thanks to Dr. Mark Craven Gene Epresson Proles we ll assume we have a D matr o gene epresson measurements rows represent genes columns represent derent eperments tme
More informationFace Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)
Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton
More informationCS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements
Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there
More informationTime Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University
Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton
More informationForecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network
700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School
More informationNonlinear data mapping by neural networks
Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationCommunication Networks II Contents
8 / 1  Communcaton Networs II (Görg)  www.comnets.unbremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP
More informationCS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering
Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that
More informationReview of Hierarchical Models for Data Clustering and Visualization
Revew of Herarchcal Models for Data Clusterng and Vsualzaton Lola Vcente & Alfredo Velldo Grup de Soft Computng Seccó d Intel lgènca Artfcal Departament de Llenguatges Sstemes Informàtcs Unverstat Poltècnca
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract  Stock market s one of the most complcated systems
More informationMixtures of Factor Analyzers with Common Factor Loadings for the Clustering and Visualisation of HighDimensional Data
Mxtures of Factor Analyzers wth Common Factor Loadngs for the Clusterng and Vsualsaton of HghDmensonal Data Jangsun Baek 1 and Geoffrey J. McLachlan 2 1 Department of Statstcs, Chonnam Natonal Unversty,
More informationActive Learning for Interactive Visualization
Actve Learnng for Interactve Vsualzaton Tomoharu Iwata Nel Houlsby Zoubn Ghahraman Unversty of Cambrdge Unversty of Cambrdge Unversty of Cambrdge Abstract Many automatc vsualzaton methods have been. However,
More informationThe Development of Web Log Mining Based on ImproveKMeans Clustering Analysis
The Development of Web Log Mnng Based on ImproveKMeans Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.
More informationOutofSample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering
OutofSample Extensons for LLE, Isomap, MDS, Egenmaps, and Spectral Clusterng Yoshua Bengo, JeanFranços Paement, Pascal Vncent Olver Delalleau, Ncolas Le Roux and Mare Oumet Département d Informatque
More informationA DATA MINING APPLICATION IN A STUDENT DATABASE
JOURNAL OF AERONAUTICS AND SPACE TECHNOLOGIES JULY 005 VOLUME NUMBER (5357) A DATA MINING APPLICATION IN A STUDENT DATABASE Şenol Zafer ERDOĞAN Maltepe Ünversty Faculty of Engneerng BüyükbakkalköyIstanbul
More informationThe Analysis of Outliers in Statistical Data
THALES Project No. xxxx The Analyss of Outlers n Statstcal Data Research Team Chrysses Caron, Assocate Professor (P.I.) Vaslk Karot, Doctoral canddate Polychrons Economou, Chrstna Perrakou, Postgraduate
More informationFast Fuzzy Clustering of Web Page Collections
Fast Fuzzy Clusterng of Web Page Collectons Chrstan Borgelt and Andreas Nürnberger Dept. of Knowledge Processng and Language Engneerng OttovonGuerckeUnversty of Magdeburg Unverstätsplatz, D396 Magdeburg,
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationDEFINING %COMPLETE IN MICROSOFT PROJECT
CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMISP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,
More informationStatistical Methods to Develop Rating Models
Statstcal Methods to Develop Ratng Models [Evelyn Hayden and Danel Porath, Österrechsche Natonalbank and Unversty of Appled Scences at Manz] Source: The Basel II Rsk Parameters Estmaton, Valdaton, and
More informationSingle and multiple stage classifiers implementing logistic discrimination
Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul  PUCRS Av. Ipranga,
More informationPerformance Analysis of Energy Consumption of Smartphone Running Mobile Hotspot Application
Internatonal Journal of mart Grd and lean Energy Performance Analyss of Energy onsumpton of martphone Runnng Moble Hotspot Applcaton Yun on hung a chool of Electronc Engneerng, oongsl Unversty, 511 angdodong,
More informationInequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.
Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.
More information1 Approximation Algorithms
CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons
More informationThe eigenvalue derivatives of linear damped systems
Control and Cybernetcs vol. 32 (2003) No. 4 The egenvalue dervatves of lnear damped systems by YeongJeu Sun Department of Electrcal Engneerng IShou Unversty Kaohsung, Tawan 840, R.O.C emal: yjsun@su.edu.tw
More informationMAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date
Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPPATBDClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller
More informationLearning from Multiple Outlooks
Learnng from Multple Outlooks Maayan Harel Department of Electrcal Engneerng, Technon, Hafa, Israel She Mannor Department of Electrcal Engneerng, Technon, Hafa, Israel maayanga@tx.technon.ac.l she@ee.technon.ac.l
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
More informationFORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER
FORCED CONVECION HEA RANSFER IN A DOUBLE PIPE HEA EXCHANGER Dr. J. Mchael Doster Department of Nuclear Engneerng Box 7909 North Carolna State Unversty Ralegh, NC 276957909 Introducton he convectve heat
More informationLogistic Regression. Steve Kroon
Logstc Regresson Steve Kroon Course notes sectons: 24.324.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro
More informationCalculating the high frequency transmission line parameters of power cables
< ' Calculatng the hgh frequency transmsson lne parameters of power cables Authors: Dr. John Dcknson, Laboratory Servces Manager, N 0 RW E B Communcatons Mr. Peter J. Ncholson, Project Assgnment Manager,
More informationUsing Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions
Usng Mxture Covarance Matrces to Improve Face and Facal Expresson Recogntons Carlos E. homaz, Duncan F. Glles and Raul Q. Fetosa 2 Imperal College of Scence echnology and Medcne, Department of Computng,
More informationLecture 5,6 Linear Methods for Classification. Summary
Lecture 5,6 Lnear Methods for Classfcaton Rce ELEC 697 Farnaz Koushanfar Fall 2006 Summary Bayes Classfers Lnear Classfers Lnear regresson of an ndcator matrx Lnear dscrmnant analyss (LDA) Logstc regresson
More informationNeural Networks. Hervé Abdi. The University of Texas at Dallas
Neural Netorks. Hervé Abd The Unversty o Texas at Dallas Introducton Neural netorks are adaptve statstcal models based on an analogy th the structure o the bran. They are adaptve because they can learn
More informationMATHEMATICAL ENGINEERING TECHNICAL REPORTS. Sequential Optimizing Investing Strategy with Neural Networks
MATHEMATICAL ENGINEERING TECHNICAL REPORTS Sequental Optmzng Investng Strategy wth Neural Networks Ryo ADACHI and Akmch TAKEMURA METR 2010 03 February 2010 DEPARTMENT OF MATHEMATICAL INFORMATICS GRADUATE
More informationx f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60
BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true
More informationMean Field Theory for Sigmoid Belief Networks. Abstract
Journal of Artæcal Intellgence Research 4 è1996è 61 76 Submtted 11è95; publshed 3è96 Mean Feld Theory for Sgmod Belef Networks Lawrence K. Saul Tomm Jaakkola Mchael I. Jordan Center for Bologcal and Computatonal
More informationv a 1 b 1 i, a 2 b 2 i,..., a n b n i.
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are
More information1 De nitions and Censoring
De ntons and Censorng. Survval Analyss We begn by consderng smple analyses but we wll lead up to and take a look at regresson on explanatory factors., as n lnear regresson part A. The mportant d erence
More informationA cooperative connectionist IDS model to identify independent anomalous SNMP situations
A cooperatve connectonst IDS model to dentfy ndependent anomalous SNMP stuatons Álvaro Herrero, Emlo Corchado, José Manuel Sáz Department of Cvl Engneerng, Unversty of Burgos, Span escorchado@ubu.es Abstract
More information8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by
6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng
More informationEE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN
EE201 Crcut Theory I 2015 Sprng Dr. Yılmaz KALKAN 1. Basc Concepts (Chapter 1 of Nlsson  3 Hrs.) Introducton, Current and Voltage, Power and Energy 2. Basc Laws (Chapter 2&3 of Nlsson  6 Hrs.) Voltage
More informationTable of Contents EQ.10...46 EQ.6...46 EQ.8...46
Table of Contents CHAPTER II  PATTERN RECOGNITION.... THE PATTERN RECOGNITION PROBLEM.... STATISTICAL FORMULATION OF CLASSIFIERS...6 3. CONCLUSIONS...30 UNDERSTANDING BAYES RULE...3 BAYESIAN THRESHOLD...33
More informationErrorPropagation.nb 1. Error Propagation
ErrorPropagaton.nb Error Propagaton Suppose that we make observatons of a quantty x that s subject to random fluctuatons or measurement errors. Our best estmate of the true value for ths quantty s then
More informationCalculation of Sampling Weights
Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a twostage stratfed cluster desgn. 1 The frst stage conssted of a sample
More informationLoop Parallelization
  Loop Parallelzaton C52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I,J]+B[I,J] ED FOR ED FOR analyze
More informationOnLine Fault Detection in Wind Turbine Transmission System using Adaptive Filter and Robust Statistical Features
OnLne Fault Detecton n Wnd Turbne Transmsson System usng Adaptve Flter and Robust Statstcal Features Ruoyu L Remote Dagnostcs Center SKF USA Inc. 3443 N. Sam Houston Pkwy., Houston TX 77086 Emal: ruoyu.l@skf.com
More informationLatent Class Regression. Statistics for Psychosocial Research II: Structural Models December 4 and 6, 2006
Latent Class Regresson Statstcs for Psychosocal Research II: Structural Models December 4 and 6, 2006 Latent Class Regresson (LCR) What s t and when do we use t? Recall the standard latent class model
More informationCausal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting
Causal, Explanatory Forecastng Assumes causeandeffect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of
More informationRegression Models for a Binary Response Using EXCEL and JMP
SEMATECH 997 Statstcal Methods Symposum Austn Regresson Models for a Bnary Response Usng EXCEL and JMP Davd C. Trndade, Ph.D. STATTECH Consultng and Tranng n Appled Statstcs San Jose, CA Topcs Practcal
More informationThe Distribution of Eigenvalues of Covariance Matrices of Residuals in Analysis of Variance
JOURNAL OF RESEARCH of the Natonal Bureau of Standards  B. Mathem atca l Scence s Vol. 74B, No.3, JulySeptember 1970 The Dstrbuton of Egenvalues of Covarance Matrces of Resduals n Analyss of Varance
More informationLogistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification
Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson
More informationCan Auto Liability Insurance Purchases Signal Risk Attitude?
Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? ChuShu L Department of Internatonal Busness, Asa Unversty, Tawan ShengChang
More informationAn Algorithm for DataDriven Bandwidth Selection
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 25, NO. 2, FEBRUARY 2003 An Algorthm for DataDrven Bandwdth Selecton Dorn Comancu, Member, IEEE Abstract The analyss of a feature space
More informationBayesian Cluster Ensembles
Bayesan Cluster Ensembles Hongjun Wang 1, Hanhua Shan 2 and Arndam Banerjee 2 1 Informaton Research Insttute, Southwest Jaotong Unversty, Chengdu, Schuan, 610031, Chna 2 Department of Computer Scence &
More informationNPAR TESTS. OneSample ChiSquare Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6
PAR TESTS If a WEIGHT varable s specfed, t s used to replcate a case as many tmes as ndcated by the weght value rounded to the nearest nteger. If the workspace requrements are exceeded and samplng has
More informationA DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATIONBASED OPTIMIZATION. Michael E. Kuhl Radhamés A. TolentinoPeña
Proceedngs of the 2008 Wnter Smulaton Conference S. J. Mason, R. R. Hll, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler eds. A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATIONBASED OPTIMIZATION
More informationAn InterestOriented Network Evolution Mechanism for Online Communities
An InterestOrented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne
More informationMultiView Regression via Canonical Correlation Analysis
MultVew Regresson va Canoncal Correlaton Analyss Sham M. Kakade 1 and Dean P. Foster 2 1 Toyota Technologcal Insttute at Chcago Chcago, IL 60637 2 Unversty of Pennsylvana Phladelpha, PA 19104 Abstract.
More informationPortfolio Loss Distribution
Portfolo Loss Dstrbuton Rsky assets n loan ortfolo hghly llqud assets holdtomaturty n the bank s balance sheet Outstandngs The orton of the bank asset that has already been extended to borrowers. Commtment
More informationThe Application of Fractional Brownian Motion in Option Pricing
Vol. 0, No. (05), pp. 738 http://dx.do.org/0.457/jmue.05.0..6 The Applcaton of Fractonal Brownan Moton n Opton Prcng Qngxn Zhou School of Basc Scence,arbn Unversty of Commerce,arbn zhouqngxn98@6.com
More informationGaining Insights to the Tea Industry of Sri Lanka using Data Mining
Proceedngs of the Internatonal MultConference of Engneers and Computer Scentsts 2008 Vol I Ganng Insghts to the Tea Industry of Sr Lanka usng Data Mnng H.C. Fernando, W. M. R Tssera, and R. I. Athauda
More informationA Structure for General and Specc Market Rsk Eckhard Platen 1 and Gerhard Stahl Summary. The paper presents a consstent approach to the modelng of general and specc market rsk as dened n regulatory documents.
More informationA FEATURE SELECTION AGENTBASED IDS
A FEATURE SELECTION AGENTBASED IDS Emlo Corchado, Álvaro Herrero and José Manuel Sáz Department of Cvl Engneerng, Unversty of Burgos C/Francsco de Vtora s/n., 09006, Burgos, Span Phone: +34 947259395,
More informationCHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol
CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK Sample Stablty Protocol Background The Cholesterol Reference Method Laboratory Network (CRMLN) developed certfcaton protocols for total cholesterol, HDL
More informationA statistical approach to determine Microbiologically Influenced Corrosion (MIC) Rates of underground gas pipelines.
A statstcal approach to determne Mcrobologcally Influenced Corroson (MIC) Rates of underground gas ppelnes. by Lech A. Grzelak A thess submtted to the Delft Unversty of Technology n conformty wth the requrements
More informationAn interactive system for structurebased ASCII art creation
An nteractve system for structurebased ASCII art creaton Katsunor Myake Henry Johan Tomoyuk Nshta The Unversty of Tokyo Nanyang Technologcal Unversty Abstract NonPhotorealstc Renderng (NPR), whose am
More informationBypassing Synthesis: PLS for Face Recognition with Pose, LowResolution and Sketch
Bypassng Synthess: PLS for Face Recognton wth Pose, LowResoluton and Setch Abhshe Sharma Insttute of Advanced Computer Scence Unversty of Maryland, USA bhoaal@umacs.umd.edu Davd W Jacobs Insttute of Advanced
More informationRealistic Image Synthesis
Realstc Image Synthess  Combned Samplng and Path Tracng  Phlpp Slusallek Karol Myszkowsk Vncent Pegoraro Overvew: Today Combned Samplng (Multple Importance Samplng) Renderng and Measurng Equaton Random
More informationPREDICTION OF MISSING DATA IN CARDIOTOCOGRAMS USING THE EXPECTATION MAXIMIZATION ALGORITHM
1819 October 2001, Hotel Kontokal Bay, Corfu PREDICTIO OF MISSIG DATA I CARDIOTOCOGRAMS USIG THE EXPECTATIO MAXIMIZATIO ALGORITHM G. okas Department of Electrcal and Computer Engneerng, Unversty of Patras,
More informationStatistical Approach for Offline Handwritten Signature Verification
Journal of Computer Scence 4 (3): 181185, 2008 ISSN 15493636 2008 Scence Publcatons Statstcal Approach for Offlne Handwrtten Sgnature Verfcaton 2 Debnath Bhattacharyya, 1 Samr Kumar Bandyopadhyay, 2
More informationA study on the ability of Support Vector Regression and Neural Networks to Forecast Basic Time Series Patterns
A study on the ablty of Support Vector Regresson and Neural Networks to Forecast Basc Tme Seres Patterns Sven F. Crone, Jose Guajardo 2, and Rchard Weber 2 Lancaster Unversty, Department of Management
More informationAnalysis of Premium Liabilities for Australian Lines of Business
Summary of Analyss of Premum Labltes for Australan Lnes of Busness Emly Tao Honours Research Paper, The Unversty of Melbourne Emly Tao Acknowledgements I am grateful to the Australan Prudental Regulaton
More informationOffline Verification of Hand Written Signature using Adaptive Resonance Theory Net (Type1)
Internatonal Journal of Sgnal Processng Systems Vol, No June 203 Offlne Verfcaton of Hand Wrtten Sgnature usng Adaptve Resonance Theory Net (Type) Trtharaj Dash Veer Surendra Sa Unversty of Technology,
More information8 Algorithm for Binary Searching in Trees
8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the
More informationApproximating Crossvalidatory Predictive Evaluation in Bayesian Latent Variables Models with Integrated IS and WAIC
Approxmatng Crossvaldatory Predctve Evaluaton n Bayesan Latent Varables Models wth Integrated IS and WAIC Longha L Department of Mathematcs and Statstcs Unversty of Saskatchewan Saskatoon, SK, CANADA
More informationTime Domain simulation of PD Propagation in XLPE Cables Considering Frequency Dependent Parameters
Internatonal Journal of Smart Grd and Clean Energy Tme Doman smulaton of PD Propagaton n XLPE Cables Consderng Frequency Dependent Parameters We Zhang a, Jan He b, Ln Tan b, Xuejun Lv b, HongJe L a *
More informationRiskbased Fatigue Estimate of Deep Water Risers  Course Project for EM388F: Fracture Mechanics, Spring 2008
Rskbased Fatgue Estmate of Deep Water Rsers  Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn
More informationLearning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing
Internatonal Journal of Machne Learnng and Computng, Vol. 4, No. 3, June 04 Learnng from Large Dstrbuted Data: A Scalng Down Samplng Scheme for Effcent Data Processng Che Ngufor and Janusz Wojtusak part
More informationVariance estimation for the instrumental variables approach to measurement error in generalized linear models
he Stata Journal (2003) 3, Number 4, pp. 342 350 Varance estmaton for the nstrumental varables approach to measurement error n generalzed lnear models James W. Hardn Arnold School of Publc Health Unversty
More informationA. Te densty matrx and densty operator In general, te manybody wave functon (q 1 ; :::; q 3N ; t) s far too large to calculate for a macroscopc syste
G25.2651: Statstcal Mecancs Notes for Lecture 13 I. PRINCIPLES OF QUANTUM STATISTICAL MECHANICS Te problem of quantum statstcal mecancs s te quantum mecancal treatment of an Npartcle system. Suppose te
More informationIDENTIFICATION AND CONTROL OF A FLEXIBLE TRANSMISSION SYSTEM
Abstract IDENTIFICATION AND CONTROL OF A FLEXIBLE TRANSMISSION SYSTEM Alca Esparza Pedro Dept. Sstemas y Automátca, Unversdad Poltécnca de Valenca, Span alespe@sa.upv.es The dentfcaton and control of a
More informationHow Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence
1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh
More informationTesting CABIDS through Mutations: on the Identification of Network Scans
Testng CABIDS through Mutatons: on the Identfcaton of Network Scans Emlo Corchado, Álvaro Herrero, José Manuel Sáz Department of Cvl Engneerng, Unversty of Burgos, Span {escorchado, ahcoso, msaz}@ubu.es
More informationBayesian Network Based Causal Relationship Identification and Funding Success Prediction in P2P Lending
Proceedngs of 2012 4th Internatonal Conference on Machne Learnng and Computng IPCSIT vol. 25 (2012) (2012) IACSIT Press, Sngapore Bayesan Network Based Causal Relatonshp Identfcaton and Fundng Success
More informationOnline Inference of Topics with Latent Dirichlet Allocation
Onlne Inference of Topcs wth Latent Drchlet Allocaton Kevn R. Cann Computer Scence Dvson Unversty of Calforna Berkeley, CA 94720 kevn@cs.berkeley.edu Le Sh Helen Wlls Neuroscence Insttute Unversty of Calforna
More informationChapter XX More advanced approaches to the analysis of survey data. Gad Nathan Hebrew University Jerusalem, Israel. Abstract
Household Sample Surveys n Developng and Transton Countres Chapter More advanced approaches to the analyss of survey data Gad Nathan Hebrew Unversty Jerusalem, Israel Abstract In the present chapter, we
More informationClustering Gene Expression Data
Clusterng Gene Expresson Data BMI/CS 576 www.bostat.wsc.edu/bm576/ Mark Craven craven@bostat.wsc.edu Fall 2011 Gene expresson profles we ll assume we have a 2D matrx of gene expresson measurements rows
More informationFaraday's Law of Induction
Introducton Faraday's Law o Inducton In ths lab, you wll study Faraday's Law o nducton usng a wand wth col whch swngs through a magnetc eld. You wll also examne converson o mechanc energy nto electrc energy
More informationTHE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek
HE DISRIBUION OF LOAN PORFOLIO VALUE * Oldrch Alfons Vascek he amount of captal necessary to support a portfolo of debt securtes depends on the probablty dstrbuton of the portfolo loss. Consder a portfolo
More informationTraffic State Estimation in the Traffic Management Center of Berlin
Traffc State Estmaton n the Traffc Management Center of Berln Authors: Peter Vortsch, PTV AG, Stumpfstrasse, D763 Karlsruhe, Germany phone ++49/72/965/35, emal peter.vortsch@ptv.de Peter Möhl, PTV AG,
More informationPrediction of Stock Market Index Movement by Ten Data Mining Techniques
Vol. 3, o. Modern Appled Scence Predcton of Stoc Maret Index Movement by en Data Mnng echnques Phchhang Ou (Correspondng author) School of Busness, Unversty of Shangha for Scence and echnology Rm 0, Internatonal
More informationExhaustive Regression. An Exploration of RegressionBased Data Mining Techniques Using Super Computation
Exhaustve Regresson An Exploraton of RegressonBased Data Mnng Technques Usng Super Computaton Antony Daves, Ph.D. Assocate Professor of Economcs Duquesne Unversty Pttsburgh, PA 58 Research Fellow The
More informationA random variable is a variable whose value depends on the outcome of a random event/experiment.
Random varables and Probablty dstrbutons A random varable s a varable whose value depends on the outcome of a random event/experment. For example, the score on the roll of a de, the heght of a randomly
More information