Lecture 6: Bayesian Logistic Regression
|
|
- Rosemary Hutchinson
- 7 years ago
- Views:
Transcription
1 CSE : Topcs n Machne Learnng Lecture Date: Aprl th, 202 Lecture 6: Bayesan Logstc Regresson Lecturer: Bran Kuls Scrbe: Zq Huang Logstc Regresson Logstc Regresson s an approach to learnng functons of the form f : X Y or P (Y X), n the case where Y s dscrete-valued, and X < X...X n > s any vector contanng dscrete or contuous varables. For a two-class classfcaton problem, the posteror probablty of Y can be wrtten as follows: and P (Y X) + exp( ω n ω X ) σ(ω T X ) P (Y 0 X) exp( ω n ω X ) + exp(ω + n ω X ) σ(ω T X ) () (2) where σ( ) s the logstc sgmod functon defned by σ(a) + exp( a). (3) whch s plotted n Fgure. (Note we are mplctly redefnng the data X to add an extra dmenson wth Fgure : Plot of the logstc sgmod functon. a, as n lnear regresson, and then redefnng ω approprately.) The term sgmod means S-shaped. Ths
2 2 Lecture 6: Bayesan Logstc Regresson type of functon s sometmes also called a squashng functon because t maps the whole real axs nto a fnte nterval.it satsfes the followng symmetry property σ( a) σ(a) (4) Interestngly, the parametrc form of P (Y X) used by Logstc Regresson s precsely the form mpled by the assumpton of a Gaussan Nave Bayes classfer.. Form of P (Y X) for Gaussan Nave Bayes Classfer We derve the form of P (Y X) entaled by the assumpton of a Gaussan Nave Bayes (GNB) classfer. Consder a GNB based on the followng modelng assumpton: Y s Boolean, governed by a Bernoull dstrbuton, wth parameter π P (Y ). X < X...X n >, where each X s a contnuous random varable. For each X, P (X Y y k ) s a Guassan dstrbuton of the form N(µ k, σ ) (n many cases, ths wll smply be N(µ k, σ)). For all and j, X and X j are condtonally ndependent gven Y. Note here we are assumng the standard devatons σ vary from pont to pont, but do not depend on Y. We now derve the parametrc form of P (Y X) that follows from ths set of GNB assumptons. In general, Bayes rule allows us to wrte P (Y X) P (Y )P (X Y ) P (Y )P (X Y ) + P (Y 0)P (X Y 0) (5) Dvdng both the numerator and denomnator by the numerator yelds: P (Y X) + P (Y 0)P (X Y 0) P (Y )P (X Y ) P (Y 0)P (X Y 0) + exp(ln P (Y )P (X Y ) ) (7) P (Y 0) + exp(ln P (Y ) + P (X Y 0) ln + exp(ln π π + P (X Y 0) ln P (X Y ) ) (9) (6) P (X Y ) ) (8) Note the fnal step expresses P(Y0) and P(Y) n terms of the bnomal parameter π.
3 Lecture 6: Bayesan Logstc Regresson 3 Now consder just the summaton n the denomnator of equaton (9). Gven our assumpton that P (X Y y k ) s Gaussan, we can expand ths term as follows: ln P (X Y 0) P (X Y ) ln exp( (X µ2 0 ) 2πσ ) 2 2 exp( (X µ2 ) 2πσ ) 2 2σ 2 (0) ln exp( (X µ ) 2 (X µ 0 ) 2 2 ) () ( (X µ ) 2 (X µ 0 ) 2 2 ) (2) ( (X2 2X µ + µ 2 ) (X2 2X µ0 + µ 2 0 ) 2 ) (3) ( 2X (µ 0 µ ) + µ 2 µ2 0 2 ) (4) ( µ 0 µ sgma 2 X + µ2 2 ) (5) Note ths expresson s a lnear weghted sum of the X s. Substtutng expresson (5) back nto equaton (9), we have Or equvalently, P (Y X) + exp(ln π π + ( µ0 µ σ 2 X + µ2 µ2 0 )) 2 (6) where the weghts ω...ω n are gven by P (Y X) + exp(ω 0 + n ω X ) (7) ω µ 0 µ σ 2 (8) and ω 0 ln π π + µ 2 µ02 2σ 2 (9) Then we can derve P (Y X) σ(ω T X ) (20) And also we have P (Y 0 X) σ(ω T X ) (2) To summarze, the logstc form arses naturally from a generatve model. However, snce the number of parameters n a generatve model s often more than the number of parameters n the logstc regresson model, one often prefers workng drectly wth the logstc regresson model to fnd the parameters W. Ths s a dscrmnatve approach to classfcaton, as we drectly model the probabltes over the class labels.
4 4 Lecture 6: Bayesan Logstc Regresson 2 Estmatng Parameters for Logstc Regresson One reasonable approach to tranng Logstc Regresson s to choose parameter values that maxmze the condtonal data lkelhood. We choose parameters W arg max P (Y X, W ) W where W < ω 0, ω...ω n > s the vector of parameters to be estmated, Y l denotes the observed value of Y n the l th tranng example, and X l denotes the observed value n the l th tranng example. Equvalently, we can work wth the log of the condtonal lkelhood: And ln P (Y X, W ) W arg max W ln P (Y X, W ) n Y ln P (Y X, W ) + ( Y ) ln P (Y 0 X, W ) (22) n Y ln σ(ω T X ) + ( Y ) ln ( σ(ω T X )) (23) n σ(ω T X ) Y ln ( σ(ω T X ) + ln ( σ(ωt X )) (24) As usual, we can defne an error functon by takng the negatve logarthm of the lkelhood, whch gves the crossentropy error functon n the form E(W ) ln p(y W ) (25) y n ln y n + ( y n) ln ( y n ) (26) n Unfortunately, there s no closed form soluton to maxmzng the lkelhood wth respect to W. The sngularty can be avoded by ncluson of a pror and fndng a MAP soluton for w, or equvalently by addng a regularzaton term to the error functon. 2. Iteratve reweghted least squares In the case of the lnear regresson models, the maxmum lkelhood soluton, on the assumpton of a Gaussan nose model, leads to a closed-form soluton. However, for logstc regresson, there s no longer a closed-form soluton, due to the nonlnearty of the logstc sgmod functon. However, the departure from a quadratc form s not substantal. To be precse, the error functon s concave, as we shall see shortly, and hence has a unque mnmum. Furthermore, the error functon can be mnmzed by an effcent teratve technque based on the N ewton Raphson teratve optmzaton scheme, whch uses a local quadratc approxmaton to the log lkelhood functon. W (new) W (old) [ 2 σ(w (old) )] f(w (old)) (27) Then we can derve E(W ) (W T X n Y )X n (28) n X T XW X T Y (29)
5 Lecture 6: Bayesan Logstc Regresson 5 Plug n equaton (27), we can derve 2 E(W ) X n Xn T (30) n X T X (3) W (new) W (old) (X T X) {X T XW (old) X T Y } (32) (X T X) X T Y (33) Now let us apply the Newton-Raphson update to the cross-entropy error functon (26) for the logstc regresson model. E(W ) (y n y n)x n (34) n X T (Y Y ) (35) H E(W ) y n ( y n )x n x T n (36) X T RX (37) where R y n ( y n ), then we can derve W (new) W (old) (X T RX) X T (Y Y ) (38) (X T RX) {X T RXW (old) X T (Y Y )} (39) (X T RX) X T Rz (40) where z XW (old) R (Y Y ). n 2.2 Regularzaton n Logstc Regresson Overfttng the tranng data s a problem that can arse n Logstc Regresson, especally when data s very hgh dmensonal and tranng data s sparse. One approach to reducng overfttng s regularzaton, n whch we create a modfed penalzed log lkelhood functon, whch penelzes large value of W. One approach s to use the penalzed log lkelhood functon W arg max ln P (Y X, w) λ W 2 W 2 (4) Whch adds a penalty proportonal to the squared magntude of W. Here λ s a constant that determnes the strength of ths penalty term. 3 The Bayesan Settng Now we have some assmputons: P (W ) N(µ 0, σ 2 0)
6 6 Lecture 6: Bayesan Logstc Regresson P (Y X, W ) σ(w T X) P (W Y, X) P (Y X, W )P (W ) σ(w T X)P (W ) Then for a new data X new, we can derve the predctve dstrbuton: P (Y new X, Ȳ, X new) P (Y new Ȳ, X new)p (W Ȳ, X)d W (42) Snce P (Y new Ȳ, X new) s proportonal to logstc sgmod dstrbuton and P (W Ȳ, X) s proportonal to Normal dstrbuton, there s no closed form for P (Y new X, Ȳ, X new). There are several approaches to approxmatng the predctve dstrbuton: the Laplace approxmaton, varatonal methods, and Monte Carlo samplng are three of the man ones. Below we focus on the Laplace approxmaton. 4 The Laplace Approxmaton In ths secton, we ntroduce a framework called the Laplace approxmaton, that ams to fnd a Gaussan approxmaton to a probablty densty defned over a set of contnuous varables. We assume that there s a f(x) where exp(nf(x))dx has no closed form. In the Laplace method the goal s to fnd a Gaussan approxmaton g(z) whch s centred on a mode of the dstrbuton f(x). The frst step s to fnd a mode for f(x), n other words a pont x 0 such that f (x 0 ) 0. A Gaussan dstrbuton has the property that ts logarthm s a quadratc functon of the varables. We therefore consder a Taylor expanson of f(x) Snce f (x 0 ) 0 Therefore, f(x) f(x 0 ) + f (x 0 )! (x x 0 ) + f (x 0 ) (x x 0 ) 2 (43) f(x) f(x 0 ) + f (x 0 ) (x x 0 ) 2 (44) ( ( exp( Nf(x))dx exp N f(x 0 ) + f (x 0 ) (x x 0 ) ))dx 2 (45) ( exp( Nf(x 0 ) exp N f (x 0 ) (x x 0 ) )dx 2 (46) 2π N f (x 0 ) exp( Nf(x 0)) (47) So we can get a approxmate closed form soluton for f(x). Note that the approxmate ntegral s accurate to order O(/N). Fnally, gven a dstrbuton p(x), usng the Laplace approxmaton we form a Gaussan approxmaton wth mean x 0 and precson p (x 0 ), where x 0 s a mode of p. 4. Example: P (W Y ) P (Y W, X)P (W ) As we ntroduced before, for the predctve dstrbuton P (Y new X, Ȳ, X new) P (Y new W, X new )P (W Ȳ, X)d W (48) σ(w T X)P (W Ȳ, X)d w (49)
7 Lecture 6: Bayesan Logstc Regresson 7 we cannot get a closed form soluton. We wll use the Laplace approxmaton to approxmate the posteror P (W Ȳ, X) as a Gaussan. We further approxmate P (Y new Ȳ, X new) as a probt functon φ(λa) and get an approxmate soluton for the predctve dstrbuton. Because P (W Ȳ, X)d W s approxmated as Gaussan, we know that the margnal dstrbuton wll also be Gaussan. Snce σ(w T X) δ(a W T X)σ(a)d a (50) Then we derve σ(w T X)P (W Ȳ, X)d w σ(a)p (a)d a (5) where P (a) δ(a W T X)P (W Ȳ, X)d w then we can derve φ(λa)n(a µ, σ)d a σ(k(σ 2 )µ) (52) where k(σ 2 ) ( + πσ2 8 ) 2 and µ a p(a)ad a (53) P (W Ȳ, X)W T Xd W (54) W T MAP X (55) and also we can derve σa 2 p(a)(a 2 µ 2 )d a (56) P (W Ȳ, X)(W T X 2 m T N X 2 )d w (57) X T S N X (58)
Logistic Regression. Steve Kroon
Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro
More informationLogistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification
Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson
More informationCS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements
Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationTHE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES
The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered
More informationbenefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
More informationLecture 5,6 Linear Methods for Classification. Summary
Lecture 5,6 Lnear Methods for Classfcaton Rce ELEC 697 Farnaz Koushanfar Fall 2006 Summary Bayes Classfers Lnear Classfers Lnear regresson of an ndcator matrx Lnear dscrmnant analyss (LDA) Logstc regresson
More informationPSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12
14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed
More informationSupport Vector Machines
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.
More informationProbabilistic Linear Classifier: Logistic Regression. CS534-Machine Learning
robablstc Lnear Classfer: Logstc Regresson CS534-Machne Learnng Three Man Approaches to learnng a Classfer Learn a classfer: a functon f, ŷ f Learn a probablstc dscrmnatve model,.e., the condtonal dstrbuton
More informationL10: Linear discriminants analysis
L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss
More informationLecture 3: Force of Interest, Real Interest Rate, Annuity
Lecture 3: Force of Interest, Real Interest Rate, Annuty Goals: Study contnuous compoundng and force of nterest Dscuss real nterest rate Learn annuty-mmedate, and ts present value Study annuty-due, and
More informationThe OC Curve of Attribute Acceptance Plans
The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4
More information1 Example 1: Axis-aligned rectangles
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton
More informationTHE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek
HE DISRIBUION OF LOAN PORFOLIO VALUE * Oldrch Alfons Vascek he amount of captal necessary to support a portfolo of debt securtes depends on the probablty dstrbuton of the portfolo loss. Consder a portfolo
More informationProduction. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem.
Producer Theory Producton ASSUMPTION 2.1 Propertes of the Producton Set The producton set Y satsfes the followng propertes 1. Y s non-empty If Y s empty, we have nothng to talk about 2. Y s closed A set
More informationRegression Models for a Binary Response Using EXCEL and JMP
SEMATECH 997 Statstcal Methods Symposum Austn Regresson Models for a Bnary Response Usng EXCEL and JMP Davd C. Trndade, Ph.D. STAT-TECH Consultng and Tranng n Appled Statstcs San Jose, CA Topcs Practcal
More information) of the Cell class is created containing information about events associated with the cell. Events are added to the Cell instance
Calbraton Method Instances of the Cell class (one nstance for each FMS cell) contan ADC raw data and methods assocated wth each partcular FMS cell. The calbraton method ncludes event selecton (Class Cell
More informationHow Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence
1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh
More informationApproximating Cross-validatory Predictive Evaluation in Bayesian Latent Variables Models with Integrated IS and WAIC
Approxmatng Cross-valdatory Predctve Evaluaton n Bayesan Latent Varables Models wth Integrated IS and WAIC Longha L Department of Mathematcs and Statstcs Unversty of Saskatchewan Saskatoon, SK, CANADA
More informationAddendum to: Importing Skill-Biased Technology
Addendum to: Importng Skll-Based Technology Arel Bursten UCLA and NBER Javer Cravno UCLA August 202 Jonathan Vogel Columba and NBER Abstract Ths Addendum derves the results dscussed n secton 3.3 of our
More informationv a 1 b 1 i, a 2 b 2 i,..., a n b n i.
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are
More informationLuby s Alg. for Maximal Independent Sets using Pairwise Independence
Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent
More informationRealistic Image Synthesis
Realstc Image Synthess - Combned Samplng and Path Tracng - Phlpp Slusallek Karol Myszkowsk Vncent Pegoraro Overvew: Today Combned Samplng (Multple Importance Samplng) Renderng and Measurng Equaton Random
More informationCausal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting
Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of
More informationECE544NA Final Project: Robust Machine Learning Hardware via Classifier Ensemble
1 ECE544NA Fnal Project: Robust Machne Learnng Hardware va Classfer Ensemble Sa Zhang, szhang12@llnos.edu Dept. of Electr. & Comput. Eng., Unv. of Illnos at Urbana-Champagn, Urbana, IL, USA Abstract In
More informationGRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM
GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM BARRIOT Jean-Perre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jean-perre.barrot@cnes.fr 1/Introducton The
More informationFace Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)
Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton
More informationMachine Learning and Data Mining Lecture Notes
Machne Learnng and Data Mnng Lecture Notes CSC 411/D11 Computer Scence Department Unversty of Toronto Verson: February 6, 2012 Copyrght c 2010 Aaron Hertzmann and Davd Fleet CONTENTS Contents Conventons
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems
More informationRecurrence. 1 Definitions and main statements
Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.
More informationQuantization Effects in Digital Filters
Quantzaton Effects n Dgtal Flters Dstrbuton of Truncaton Errors In two's complement representaton an exact number would have nfntely many bts (n general). When we lmt the number of bts to some fnte value
More informationRisk-based Fatigue Estimate of Deep Water Risers -- Course Project for EM388F: Fracture Mechanics, Spring 2008
Rsk-based Fatgue Estmate of Deep Water Rsers -- Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn
More informationLatent Class Regression. Statistics for Psychosocial Research II: Structural Models December 4 and 6, 2006
Latent Class Regresson Statstcs for Psychosocal Research II: Structural Models December 4 and 6, 2006 Latent Class Regresson (LCR) What s t and when do we use t? Recall the standard latent class model
More informationSection 5.4 Annuities, Present Value, and Amortization
Secton 5.4 Annutes, Present Value, and Amortzaton Present Value In Secton 5.2, we saw that the present value of A dollars at nterest rate per perod for n perods s the amount that must be deposted today
More informationCalculation of Sampling Weights
Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a two-stage stratfed cluster desgn. 1 The frst stage conssted of a sample
More informationCHAPTER 14 MORE ABOUT REGRESSION
CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp
More informationActiveClean: Interactive Data Cleaning While Learning Convex Loss Models
ActveClean: Interactve Data Cleanng Whle Learnng Convex Loss Models Sanjay Krshnan, Jannan Wang, Eugene Wu, Mchael J. Frankln, Ken Goldberg UC Berkeley, Columba Unversty {sanjaykrshnan, jnwang, frankln,
More information1 De nitions and Censoring
De ntons and Censorng. Survval Analyss We begn by consderng smple analyses but we wll lead up to and take a look at regresson on explanatory factors., as n lnear regresson part A. The mportant d erence
More informationNPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6
PAR TESTS If a WEIGHT varable s specfed, t s used to replcate a case as many tmes as ndcated by the weght value rounded to the nearest nteger. If the workspace requrements are exceeded and samplng has
More information8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by
6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng
More informationRing structure of splines on triangulations
www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon
More informationSingle and multiple stage classifiers implementing logistic discrimination
Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul - PUCRS Av. Ipranga,
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationExhaustive Regression. An Exploration of Regression-Based Data Mining Techniques Using Super Computation
Exhaustve Regresson An Exploraton of Regresson-Based Data Mnng Technques Usng Super Computaton Antony Daves, Ph.D. Assocate Professor of Economcs Duquesne Unversty Pttsburgh, PA 58 Research Fellow The
More informationHow To Calculate The Accountng Perod Of Nequalty
Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.
More informationSIMPLE LINEAR CORRELATION
SIMPLE LINEAR CORRELATION Smple lnear correlaton s a measure of the degree to whch two varables vary together, or a measure of the ntensty of the assocaton between two varables. Correlaton often s abused.
More informationSection 5.3 Annuities, Future Value, and Sinking Funds
Secton 5.3 Annutes, Future Value, and Snkng Funds Ordnary Annutes A sequence of equal payments made at equal perods of tme s called an annuty. The tme between payments s the payment perod, and the tme
More information8 Algorithm for Binary Searching in Trees
8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the
More informationTrade Adjustment and Productivity in Large Crises. Online Appendix May 2013. Appendix A: Derivation of Equations for Productivity
Trade Adjustment Productvty n Large Crses Gta Gopnath Department of Economcs Harvard Unversty NBER Brent Neman Booth School of Busness Unversty of Chcago NBER Onlne Appendx May 2013 Appendx A: Dervaton
More informationForecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network
700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School
More informationGeorey E. Hinton. University oftoronto. Email: zoubin@cs.toronto.edu. Technical Report CRG-TR-96-1. May 21, 1996 (revised Feb 27, 1997) Abstract
The EM Algorthm for Mxtures of Factor Analyzers Zoubn Ghahraman Georey E. Hnton Department of Computer Scence Unversty oftoronto 6 Kng's College Road Toronto, Canada M5S A4 Emal: zoubn@cs.toronto.edu Techncal
More informationLeast Squares Fitting of Data
Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2016. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng
More informationInstitute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic
Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange
More informationNew Approaches to Support Vector Ordinal Regression
New Approaches to Support Vector Ordnal Regresson We Chu chuwe@gatsby.ucl.ac.uk Gatsby Computatonal Neuroscence Unt, Unversty College London, London, WCN 3AR, UK S. Sathya Keerth selvarak@yahoo-nc.com
More informationsubstances (among other variables as well). ( ) Thus the change in volume of a mixture can be written as
Mxtures and Solutons Partal Molar Quanttes Partal molar volume he total volume of a mxture of substances s a functon of the amounts of both V V n,n substances (among other varables as well). hus the change
More informationBERNSTEIN POLYNOMIALS
On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful
More informationA Practitioner's Guide to Generalized Linear Models
A Practtoner's Gude to Generalzed Lnear Models A CAS Study Note Duncan Anderson, FIA Sholom Feldblum, FCAS Claudne Modln, FCAS Dors Schrmacher, FCAS Ernesto Schrmacher, ASA Neeza Thand, FCAS Thrd Edton
More informationTexas Instruments 30X IIS Calculator
Texas Instruments 30X IIS Calculator Keystrokes for the TI-30X IIS are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the
More informationSection 2 Introduction to Statistical Mechanics
Secton 2 Introducton to Statstcal Mechancs 2.1 Introducng entropy 2.1.1 Boltzmann s formula A very mportant thermodynamc concept s that of entropy S. Entropy s a functon of state, lke the nternal energy.
More informationExtending Probabilistic Dynamic Epistemic Logic
Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set
More informationThe Cox-Ross-Rubinstein Option Pricing Model
Fnance 400 A. Penat - G. Pennacc Te Cox-Ross-Rubnsten Opton Prcng Model Te prevous notes sowed tat te absence o arbtrage restrcts te prce o an opton n terms o ts underlyng asset. However, te no-arbtrage
More informationEnergies of Network Nastsemble
Supplementary materal: Assessng the relevance of node features for network structure Gnestra Bancon, 1 Paolo Pn,, 3 and Matteo Marsl 1 1 The Abdus Salam Internatonal Center for Theoretcal Physcs, Strada
More informationLecture 2: Single Layer Perceptrons Kevin Swingler
Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCulloch-Ptts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses
More informationHallucinating Multiple Occluded CCTV Face Images of Different Resolutions
In Proc. IEEE Internatonal Conference on Advanced Vdeo and Sgnal based Survellance (AVSS 05), September 2005 Hallucnatng Multple Occluded CCTV Face Images of Dfferent Resolutons Ku Ja Shaogang Gong Computer
More informationFisher Markets and Convex Programs
Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and
More informationEstimation of Dispersion Parameters in GLMs with and without Random Effects
Mathematcal Statstcs Stockholm Unversty Estmaton of Dsperson Parameters n GLMs wth and wthout Random Effects Meng Ruoyan Examensarbete 2004:5 Postal address: Mathematcal Statstcs Dept. of Mathematcs Stockholm
More informationSolution: Let i = 10% and d = 5%. By definition, the respective forces of interest on funds A and B are. i 1 + it. S A (t) = d (1 dt) 2 1. = d 1 dt.
Chapter 9 Revew problems 9.1 Interest rate measurement Example 9.1. Fund A accumulates at a smple nterest rate of 10%. Fund B accumulates at a smple dscount rate of 5%. Fnd the pont n tme at whch the forces
More informationCharacterization of Assembly. Variation Analysis Methods. A Thesis. Presented to the. Department of Mechanical Engineering. Brigham Young University
Characterzaton of Assembly Varaton Analyss Methods A Thess Presented to the Department of Mechancal Engneerng Brgham Young Unversty In Partal Fulfllment of the Requrements for the Degree Master of Scence
More informationSeries Solutions of ODEs 2 the Frobenius method. The basic idea of the Frobenius method is to look for solutions of the form 3
Royal Holloway Unversty of London Department of Physs Seres Solutons of ODEs the Frobenus method Introduton to the Methodology The smple seres expanson method works for dfferental equatons whose solutons
More informationHedging Interest-Rate Risk with Duration
FIXED-INCOME SECURITIES Chapter 5 Hedgng Interest-Rate Rsk wth Duraton Outlne Prcng and Hedgng Prcng certan cash-flows Interest rate rsk Hedgng prncples Duraton-Based Hedgng Technques Defnton of duraton
More informationSolving Factored MDPs with Continuous and Discrete Variables
Solvng Factored MPs wth Contnuous and screte Varables Carlos Guestrn Berkeley Research Center Intel Corporaton Mlos Hauskrecht epartment of Computer Scence Unversty of Pttsburgh Branslav Kveton Intellgent
More information1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP)
6.3 / -- Communcaton Networks II (Görg) SS20 -- www.comnets.un-bremen.de Communcaton Networks II Contents. Fundamentals of probablty theory 2. Emergence of communcaton traffc 3. Stochastc & Markovan Processes
More informationGENETIC ALGORITHM FOR PROJECT SCHEDULING AND RESOURCE ALLOCATION UNDER UNCERTAINTY
Int. J. Mech. Eng. & Rob. Res. 03 Fady Safwat et al., 03 Research Paper ISS 78 049 www.jmerr.com Vol., o. 3, July 03 03 IJMERR. All Rghts Reserved GEETIC ALGORITHM FOR PROJECT SCHEDULIG AD RESOURCE ALLOCATIO
More informationSPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:
SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and
More informationBag-of-Words models. Lecture 9. Slides from: S. Lazebnik, A. Torralba, L. Fei-Fei, D. Lowe, C. Szurka
Bag-of-Words models Lecture 9 Sldes from: S. Lazebnk, A. Torralba, L. Fe-Fe, D. Lowe, C. Szurka Bag-of-features models Overvew: Bag-of-features models Orgns and motvaton Image representaton Dscrmnatve
More informationPRIVATE SCHOOL CHOICE: THE EFFECTS OF RELIGIOUS AFFILIATION AND PARTICIPATION
PRIVATE SCHOOL CHOICE: THE EFFECTS OF RELIIOUS AFFILIATION AND PARTICIPATION Danny Cohen-Zada Department of Economcs, Ben-uron Unversty, Beer-Sheva 84105, Israel Wllam Sander Department of Economcs, DePaul
More informationStatistical Methods to Develop Rating Models
Statstcal Methods to Develop Ratng Models [Evelyn Hayden and Danel Porath, Österrechsche Natonalbank and Unversty of Appled Scences at Manz] Source: The Basel II Rsk Parameters Estmaton, Valdaton, and
More informationMean Molecular Weight
Mean Molecular Weght The thermodynamc relatons between P, ρ, and T, as well as the calculaton of stellar opacty requres knowledge of the system s mean molecular weght defned as the mass per unt mole of
More informationProject Networks With Mixed-Time Constraints
Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa
More informationSurvival analysis methods in Insurance Applications in car insurance contracts
Survval analyss methods n Insurance Applcatons n car nsurance contracts Abder OULIDI 1 Jean-Mare MARION 2 Hervé GANACHAUD 3 Abstract In ths wor, we are nterested n survval models and ther applcatons on
More informationCHAPTER 5 RELATIONSHIPS BETWEEN QUANTITATIVE VARIABLES
CHAPTER 5 RELATIONSHIPS BETWEEN QUANTITATIVE VARIABLES In ths chapter, we wll learn how to descrbe the relatonshp between two quanttatve varables. Remember (from Chapter 2) that the terms quanttatve varable
More informationApplied Research Laboratory. Decision Theory and Receiver Design
Decson Theor and Recever Desgn Sgnal Detecton and Performance Estmaton Sgnal Processor Decde Sgnal s resent or Sgnal s not resent Nose Nose Sgnal? Problem: How should receved sgnals be rocessed n order
More informationPrediction of Stock Market Index Movement by Ten Data Mining Techniques
Vol. 3, o. Modern Appled Scence Predcton of Stoc Maret Index Movement by en Data Mnng echnques Phchhang Ou (Correspondng author) School of Busness, Unversty of Shangha for Scence and echnology Rm 0, Internatonal
More informationHow To Calculate An Approxmaton Factor Of 1 1/E
Approxmaton algorthms for allocaton problems: Improvng the factor of 1 1/e Urel Fege Mcrosoft Research Redmond, WA 98052 urfege@mcrosoft.com Jan Vondrák Prnceton Unversty Prnceton, NJ 08540 jvondrak@gmal.com
More informationPERRON FROBENIUS THEOREM
PERRON FROBENIUS THEOREM R. CLARK ROBINSON Defnton. A n n matrx M wth real entres m, s called a stochastc matrx provded () all the entres m satsfy 0 m, () each of the columns sum to one, m = for all, ()
More informationOptimal Bidding Strategies for Generation Companies in a Day-Ahead Electricity Market with Risk Management Taken into Account
Amercan J. of Engneerng and Appled Scences (): 8-6, 009 ISSN 94-700 009 Scence Publcatons Optmal Bddng Strateges for Generaton Companes n a Day-Ahead Electrcty Market wth Rsk Management Taken nto Account
More informationPrediction of Disability Frequencies in Life Insurance
Predcton of Dsablty Frequences n Lfe Insurance Bernhard Köng Fran Weber Maro V. Wüthrch October 28, 2011 Abstract For the predcton of dsablty frequences, not only the observed, but also the ncurred but
More informationWe are now ready to answer the question: What are the possible cardinalities for finite fields?
Chapter 3 Fnte felds We have seen, n the prevous chapters, some examples of fnte felds. For example, the resdue class rng Z/pZ (when p s a prme) forms a feld wth p elements whch may be dentfed wth the
More informationRotation Kinematics, Moment of Inertia, and Torque
Rotaton Knematcs, Moment of Inerta, and Torque Mathematcally, rotaton of a rgd body about a fxed axs s analogous to a lnear moton n one dmenson. Although the physcal quanttes nvolved n rotaton are qute
More informationThe Development of Web Log Mining Based on Improve-K-Means Clustering Analysis
The Development of Web Log Mnng Based on Improve-K-Means Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.
More informationQuestion 2: What is the variance and standard deviation of a dataset?
Queston 2: What s the varance and standard devaton of a dataset? The varance of the data uses all of the data to compute a measure of the spread n the data. The varance may be computed for a sample of
More informationHow To Find The Dsablty Frequency Of A Clam
1 Predcton of Dsablty Frequences n Lfe Insurance Bernhard Köng 1, Fran Weber 1, Maro V. Wüthrch 2 Abstract: For the predcton of dsablty frequences, not only the observed, but also the ncurred but not yet
More informationNMT EE 589 & UNM ME 482/582 ROBOT ENGINEERING. Dr. Stephen Bruder NMT EE 589 & UNM ME 482/582
NMT EE 589 & UNM ME 482/582 ROBOT ENGINEERING Dr. Stephen Bruder NMT EE 589 & UNM ME 482/582 7. Root Dynamcs 7.2 Intro to Root Dynamcs We now look at the forces requred to cause moton of the root.e. dynamcs!!
More informationSupport vector domain description
Pattern Recognton Letters 20 (1999) 1191±1199 www.elsever.nl/locate/patrec Support vector doman descrpton Davd M.J. Tax *,1, Robert P.W. Dun Pattern Recognton Group, Faculty of Appled Scence, Delft Unversty
More informationSVM Tutorial: Classification, Regression, and Ranking
SVM Tutoral: Classfcaton, Regresson, and Rankng Hwanjo Yu and Sungchul Km 1 Introducton Support Vector Machnes(SVMs) have been extensvely researched n the data mnng and machne learnng communtes for the
More informationDiscussion Papers. Support Vector Machines (SVM) as a Technique for Solvency Analysis. Laura Auria Rouslan A. Moro. Berlin, August 2008
Deutsches Insttut für Wrtschaftsforschung www.dw.de Dscusson Papers 8 Laura Aura Rouslan A. Moro Support Vector Machnes (SVM) as a Technque for Solvency Analyss Berln, August 2008 Opnons expressed n ths
More informationFaraday's Law of Induction
Introducton Faraday's Law o Inducton In ths lab, you wll study Faraday's Law o nducton usng a wand wth col whch swngs through a magnetc eld. You wll also examne converson o mechanc energy nto electrc energy
More informationIntroduction to Statistical Physics (2SP)
Introducton to Statstcal Physcs (2SP) Rchard Sear March 5, 20 Contents What s the entropy (aka the uncertanty)? 2. One macroscopc state s the result of many many mcroscopc states.......... 2.2 States wth
More informationn + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)
MATH 16T Exam 1 : Part I (In-Class) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total
More information