Online Learning from Experts: Minimax Regret

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Online Learning from Experts: Minimax Regret"

Transcription

1 E0 370 tatstcal Learnng Theory Lecture 2 Nov 24, 20) Onlne Learnng from Experts: Mn Regret Lecturer: hvan garwal crbe: Nkhl Vdhan Introducton In the last three lectures we have been dscussng the onlne learnng algorthms where we receve the nstance x t and then ts label y t for t =,..., T. pecfcally n the last lecture we talked about onlne learnng from experts and onlne predcton. We saw many algorthms lke Halvng algorthm, Weghted Majorty WM) algorthm and lastly Weghted Majorty Contnuous WMC) algorthm. We also saw bounds on the cumulatve loss ncurred by these algorthms. Today, we wll focus on onlne predcton. For the WMC algorthm the settng s: we have N experts who predct the outcome label) n [0,], then we combne these predctons usng a weghted average of these. Then we receve the true label and ncur some loss absolute loss n our settng), then we make an update to the weght vectors based on the loss. The ntuton s that the hgher the loss ncurred by an expert, the more drastcally we reduce ts weght. For the WMC algorthm, we proved that: [W MCη)] lnn) e η + η mn [ξ ] e η where the symbols have ther meanngs as explaned n the last lecture. lso f the number of examples T s known n advance, then we can choose the WMC parameter η s.t. [W MCη )] mn [ξ ] + 2T lnn) + lnn) 2 Mn Regret Lets defne the regret of an algorthm w.r.t. the set of experts ξ = {ξ,..., ξ N } on the sample = x, y ),..., x T, y T )) as: R l ξ,[] = L l [] mn L l [ξ ].e. the dfference between the total loss of your algorthm and the total loss of the best expert. Thus f we had chosen the best expert always for our predcton then our regret would be zero; ths makes sense as n onlne settng absolute loss s nsgnfcant as the adversary may always choose the label other than what we predcted. The goal of the predctor s to mnmze the regret whle the goal of adversary s to mze t. In general, we can formalze t as a game where at each tral you receve expert s predctons, then you make your own predcton and fnally the true label s revealed by the adversary. In ths settng we want to get close to the best possble loss mnmum regret). Thus we want to choose an algorthm that mnmzes the total regret, however we do not have any control over the set of expert predctons and the sample as adversary can choose a set of experts or sample whch gves a larger regret over the algorthm. Here, we can defne the mn value regret) of the game as: V l = mn ξ, Rl ξ,[] It s the worst case guarantee on the regret over all the sequences of labels and expert predctons. From the last lecture we know that: 2T lnn) + lnn) natural queston here s can we bound the mn regret for other loss functon? Is ths the best possble mn regret?

2 2 Onlne Learnng from Experts: Mn Regret 3 Mn regret for varous loss functons In ths secton we wll look at upper bound on mn regret for some other loss functons whch we have seen before. We wll focus on bnary outcomes y t {0, } wth experts predctng n [0,]; ξ t [0, ] and ŷ t [0, ]. ξ t can be seen as probablty of th expert predctng the label as. Defne the loss functon l : {0, } [0, ] [0, ) Lets us also defne l 0 ŷ) = l0, ŷ) and l ŷ) = l, ŷ). 3. quared Loss For squared loss ly, ŷ) = y ŷ) 2, t can be easly seen that l 0 ŷ) = ŷ 2 and l ŷ) = ŷ) 2. Let us defne some new quanttes assumng l 0 ) and l ) beng twce dfferentable): l ŷ) = l 0ŷ) l ŷ) l ŷ) l 0ŷ) R l ŷ) = l 0ŷ) l ŷ) 2 l ŷ) l 0ŷ) 2 l ŷ) c l = sup R l ŷ) 0<ŷ< Theorem 3.. Haussler et al., 998) Let l be such that l 0 0) = l ) = 0. Let l 0 ) be strctly ncreasng n 0,) and l ) be strctly decreasng n 0,). Let l 0 ) and l ) be three tmes dfferentable n 0,). If l ŷ) > 0 n 0,) and c l <, then: V l = ΘlnN)) pecfcally, we have V l c l lnn) and V l c l o)) lnn) For the squared loss: l 0ŷ) = 2ŷ, l ŷ) = 2ŷ ), l 0ŷ) = 2, l ŷ) = 2 Thus, l ŷ) = 4 > 0, and R l ŷ) = 2ŷ ŷ). Clearly, sup 0<ŷ< R l ŷ) = /2 <. Therefore, for squared loss we have, V lsq 2 lnn) 3.2 Logarthmc Loss Ths type of loss functon haven t been dscussed before n the lectures. Let us defne t frst. { lnŷ) f y = l log y, ŷ) = ln ŷ) f y = 0 For l log loss, l 0 ŷ) = ln ŷ) and l ŷ) = ln ŷ). mple mathematcs wll lead us to: Thus, we get l 0ŷ) = ŷ, l ŷ) = ŷ, l 0ŷ) = ŷ) 2, l ŷ) = ŷ 2 l ŷ) = Therefore for the logarthmc loss we get, ŷ) 2 ŷ 2 and subsequently R lŷ) =, c l = V l log lnn)

3 Onlne Learnng from Experts: Mn Regret bsolute Loss For the absolute loss we have l abs y, ŷ) = y ŷ, here t s easy to see that l 0 ŷ) = ŷ and l ŷ) = ŷ, for ŷ 0, ). lso l 0ŷ) =, l ŷ) =, l 0ŷ) = 0, l ŷ) = 0 nd therefore we get l ŷ) = 0 c l =. Thus, the above theorem doesn t apply n the absolute loss settng. Thus for the absolute loss we have the result of WMC algorthm from the last lecture whch gves an upper bound on the regret. Clearly for the squared and logarthmc loss we saw a much tghter bound; then a natural queston arses: can we do better for the absolute loss too? It turns out that we can only mprove the constants n the WMC regret bound for the absolute error case. Theorem 3.2. Let l be s.t. l 0 ) s strctly ncreasng n 0,) whle l ) s strctly decreasng n 0,) and both beng three tmes dfferentable n 0,). Then, If l ŷ) = 0 for some ŷ 0, ), then V = Ω T /6 ) lnn) T ) If l ŷ) < 0 or a < b, s.t. l ŷ) = 0 ŷ [a, b], then V = Ω lnn) Note. Clearly, l abs falls n the second category of the above theorem. T ) T ) = O lnn). Thus, = Θ lnn) []. nce, we have already seen that 4 Vovk s lgorthm, 990 Ths algorthm s very smlar to the Weghted Majorty algorthm. The dfference les n the update,.e. how do we combne the predctons from the experts to make our own predctons. lgorthm Vovk Parameters: c, η > 0 Intal weght vector w = [N] Loss functon l : {0, } [0, ] [0, ) For t =,..., T : Receve expert predctons ξ, t..., ξn t )[0, ] Compute y = c ln for y = 0,. wt e ηly,ξt ) wt Predct any ŷ satsfyng: ly, ŷ t ) y y {0, }. If such a ŷ, then the algorthm fals. Receve true label y t {0, } Incur loss ly t, ŷ) Update [N] : w t+ w t e ηlyt,ξ t ) nalyss of Vovk s lgorthm uppose that the algorthm doesn t fals. Defne U t = c lnw t ), where W t = N = wt. Then for each tral t, ly t, ŷ t ) U t+ U t t y ummng over t =,..., T, Now, L l [Vovkc, η)] U T + U T U T + = c lnw T + ) c lnw T + ), W T + w T +

4 4 Onlne Learnng from Experts: Mn Regret lso, U = c lnn) Thus, ) L l [Vovkc, η)] c lnn) + η mn L l [ξ ] Note. The above s true for all for whch the algorthm wth parameters c, η) doesn t fal. Thus gven the loss functon of nterest, f one can fnd the parameters c and η such that the algorthm never fals, then we get the above bound for all possble sequence. The proof of above nvolves takng c = c l, η = /c l wth l beng c, η)-realzable. howng n general that any loss functon satsfyng these condtons s realzable w.r.t. these values nvolves a sgnfcant amount of work. Defnton. Defne l to be c, η)-realzable f, ξ; Vovkc, η) doesn t fal. Now, we wll see what condtons are suffcent for the Vovk s algorthm not to fal. We wll verfy t for some loss functons. Logarthmc Loss Run Vovk algorthm wth c = c l and η = /c l. For the l log loss we have, c =, η =. Defne p t = w t N j= wt j to be the normalzed verson of the weght vector. ) ) 0 = c ln p t e ηl0ξt ) = ln p t e ln ξt ) = ln p t ξ t) We bascally need: whch s same as requrng: ) ) = c ln p t e ηlξt ) = ln p t e lnξt ) = ln p t ξ t) l 0 ŷ t ) ln p t ξ t) l ŷ t ) ln p t ξ t) l 0 ln pt ξ t)) ŷ t ŷ p t ξ t ln pt ξ t)) ŷ t ŷ p t ξ t l Thus there s only one value ŷ t = p t ξ t satsfyng t. Exercse. Work out the condtons for the Vovk s algorthm to be c, η)-realzable n the squared loss settng. bsolute Loss We start wth some η > 0 and c abs η) = 2. 2 ln +e η For the above choce of c, l abs turns out to be realzable for any η > 0. The bound that we get here s [Vovkη, c absη))] lnn) + η mn [ξ ] 2 2 ln +e η ) The above holds for any η > 0. The bound seems qute smlar to the one n Weghted Majorty algorthm except for the denomnator term. nce, t holds for all η, f we choose η carefully, we can get a bound of the form [Vovkη, c abs η ))] T lnn)+lnn)/2. For the absolute loss the best possble bound happens to be T/2) lnn + ) + lnn + )/2, whch s obtaned when we add an addtonal arbtrary expert ξ N+, who always predct the nverse of one of the experts say ξ ). Here, at least one of the two experts ξ and ξ N+ ) must have loss at most T/2 tmes.

5 Onlne Learnng from Experts: Mn Regret 5 5 Lower Bound on Mn Regret for bsolute Loss From earler secton we know that, = mn ξ, [] mn ) [ξ ] mn ξ, Rl abs ξ, [] Clam. For any and any ξ,..., ξ N [0, ] T some label sequence for whch the regret follows: [0,] T R l abs ξ, [] T 2 E [0,] T [mn ] [ξ ] Proof. t each tral the algorthm s predctng some value ŷ t. The label for the tral s unformly ether 0 or,.e. w.p. 2 the label s 0 and w.p. 2 t s. Thus, [ ] [ E [0,] T = 2 ŷt 0) + ] 2 ŷt ) T = T/2 The clam follows by observng that, [ ] R l abs [0,] T ξ, [] E R l abs [0,] T ξ, [] Now, = mn mn mn mn ξ { R l abs ξ {0,} T ) N E ξ {0,} T ) N ξ, [] } { } R l abs ξ, [] [ T [ 2 E [0,] T [ T 2 E ξ {0,} T ) N, [0,] T [ mn mn ] ] [ξ ] ] ] [ξ ] 6 Next Lecture In the next lecture we wll study the connectons of the onlne learnng wth the batch learnng and also see some methods to transform the onlne learnng problem nto a batch learnng problem. References [] Ncolo Cesa-Banch, Yoav Freund, Davd P. Helmbold, Davd Haussler, Robert E. chapre and Manfred K. Warmuth. How to use expert advce. Journal of the CM, 443): , 997. [2] Davd Haussler, Jyrk Kvnen and Manfred K. Warmuth. equental predcton of ndvdual sequences under general loss functons. IEEE Transactons on Informaton Theory, 445): , 998.

U.C. Berkeley CS270: Algorithms Lecture 4 Professor Vazirani and Professor Rao Jan 27,2011 Lecturer: Umesh Vazirani Last revised February 10, 2012

U.C. Berkeley CS270: Algorithms Lecture 4 Professor Vazirani and Professor Rao Jan 27,2011 Lecturer: Umesh Vazirani Last revised February 10, 2012 U.C. Berkeley CS270: Algorthms Lecture 4 Professor Vazran and Professor Rao Jan 27,2011 Lecturer: Umesh Vazran Last revsed February 10, 2012 Lecture 4 1 The multplcatve weghts update method The multplcatve

More information

Graph Theory and Cayley s Formula

Graph Theory and Cayley s Formula Graph Theory and Cayley s Formula Chad Casarotto August 10, 2006 Contents 1 Introducton 1 2 Bascs and Defntons 1 Cayley s Formula 4 4 Prüfer Encodng A Forest of Trees 7 1 Introducton In ths paper, I wll

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2

Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2 18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

1. Measuring association using correlation and regression

1. Measuring association using correlation and regression How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a

More information

II. PROBABILITY OF AN EVENT

II. PROBABILITY OF AN EVENT II. PROBABILITY OF AN EVENT As ndcated above, probablty s a quantfcaton, or a mathematcal model, of a random experment. Ths quantfcaton s a measure of the lkelhood that a gven event wll occur when the

More information

Solution : (a) FALSE. Let C be a binary one-error correcting code of length 9. Then it follows from the Sphere packing bound that.

Solution : (a) FALSE. Let C be a binary one-error correcting code of length 9. Then it follows from the Sphere packing bound that. MATH 29T Exam : Part I Solutons. TRUE/FALSE? Prove your answer! (a) (5 pts) There exsts a bnary one-error correctng code of length 9 wth 52 codewords. (b) (5 pts) There exsts a ternary one-error correctng

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

1 Example 1: Axis-aligned rectangles

1 Example 1: Axis-aligned rectangles COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei

b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei Mathematcal Propertes of the Least Squares Regresson The least squares regresson lne obeys certan mathematcal propertes whch are useful to know n practce. The followng propertes can be establshed algebracally:

More information

Formula of Total Probability, Bayes Rule, and Applications

Formula of Total Probability, Bayes Rule, and Applications 1 Formula of Total Probablty, Bayes Rule, and Applcatons Recall that for any event A, the par of events A and A has an ntersecton that s empty, whereas the unon A A represents the total populaton of nterest.

More information

The Mathematical Derivation of Least Squares

The Mathematical Derivation of Least Squares Pscholog 885 Prof. Federco The Mathematcal Dervaton of Least Squares Back when the powers that e forced ou to learn matr algera and calculus, I et ou all asked ourself the age-old queston: When the hell

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

Communication Networks II Contents

Communication Networks II Contents 8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

More information

A Note on the Decomposition of a Random Sample Size

A Note on the Decomposition of a Random Sample Size A Note on the Decomposton of a Random Sample Sze Klaus Th. Hess Insttut für Mathematsche Stochastk Technsche Unverstät Dresden Abstract Ths note addresses some results of Hess 2000) on the decomposton

More information

An Alternative Way to Measure Private Equity Performance

An Alternative Way to Measure Private Equity Performance An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate

More information

Finite Math Chapter 10: Study Guide and Solution to Problems

Finite Math Chapter 10: Study Guide and Solution to Problems Fnte Math Chapter 10: Study Gude and Soluton to Problems Basc Formulas and Concepts 10.1 Interest Basc Concepts Interest A fee a bank pays you for money you depost nto a savngs account. Prncpal P The amount

More information

Questions that we may have about the variables

Questions that we may have about the variables Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent

More information

Binary Dependent Variables. In some cases the outcome of interest rather than one of the right hand side variables is discrete rather than continuous

Binary Dependent Variables. In some cases the outcome of interest rather than one of the right hand side variables is discrete rather than continuous Bnary Dependent Varables In some cases the outcome of nterest rather than one of the rght hand sde varables s dscrete rather than contnuous The smplest example of ths s when the Y varable s bnary so that

More information

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

More information

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001. Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.

More information

Lossless Data Compression

Lossless Data Compression Lossless Data Compresson Lecture : Unquely Decodable and Instantaneous Codes Sam Rowes September 5, 005 Let s focus on the lossless data compresson problem for now, and not worry about nosy channel codng

More information

THE TITANIC SHIPWRECK: WHO WAS

THE TITANIC SHIPWRECK: WHO WAS THE TITANIC SHIPWRECK: WHO WAS MOST LIKELY TO SURVIVE? A STATISTICAL ANALYSIS Ths paper examnes the probablty of survvng the Ttanc shpwreck usng lmted dependent varable regresson analyss. Ths appled analyss

More information

v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

v a 1 b 1 i, a 2 b 2 i,..., a n b n i. SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

More information

Naïve Bayes classifier & Evaluation framework

Naïve Bayes classifier & Evaluation framework Lecture aïve Bayes classfer & Evaluaton framework Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Generatve approach to classfcaton Idea:. Represent and learn the dstrbuton p x, y. Use t to defne probablstc

More information

State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness

State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness Schroednger equaton Basc postulates of quantum mechancs. Operators: Hermtan operators, commutators State functon: egenfunctons of hermtan operators-> normalzaton, orthogonalty completeness egenvalues and

More information

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

EE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN

EE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN EE201 Crcut Theory I 2015 Sprng Dr. Yılmaz KALKAN 1. Basc Concepts (Chapter 1 of Nlsson - 3 Hrs.) Introducton, Current and Voltage, Power and Energy 2. Basc Laws (Chapter 2&3 of Nlsson - 6 Hrs.) Voltage

More information

Solution of Algebraic and Transcendental Equations

Solution of Algebraic and Transcendental Equations CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton

More information

Moment of a force about a point and about an axis

Moment of a force about a point and about an axis 3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should

More information

Extending Probabilistic Dynamic Epistemic Logic

Extending Probabilistic Dynamic Epistemic Logic Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set

More information

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2) MATH 16T Exam 1 : Part I (In-Class) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total

More information

The Probit Model. Alexander Spermann. SoSe 2009

The Probit Model. Alexander Spermann. SoSe 2009 The Probt Model Aleander Spermann Unversty of Freburg SoSe 009 Course outlne. Notaton and statstcal foundatons. Introducton to the Probt model 3. Applcaton 4. Coeffcents and margnal effects 5. Goodness-of-ft

More information

Math 31 Lesson Plan. Day 27: Fundamental Theorem of Finite Abelian Groups. Elizabeth Gillaspy. November 11, 2011

Math 31 Lesson Plan. Day 27: Fundamental Theorem of Finite Abelian Groups. Elizabeth Gillaspy. November 11, 2011 Math 31 Lesson Plan Day 27: Fundamental Theorem of Fnte Abelan Groups Elzabeth Gllaspy November 11, 2011 Supples needed: Colored chal Quzzes Homewor 4 envelopes: evals, HW, presentaton rubrcs, * probs

More information

The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method. Introduction. 0/1 Knapsack Problem The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

More information

Lecture 3: Annuity. Study annuities whose payments form a geometric progression or a arithmetic progression.

Lecture 3: Annuity. Study annuities whose payments form a geometric progression or a arithmetic progression. Lecture 3: Annuty Goals: Learn contnuous annuty and perpetuty. Study annutes whose payments form a geometrc progresson or a arthmetc progresson. Dscuss yeld rates. Introduce Amortzaton Suggested Textbook

More information

Logistic Regression. Steve Kroon

Logistic Regression. Steve Kroon Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro

More information

Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect

Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect Chapter 3 Group Theory p. - 3. Compact Course: Groups Theory emark: Ths s only a bref summary of most mportant results of groups theory wth respect to the applcatons dscussed n the followng chapters. For

More information

Learning Permutations with Exponential Weights

Learning Permutations with Exponential Weights Journal of Machne Learnng Research 2009 (10) 1705-1736 Submtted 9/08; Publshed 7/09 Learnng Permutatons wth Exponental Weghts Davd P. Helmbold Manfred K. Warmuth Computer Scence Department Unversty of

More information

2.4 Bivariate distributions

2.4 Bivariate distributions page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together

More information

8 Algorithm for Binary Searching in Trees

8 Algorithm for Binary Searching in Trees 8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

More information

Gibbs Free Energy and Chemical Equilibrium (or how to predict chemical reactions without doing experiments)

Gibbs Free Energy and Chemical Equilibrium (or how to predict chemical reactions without doing experiments) Gbbs Free Energy and Chemcal Equlbrum (or how to predct chemcal reactons wthout dong experments) OCN 623 Chemcal Oceanography Readng: Frst half of Chapter 3, Snoeynk and Jenkns (1980) Introducton We want

More information

An Interest-Oriented Network Evolution Mechanism for Online Communities

An Interest-Oriented Network Evolution Mechanism for Online Communities An Interest-Orented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne

More information

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

More information

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network 700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School

More information

Describing Communities. Species Diversity Concepts. Species Richness. Species Richness. Species-Area Curve. Species-Area Curve

Describing Communities. Species Diversity Concepts. Species Richness. Species Richness. Species-Area Curve. Species-Area Curve peces versty Concepts peces Rchness peces-area Curves versty Indces - mpson's Index - hannon-wener Index - rlloun Index peces Abundance Models escrbng Communtes There are two mportant descrptors of a communty:

More information

Section 2.3 Present Value of an Annuity; Amortization

Section 2.3 Present Value of an Annuity; Amortization Secton 2.3 Present Value of an Annuty; Amortzaton Prncpal Intal Value PV s the present value or present sum of the payments. PMT s the perodc payments. Gven r = 6% semannually, n order to wthdraw $1,000.00

More information

IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS 1

IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS 1 Nov Sad J. Math. Vol. 36, No. 2, 2006, 0-09 IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS Modrag S. Petkovć 2, Dušan M. Mloševć 3 Abstract. A new theorem concerned

More information

General Auction Mechanism for Search Advertising

General Auction Mechanism for Search Advertising General Aucton Mechansm for Search Advertsng Gagan Aggarwal S. Muthukrshnan Dávd Pál Martn Pál Keywords game theory, onlne auctons, stable matchngs ABSTRACT Internet search advertsng s often sold by an

More information

Texas Instruments 30X IIS Calculator

Texas Instruments 30X IIS Calculator Texas Instruments 30X IIS Calculator Keystrokes for the TI-30X IIS are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the

More information

Nonlinear data mapping by neural networks

Nonlinear data mapping by neural networks Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal

More information

New bounds in Balog-Szemerédi-Gowers theorem

New bounds in Balog-Szemerédi-Gowers theorem New bounds n Balog-Szemeréd-Gowers theorem By Tomasz Schoen Abstract We prove, n partcular, that every fnte subset A of an abelan group wth the addtve energy κ A 3 contans a set A such that A κ A and A

More information

9 Arithmetic and Geometric Sequence

9 Arithmetic and Geometric Sequence AAU - Busness Mathematcs I Lecture #5, Aprl 4, 010 9 Arthmetc and Geometrc Sequence Fnte sequence: 1, 5, 9, 13, 17 Fnte seres: 1 + 5 + 9 + 13 +17 Infnte sequence: 1,, 4, 8, 16,... Infnte seres: 1 + + 4

More information

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

More information

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that

More information

Nasdaq Iceland Bond Indices 01 April 2015

Nasdaq Iceland Bond Indices 01 April 2015 Nasdaq Iceland Bond Indces 01 Aprl 2015 -Fxed duraton Indces Introducton Nasdaq Iceland (the Exchange) began calculatng ts current bond ndces n the begnnng of 2005. They were a response to recent changes

More information

Using Alpha-Beta Associative Memories to Learn and Recall RGB Images

Using Alpha-Beta Associative Memories to Learn and Recall RGB Images Usng Alpha-Beta Assocatve Memores to Learn and Recall RGB Images Cornelo Yáñez-Márquez, María Elena Cruz-Meza, Flavo Arturo Sánchez-Garfas, and Itzamá López-Yáñez Centro de Investgacón en Computacón, Insttuto

More information

Implementation of Deutsch's Algorithm Using Mathcad

Implementation of Deutsch's Algorithm Using Mathcad Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages - n "Machnes, Logc and Quantum Physcs"

More information

A linear recurrence sequence of composite numbers

A linear recurrence sequence of composite numbers LMS J Comput Math 15 (2012) 360 373 C 2012 Author do:101112/s1461157012001143 A lnear recurrence sequence of composte numbers Jonas Šurys Abstract We prove that for each postve nteger k n the range 2 k

More information

A Simple Economic Model about the Teamwork Pedagogy

A Simple Economic Model about the Teamwork Pedagogy Appled Mathematcal Scences, Vol. 6, 01, no. 1, 13-0 A Smple Economc Model about the Teamwork Pedagog Gregor L. Lght Department of Management, Provdence College Provdence, Rhode Island 0918, USA glght@provdence.edu

More information

DEFINING %COMPLETE IN MICROSOFT PROJECT

DEFINING %COMPLETE IN MICROSOFT PROJECT CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMI-SP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

VLSI Technology Dr. Nandita Dasgupta Department of Electrical Engineering Indian Institute of Technology, Madras

VLSI Technology Dr. Nandita Dasgupta Department of Electrical Engineering Indian Institute of Technology, Madras VLI Technology Dr. Nandta Dasgupta Department of Electrcal Engneerng Indan Insttute of Technology, Madras Lecture - 11 Oxdaton I netcs of Oxdaton o, the unt process step that we are gong to dscuss today

More information

Linear Regression, Regularization Bias-Variance Tradeoff

Linear Regression, Regularization Bias-Variance Tradeoff HTF: Ch3, 7 B: Ch3 Lnear Regresson, Regularzaton Bas-Varance Tradeoff Thanks to C Guestrn, T Detterch, R Parr, N Ray 1 Outlne Lnear Regresson MLE = Least Squares! Bass functons Evaluatng Predctors Tranng

More information

INVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMA-HDR NETWORKS

INVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMA-HDR NETWORKS 21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech-2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS

More information

Passive Filters. References: Barbow (pp 265-275), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6)

Passive Filters. References: Barbow (pp 265-275), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6) Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called

More information

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60 BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.

More information

Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier

Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier Lesson 2 Chapter Two Three Phase Uncontrolled Rectfer. Operatng prncple of three phase half wave uncontrolled rectfer The half wave uncontrolled converter s the smplest of all three phase rectfer topologes.

More information

Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006

Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006 Aryabhata s Root Extracton Methods Abhshek Parakh Lousana State Unversty Aug 1 st 1 Introducton Ths artcle presents an analyss of the root extracton algorthms of Aryabhata gven n hs book Āryabhatīya [1,

More information

Usage of LCG/CLCG numbers for electronic gambling applications

Usage of LCG/CLCG numbers for electronic gambling applications Usage of LCG/CLCG numbers for electronc gamblng applcatons Anders Knutsson Smovts Consultng, Wenner-Gren Center, Sveavägen 166, 113 46 Stockholm, Sweden anders.knutsson@smovts.com Abstract. Several attacks

More information

Fisher Markets and Convex Programs

Fisher Markets and Convex Programs Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

More information

Online Multiple Kernel Learning: Algorithms and Mistake Bounds

Online Multiple Kernel Learning: Algorithms and Mistake Bounds Onlne Multple Kernel Learnng: Algorthms and Mstake Bounds Rong Jn 1, Steven C.H. Ho 2, and Tanbao Yang 1 1 Department of Computer Scence and Engneerng, Mchgan State Unversty, MI, 48824, USA 2 School of

More information

The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets

The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets . The Magnetc Feld Concepts and Prncples Movng Charges All charged partcles create electrc felds, and these felds can be detected by other charged partcles resultng n electrc force. However, a completely

More information

Getting Started. with Penfriend XP. 1 Installing 2 Trying it out! 3 Word Prediction. Reviewing Lexicons 5 Preferences 6 Finding More Help 7

Getting Started. with Penfriend XP. 1 Installing 2 Trying it out! 3 Word Prediction. Reviewing Lexicons 5 Preferences 6 Finding More Help 7 Gettng Started wth Penfrend XP Installng Tryng t out! Word Predcton Other Features Revewng Lexcons 5 Preferences 6 Fndng More Help 7 Installng 5 6 7 Before you can use Penfrend XP on your computer the

More information

Lecture 2: Absorbing states in Markov chains. Mean time to absorption. Wright-Fisher Model. Moran Model.

Lecture 2: Absorbing states in Markov chains. Mean time to absorption. Wright-Fisher Model. Moran Model. Lecture 2: Absorbng states n Markov chans. Mean tme to absorpton. Wrght-Fsher Model. Moran Model. Antonna Mtrofanova, NYU, department of Computer Scence December 8, 2007 Hgher Order Transton Probabltes

More information

Joe Pimbley, unpublished, 2005. Yield Curve Calculations

Joe Pimbley, unpublished, 2005. Yield Curve Calculations Joe Pmbley, unpublshed, 005. Yeld Curve Calculatons Background: Everythng s dscount factors Yeld curve calculatons nclude valuaton of forward rate agreements (FRAs), swaps, nterest rate optons, and forward

More information

Texas Instruments 30Xa Calculator

Texas Instruments 30Xa Calculator Teas Instruments 30Xa Calculator Keystrokes for the TI-30Xa are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the tet, check

More information

Simple Interest Loans (Section 5.1) :

Simple Interest Loans (Section 5.1) : Chapter 5 Fnance The frst part of ths revew wll explan the dfferent nterest and nvestment equatons you learned n secton 5.1 through 5.4 of your textbook and go through several examples. The second part

More information

Sensitivity Analysis in a Generic Multi-Attribute Decision Support System

Sensitivity Analysis in a Generic Multi-Attribute Decision Support System Senstvty Analyss n a Generc Mult-Attrbute Decson Support System Sxto Ríos-Insua, Antono Jménez and Alfonso Mateos Department of Artfcal Intellgence, Madrd Techncal Unversty Campus de Montegancedo s/n,

More information

14.74 Lecture 5: Health (2)

14.74 Lecture 5: Health (2) 14.74 Lecture 5: Health (2) Esther Duflo February 17, 2004 1 Possble Interventons Last tme we dscussed possble nterventons. Let s take one: provdng ron supplements to people, for example. From the data,

More information

SPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:

SPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background: SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and

More information

Marginal Returns to Education For Teachers

Marginal Returns to Education For Teachers The Onlne Journal of New Horzons n Educaton Volume 4, Issue 3 MargnalReturnstoEducatonForTeachers RamleeIsmal,MarnahAwang ABSTRACT FacultyofManagementand Economcs UnverstPenddkanSultan Idrs ramlee@fpe.ups.edu.my

More information

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.

More information

Capital asset pricing model, arbitrage pricing theory and portfolio management

Capital asset pricing model, arbitrage pricing theory and portfolio management Captal asset prcng model, arbtrage prcng theory and portfolo management Vnod Kothar The captal asset prcng model (CAPM) s great n terms of ts understandng of rsk decomposton of rsk nto securty-specfc rsk

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

The example below solves a system in the unknowns α and β:

The example below solves a system in the unknowns α and β: The Fnd Functon The functon Fnd returns a soluton to a system of equatons gven by a solve block. You can use Fnd to solve a lnear system, as wth lsolve, or to solve nonlnear systems. The example below

More information

LETTER IMAGE RECOGNITION

LETTER IMAGE RECOGNITION LETTER IMAGE RECOGNITION 1. Introducton. 1. Introducton. Objectve: desgn classfers for letter mage recognton. consder accuracy and tme n takng the decson. 20,000 samples: Startng set: mages based on 20

More information

Exceptionality and Natural Language Learning

Exceptionality and Natural Language Learning Exceptonalty and Natural Language Learnng Mha Rotaru Dane J. Ltman Computer Scence Department Unversty of Pttsburgh Pttsburgh, PA 5260 mrotaru, ltman @cs.ptt.edu Abstract Prevous work has argued that memory-based

More information

Properties of American Derivative Securities

Properties of American Derivative Securities Capter 6 Propertes of Amercan Dervatve Securtes 6.1 Te propertes Defnton 6.1 An Amercan dervatve securty s a sequence of non-negatve random varables fg k g n k= suc tat eac G k s F k -measurable. Te owner

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2 EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a non-zero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :

More information

Stochastic Bandits with Side Observations on Networks

Stochastic Bandits with Side Observations on Networks Stochastc Bandts wth Sde Observatons on Networks Swapna Buccapatnam, Atlla Erylmaz Department of ECE The Oho State Unversty Columbus, OH - 430 buccapat@eceosuedu, erylmaz@osuedu Ness B Shroff Departments

More information

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there

More information

Multivariate EWMA Control Chart

Multivariate EWMA Control Chart Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant

More information