# Online Learning from Experts: Minimax Regret

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 E0 370 tatstcal Learnng Theory Lecture 2 Nov 24, 20) Onlne Learnng from Experts: Mn Regret Lecturer: hvan garwal crbe: Nkhl Vdhan Introducton In the last three lectures we have been dscussng the onlne learnng algorthms where we receve the nstance x t and then ts label y t for t =,..., T. pecfcally n the last lecture we talked about onlne learnng from experts and onlne predcton. We saw many algorthms lke Halvng algorthm, Weghted Majorty WM) algorthm and lastly Weghted Majorty Contnuous WMC) algorthm. We also saw bounds on the cumulatve loss ncurred by these algorthms. Today, we wll focus on onlne predcton. For the WMC algorthm the settng s: we have N experts who predct the outcome label) n [0,], then we combne these predctons usng a weghted average of these. Then we receve the true label and ncur some loss absolute loss n our settng), then we make an update to the weght vectors based on the loss. The ntuton s that the hgher the loss ncurred by an expert, the more drastcally we reduce ts weght. For the WMC algorthm, we proved that: [W MCη)] lnn) e η + η mn [ξ ] e η where the symbols have ther meanngs as explaned n the last lecture. lso f the number of examples T s known n advance, then we can choose the WMC parameter η s.t. [W MCη )] mn [ξ ] + 2T lnn) + lnn) 2 Mn Regret Lets defne the regret of an algorthm w.r.t. the set of experts ξ = {ξ,..., ξ N } on the sample = x, y ),..., x T, y T )) as: R l ξ,[] = L l [] mn L l [ξ ].e. the dfference between the total loss of your algorthm and the total loss of the best expert. Thus f we had chosen the best expert always for our predcton then our regret would be zero; ths makes sense as n onlne settng absolute loss s nsgnfcant as the adversary may always choose the label other than what we predcted. The goal of the predctor s to mnmze the regret whle the goal of adversary s to mze t. In general, we can formalze t as a game where at each tral you receve expert s predctons, then you make your own predcton and fnally the true label s revealed by the adversary. In ths settng we want to get close to the best possble loss mnmum regret). Thus we want to choose an algorthm that mnmzes the total regret, however we do not have any control over the set of expert predctons and the sample as adversary can choose a set of experts or sample whch gves a larger regret over the algorthm. Here, we can defne the mn value regret) of the game as: V l = mn ξ, Rl ξ,[] It s the worst case guarantee on the regret over all the sequences of labels and expert predctons. From the last lecture we know that: 2T lnn) + lnn) natural queston here s can we bound the mn regret for other loss functon? Is ths the best possble mn regret?

2 2 Onlne Learnng from Experts: Mn Regret 3 Mn regret for varous loss functons In ths secton we wll look at upper bound on mn regret for some other loss functons whch we have seen before. We wll focus on bnary outcomes y t {0, } wth experts predctng n [0,]; ξ t [0, ] and ŷ t [0, ]. ξ t can be seen as probablty of th expert predctng the label as. Defne the loss functon l : {0, } [0, ] [0, ) Lets us also defne l 0 ŷ) = l0, ŷ) and l ŷ) = l, ŷ). 3. quared Loss For squared loss ly, ŷ) = y ŷ) 2, t can be easly seen that l 0 ŷ) = ŷ 2 and l ŷ) = ŷ) 2. Let us defne some new quanttes assumng l 0 ) and l ) beng twce dfferentable): l ŷ) = l 0ŷ) l ŷ) l ŷ) l 0ŷ) R l ŷ) = l 0ŷ) l ŷ) 2 l ŷ) l 0ŷ) 2 l ŷ) c l = sup R l ŷ) 0<ŷ< Theorem 3.. Haussler et al., 998) Let l be such that l 0 0) = l ) = 0. Let l 0 ) be strctly ncreasng n 0,) and l ) be strctly decreasng n 0,). Let l 0 ) and l ) be three tmes dfferentable n 0,). If l ŷ) > 0 n 0,) and c l <, then: V l = ΘlnN)) pecfcally, we have V l c l lnn) and V l c l o)) lnn) For the squared loss: l 0ŷ) = 2ŷ, l ŷ) = 2ŷ ), l 0ŷ) = 2, l ŷ) = 2 Thus, l ŷ) = 4 > 0, and R l ŷ) = 2ŷ ŷ). Clearly, sup 0<ŷ< R l ŷ) = /2 <. Therefore, for squared loss we have, V lsq 2 lnn) 3.2 Logarthmc Loss Ths type of loss functon haven t been dscussed before n the lectures. Let us defne t frst. { lnŷ) f y = l log y, ŷ) = ln ŷ) f y = 0 For l log loss, l 0 ŷ) = ln ŷ) and l ŷ) = ln ŷ). mple mathematcs wll lead us to: Thus, we get l 0ŷ) = ŷ, l ŷ) = ŷ, l 0ŷ) = ŷ) 2, l ŷ) = ŷ 2 l ŷ) = Therefore for the logarthmc loss we get, ŷ) 2 ŷ 2 and subsequently R lŷ) =, c l = V l log lnn)

3 Onlne Learnng from Experts: Mn Regret bsolute Loss For the absolute loss we have l abs y, ŷ) = y ŷ, here t s easy to see that l 0 ŷ) = ŷ and l ŷ) = ŷ, for ŷ 0, ). lso l 0ŷ) =, l ŷ) =, l 0ŷ) = 0, l ŷ) = 0 nd therefore we get l ŷ) = 0 c l =. Thus, the above theorem doesn t apply n the absolute loss settng. Thus for the absolute loss we have the result of WMC algorthm from the last lecture whch gves an upper bound on the regret. Clearly for the squared and logarthmc loss we saw a much tghter bound; then a natural queston arses: can we do better for the absolute loss too? It turns out that we can only mprove the constants n the WMC regret bound for the absolute error case. Theorem 3.2. Let l be s.t. l 0 ) s strctly ncreasng n 0,) whle l ) s strctly decreasng n 0,) and both beng three tmes dfferentable n 0,). Then, If l ŷ) = 0 for some ŷ 0, ), then V = Ω T /6 ) lnn) T ) If l ŷ) < 0 or a < b, s.t. l ŷ) = 0 ŷ [a, b], then V = Ω lnn) Note. Clearly, l abs falls n the second category of the above theorem. T ) T ) = O lnn). Thus, = Θ lnn) []. nce, we have already seen that 4 Vovk s lgorthm, 990 Ths algorthm s very smlar to the Weghted Majorty algorthm. The dfference les n the update,.e. how do we combne the predctons from the experts to make our own predctons. lgorthm Vovk Parameters: c, η > 0 Intal weght vector w = [N] Loss functon l : {0, } [0, ] [0, ) For t =,..., T : Receve expert predctons ξ, t..., ξn t )[0, ] Compute y = c ln for y = 0,. wt e ηly,ξt ) wt Predct any ŷ satsfyng: ly, ŷ t ) y y {0, }. If such a ŷ, then the algorthm fals. Receve true label y t {0, } Incur loss ly t, ŷ) Update [N] : w t+ w t e ηlyt,ξ t ) nalyss of Vovk s lgorthm uppose that the algorthm doesn t fals. Defne U t = c lnw t ), where W t = N = wt. Then for each tral t, ly t, ŷ t ) U t+ U t t y ummng over t =,..., T, Now, L l [Vovkc, η)] U T + U T U T + = c lnw T + ) c lnw T + ), W T + w T +

4 4 Onlne Learnng from Experts: Mn Regret lso, U = c lnn) Thus, ) L l [Vovkc, η)] c lnn) + η mn L l [ξ ] Note. The above s true for all for whch the algorthm wth parameters c, η) doesn t fal. Thus gven the loss functon of nterest, f one can fnd the parameters c and η such that the algorthm never fals, then we get the above bound for all possble sequence. The proof of above nvolves takng c = c l, η = /c l wth l beng c, η)-realzable. howng n general that any loss functon satsfyng these condtons s realzable w.r.t. these values nvolves a sgnfcant amount of work. Defnton. Defne l to be c, η)-realzable f, ξ; Vovkc, η) doesn t fal. Now, we wll see what condtons are suffcent for the Vovk s algorthm not to fal. We wll verfy t for some loss functons. Logarthmc Loss Run Vovk algorthm wth c = c l and η = /c l. For the l log loss we have, c =, η =. Defne p t = w t N j= wt j to be the normalzed verson of the weght vector. ) ) 0 = c ln p t e ηl0ξt ) = ln p t e ln ξt ) = ln p t ξ t) We bascally need: whch s same as requrng: ) ) = c ln p t e ηlξt ) = ln p t e lnξt ) = ln p t ξ t) l 0 ŷ t ) ln p t ξ t) l ŷ t ) ln p t ξ t) l 0 ln pt ξ t)) ŷ t ŷ p t ξ t ln pt ξ t)) ŷ t ŷ p t ξ t l Thus there s only one value ŷ t = p t ξ t satsfyng t. Exercse. Work out the condtons for the Vovk s algorthm to be c, η)-realzable n the squared loss settng. bsolute Loss We start wth some η > 0 and c abs η) = 2. 2 ln +e η For the above choce of c, l abs turns out to be realzable for any η > 0. The bound that we get here s [Vovkη, c absη))] lnn) + η mn [ξ ] 2 2 ln +e η ) The above holds for any η > 0. The bound seems qute smlar to the one n Weghted Majorty algorthm except for the denomnator term. nce, t holds for all η, f we choose η carefully, we can get a bound of the form [Vovkη, c abs η ))] T lnn)+lnn)/2. For the absolute loss the best possble bound happens to be T/2) lnn + ) + lnn + )/2, whch s obtaned when we add an addtonal arbtrary expert ξ N+, who always predct the nverse of one of the experts say ξ ). Here, at least one of the two experts ξ and ξ N+ ) must have loss at most T/2 tmes.

5 Onlne Learnng from Experts: Mn Regret 5 5 Lower Bound on Mn Regret for bsolute Loss From earler secton we know that, = mn ξ, [] mn ) [ξ ] mn ξ, Rl abs ξ, [] Clam. For any and any ξ,..., ξ N [0, ] T some label sequence for whch the regret follows: [0,] T R l abs ξ, [] T 2 E [0,] T [mn ] [ξ ] Proof. t each tral the algorthm s predctng some value ŷ t. The label for the tral s unformly ether 0 or,.e. w.p. 2 the label s 0 and w.p. 2 t s. Thus, [ ] [ E [0,] T = 2 ŷt 0) + ] 2 ŷt ) T = T/2 The clam follows by observng that, [ ] R l abs [0,] T ξ, [] E R l abs [0,] T ξ, [] Now, = mn mn mn mn ξ { R l abs ξ {0,} T ) N E ξ {0,} T ) N ξ, [] } { } R l abs ξ, [] [ T [ 2 E [0,] T [ T 2 E ξ {0,} T ) N, [0,] T [ mn mn ] ] [ξ ] ] ] [ξ ] 6 Next Lecture In the next lecture we wll study the connectons of the onlne learnng wth the batch learnng and also see some methods to transform the onlne learnng problem nto a batch learnng problem. References [] Ncolo Cesa-Banch, Yoav Freund, Davd P. Helmbold, Davd Haussler, Robert E. chapre and Manfred K. Warmuth. How to use expert advce. Journal of the CM, 443): , 997. [2] Davd Haussler, Jyrk Kvnen and Manfred K. Warmuth. equental predcton of ndvdual sequences under general loss functons. IEEE Transactons on Informaton Theory, 445): , 998.

### U.C. Berkeley CS270: Algorithms Lecture 4 Professor Vazirani and Professor Rao Jan 27,2011 Lecturer: Umesh Vazirani Last revised February 10, 2012

U.C. Berkeley CS270: Algorthms Lecture 4 Professor Vazran and Professor Rao Jan 27,2011 Lecturer: Umesh Vazran Last revsed February 10, 2012 Lecture 4 1 The multplcatve weghts update method The multplcatve

### Graph Theory and Cayley s Formula

Graph Theory and Cayley s Formula Chad Casarotto August 10, 2006 Contents 1 Introducton 1 2 Bascs and Defntons 1 Cayley s Formula 4 4 Prüfer Encodng A Forest of Trees 7 1 Introducton In ths paper, I wll

### 8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

### Lecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2

18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest

### Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

### 1. Measuring association using correlation and regression

How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a

### II. PROBABILITY OF AN EVENT

II. PROBABILITY OF AN EVENT As ndcated above, probablty s a quantfcaton, or a mathematcal model, of a random experment. Ths quantfcaton s a measure of the lkelhood that a gven event wll occur when the

### Solution : (a) FALSE. Let C be a binary one-error correcting code of length 9. Then it follows from the Sphere packing bound that.

MATH 29T Exam : Part I Solutons. TRUE/FALSE? Prove your answer! (a) (5 pts) There exsts a bnary one-error correctng code of length 9 wth 52 codewords. (b) (5 pts) There exsts a ternary one-error correctng

### Forecasting the Direction and Strength of Stock Market Movement

Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

### 1 Approximation Algorithms

CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

### 1 Example 1: Axis-aligned rectangles

COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton

### What is Candidate Sampling

What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

### b) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei

Mathematcal Propertes of the Least Squares Regresson The least squares regresson lne obeys certan mathematcal propertes whch are useful to know n practce. The followng propertes can be establshed algebracally:

### Formula of Total Probability, Bayes Rule, and Applications

1 Formula of Total Probablty, Bayes Rule, and Applcatons Recall that for any event A, the par of events A and A has an ntersecton that s empty, whereas the unon A A represents the total populaton of nterest.

### The Mathematical Derivation of Least Squares

Pscholog 885 Prof. Federco The Mathematcal Dervaton of Least Squares Back when the powers that e forced ou to learn matr algera and calculus, I et ou all asked ourself the age-old queston: When the hell

### Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

### Communication Networks II Contents

8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

### A Note on the Decomposition of a Random Sample Size

A Note on the Decomposton of a Random Sample Sze Klaus Th. Hess Insttut für Mathematsche Stochastk Technsche Unverstät Dresden Abstract Ths note addresses some results of Hess 2000) on the decomposton

### An Alternative Way to Measure Private Equity Performance

An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate

### Finite Math Chapter 10: Study Guide and Solution to Problems

Fnte Math Chapter 10: Study Gude and Soluton to Problems Basc Formulas and Concepts 10.1 Interest Basc Concepts Interest A fee a bank pays you for money you depost nto a savngs account. Prncpal P The amount

### Questions that we may have about the variables

Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent

### Binary Dependent Variables. In some cases the outcome of interest rather than one of the right hand side variables is discrete rather than continuous

Bnary Dependent Varables In some cases the outcome of nterest rather than one of the rght hand sde varables s dscrete rather than contnuous The smplest example of ths s when the Y varable s bnary so that

### Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

### Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.

Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.

### Lossless Data Compression

Lossless Data Compresson Lecture : Unquely Decodable and Instantaneous Codes Sam Rowes September 5, 005 Let s focus on the lossless data compresson problem for now, and not worry about nosy channel codng

### THE TITANIC SHIPWRECK: WHO WAS

THE TITANIC SHIPWRECK: WHO WAS MOST LIKELY TO SURVIVE? A STATISTICAL ANALYSIS Ths paper examnes the probablty of survvng the Ttanc shpwreck usng lmted dependent varable regresson analyss. Ths appled analyss

### v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

### Naïve Bayes classifier & Evaluation framework

Lecture aïve Bayes classfer & Evaluaton framework Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Generatve approach to classfcaton Idea:. Represent and learn the dstrbuton p x, y. Use t to defne probablstc

### State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness

Schroednger equaton Basc postulates of quantum mechancs. Operators: Hermtan operators, commutators State functon: egenfunctons of hermtan operators-> normalzaton, orthogonalty completeness egenvalues and

### Recurrence. 1 Definitions and main statements

Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

### EE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN

EE201 Crcut Theory I 2015 Sprng Dr. Yılmaz KALKAN 1. Basc Concepts (Chapter 1 of Nlsson - 3 Hrs.) Introducton, Current and Voltage, Power and Energy 2. Basc Laws (Chapter 2&3 of Nlsson - 6 Hrs.) Voltage

### Solution of Algebraic and Transcendental Equations

CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton

### Moment of a force about a point and about an axis

3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should

### Extending Probabilistic Dynamic Epistemic Logic

Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set

### n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)

MATH 16T Exam 1 : Part I (In-Class) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total

### The Probit Model. Alexander Spermann. SoSe 2009

The Probt Model Aleander Spermann Unversty of Freburg SoSe 009 Course outlne. Notaton and statstcal foundatons. Introducton to the Probt model 3. Applcaton 4. Coeffcents and margnal effects 5. Goodness-of-ft

### Math 31 Lesson Plan. Day 27: Fundamental Theorem of Finite Abelian Groups. Elizabeth Gillaspy. November 11, 2011

Math 31 Lesson Plan Day 27: Fundamental Theorem of Fnte Abelan Groups Elzabeth Gllaspy November 11, 2011 Supples needed: Colored chal Quzzes Homewor 4 envelopes: evals, HW, presentaton rubrcs, * probs

### The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

### Lecture 3: Annuity. Study annuities whose payments form a geometric progression or a arithmetic progression.

Lecture 3: Annuty Goals: Learn contnuous annuty and perpetuty. Study annutes whose payments form a geometrc progresson or a arthmetc progresson. Dscuss yeld rates. Introduce Amortzaton Suggested Textbook

### Logistic Regression. Steve Kroon

Logstc Regresson Steve Kroon Course notes sectons: 24.3-24.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro

### Chapter 3 Group Theory p. 1 - Remark: This is only a brief summary of most important results of groups theory with respect

Chapter 3 Group Theory p. - 3. Compact Course: Groups Theory emark: Ths s only a bref summary of most mportant results of groups theory wth respect to the applcatons dscussed n the followng chapters. For

### Learning Permutations with Exponential Weights

Journal of Machne Learnng Research 2009 (10) 1705-1736 Submtted 9/08; Publshed 7/09 Learnng Permutatons wth Exponental Weghts Davd P. Helmbold Manfred K. Warmuth Computer Scence Department Unversty of

### 2.4 Bivariate distributions

page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together

### 8 Algorithm for Binary Searching in Trees

8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

### Gibbs Free Energy and Chemical Equilibrium (or how to predict chemical reactions without doing experiments)

Gbbs Free Energy and Chemcal Equlbrum (or how to predct chemcal reactons wthout dong experments) OCN 623 Chemcal Oceanography Readng: Frst half of Chapter 3, Snoeynk and Jenkns (1980) Introducton We want

### An Interest-Oriented Network Evolution Mechanism for Online Communities

An Interest-Orented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne

### THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

### Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School

### Describing Communities. Species Diversity Concepts. Species Richness. Species Richness. Species-Area Curve. Species-Area Curve

peces versty Concepts peces Rchness peces-area Curves versty Indces - mpson's Index - hannon-wener Index - rlloun Index peces Abundance Models escrbng Communtes There are two mportant descrptors of a communty:

### Section 2.3 Present Value of an Annuity; Amortization

Secton 2.3 Present Value of an Annuty; Amortzaton Prncpal Intal Value PV s the present value or present sum of the payments. PMT s the perodc payments. Gven r = 6% semannually, n order to wthdraw \$1,000.00

### IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS 1

Nov Sad J. Math. Vol. 36, No. 2, 2006, 0-09 IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS Modrag S. Petkovć 2, Dušan M. Mloševć 3 Abstract. A new theorem concerned

### General Auction Mechanism for Search Advertising

General Aucton Mechansm for Search Advertsng Gagan Aggarwal S. Muthukrshnan Dávd Pál Martn Pál Keywords game theory, onlne auctons, stable matchngs ABSTRACT Internet search advertsng s often sold by an

### Texas Instruments 30X IIS Calculator

Texas Instruments 30X IIS Calculator Keystrokes for the TI-30X IIS are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the

### Nonlinear data mapping by neural networks

Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal

### New bounds in Balog-Szemerédi-Gowers theorem

New bounds n Balog-Szemeréd-Gowers theorem By Tomasz Schoen Abstract We prove, n partcular, that every fnte subset A of an abelan group wth the addtve energy κ A 3 contans a set A such that A κ A and A

### 9 Arithmetic and Geometric Sequence

AAU - Busness Mathematcs I Lecture #5, Aprl 4, 010 9 Arthmetc and Geometrc Sequence Fnte sequence: 1, 5, 9, 13, 17 Fnte seres: 1 + 5 + 9 + 13 +17 Infnte sequence: 1,, 4, 8, 16,... Infnte seres: 1 + + 4

### Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

### CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering

Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that

### Nasdaq Iceland Bond Indices 01 April 2015

Nasdaq Iceland Bond Indces 01 Aprl 2015 -Fxed duraton Indces Introducton Nasdaq Iceland (the Exchange) began calculatng ts current bond ndces n the begnnng of 2005. They were a response to recent changes

### Using Alpha-Beta Associative Memories to Learn and Recall RGB Images

Usng Alpha-Beta Assocatve Memores to Learn and Recall RGB Images Cornelo Yáñez-Márquez, María Elena Cruz-Meza, Flavo Arturo Sánchez-Garfas, and Itzamá López-Yáñez Centro de Investgacón en Computacón, Insttuto

### Implementation of Deutsch's Algorithm Using Mathcad

Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages - n "Machnes, Logc and Quantum Physcs"

### A linear recurrence sequence of composite numbers

LMS J Comput Math 15 (2012) 360 373 C 2012 Author do:101112/s1461157012001143 A lnear recurrence sequence of composte numbers Jonas Šurys Abstract We prove that for each postve nteger k n the range 2 k

### A Simple Economic Model about the Teamwork Pedagogy

Appled Mathematcal Scences, Vol. 6, 01, no. 1, 13-0 A Smple Economc Model about the Teamwork Pedagog Gregor L. Lght Department of Management, Provdence College Provdence, Rhode Island 0918, USA glght@provdence.edu

### Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

### A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

### PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

### VLSI Technology Dr. Nandita Dasgupta Department of Electrical Engineering Indian Institute of Technology, Madras

VLI Technology Dr. Nandta Dasgupta Department of Electrcal Engneerng Indan Insttute of Technology, Madras Lecture - 11 Oxdaton I netcs of Oxdaton o, the unt process step that we are gong to dscuss today

### Linear Regression, Regularization Bias-Variance Tradeoff

HTF: Ch3, 7 B: Ch3 Lnear Regresson, Regularzaton Bas-Varance Tradeoff Thanks to C Guestrn, T Detterch, R Parr, N Ray 1 Outlne Lnear Regresson MLE = Least Squares! Bass functons Evaluatng Predctors Tranng

### INVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMA-HDR NETWORKS

21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech-2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS

### Passive Filters. References: Barbow (pp 265-275), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6)

Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called

### x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

### Support Vector Machines

Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.

### Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier

Lesson 2 Chapter Two Three Phase Uncontrolled Rectfer. Operatng prncple of three phase half wave uncontrolled rectfer The half wave uncontrolled converter s the smplest of all three phase rectfer topologes.

### Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006

Aryabhata s Root Extracton Methods Abhshek Parakh Lousana State Unversty Aug 1 st 1 Introducton Ths artcle presents an analyss of the root extracton algorthms of Aryabhata gven n hs book Āryabhatīya [1,

### Usage of LCG/CLCG numbers for electronic gambling applications

Usage of LCG/CLCG numbers for electronc gamblng applcatons Anders Knutsson Smovts Consultng, Wenner-Gren Center, Sveavägen 166, 113 46 Stockholm, Sweden anders.knutsson@smovts.com Abstract. Several attacks

### Fisher Markets and Convex Programs

Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

### Online Multiple Kernel Learning: Algorithms and Mistake Bounds

Onlne Multple Kernel Learnng: Algorthms and Mstake Bounds Rong Jn 1, Steven C.H. Ho 2, and Tanbao Yang 1 1 Department of Computer Scence and Engneerng, Mchgan State Unversty, MI, 48824, USA 2 School of

### The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets

. The Magnetc Feld Concepts and Prncples Movng Charges All charged partcles create electrc felds, and these felds can be detected by other charged partcles resultng n electrc force. However, a completely

### Getting Started. with Penfriend XP. 1 Installing 2 Trying it out! 3 Word Prediction. Reviewing Lexicons 5 Preferences 6 Finding More Help 7

Gettng Started wth Penfrend XP Installng Tryng t out! Word Predcton Other Features Revewng Lexcons 5 Preferences 6 Fndng More Help 7 Installng 5 6 7 Before you can use Penfrend XP on your computer the

### Lecture 2: Absorbing states in Markov chains. Mean time to absorption. Wright-Fisher Model. Moran Model.

Lecture 2: Absorbng states n Markov chans. Mean tme to absorpton. Wrght-Fsher Model. Moran Model. Antonna Mtrofanova, NYU, department of Computer Scence December 8, 2007 Hgher Order Transton Probabltes

### Joe Pimbley, unpublished, 2005. Yield Curve Calculations

Joe Pmbley, unpublshed, 005. Yeld Curve Calculatons Background: Everythng s dscount factors Yeld curve calculatons nclude valuaton of forward rate agreements (FRAs), swaps, nterest rate optons, and forward

### Texas Instruments 30Xa Calculator

Teas Instruments 30Xa Calculator Keystrokes for the TI-30Xa are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the tet, check

### Simple Interest Loans (Section 5.1) :

Chapter 5 Fnance The frst part of ths revew wll explan the dfferent nterest and nvestment equatons you learned n secton 5.1 through 5.4 of your textbook and go through several examples. The second part

### Sensitivity Analysis in a Generic Multi-Attribute Decision Support System

Senstvty Analyss n a Generc Mult-Attrbute Decson Support System Sxto Ríos-Insua, Antono Jménez and Alfonso Mateos Department of Artfcal Intellgence, Madrd Techncal Unversty Campus de Montegancedo s/n,

### 14.74 Lecture 5: Health (2)

14.74 Lecture 5: Health (2) Esther Duflo February 17, 2004 1 Possble Interventons Last tme we dscussed possble nterventons. Let s take one: provdng ron supplements to people, for example. From the data,

### SPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:

SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and

### Marginal Returns to Education For Teachers

The Onlne Journal of New Horzons n Educaton Volume 4, Issue 3 MargnalReturnstoEducatonForTeachers RamleeIsmal,MarnahAwang ABSTRACT FacultyofManagementand Economcs UnverstPenddkanSultan Idrs ramlee@fpe.ups.edu.my

### The covariance is the two variable analog to the variance. The formula for the covariance between two variables is

Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.

### Capital asset pricing model, arbitrage pricing theory and portfolio management

Captal asset prcng model, arbtrage prcng theory and portfolo management Vnod Kothar The captal asset prcng model (CAPM) s great n terms of ts understandng of rsk decomposton of rsk nto securty-specfc rsk

### BERNSTEIN POLYNOMIALS

On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

### The example below solves a system in the unknowns α and β:

The Fnd Functon The functon Fnd returns a soluton to a system of equatons gven by a solve block. You can use Fnd to solve a lnear system, as wth lsolve, or to solve nonlnear systems. The example below

### LETTER IMAGE RECOGNITION

LETTER IMAGE RECOGNITION 1. Introducton. 1. Introducton. Objectve: desgn classfers for letter mage recognton. consder accuracy and tme n takng the decson. 20,000 samples: Startng set: mages based on 20

### Exceptionality and Natural Language Learning

Exceptonalty and Natural Language Learnng Mha Rotaru Dane J. Ltman Computer Scence Department Unversty of Pttsburgh Pttsburgh, PA 5260 mrotaru, ltman @cs.ptt.edu Abstract Prevous work has argued that memory-based

### Properties of American Derivative Securities

Capter 6 Propertes of Amercan Dervatve Securtes 6.1 Te propertes Defnton 6.1 An Amercan dervatve securty s a sequence of non-negatve random varables fg k g n k= suc tat eac G k s F k -measurable. Te owner

### J. Parallel Distrib. Comput.

J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

### 6. EIGENVALUES AND EIGENVECTORS 3 = 3 2

EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a non-zero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :

### Stochastic Bandits with Side Observations on Networks

Stochastc Bandts wth Sde Observatons on Networks Swapna Buccapatnam, Atlla Erylmaz Department of ECE The Oho State Unversty Columbus, OH - 430 buccapat@eceosuedu, erylmaz@osuedu Ness B Shroff Departments