Online Learning from Experts: Minimax Regret


 Darren Cannon
 2 years ago
 Views:
Transcription
1 E0 370 tatstcal Learnng Theory Lecture 2 Nov 24, 20) Onlne Learnng from Experts: Mn Regret Lecturer: hvan garwal crbe: Nkhl Vdhan Introducton In the last three lectures we have been dscussng the onlne learnng algorthms where we receve the nstance x t and then ts label y t for t =,..., T. pecfcally n the last lecture we talked about onlne learnng from experts and onlne predcton. We saw many algorthms lke Halvng algorthm, Weghted Majorty WM) algorthm and lastly Weghted Majorty Contnuous WMC) algorthm. We also saw bounds on the cumulatve loss ncurred by these algorthms. Today, we wll focus on onlne predcton. For the WMC algorthm the settng s: we have N experts who predct the outcome label) n [0,], then we combne these predctons usng a weghted average of these. Then we receve the true label and ncur some loss absolute loss n our settng), then we make an update to the weght vectors based on the loss. The ntuton s that the hgher the loss ncurred by an expert, the more drastcally we reduce ts weght. For the WMC algorthm, we proved that: [W MCη)] lnn) e η + η mn [ξ ] e η where the symbols have ther meanngs as explaned n the last lecture. lso f the number of examples T s known n advance, then we can choose the WMC parameter η s.t. [W MCη )] mn [ξ ] + 2T lnn) + lnn) 2 Mn Regret Lets defne the regret of an algorthm w.r.t. the set of experts ξ = {ξ,..., ξ N } on the sample = x, y ),..., x T, y T )) as: R l ξ,[] = L l [] mn L l [ξ ].e. the dfference between the total loss of your algorthm and the total loss of the best expert. Thus f we had chosen the best expert always for our predcton then our regret would be zero; ths makes sense as n onlne settng absolute loss s nsgnfcant as the adversary may always choose the label other than what we predcted. The goal of the predctor s to mnmze the regret whle the goal of adversary s to mze t. In general, we can formalze t as a game where at each tral you receve expert s predctons, then you make your own predcton and fnally the true label s revealed by the adversary. In ths settng we want to get close to the best possble loss mnmum regret). Thus we want to choose an algorthm that mnmzes the total regret, however we do not have any control over the set of expert predctons and the sample as adversary can choose a set of experts or sample whch gves a larger regret over the algorthm. Here, we can defne the mn value regret) of the game as: V l = mn ξ, Rl ξ,[] It s the worst case guarantee on the regret over all the sequences of labels and expert predctons. From the last lecture we know that: 2T lnn) + lnn) natural queston here s can we bound the mn regret for other loss functon? Is ths the best possble mn regret?
2 2 Onlne Learnng from Experts: Mn Regret 3 Mn regret for varous loss functons In ths secton we wll look at upper bound on mn regret for some other loss functons whch we have seen before. We wll focus on bnary outcomes y t {0, } wth experts predctng n [0,]; ξ t [0, ] and ŷ t [0, ]. ξ t can be seen as probablty of th expert predctng the label as. Defne the loss functon l : {0, } [0, ] [0, ) Lets us also defne l 0 ŷ) = l0, ŷ) and l ŷ) = l, ŷ). 3. quared Loss For squared loss ly, ŷ) = y ŷ) 2, t can be easly seen that l 0 ŷ) = ŷ 2 and l ŷ) = ŷ) 2. Let us defne some new quanttes assumng l 0 ) and l ) beng twce dfferentable): l ŷ) = l 0ŷ) l ŷ) l ŷ) l 0ŷ) R l ŷ) = l 0ŷ) l ŷ) 2 l ŷ) l 0ŷ) 2 l ŷ) c l = sup R l ŷ) 0<ŷ< Theorem 3.. Haussler et al., 998) Let l be such that l 0 0) = l ) = 0. Let l 0 ) be strctly ncreasng n 0,) and l ) be strctly decreasng n 0,). Let l 0 ) and l ) be three tmes dfferentable n 0,). If l ŷ) > 0 n 0,) and c l <, then: V l = ΘlnN)) pecfcally, we have V l c l lnn) and V l c l o)) lnn) For the squared loss: l 0ŷ) = 2ŷ, l ŷ) = 2ŷ ), l 0ŷ) = 2, l ŷ) = 2 Thus, l ŷ) = 4 > 0, and R l ŷ) = 2ŷ ŷ). Clearly, sup 0<ŷ< R l ŷ) = /2 <. Therefore, for squared loss we have, V lsq 2 lnn) 3.2 Logarthmc Loss Ths type of loss functon haven t been dscussed before n the lectures. Let us defne t frst. { lnŷ) f y = l log y, ŷ) = ln ŷ) f y = 0 For l log loss, l 0 ŷ) = ln ŷ) and l ŷ) = ln ŷ). mple mathematcs wll lead us to: Thus, we get l 0ŷ) = ŷ, l ŷ) = ŷ, l 0ŷ) = ŷ) 2, l ŷ) = ŷ 2 l ŷ) = Therefore for the logarthmc loss we get, ŷ) 2 ŷ 2 and subsequently R lŷ) =, c l = V l log lnn)
3 Onlne Learnng from Experts: Mn Regret bsolute Loss For the absolute loss we have l abs y, ŷ) = y ŷ, here t s easy to see that l 0 ŷ) = ŷ and l ŷ) = ŷ, for ŷ 0, ). lso l 0ŷ) =, l ŷ) =, l 0ŷ) = 0, l ŷ) = 0 nd therefore we get l ŷ) = 0 c l =. Thus, the above theorem doesn t apply n the absolute loss settng. Thus for the absolute loss we have the result of WMC algorthm from the last lecture whch gves an upper bound on the regret. Clearly for the squared and logarthmc loss we saw a much tghter bound; then a natural queston arses: can we do better for the absolute loss too? It turns out that we can only mprove the constants n the WMC regret bound for the absolute error case. Theorem 3.2. Let l be s.t. l 0 ) s strctly ncreasng n 0,) whle l ) s strctly decreasng n 0,) and both beng three tmes dfferentable n 0,). Then, If l ŷ) = 0 for some ŷ 0, ), then V = Ω T /6 ) lnn) T ) If l ŷ) < 0 or a < b, s.t. l ŷ) = 0 ŷ [a, b], then V = Ω lnn) Note. Clearly, l abs falls n the second category of the above theorem. T ) T ) = O lnn). Thus, = Θ lnn) []. nce, we have already seen that 4 Vovk s lgorthm, 990 Ths algorthm s very smlar to the Weghted Majorty algorthm. The dfference les n the update,.e. how do we combne the predctons from the experts to make our own predctons. lgorthm Vovk Parameters: c, η > 0 Intal weght vector w = [N] Loss functon l : {0, } [0, ] [0, ) For t =,..., T : Receve expert predctons ξ, t..., ξn t )[0, ] Compute y = c ln for y = 0,. wt e ηly,ξt ) wt Predct any ŷ satsfyng: ly, ŷ t ) y y {0, }. If such a ŷ, then the algorthm fals. Receve true label y t {0, } Incur loss ly t, ŷ) Update [N] : w t+ w t e ηlyt,ξ t ) nalyss of Vovk s lgorthm uppose that the algorthm doesn t fals. Defne U t = c lnw t ), where W t = N = wt. Then for each tral t, ly t, ŷ t ) U t+ U t t y ummng over t =,..., T, Now, L l [Vovkc, η)] U T + U T U T + = c lnw T + ) c lnw T + ), W T + w T +
4 4 Onlne Learnng from Experts: Mn Regret lso, U = c lnn) Thus, ) L l [Vovkc, η)] c lnn) + η mn L l [ξ ] Note. The above s true for all for whch the algorthm wth parameters c, η) doesn t fal. Thus gven the loss functon of nterest, f one can fnd the parameters c and η such that the algorthm never fals, then we get the above bound for all possble sequence. The proof of above nvolves takng c = c l, η = /c l wth l beng c, η)realzable. howng n general that any loss functon satsfyng these condtons s realzable w.r.t. these values nvolves a sgnfcant amount of work. Defnton. Defne l to be c, η)realzable f, ξ; Vovkc, η) doesn t fal. Now, we wll see what condtons are suffcent for the Vovk s algorthm not to fal. We wll verfy t for some loss functons. Logarthmc Loss Run Vovk algorthm wth c = c l and η = /c l. For the l log loss we have, c =, η =. Defne p t = w t N j= wt j to be the normalzed verson of the weght vector. ) ) 0 = c ln p t e ηl0ξt ) = ln p t e ln ξt ) = ln p t ξ t) We bascally need: whch s same as requrng: ) ) = c ln p t e ηlξt ) = ln p t e lnξt ) = ln p t ξ t) l 0 ŷ t ) ln p t ξ t) l ŷ t ) ln p t ξ t) l 0 ln pt ξ t)) ŷ t ŷ p t ξ t ln pt ξ t)) ŷ t ŷ p t ξ t l Thus there s only one value ŷ t = p t ξ t satsfyng t. Exercse. Work out the condtons for the Vovk s algorthm to be c, η)realzable n the squared loss settng. bsolute Loss We start wth some η > 0 and c abs η) = 2. 2 ln +e η For the above choce of c, l abs turns out to be realzable for any η > 0. The bound that we get here s [Vovkη, c absη))] lnn) + η mn [ξ ] 2 2 ln +e η ) The above holds for any η > 0. The bound seems qute smlar to the one n Weghted Majorty algorthm except for the denomnator term. nce, t holds for all η, f we choose η carefully, we can get a bound of the form [Vovkη, c abs η ))] T lnn)+lnn)/2. For the absolute loss the best possble bound happens to be T/2) lnn + ) + lnn + )/2, whch s obtaned when we add an addtonal arbtrary expert ξ N+, who always predct the nverse of one of the experts say ξ ). Here, at least one of the two experts ξ and ξ N+ ) must have loss at most T/2 tmes.
5 Onlne Learnng from Experts: Mn Regret 5 5 Lower Bound on Mn Regret for bsolute Loss From earler secton we know that, = mn ξ, [] mn ) [ξ ] mn ξ, Rl abs ξ, [] Clam. For any and any ξ,..., ξ N [0, ] T some label sequence for whch the regret follows: [0,] T R l abs ξ, [] T 2 E [0,] T [mn ] [ξ ] Proof. t each tral the algorthm s predctng some value ŷ t. The label for the tral s unformly ether 0 or,.e. w.p. 2 the label s 0 and w.p. 2 t s. Thus, [ ] [ E [0,] T = 2 ŷt 0) + ] 2 ŷt ) T = T/2 The clam follows by observng that, [ ] R l abs [0,] T ξ, [] E R l abs [0,] T ξ, [] Now, = mn mn mn mn ξ { R l abs ξ {0,} T ) N E ξ {0,} T ) N ξ, [] } { } R l abs ξ, [] [ T [ 2 E [0,] T [ T 2 E ξ {0,} T ) N, [0,] T [ mn mn ] ] [ξ ] ] ] [ξ ] 6 Next Lecture In the next lecture we wll study the connectons of the onlne learnng wth the batch learnng and also see some methods to transform the onlne learnng problem nto a batch learnng problem. References [] Ncolo CesaBanch, Yoav Freund, Davd P. Helmbold, Davd Haussler, Robert E. chapre and Manfred K. Warmuth. How to use expert advce. Journal of the CM, 443): , 997. [2] Davd Haussler, Jyrk Kvnen and Manfred K. Warmuth. equental predcton of ndvdual sequences under general loss functons. IEEE Transactons on Informaton Theory, 445): , 998.
U.C. Berkeley CS270: Algorithms Lecture 4 Professor Vazirani and Professor Rao Jan 27,2011 Lecturer: Umesh Vazirani Last revised February 10, 2012
U.C. Berkeley CS270: Algorthms Lecture 4 Professor Vazran and Professor Rao Jan 27,2011 Lecturer: Umesh Vazran Last revsed February 10, 2012 Lecture 4 1 The multplcatve weghts update method The multplcatve
More informationGraph Theory and Cayley s Formula
Graph Theory and Cayley s Formula Chad Casarotto August 10, 2006 Contents 1 Introducton 1 2 Bascs and Defntons 1 Cayley s Formula 4 4 Prüfer Encodng A Forest of Trees 7 1 Introducton In ths paper, I wll
More information8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by
6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng
More informationLecture 3. 1 Largest singular value The Behavior of Algorithms in Practice 2/14/2
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationLuby s Alg. for Maximal Independent Sets using Pairwise Independence
Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent
More information1. Measuring association using correlation and regression
How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a
More informationII. PROBABILITY OF AN EVENT
II. PROBABILITY OF AN EVENT As ndcated above, probablty s a quantfcaton, or a mathematcal model, of a random experment. Ths quantfcaton s a measure of the lkelhood that a gven event wll occur when the
More informationSolution : (a) FALSE. Let C be a binary oneerror correcting code of length 9. Then it follows from the Sphere packing bound that.
MATH 29T Exam : Part I Solutons. TRUE/FALSE? Prove your answer! (a) (5 pts) There exsts a bnary oneerror correctng code of length 9 wth 52 codewords. (b) (5 pts) There exsts a ternary oneerror correctng
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract  Stock market s one of the most complcated systems
More information1 Approximation Algorithms
CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons
More information1 Example 1: Axisaligned rectangles
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationb) The mean of the fitted (predicted) values of Y is equal to the mean of the Y values: c) The residuals of the regression line sum up to zero: = ei
Mathematcal Propertes of the Least Squares Regresson The least squares regresson lne obeys certan mathematcal propertes whch are useful to know n practce. The followng propertes can be establshed algebracally:
More informationFormula of Total Probability, Bayes Rule, and Applications
1 Formula of Total Probablty, Bayes Rule, and Applcatons Recall that for any event A, the par of events A and A has an ntersecton that s empty, whereas the unon A A represents the total populaton of nterest.
More informationThe Mathematical Derivation of Least Squares
Pscholog 885 Prof. Federco The Mathematcal Dervaton of Least Squares Back when the powers that e forced ou to learn matr algera and calculus, I et ou all asked ourself the ageold queston: When the hell
More informationLogistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification
Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson
More informationCommunication Networks II Contents
8 / 1  Communcaton Networs II (Görg)  www.comnets.unbremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP
More informationA Note on the Decomposition of a Random Sample Size
A Note on the Decomposton of a Random Sample Sze Klaus Th. Hess Insttut für Mathematsche Stochastk Technsche Unverstät Dresden Abstract Ths note addresses some results of Hess 2000) on the decomposton
More informationAn Alternative Way to Measure Private Equity Performance
An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate
More informationFinite Math Chapter 10: Study Guide and Solution to Problems
Fnte Math Chapter 10: Study Gude and Soluton to Problems Basc Formulas and Concepts 10.1 Interest Basc Concepts Interest A fee a bank pays you for money you depost nto a savngs account. Prncpal P The amount
More informationQuestions that we may have about the variables
Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent
More informationBinary Dependent Variables. In some cases the outcome of interest rather than one of the right hand side variables is discrete rather than continuous
Bnary Dependent Varables In some cases the outcome of nterest rather than one of the rght hand sde varables s dscrete rather than contnuous The smplest example of ths s when the Y varable s bnary so that
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
More informationInequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.
Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.
More informationLossless Data Compression
Lossless Data Compresson Lecture : Unquely Decodable and Instantaneous Codes Sam Rowes September 5, 005 Let s focus on the lossless data compresson problem for now, and not worry about nosy channel codng
More informationTHE TITANIC SHIPWRECK: WHO WAS
THE TITANIC SHIPWRECK: WHO WAS MOST LIKELY TO SURVIVE? A STATISTICAL ANALYSIS Ths paper examnes the probablty of survvng the Ttanc shpwreck usng lmted dependent varable regresson analyss. Ths appled analyss
More informationv a 1 b 1 i, a 2 b 2 i,..., a n b n i.
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are
More informationNaïve Bayes classifier & Evaluation framework
Lecture aïve Bayes classfer & Evaluaton framework Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Generatve approach to classfcaton Idea:. Represent and learn the dstrbuton p x, y. Use t to defne probablstc
More informationState function: eigenfunctions of hermitian operators> normalization, orthogonality completeness
Schroednger equaton Basc postulates of quantum mechancs. Operators: Hermtan operators, commutators State functon: egenfunctons of hermtan operators> normalzaton, orthogonalty completeness egenvalues and
More informationRecurrence. 1 Definitions and main statements
Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.
More informationEE201 Circuit Theory I 2015 Spring. Dr. Yılmaz KALKAN
EE201 Crcut Theory I 2015 Sprng Dr. Yılmaz KALKAN 1. Basc Concepts (Chapter 1 of Nlsson  3 Hrs.) Introducton, Current and Voltage, Power and Energy 2. Basc Laws (Chapter 2&3 of Nlsson  6 Hrs.) Voltage
More informationSolution of Algebraic and Transcendental Equations
CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton
More informationMoment of a force about a point and about an axis
3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should
More informationExtending Probabilistic Dynamic Epistemic Logic
Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σalgebra: a set
More informationn + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)
MATH 16T Exam 1 : Part I (InClass) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total
More informationThe Probit Model. Alexander Spermann. SoSe 2009
The Probt Model Aleander Spermann Unversty of Freburg SoSe 009 Course outlne. Notaton and statstcal foundatons. Introducton to the Probt model 3. Applcaton 4. Coeffcents and margnal effects 5. Goodnessofft
More informationMath 31 Lesson Plan. Day 27: Fundamental Theorem of Finite Abelian Groups. Elizabeth Gillaspy. November 11, 2011
Math 31 Lesson Plan Day 27: Fundamental Theorem of Fnte Abelan Groups Elzabeth Gllaspy November 11, 2011 Supples needed: Colored chal Quzzes Homewor 4 envelopes: evals, HW, presentaton rubrcs, * probs
More informationThe Greedy Method. Introduction. 0/1 Knapsack Problem
The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton
More informationLecture 3: Annuity. Study annuities whose payments form a geometric progression or a arithmetic progression.
Lecture 3: Annuty Goals: Learn contnuous annuty and perpetuty. Study annutes whose payments form a geometrc progresson or a arthmetc progresson. Dscuss yeld rates. Introduce Amortzaton Suggested Textbook
More informationLogistic Regression. Steve Kroon
Logstc Regresson Steve Kroon Course notes sectons: 24.324.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro
More informationChapter 3 Group Theory p. 1  Remark: This is only a brief summary of most important results of groups theory with respect
Chapter 3 Group Theory p.  3. Compact Course: Groups Theory emark: Ths s only a bref summary of most mportant results of groups theory wth respect to the applcatons dscussed n the followng chapters. For
More informationLearning Permutations with Exponential Weights
Journal of Machne Learnng Research 2009 (10) 17051736 Submtted 9/08; Publshed 7/09 Learnng Permutatons wth Exponental Weghts Davd P. Helmbold Manfred K. Warmuth Computer Scence Department Unversty of
More information2.4 Bivariate distributions
page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together
More information8 Algorithm for Binary Searching in Trees
8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the
More informationGibbs Free Energy and Chemical Equilibrium (or how to predict chemical reactions without doing experiments)
Gbbs Free Energy and Chemcal Equlbrum (or how to predct chemcal reactons wthout dong experments) OCN 623 Chemcal Oceanography Readng: Frst half of Chapter 3, Snoeynk and Jenkns (1980) Introducton We want
More informationAn InterestOriented Network Evolution Mechanism for Online Communities
An InterestOrented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne
More informationTHE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES
The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered
More informationForecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network
700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School
More informationDescribing Communities. Species Diversity Concepts. Species Richness. Species Richness. SpeciesArea Curve. SpeciesArea Curve
peces versty Concepts peces Rchness pecesarea Curves versty Indces  mpson's Index  hannonwener Index  rlloun Index peces Abundance Models escrbng Communtes There are two mportant descrptors of a communty:
More informationSection 2.3 Present Value of an Annuity; Amortization
Secton 2.3 Present Value of an Annuty; Amortzaton Prncpal Intal Value PV s the present value or present sum of the payments. PMT s the perodc payments. Gven r = 6% semannually, n order to wthdraw $1,000.00
More informationIMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUAREROOT INTERVAL METHOD FOR MULTIPLE ZEROS 1
Nov Sad J. Math. Vol. 36, No. 2, 2006, 009 IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUAREROOT INTERVAL METHOD FOR MULTIPLE ZEROS Modrag S. Petkovć 2, Dušan M. Mloševć 3 Abstract. A new theorem concerned
More informationGeneral Auction Mechanism for Search Advertising
General Aucton Mechansm for Search Advertsng Gagan Aggarwal S. Muthukrshnan Dávd Pál Martn Pál Keywords game theory, onlne auctons, stable matchngs ABSTRACT Internet search advertsng s often sold by an
More informationTexas Instruments 30X IIS Calculator
Texas Instruments 30X IIS Calculator Keystrokes for the TI30X IIS are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the
More informationNonlinear data mapping by neural networks
Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal
More informationNew bounds in BalogSzemerédiGowers theorem
New bounds n BalogSzemerédGowers theorem By Tomasz Schoen Abstract We prove, n partcular, that every fnte subset A of an abelan group wth the addtve energy κ A 3 contans a set A such that A κ A and A
More information9 Arithmetic and Geometric Sequence
AAU  Busness Mathematcs I Lecture #5, Aprl 4, 010 9 Arthmetc and Geometrc Sequence Fnte sequence: 1, 5, 9, 13, 17 Fnte seres: 1 + 5 + 9 + 13 +17 Infnte sequence: 1,, 4, 8, 16,... Infnte seres: 1 + + 4
More informationInstitute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic
Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange
More informationCS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering
Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that
More informationNasdaq Iceland Bond Indices 01 April 2015
Nasdaq Iceland Bond Indces 01 Aprl 2015 Fxed duraton Indces Introducton Nasdaq Iceland (the Exchange) began calculatng ts current bond ndces n the begnnng of 2005. They were a response to recent changes
More informationUsing AlphaBeta Associative Memories to Learn and Recall RGB Images
Usng AlphaBeta Assocatve Memores to Learn and Recall RGB Images Cornelo YáñezMárquez, María Elena CruzMeza, Flavo Arturo SánchezGarfas, and Itzamá LópezYáñez Centro de Investgacón en Computacón, Insttuto
More informationImplementation of Deutsch's Algorithm Using Mathcad
Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages  n "Machnes, Logc and Quantum Physcs"
More informationA linear recurrence sequence of composite numbers
LMS J Comput Math 15 (2012) 360 373 C 2012 Author do:101112/s1461157012001143 A lnear recurrence sequence of composte numbers Jonas Šurys Abstract We prove that for each postve nteger k n the range 2 k
More informationA Simple Economic Model about the Teamwork Pedagogy
Appled Mathematcal Scences, Vol. 6, 01, no. 1, 130 A Smple Economc Model about the Teamwork Pedagog Gregor L. Lght Department of Management, Provdence College Provdence, Rhode Island 0918, USA glght@provdence.edu
More informationDEFINING %COMPLETE IN MICROSOFT PROJECT
CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMISP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationA hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm
Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):18841889 Research Artcle ISSN : 09757384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel
More informationPSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 12
14 The Chsquared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed
More informationVLSI Technology Dr. Nandita Dasgupta Department of Electrical Engineering Indian Institute of Technology, Madras
VLI Technology Dr. Nandta Dasgupta Department of Electrcal Engneerng Indan Insttute of Technology, Madras Lecture  11 Oxdaton I netcs of Oxdaton o, the unt process step that we are gong to dscuss today
More informationLinear Regression, Regularization BiasVariance Tradeoff
HTF: Ch3, 7 B: Ch3 Lnear Regresson, Regularzaton BasVarance Tradeoff Thanks to C Guestrn, T Detterch, R Parr, N Ray 1 Outlne Lnear Regresson MLE = Least Squares! Bass functons Evaluatng Predctors Tranng
More informationINVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMAHDR NETWORKS
21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS
More informationPassive Filters. References: Barbow (pp 265275), Hayes & Horowitz (pp 3260), Rizzoni (Chap. 6)
Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called
More informationx f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60
BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true
More informationSupport Vector Machines
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.
More informationLesson 2 Chapter Two Three Phase Uncontrolled Rectifier
Lesson 2 Chapter Two Three Phase Uncontrolled Rectfer. Operatng prncple of three phase half wave uncontrolled rectfer The half wave uncontrolled converter s the smplest of all three phase rectfer topologes.
More informationAryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006
Aryabhata s Root Extracton Methods Abhshek Parakh Lousana State Unversty Aug 1 st 1 Introducton Ths artcle presents an analyss of the root extracton algorthms of Aryabhata gven n hs book Āryabhatīya [1,
More informationUsage of LCG/CLCG numbers for electronic gambling applications
Usage of LCG/CLCG numbers for electronc gamblng applcatons Anders Knutsson Smovts Consultng, WennerGren Center, Sveavägen 166, 113 46 Stockholm, Sweden anders.knutsson@smovts.com Abstract. Several attacks
More informationFisher Markets and Convex Programs
Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and
More informationOnline Multiple Kernel Learning: Algorithms and Mistake Bounds
Onlne Multple Kernel Learnng: Algorthms and Mstake Bounds Rong Jn 1, Steven C.H. Ho 2, and Tanbao Yang 1 1 Department of Computer Scence and Engneerng, Mchgan State Unversty, MI, 48824, USA 2 School of
More informationThe Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets
. The Magnetc Feld Concepts and Prncples Movng Charges All charged partcles create electrc felds, and these felds can be detected by other charged partcles resultng n electrc force. However, a completely
More informationGetting Started. with Penfriend XP. 1 Installing 2 Trying it out! 3 Word Prediction. Reviewing Lexicons 5 Preferences 6 Finding More Help 7
Gettng Started wth Penfrend XP Installng Tryng t out! Word Predcton Other Features Revewng Lexcons 5 Preferences 6 Fndng More Help 7 Installng 5 6 7 Before you can use Penfrend XP on your computer the
More informationLecture 2: Absorbing states in Markov chains. Mean time to absorption. WrightFisher Model. Moran Model.
Lecture 2: Absorbng states n Markov chans. Mean tme to absorpton. WrghtFsher Model. Moran Model. Antonna Mtrofanova, NYU, department of Computer Scence December 8, 2007 Hgher Order Transton Probabltes
More informationJoe Pimbley, unpublished, 2005. Yield Curve Calculations
Joe Pmbley, unpublshed, 005. Yeld Curve Calculatons Background: Everythng s dscount factors Yeld curve calculatons nclude valuaton of forward rate agreements (FRAs), swaps, nterest rate optons, and forward
More informationTexas Instruments 30Xa Calculator
Teas Instruments 30Xa Calculator Keystrokes for the TI30Xa are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the tet, check
More informationSimple Interest Loans (Section 5.1) :
Chapter 5 Fnance The frst part of ths revew wll explan the dfferent nterest and nvestment equatons you learned n secton 5.1 through 5.4 of your textbook and go through several examples. The second part
More informationSensitivity Analysis in a Generic MultiAttribute Decision Support System
Senstvty Analyss n a Generc MultAttrbute Decson Support System Sxto RíosInsua, Antono Jménez and Alfonso Mateos Department of Artfcal Intellgence, Madrd Techncal Unversty Campus de Montegancedo s/n,
More information14.74 Lecture 5: Health (2)
14.74 Lecture 5: Health (2) Esther Duflo February 17, 2004 1 Possble Interventons Last tme we dscussed possble nterventons. Let s take one: provdng ron supplements to people, for example. From the data,
More informationSPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:
SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and
More informationMarginal Returns to Education For Teachers
The Onlne Journal of New Horzons n Educaton Volume 4, Issue 3 MargnalReturnstoEducatonForTeachers RamleeIsmal,MarnahAwang ABSTRACT FacultyofManagementand Economcs UnverstPenddkanSultan Idrs ramlee@fpe.ups.edu.my
More informationThe covariance is the two variable analog to the variance. The formula for the covariance between two variables is
Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.
More informationCapital asset pricing model, arbitrage pricing theory and portfolio management
Captal asset prcng model, arbtrage prcng theory and portfolo management Vnod Kothar The captal asset prcng model (CAPM) s great n terms of ts understandng of rsk decomposton of rsk nto securtyspecfc rsk
More informationBERNSTEIN POLYNOMIALS
OnLne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful
More informationThe example below solves a system in the unknowns α and β:
The Fnd Functon The functon Fnd returns a soluton to a system of equatons gven by a solve block. You can use Fnd to solve a lnear system, as wth lsolve, or to solve nonlnear systems. The example below
More informationLETTER IMAGE RECOGNITION
LETTER IMAGE RECOGNITION 1. Introducton. 1. Introducton. Objectve: desgn classfers for letter mage recognton. consder accuracy and tme n takng the decson. 20,000 samples: Startng set: mages based on 20
More informationExceptionality and Natural Language Learning
Exceptonalty and Natural Language Learnng Mha Rotaru Dane J. Ltman Computer Scence Department Unversty of Pttsburgh Pttsburgh, PA 5260 mrotaru, ltman @cs.ptt.edu Abstract Prevous work has argued that memorybased
More informationProperties of American Derivative Securities
Capter 6 Propertes of Amercan Dervatve Securtes 6.1 Te propertes Defnton 6.1 An Amercan dervatve securty s a sequence of nonnegatve random varables fg k g n k= suc tat eac G k s F k measurable. Te owner
More informationJ. Parallel Distrib. Comput.
J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n
More information6. EIGENVALUES AND EIGENVECTORS 3 = 3 2
EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a nonzero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :
More informationStochastic Bandits with Side Observations on Networks
Stochastc Bandts wth Sde Observatons on Networks Swapna Buccapatnam, Atlla Erylmaz Department of ECE The Oho State Unversty Columbus, OH  430 buccapat@eceosuedu, erylmaz@osuedu Ness B Shroff Departments
More informationCS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements
Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there
More informationMultivariate EWMA Control Chart
Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant
More information