Optimal Strategies from Random Walks

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Optimal Strategies from Random Walks"

Transcription

1 Optimal Strategies from Radom Walks Jacob Aberethy Divisio of Computer Sciece UC Berkeley Mafred K Warmuth Departmet of Computer Sciece UC Sata Cruz Joel Yelli Divisio of Physical ad Biological Scieces UC Sata Cruz Abstract We aalyze a sequetial game betwee a Gambler ad a Casio The Gambler allocates bets from a limited budget over a fixed meu of gamblig evets that are offered at equal time itervals, ad the Casio chooses a biary loss outcome for each of the evets We derive the optimal mi-max strategies for both participats We the prove that the miimum cumulative loss of the Gambler, assumig optimal play by the Casio, is exactly a well-kow combiatorial quatity: the expected umber of draws eeded to complete a multiple set of cards i the geeralized Coupo Collector s Problem We show that this quatity ad the optimal strategy of the Gambler ca be efficietly estimated from a simple radom walk 1 Itroductio This paper aalyzes the problem of sequetial predictio ad decisio makig from the perspective of a two player game The game is played by a learer, called here the Gambler, who makes a sequece of bettig decisios The Gambler s oppoet is the Casio i which he plays Gambler vs Casio: 1 O each day, the Gambler arrives at the Casio with $1 The Casio presets evets ad each evet is played oce per day The Gambler chooses a distributio vector w [0, 1], where w i = 1, ad bets the portio w i of his $1 budget o evet i 2 O each day the Casio determies the outcome of each evet with the objective of wiig as much moey from the Gambler as possible I particular, after observig the distributio of the Gambler s bets the Casio decides betwee a loss or a o loss for all daily evets These choices are summarized by a loss vector l {1, 0} where l i = 1 implies that o evet i, the Gambler lost (For simplicity, we assume the oly relevat quatities are losses By shiftig our baselie we ca model wis as o-losses) Supported by DARPA grat FA ad NSF grat DMS Supported by NSF grat CCR At the ed of each day, the Gambler leaves the Casio havig lost w l = i w il i ad the cumulative loss of the gambler is updated as L L + w l The Gambler also moitors the cumulative performace of each evet with a state vector s N, where s i is the curret total loss of evet i After icurrig loss l at the curret day, the state vector is updated to s s + l 4 The Gambler stops playig as soo as he observes that each eve has suffered more tha k losses, where k is some fixed positive iteger kow to both The Casio is aware of this decisio ad behaves accordigly Gamblig agaist a casio may seem a ulikely startig poit for a model of sequetial decisio makig we geerally cosider the typical eviromet for learig to be stochastic rather tha adversarial Yet are these two eviromets ecessarily icompatible? Amog the objectives of this paper is to address questios such as: What will be the Gambler s worst-case cumulative loss? ; ad What is the optimal bettig strategy? These questios, while clearly game-theoretic, are ultimately aswered here by cosiderig a radomized Casio rather tha a adversarial oe From this perspective, radomess may ideed be the Gambler s worst adversary Early work o sequetial decisio makig focused o the problem of predictig a biary outcome give advice from a set of experts I that settig, the goal of the predictor is to combie the predictios of the experts to make his ow predictio, with the objective of performig well, i hidsight, compared to the best expert The performace of both the learer ad the experts is measured by a loss fuctio that compares predictios to outcomes Oe of the early algorithms, the Weighted Majority algorithm [LW94], utilizes a distributio correspodig to the degree of trust i each expert It was observed by Freud ad Schapire [FS97] that the aalysis of the Weighted Majority algorithm ca be applied to the so-called hedge settig Rather tha predict a biary outcome, the learer ow plays some distributio over the experts o every roud, a loss value is assiged to each expert idepedetly, ad the learer suffers the expected loss accordig to his chose distributio I this case, the learer bears the exact burde of the Gambler - that of hedgig his bets so as to miimize his cumulative loss To emphasize that the Gambler/Casio game is useful for settigs other tha predictio, we use the term evet rather tha expert

2 A cetral theme of much of the sequetial decisio makig literature is the use of so-called expoetial weights to determie the learer s distributio o each roud Use of the expoetial weightig scheme i the case of the Casio game results i the followig strategy for the Gambler: At a state s, bet w i = βsi j βsj o evet i, (1) where the factor β lies i [0, 1) From the aalysis of the Weighted Majority algorithm it follows that the cumulative loss of the Gambler usig the above strategy is bouded by l + k l 1 β 1 β Uder the assumptio that the loss of the best evet is at most k, the factor β ca be tued [FS97] so that the above boud becomes k + 2k l + l The expoetial weights framework, as well as other olie learig techiques, ca be motivated usig the method of relative etropy regularizatio [KW99] While the resultig algorithms are elegat ad i some cases ca be show to be asymptotically optimal [CBFHW96], they do ot optimally solve the uderlyig game Some improvemets have bee made usig, for example, biomial weights that lead to slightly better but still o-optimal solutios [CBFHW96] i a settig where the experts must produce a predictio While it is formally easy to defie the optimal algorithms usig miimax expressios, it has geerally bee assumed that actually computig a efficiet solutio is quite challegig [CBFH + 97] More recetly, however, a miimax result [ALW07] was obtaied for the specific game of predictio with absolute loss The resultig algorithm, Biig, is efficiet ad optimal i a slightly relaxed settig I this paper we show that the miimax solutio to the Gambler/Casio predictio game, which is idetical to the uderlyig game of the hedge settig with biary losses, ca be obtaied efficietly I additio, the game ca be fully aalyzed usig a simple Markov process: a radom walk o a -dimesioal lattice The value of the game, that is the cumulative loss of a optimal Gambler, ca be iterpreted as the expected legth of such a radom walk The Gambler s optimal play, the portio of his budget he should bet o a give evet, ca similarly be iterpreted as maifestig a assessmet of the probability of a specific radom outcome of this walk The game s stoppig criterio, that is whe all evets have lost at least k + 1 times, may seem uusual at first yet fits quite aturally withi the experts framework Ideed, olie learig bouds are ofte tued with a explicit a priori kowledge of the cumulative loss of the best expert, which here would be k 1 While perhaps ot realistic i prac- 1 Strictly speakig, i the expert settig it is assumed that at least oe expert has ot crossed the k-mistake threshold, while here we stop the Gambler/Casio game whe the loss of the last expert/evet goes beyod this threshold It is easy to show that this slight modificatio, made for coveiece, icreases the worst-case loss of the Gambler by exactly 1 tice, k ca be estimated ad various techiques such as successive doublig ca be used to obtai ear-optimal bouds [CBFH + 97] The paper is structured as follows I Sectio 2 we give a miimax defiitio of the optimal value of the game cosidered here I Sectio 21 we modify the game by restrictig the adversary s choices to uit loss vectors I Sectio 3, we the tur our attetio to a specific Markov process with a umber of relevat properties We apply this radomized approach to the Casio game i Sectio 4, where we prove our mai results I Sectio 5 we give recurreces ad exact formulas, based o sums over multiomials, for the value of the game ad for the optimal probabilities We set out a efficiet method to compute both the optimal strategy of the Gambler ad the value of the game I Sectio 6 we compare the optimal regret boud to previous results, ad i Sectio 7 we draw a coectio betwee our game ad a well studied versio of the coupo collector problem We also briefly summarize what is kow about the asymptotics of this problem We coclude with a discussio of our results ad list ope problems (Sectio 8) 2 The Value of the Game Assume that i each evet the Gambler has already suffered some losses specified by state vector s Defie V (s) to be the total moey lost by a optimal Gambler playig agaist a optimal Casio startig from the state s That is, V (s) is the amout of moey that the optimal Gambler will lose (agaist a optimal Casio) from ow util the ed of the game Roughly speakig, the value of the game is computed as: V (s)? = mi dist w max w l + V (s + l) l {0,1} The Gambler chooses w to miimize the loss while the Casio chooses l to maximize the loss, where the loss is computed as the loss w l o this roud plus the worst case loss V (s+l) o future rouds However, we have to be careful, as this recursive defiitio does t address the followig issues: Whe is the game over? What is the base case of V ( )? Is this recursio bouded? Do we eed to record the losses s i that go above k? We address these issues begiig with some simplificatios ad otatioal covetios First, we assume that the state vector s lies withi the set S = {0, 1,, k+1} Note that it is ot ecessary to record the losses of evets that have already crossed the k threshold We call such evets dead Sice the losses of dead evets are ot restricted, havig loss k + 3 is the same as loss k We therefore roud all states s ito the state space {0, 1,, k + 1} usig the otatio + which we defie below We use the otatio λ(s) to record the set of live evets; the statemet i / λ(s) is exactly the statemet s i = k + 1 Secod, as the game is defied recursively, we must guaratee that this recursio termiates If the Casio repeatedly chose l = 0,, 0, for example, the game would make o progress The same problem occurs if the Casio causes

3 losses o oly dead evets We must therefore place additioal restrictios so that the dead state is reached evetually The simplest way to esure this is to forbid the Casio from iflictig loss o oly dead evets Yet this is ot sufficiet: with this restrictio aloe the Gambler would have a guarateed o-losig strategy by bettig solely o dead evets We thus assume that either ca the Casio ca iflict losses o dead evets or ca the Gambler bet moey o them (keepig i mid that all such bets are i ay case ooptimal) We must eforce this explicitly i order to have a well-defied game We use two otatioal covetios to describe the above restrictios First, we write w λ(s) to describe the set {w w i = 0 i / λ(s)} where is the -simplex We also abuse otatio slightly ad write l λ(s) to mea that l {0, 1} ad l i = 0 for all i / λ(s) We ow defie the value of the game precisely Defiitio 1 Defie the value V (s) of the game as follows At the dead state, V (d) := 0 For ay other s S, we defie V (s) recursively as V (s) := mi max w l + V (s + l) (2) 0 l λ(s) I our otatio, we commoly make use of several special states The state where the game begis is the iitial state, s = 0 Oce all evets have lost more tha k times the game is over ad we refer to this as the dead state d It will also be useful to cosider oe-live states o i, where all evets except i are dead, ad the remaiig evet has exactly k losses By the game defiitio, it is easy to check that V (o i ) = 1, sice the Gambler must bet all of his moey o this evet, ad the Casio must iflict a correspodig loss, chargig the Gambler $1 ad edig the game Below, we iclude a list of otatios for referece: Notatio: S :={0,, k + 1} (the state space) 0 := 0, 0,, 0 = 0 (the iitial state) d := (k + 1) (the dead state) o i := d e i (ith oe-live state) λ(s) := {i [] : s i k} (set of live evets) s +l := mi(s i + l i, k + 1) s := s i := {w R + : w = 1} 21 The Modified Game ( rouded additio) (elemetwise sum) (the -simplex) We also cosider a modified game that we make easier for the Gambler I this ew game, we restrict the Casio to iflict loss o exactly oe evet i each roud, ie l must be a basis vector e 1,, e So for l = e i we have w l = w i We ca the precisely defie the value V ( ) of the modified game: Defiitio 2 Defie V (d) := V (d) = 0 Otherwise V (s) := mi max w i + V (s + e i ) (3) i λ(s) Oe of the cetral results of this paper is that the above game, while seemigly more restricted, is ultimately just as difficult for the Gambler as the origial game It is easy to show that V (s) V (s), sice the Casio has strictly more choices i the origial game We go further ad prove as our mai result i Theorem 12 that V (s) = V (s) Thus both games have the same worst-case outcome Both the aalysis of the modified game, as well as the proof of the above result, requires a differet formulatio of the Casio s actios 3 A Radomized Casio I Sectio 2 we preseted a game-theoretic aalysis of a well-kow sequetial predictio problem characterized as a game betwee a Gambler ad a Casio I the preset sectio, we cosider a differet framework, i which the Casio uses radom evets We will show that itroducig a radomized strategy of the Casio eables us to specify the optimal strategy of the Gambler 31 A Radom Walk o the State Graph Let us ow imagie that our Casio does ot fix outcomes determiistically, but istead chooses the outcome of each evet usig the followig radom process Assume we are at state s ad that, o each day, a evet i is chose uiformly at radom from {1,, } ad a loss is assiged to evet i I other words, the loss vector l is a uiformly sampled uit vector e i, ad after the loss the ew state is s +e i This process cotiues util we reach the dead state d We ca model this behavior as a Markov process o the state space as follows Cosider ay sequece of idices I 1, I 2, [], ad let S t := tm=1 e I m, where S 0 := 0 Assumig that we start at state s, this iduces a sequece of states s = s +S 0 s +S 1 s +S 2 s +S t Notice that this process has self-loops ; ie it is quite possible that s +S t = s +S t+1 This occurs whe (s +S t ) It+1 is already at k + 1 If we imagie the state space S as a -dimesioal lattice, which we will call the state lattice, the the Markov process above ca be iterpreted as a radom walk o this lattice The walker starts at the iitial state 0, ad o every time iterval a positively directed sigle step is take alog a axis draw uiformly at radom If the walker has already reached the k + 1 boudary i this dimesio, he remais i place The walk stops oce the dead state d is reached We will show that the value V is 1/ times the expected total umber of radom draws that achieves this positio Thus V is the expected walk/path legth from s to d 32 Survival Probabilities We ow defie a survival probability at a state s We will show i the ext sectio that such probabilities are the basis for the Gambler s optimal strategy Defiitio 3 Assume we are at state s, ad let the radom state s +S t be the result of the above radom walk after t

4 steps Defie the ith survival probability p i (s) to be the probability that t : s +S t = o i Equivaletly, p i (s) = P r(λ(s +S t ) = {i} for some t) We call these survival probabilities sice p i (s) is the probability that, if the losses were assiged radomly to the evets i sequece, the ith evet would be the last o-dead evet Lemma 4 For ay s d, the vector p(s) := p i (s) i=1 defies a distributio o {1,, } Proof: The quatity i p i(s) is the probability that evetually there is exactly oe live evet This probability is exactly 1, give that the curret state is ot the dead state d We list some examples of survival probabilities: Whe s = 0 (or ay other symmetric state), we have p i (s) = 1, i bexause there is a uiform chace of survival Whe i is a dead evet, ie s i = k + 1, the p i (s) = 0 because o dead evet ca be the last remaiig live evet If there is oly oe remaiig live evet, ie λ(s) = {i}, the p i (s) = 1 Computig p i (s) for more geeral s requires a recursio, ad we leave this discussio for Sectio 5 33 Expected Path Legths Aother importat quatity we cosider is the legth of a radom path, ie the umber of steps i the radom walk o the state lattice required util the dead state d is reached Defiitio 5 For a sequece S 0, S 1,, let T (s) := mi{t 0 : s +S t = d} That is, T (s) is the legth of the radom path startig at s ad just eterig d Furthermore, let be the expected path legth τ(s) := E T (s) We ote that paths may be ifiitely log due to self-loops, yet such paths occur with probability 0 A key fact is that the expected path legth τ(s) ca be rewritte usig idicator variables: T (s) = 1[s +S t d], (4) t=0 ie T (s) is the umber of iitial segmets (icludig the empty segmet) of a radom path startig at s that has ot reached the dead state d We ow prove a relatioship betwee expected path legth τ(s) ad survival probabilities p i (s): Lemma 6 For ay state s ad evet i, p i (s) = 1 (τ(s) τ(s +e i )) Proof: Whe i / λ(s), the s = s +e i ad it is trivially true that p i (s) = 0 = 1 (τ(s) τ(s +e i )) The iterestig case is whe i λ(s) Ideed, Usig (4), we have τ(s) τ(s + e i ) = E T (s) E T (s + e i ) [ ] = E 1[s +S t d] 1[(s + e i ) +S t d] t=0 Sice the dead state d is a absorbig state we have that for ay path S, if s +S = d, the s + e i +S = d as well Equivaletly, if (s + e i ) +S d, the s +S d Thus i the differece betwee the expectatios, we oly eed be cocered with sequeces S t that are accouted for i the first expectatio but ot i the secod Therefore the above differece becomes [ ] = E 1[(s +S t d) ((s + e i ) +S t ) = d] t=0 We claim that ay sequece S t that satisfies the cojuctio must have the property that (S t ) i = k s i This is true because (s + e i ) +S t = d ad therefore (S t ) i k + 1 s i Also (S t ) j k + 1 s j, for j i This implies that s +S t = o i ad the above differece becomes E [ t=0 1[s +S t = o i ] ] The last term is exactly p i (s), the probability that s +S t evetually arrives at o i, times the expected umber of iteratios spet i state o i before arrivig at d To leave o i, the radom walk must make a step i the ith directio, ad thus the expected waitig time at o i is ca be computed as q q=1 (1 1 )q 1 } {{ } prob of q 1 loops 1 = }{{} prob of leavig The last lemma implies a importat fact about the state lattice Iterpret the state lattice as a directed graph with directed edges at all pairs (s, s + e i ) for each i λ(s) Also associate the edge (s, s + e i ) with the survival probability p i (s) Cosider startig at state s ad walkig through this directed graph: s s + e i1 s + e i1 + e i2 Corollary 7 Cosider ay two states s, s For ay path from s to s through the directed state graph, the sum of all edge weights p i ( ) alog this path is idepedet of the choice of path

5 Proof: Assume the path s = s 1, s 2,, s T, s T +1 = s defied by a sequece of moves is i 1, i 2,, i T, where s t+1 = s t + e it By Lemma 6 the total weight sum is T p it (s t ) = t=1 T t=1 1 (τ(st ) τ(s t+1 )) = 1 (τ(s) τ(s )), which is idepedet of the choice of path Note that i the defiitio of the directed state graph above ad i the corollary we igore loops, which occur whe s = s +e i (or equivaletly i / λ(s)) Such loops out of state s are immaterial because they correspod to dead evets, ad i / λ(s) iff p i (s) = 0 4 The Optimal Strategy We ow have the all the tools to express V (s) i terms of the expected path legth τ(s), prove that V (s) = V (s), ad show that the optimal bettig strategy for the gambler is p(s) We prove two major theorems i this sectio We provide the mathematically precise argumet for each but, as formality ofte obscures the true ituitio, we also provide a Eglish Versio so that the reader sees a rough sketch Our mathematical proofs require iductio o the state space S, so we eed a measure of progress for state vectors s For ay s S, defie m(s) := (k + 1) s, the umber of steps required before reachig the dead state Clearly m(s) = 0 if ad oly if s = d Theorem 8 For all states s, V (s) = 1 τ(s) Now assume that m(s) > 0 The V (s) = mi (iduc) = mi (Lem 6) = max i max w i + V (s + e i ) i λ(s) max i λ(s) w i + 1 τ(s + e i) max p i(s) + 1 i λ(s) τ(s + e i) = 1 τ(s) 1 (τ(s) τ(s + e i)) + 1 τ(s + e i) We prove V (s) 1 τ(s) by a similar iductio Assume that the Gambler chooses the optimal distributio w which may ideed be differet from p(s) For ay i / λ(s), p i (s) is defied as zero For the optimal strategy wi = 0 as well because otherwise the Casio ca icurr ubouded loss by playig e i repeatedly Sice w ad p(s) are differet distributios o the live evets λ(s), there must exist some j λ(s) for which wj > p j(s) We ow have V (s) = max i λ(s) w i + V (s + e i ) (iduc) = max i λ(s) w i + 1 τ(s + e i) w j + 1 τ(s + e i) > p j (s) + 1 τ(s + e i) (Lem6) = 1 (τ(s) τ(s + e i)) + 1 τ(s + e i) = 1 τ(s) Proof: (Eglish Versio) Assume that the Gambler always plays accordig to the distributio vector p(s) The we may thik of the Casio s choices as a walk aroud the state graph ad, as we discussed at the ed of Sectio 3, a collectio of the weights p i ( ) alog the way, edig at d But as we proved i Corollary 7 for the weights p(), it does t matter what path is take: the Casio will always receive 1 (τ(s) τ(d)) = 1 τ(s) o ay path from s that just eded i d If the Gambler ever chooses a distributio w differet from p(s) at some state s, the the Casio ca simply let l = e j for ay j for which w j > p j (s), ad o this roud the casio will force loss greater tha p j (s) This meas that o some path startig from s, the Casio will accrue total weight/loss larger tha 1 τ(s), ad therefore that the distributio w at s was o-optimal for the Gambler We coclude that for the Gambler p() is the oly optimal assigmet of distributios to states Proof: (Formal Versio) We iduct o m(s) First we check the base case s = d I this case, the expected path legth is exactly 0 sice we have already reached the dead state Thus τ(s) = 0 = V (d) as desired Corollary 9 For ay s d, p(s) is the uique optimal probability vector for the learer for the game related to V Proof: See ed of last proof Corollary 10 For all s ad all i [], p i (s) = V (s) V (s +e i ) Proof: This follows from the previous theorem ad Lemma 6 We eed oe more lemma before we ca prove our mai result Lemma 11 For ay state s ad distict evets i, j λ(s), we have p i (s) < p i (s + e j ) This fact is ituitive: if losses are radomly assiged the the probability that the ith evet will survive last strictly icreases whe aother evet suffers a loss We prove this precisely below

6 Proof: To show that p i (s) p i (s + e j ) is straightforward Ay sequece S 0, S 1, S 2, that brigs s to the oe-live state o i also brigs s + e j to o i Ideed, if s +S t = o i for some t the certaily (s + e j ) +S t = o i as well To show that this iequality is strict, we eed oly fid oe radom sequece for which s+e j is brought to o i but ot s Take ay sequece S 0, S 1, such that s +S t = d e i e j (where the oly evets remaiig are i ad j) ad where S t+1 = S t + e i The (s + e j ) +S t = o i but s +S t+1 = s +(S t + e i ) = o j Theorem 12 For all states s, V (s) = V (s) = 1 τ(s) Proof: (Eglish Versio) Imagie a gambler who plays the distributio p(s) at every state s We already kow that the Casio ca use its modified game strategy ad simply play uit vectors l = e i o each roud to force 1 τ(s) loss Yet sice l is urestricted, ca it obtai more? The aswer is No: cosider what happes if the Casio decides to choose l larger tha a uit vector, eg let l = e i + e j for simplicity The o this roud it obtais p i (s) + p j (s), but it ca do better! We proved i Lemma 11 that survival probabilities strictly icrease ad therefore p i (s) < p i (s + e j ) Thus, a more patiet Casio could choose l = e j o this roud, obtai p j (s), ad the choose l = e i o the ext roud to obtai p i (s+e j ) As p j (s)+ p i (s+e j ) > p j (s)+ p i (s), the Casio oly does worse by playig o-uit vectors Ideed, this suggests that the Gambler has a strategy by which the Casio ca iflict oly as much loss as i the modified game, ad thus the value V (s) is o differet from V (s) Proof: (Formal Versio) Certaily V (s) V (s), sice the Casio is give strictly fewer choices i the modified game Thus we are left to show that V (s) V (s) We proceed via iductio o m(s) By defiitio, V (s) = V (s) for the case s = d Now assume that, for all successive states s where m(s ) < m(s), V (s ) = V (s ) We proceed by directly aalyzig the recursive defiitio (2) Assume that the Gambler has chose the (possibly o-optimal) distributio w = p(s) to distribute his wealth o the live evets λ(s), ad let l {0, 1} be a optimal choice of the Casio (which ca deped o the Gambler s choice) By defiitio (1) of V (s), the chose loss vector ca t be 0 ad all evets with loss oe must be i λ(s) More precisely, V (s) = mi (id) = mi max w l + V (s + l) 0 l λ(s) max w l + V (s + l) 0 l λ(s) 0 l λ(s) p(s) max l + V (s + l) = p(s) l + V (s + l ) If l is ay uit vector e i, st i λ(s), the V (s) p(s) e i + V (s + e i ) = p i (s) + V (s + e i ) = V (s) ad i this case, V (s) = V (s) ad we are doe We ow prove by cotradictio that l ca have o more tha oe o-zero coordiate Assume ideed that l > 1, ie it admits a decompositio l = e i + l for some i ad bit vector l 0 with li = 0 Applyig Lemma 11 repeatedly, we have that p i (s) < p i (s + l) ad therefore p(s) l + V (s + l ) = p i (s) + p(s) l + V (s + l ) (Lem 11) < p i (s + l) + p(s) l + V (s + l ) (Cor 10) = V (s + l) V (s + l ) + p(s) l + V (s + l ) = p(s) l + V (s + l) But the statemet p(s) l + V (s + l ) < p(s) l + V (s + l) implies l is a o-optimal choice for the Casio ad this cotradicts our assumptio that l was optimum Corollary 13 For ay s d, if the learer plays with the optimum probability vector p(s), the the oly optimal resposes of the adversary i the recurrece (2) for V is to choose a uit vector of a live evet Proof: Proved at the ed of the last theorem 5 Recurreces, Combiatorics ad Radomized Algorithms The quatities V (s), τ(s) ad p i (s) have a umber of iterestig properties that we lay out i this sectio 51 Some Recurreces The expected path legth, τ(s) satisfies a very atural recursio Whe s = d, the the path legth is determiistically 0 ad therefore τ(d) = 0 Otherwise, we see that the expected path legth is i=1 τ(s) = 1 + τ(s +e i ) (5) That is, the expected path legth is 1, for the curret step i the path, plus the expected path legth of the ext radom state Sice the ext state is chose radomly from the set {s +e i : i = 1,, }, the probability of ay give state is 1, hece the ormalizatio factor Of course, our origial quatity of iterest is V (s), ad as we showed i Theorem 12 V (s) = 1 τ(s) This immediately gives us a recursio for V : ( ) V (s) = τ(s +e i ) i=1 = 1 + i=1 V (s +e i ) This recurrece, while true for the fuctio V ( ), is ambiguous because V (s) ca occur o both sides of the equatio Ideed, wheever i / λ(s), V (s +e i ) = V (s) However, we ca rearrage all V (s) terms to obtai the followig welldefied recursio: V (s) = 1 + i λ(s) V (s + e i) (6) λ(s)

7 We ca fid a similar recurrece for p i ( ) For the oelive states o i we have p j (o i ) = 1 if i = j ad 0 otherwise If λ(s) > 1, the j=1 p i (s) = p i(s +e j ) As p i (s) is the probability of edig at state o i after executig the Markov chai, this formula is obtaied by coditioig o oe step of the Markov process That is, the probability of edig at state o i is P r(j chose)p r(radom process takes s +e j to o i ) j This recurrece suffers from the same problem as did our iitial recurrece for V ( ): p(s) ca occur o both sides of the equality We agai solve this problem by rearragig terms ad obtai p i (s) = 52 Combiatorial Sums j λ(s) p i(s + e j ) λ(s) A further aalysis gives us exact expressios for both p i (s) ad V (s) i terms of ifiite sums of multiomials Propositio 51 For ay state s S, p i (s) = ( ) ( ) r +1 r 1 r 1, r 2,, r r:s +r=o i Proof: By defiitio, p i (s) is the probability that s reaches the oe-live state o i evetually To compute this probability, we cosider at what poit the Markov process exits the state o i ad ito d Recall the radom variable S t defied i Sectio 3 Take ay r for which s +r = o i ad coditio o S t = r The p i (s) = P r(s t = r)p r(s t+1 = r + e i S t = r) r:s +r=o i The first probability is exactly ( ) r r 1,r 2,,r r ad the secod probability is exactly 1/ Sice V (s) ca be writte as a expected path legth, we ca obtai a similar expressio as a sum of multiomials for V (s): Propositio 52 ( ) ( ) r +1 r 1 V (s) = ( r + 1) r i=1 1, r 2,, r r:r +s=o i 53 Radomized Approximatios Computig the exact value V (s) for large but o-asymptotic values of the state vector is difficult because we have o polyomial time algorithm O the other had, fidig a radomized approximatio to V (s) ca be doe very efficietly Ideed, as we ow have a represetatio of V (s) i terms of the legth of a radom walk, we ca simply ru the radom walk S 1, S 2, several times, ote the legth T (s), ad retur the mea Such radom approximatios require that the distributio o T (s) has low-variace, yet this certaily holds i the case at had While the radom walk requires at least (k + 1) iteratios to fiish, a simple argumet shows that with probability 1 δ the radom walk completes i less tha k log(k/δ) rouds Algorithm 1 Radom Approximatio to V (s) Iput: state s t 0 for i = 1,, NUMITER do z s repeat Sample i {1,, } uar z z +e i t t + 1 util z = d ed for t Retur NUMITER If R(s) is the radom variable retured by the above algorithm, the clealy ER(s) = V (s) By icreasig NUMITER, the variace of this estimate ca be reduced quickly A radomized approximatio for p(s) ca be obtaied similarly Agai the above algorithm approximately com- Algorithm 2 Radom Approximatio to p(s) Iput: state s d p 0 for i = 1,, NUMITER do z s repeat Sample i {1,, } uar z z +e i util z = o j for some j p p + e j ed for p Retur NUMITER putes p(s) i the followig sese: If R(s) is the radom variable retured by the above algorithm, the clearly ER(s) = p(s) Agai icreasig NUMITER, reduces the variace of the estimate 54 A Simple Strategy i a Radomized Settig I the particular case of bettig agaist the Casio, it may be ecessary for the Gambler to compute p i (s) i order to place his bets optimally I a alterative settig, however, a radomized algorithm may be sufficiet Let us cosider the case i which the Gambler chooses to bet accordig to the outcome of several coi tosses Further assume that the Casio ca observe his strategy but caot see the outcome of the coi tosses or his fial bets I this sceario, the Gambler ca eve bet all of his moey o a radom evet I {1,, } draw accordig to some distributio as log

8 22 Radomly Computig V compariso of the regret for = 2, 10, 100 ad k = 1,, The Optimal Boud vs the Hedge Boud The value V (0) k Samples 1 Sample Exact Value of V Regret Optimal, =2 Hedge, =2 Optimal, =10 Hedge, =10 Optimal, =100 Hedge, = The parameter k Figure 1: We illustrate the accuracy of the radomized approximatio to V (0) stated i Algorithm 1 The plot compares the exact value of V (0) to that obtaied by usig either 1 or 10 samples of the radom walk Here = 100 as E1[I = i] = p i (s) for all i, ad ideed his expected loss would be p(s) l For this sceario, radomly approximatig p is ot ecessary: oly oe sample is eeded! To be precise, the Gambler ca take the state s, ru the radom walk util the state reaches o i for some i, ad the bet his full dollar o evet i This bet will be correct i expectatio, ie he will pick evet i with probability p i (s), ad thus his expected loss will be exactly p l The key here is that samplig from the distributio p(s) may be quite easy eve whe computig it exactly may take more time Note that the above method based o oe sample is similar to the way the Radomized Weighted Majority algorithm approximates the Weighted Majority algorithm (more precisely the WMC algorithm of [LW94]) More precisely NUMITER=1 of Algorithm 53 correspods to WMR, ad NUMITER correspods to WMC 6 Compariso to Previous Bouds As metioed i the itroductio, the boud obtaiable base o expoetial weights [FS97] is k + 2k log + log (7) ad ca be show to be asymptotically optimal [Vov98] 2 Havig computed the miimax solutio to the same game, we ca compute the game-theoretically optimal boud of V (0) usig Algorithm 1 For small values of ad k, these bouds do differ quite substatially We preset i Figure 2 a 2 A slightly better but more complicated boud tha (7) was give i [Vov98] I the full paper we compare the optimal boud to this oe as well T he parameter k Figure 2: We compare the optimal regret boud we obtai from V (0) to that foud i [FS97], which we refer to as the hedge boud While asymptotically optimal, we observed that the hedge boud of k + 2k log + log is ot tight for small values of ad k 7 Coectios to classic problems of probabilistic eumerative combiatorics Theorem 12 shows that a optimal strategy for the Casio requires uit vector plays This leads to alterative iterpretatios of the game i terms of well studied radom processes For example, oe ca easily cofirm that our game also describes the radom process uderlyig a geeralized form of the Coupo Collector s Problem [? ] i which the collector buys cereal boxes oe by oe i order to obtai K = k+1 complete sets of baseball cards, assumig oe card is radomly placed withi each cereal box The value of our game, V (0, 0), is i fact the expected umber of cereal boxes, per baseball card, eeded to obtai the desired K complete sets Specifically, the probability geeratig fuctio for the geeralized Coupo Collector s Problem is [MW03] G,K (z) = e t/z t K 1 1 t j dt (K 1)! 0 j! j k Takig the derivative at z = 1 ad dividig by, we derive the expected umber of steps to obtai K sets, which is also the value of our game, viz V (0 ) = t K e t 1 t j dt (8) (K 1)! 0 j! j k Equatio (8) gives us a elegat closed form for the two-card case ( = 2): V ( 0, 0 ) = K + K ( ) 2K 2 2K K

9 From (8) we also obtai the well kow asymptotic expressio for the value, for large ad fixed K, V (0 ) log + (K 1) log log [1 + o(1)] The same asymptotic form appears i the aalysis of a evolvig radom graph [ER60] The radom walk o the state lattice provides yet aother iterpretatio of the same dyamics For K >> >> 1, the law of large umbers gives [NS60] V ( 0 ) = K + O(K 1/2 ) 8 Coclusio We showed i Corollary 13 that agaist the optimal learig algorithm the optimal strategy of the adversary is to choose oe of the uit loss vectors as his respose Curiously eough it ca be show that this is also true of the Weighted Majority algorithm (1) That is, ay trial i which q > 1 experts icurred a uit of loss ca be split ito q trials i which a sigle expert has a uit of loss, ad doig this always icreases the loss of the algorithm for all update factor β [0, 1) This observatio about the Weighted Majority algorithm might actually lead to improved loss bouds for this algorithm, perhaps i the way the parameter β is tued There remais also a deep questio regardig the techiques itroduced i this paper: how geeral is this method of computig the value of a game based o a radom path? Ca it hadle slightly more ivolved problems? Examples we have cosidered iclude competig agaist m-sized sets of experts, discussed i [WK06], i which the loss of the algorithm is compared to the loss of the best m-subset Aother example is the problem of competig agaist permutatios of objects [HW07], where the loss of a permutatio is liearly assiged Our prelimiary ivestigatio suggests that similar techiques ca be adapted to also hadle such more complex problems I the full paper we hope to delieate the scope of our ew method of optimal algorithm desig Refereces [ALW07] J Aberethy, J Lagford, ad M K Warmuth Cotiuous experts ad the Biig algorithm I Proceedigs of the 19th Aual Coferece o Learig Theory (COLT06), pages Spriger, Jue 2007 [CBFH + 97] Nicolò Cesa-Biachi, Yaov Freud, David Haussler, David P Helmbold, Robert E Schapire, ad Mafred K Warmuth How to use expert advice J ACM, 44(3): , 1997 [CBFHW96] N Cesa-Biachi, Y Freud, D P Helmbold, ad M K Warmuth O-lie predictio ad coversio strategies Machie Learig, 25:71 110, 1996 [ER60] P Erdos ad A Reyi O the evolutio of radom graphs Publ Math Ist Hug Acad Sci, 5A:17 61, 1960 [FS97] Yoav Freud ad Robert E Schapire A decisio-theoretic geeralizatio of o-lie learig ad a applicatio to Boostig J Comput Syst Sci, 55(1): , 1997 Special Issue for EuroCOLT 95 [HW07] D Helmbold ad M K Warmuth Learig permutatios with expoetial weights I Proceedigs of the 20th Aual Coferece o Learig Theory (COLT07) Spriger, 2007 [KW99] Jyrki Kivie ad Mafred K Warmuth Averagig expert predictios I Computatioal Learig Theory, 4th Europea Coferece, EuroCOLT 99, Nordkirche, Germay, March 29-31, 1999, Proceedigs, volume 1572 of Lecture Notes i Artificial Itelligece, pages Spriger, 1999 [LW94] N Littlestoe ad M K Warmuth The Weighted Majority algorithm Iform Comput, 108(2): , 1994 Prelimiary versio i i FOCS 89 [MW03] A Myers ad H S Wilf Some ew aspects of the Coupo-Collector s problem SIAM J Disc Math, 17:1 17, 2003 [NS60] D Newma ad L Shepp The Double Dixie Cup problem Amer Math Mothly, 67: , 1960 [Vov98] V Vovk A game of predictio with expert advice J of Comput Syst Sci, 56(2): , 1998 Special Issue: Eighth Aual Coferece o Computatioal Learig Theory [WK06] M K Warmuth ad D Kuzmi Radomized PCA algorithms with regret bouds that are logarithmic i the dimesio I Advaces i Neural Iformatio Processig Systems 19 (NIPS 06) MIT Press, December 2006

Asymptotic Growth of Functions

Asymptotic Growth of Functions CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

More information

Module 4: Mathematical Induction

Module 4: Mathematical Induction Module 4: Mathematical Iductio Theme 1: Priciple of Mathematical Iductio Mathematical iductio is used to prove statemets about atural umbers. As studets may remember, we ca write such a statemet as a predicate

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

5 Boolean Decision Trees (February 11)

5 Boolean Decision Trees (February 11) 5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

More information

Modified Line Search Method for Global Optimization

Modified Line Search Method for Global Optimization Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

More information

Soving Recurrence Relations

Soving Recurrence Relations Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

More information

THE ABRACADABRA PROBLEM

THE ABRACADABRA PROBLEM THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected

More information

Department of Computer Science, University of Otago

Department of Computer Science, University of Otago Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

More information

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

Properties of MLE: consistency, asymptotic normality. Fisher information.

Properties of MLE: consistency, asymptotic normality. Fisher information. Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

Recursion and Recurrences

Recursion and Recurrences Chapter 5 Recursio ad Recurreces 5.1 Growth Rates of Solutios to Recurreces Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer. Cosider, for example,

More information

Sequences and Series

Sequences and Series CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

More information

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13 EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

Lecture 2: Karger s Min Cut Algorithm

Lecture 2: Karger s Min Cut Algorithm priceto uiv. F 3 cos 5: Advaced Algorithm Desig Lecture : Karger s Mi Cut Algorithm Lecturer: Sajeev Arora Scribe:Sajeev Today s topic is simple but gorgeous: Karger s mi cut algorithm ad its extesio.

More information

Estimating Probability Distributions by Observing Betting Practices

Estimating Probability Distributions by Observing Betting Practices 5th Iteratioal Symposium o Imprecise Probability: Theories ad Applicatios, Prague, Czech Republic, 007 Estimatig Probability Distributios by Observig Bettig Practices Dr C Lych Natioal Uiversity of Irelad,

More information

Factors of sums of powers of binomial coefficients

Factors of sums of powers of binomial coefficients ACTA ARITHMETICA LXXXVI.1 (1998) Factors of sums of powers of biomial coefficiets by Neil J. Cali (Clemso, S.C.) Dedicated to the memory of Paul Erdős 1. Itroductio. It is well ow that if ( ) a f,a = the

More information

1 Introduction to reducing variance in Monte Carlo simulations

1 Introduction to reducing variance in Monte Carlo simulations Copyright c 007 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a uow mea µ = E(X) of a distributio by

More information

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

More information

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

The Stable Marriage Problem

The Stable Marriage Problem The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

More information

Learning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr.

Learning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr. Algorithms ad Data Structures Algorithm efficiecy Learig outcomes Able to carry out simple asymptotic aalysisof algorithms Prof. Dr. Qi Xi 2 Time Complexity Aalysis How fast is the algorithm? Code the

More information

Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis

Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.

More information

CHAPTER 3 THE TIME VALUE OF MONEY

CHAPTER 3 THE TIME VALUE OF MONEY CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

More information

Sequences II. Chapter 3. 3.1 Convergent Sequences

Sequences II. Chapter 3. 3.1 Convergent Sequences Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

Divide and Conquer. Maximum/minimum. Integer Multiplication. CS125 Lecture 4 Fall 2015

Divide and Conquer. Maximum/minimum. Integer Multiplication. CS125 Lecture 4 Fall 2015 CS125 Lecture 4 Fall 2015 Divide ad Coquer We have see oe geeral paradigm for fidig algorithms: the greedy approach. We ow cosider aother geeral paradigm, kow as divide ad coquer. We have already see a

More information

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009) 18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

Solutions to Selected Problems In: Pattern Classification by Duda, Hart, Stork

Solutions to Selected Problems In: Pattern Classification by Duda, Hart, Stork Solutios to Selected Problems I: Patter Classificatio by Duda, Hart, Stork Joh L. Weatherwax February 4, 008 Problem Solutios Chapter Bayesia Decisio Theory Problem radomized rules Part a: Let Rx be the

More information

8.1 Arithmetic Sequences

8.1 Arithmetic Sequences MCR3U Uit 8: Sequeces & Series Page 1 of 1 8.1 Arithmetic Sequeces Defiitio: A sequece is a comma separated list of ordered terms that follow a patter. Examples: 1, 2, 3, 4, 5 : a sequece of the first

More information

Trading the randomness - Designing an optimal trading strategy under a drifted random walk price model

Trading the randomness - Designing an optimal trading strategy under a drifted random walk price model Tradig the radomess - Desigig a optimal tradig strategy uder a drifted radom walk price model Yuao Wu Math 20 Project Paper Professor Zachary Hamaker Abstract: I this paper the author iteds to explore

More information

Unit 20 Hypotheses Testing

Unit 20 Hypotheses Testing Uit 2 Hypotheses Testig Objectives: To uderstad how to formulate a ull hypothesis ad a alterative hypothesis about a populatio proportio, ad how to choose a sigificace level To uderstad how to collect

More information

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

More information

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

More information

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad

More information

Designing Incentives for Online Question and Answer Forums

Designing Incentives for Online Question and Answer Forums Desigig Icetives for Olie Questio ad Aswer Forums Shaili Jai School of Egieerig ad Applied Scieces Harvard Uiversity Cambridge, MA 0238 USA shailij@eecs.harvard.edu Yilig Che School of Egieerig ad Applied

More information

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006 Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

More information

CHAPTER 3 DIGITAL CODING OF SIGNALS

CHAPTER 3 DIGITAL CODING OF SIGNALS CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity

More information

Incremental calculation of weighted mean and variance

Incremental calculation of weighted mean and variance Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

More information

4.1 Sigma Notation and Riemann Sums

4.1 Sigma Notation and Riemann Sums 0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

More information

1 Correlation and Regression Analysis

1 Correlation and Regression Analysis 1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio

More information

A Recursive Formula for Moments of a Binomial Distribution

A Recursive Formula for Moments of a Binomial Distribution A Recursive Formula for Momets of a Biomial Distributio Árpád Béyi beyi@mathumassedu, Uiversity of Massachusetts, Amherst, MA 01003 ad Saverio M Maago smmaago@psavymil Naval Postgraduate School, Moterey,

More information

Solving Divide-and-Conquer Recurrences

Solving Divide-and-Conquer Recurrences Solvig Divide-ad-Coquer Recurreces Victor Adamchik A divide-ad-coquer algorithm cosists of three steps: dividig a problem ito smaller subproblems solvig (recursively) each subproblem the combiig solutios

More information

THE ARITHMETIC OF INTEGERS. - multiplication, exponentiation, division, addition, and subtraction

THE ARITHMETIC OF INTEGERS. - multiplication, exponentiation, division, addition, and subtraction THE ARITHMETIC OF INTEGERS - multiplicatio, expoetiatio, divisio, additio, ad subtractio What to do ad what ot to do. THE INTEGERS Recall that a iteger is oe of the whole umbers, which may be either positive,

More information

Basic Elements of Arithmetic Sequences and Series

Basic Elements of Arithmetic Sequences and Series MA40S PRE-CALCULUS UNIT G GEOMETRIC SEQUENCES CLASS NOTES (COMPLETED NO NEED TO COPY NOTES FROM OVERHEAD) Basic Elemets of Arithmetic Sequeces ad Series Objective: To establish basic elemets of arithmetic

More information

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

More information

arxiv:1012.1336v2 [cs.cc] 8 Dec 2010

arxiv:1012.1336v2 [cs.cc] 8 Dec 2010 Uary Subset-Sum is i Logspace arxiv:1012.1336v2 [cs.cc] 8 Dec 2010 1 Itroductio Daiel M. Kae December 9, 2010 I this paper we cosider the Uary Subset-Sum problem which is defied as follows: Give itegers

More information

Convexity, Inequalities, and Norms

Convexity, Inequalities, and Norms Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information

A Mathematical Perspective on Gambling

A Mathematical Perspective on Gambling A Mathematical Perspective o Gamblig Molly Maxwell Abstract. This paper presets some basic topics i probability ad statistics, icludig sample spaces, probabilistic evets, expectatios, the biomial ad ormal

More information

Hypergeometric Distributions

Hypergeometric Distributions 7.4 Hypergeometric Distributios Whe choosig the startig lie-up for a game, a coach obviously has to choose a differet player for each positio. Similarly, whe a uio elects delegates for a covetio or you

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

CS103X: Discrete Structures Homework 4 Solutions

CS103X: Discrete Structures Homework 4 Solutions CS103X: Discrete Structures Homewor 4 Solutios Due February 22, 2008 Exercise 1 10 poits. Silico Valley questios: a How may possible six-figure salaries i whole dollar amouts are there that cotai at least

More information

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

9.8: THE POWER OF A TEST

9.8: THE POWER OF A TEST 9.8: The Power of a Test CD9-1 9.8: THE POWER OF A TEST I the iitial discussio of statistical hypothesis testig, the two types of risks that are take whe decisios are made about populatio parameters based

More information

Lesson 15 ANOVA (analysis of variance)

Lesson 15 ANOVA (analysis of variance) Outlie Variability -betwee group variability -withi group variability -total variability -F-ratio Computatio -sums of squares (betwee/withi/total -degrees of freedom (betwee/withi/total -mea square (betwee/withi

More information

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here). BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook - Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly

More information

Maximum Likelihood Estimators.

Maximum Likelihood Estimators. Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

A Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length

A Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length Joural o Satisfiability, Boolea Modelig ad Computatio 1 2005) 49-60 A Faster Clause-Shorteig Algorithm for SAT with No Restrictio o Clause Legth Evgey Datsi Alexader Wolpert Departmet of Computer Sciece

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

5.4 Amortization. Question 1: How do you find the present value of an annuity? Question 2: How is a loan amortized?

5.4 Amortization. Question 1: How do you find the present value of an annuity? Question 2: How is a loan amortized? 5.4 Amortizatio Questio 1: How do you fid the preset value of a auity? Questio 2: How is a loa amortized? Questio 3: How do you make a amortizatio table? Oe of the most commo fiacial istrumets a perso

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

{{1}, {2, 4}, {3}} {{1, 3, 4}, {2}} {{1}, {2}, {3, 4}} 5.4 Stirling Numbers

{{1}, {2, 4}, {3}} {{1, 3, 4}, {2}} {{1}, {2}, {3, 4}} 5.4 Stirling Numbers . Stirlig Numbers Whe coutig various types of fuctios from., we quicly discovered that eumeratig the umber of oto fuctios was a difficult problem. For a domai of five elemets ad a rage of four elemets,

More information

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or p-series (the Compariso Test), but of

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

Chapter 5 Unit 1. IET 350 Engineering Economics. Learning Objectives Chapter 5. Learning Objectives Unit 1. Annual Amount and Gradient Functions

Chapter 5 Unit 1. IET 350 Engineering Economics. Learning Objectives Chapter 5. Learning Objectives Unit 1. Annual Amount and Gradient Functions Chapter 5 Uit Aual Amout ad Gradiet Fuctios IET 350 Egieerig Ecoomics Learig Objectives Chapter 5 Upo completio of this chapter you should uderstad: Calculatig future values from aual amouts. Calculatig

More information

Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread and Boxplots Discrete Math, Section 9.4 Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

Trigonometric Form of a Complex Number. The Complex Plane. axis. ( 2, 1) or 2 i FIGURE 6.44. The absolute value of the complex number z a bi is

Trigonometric Form of a Complex Number. The Complex Plane. axis. ( 2, 1) or 2 i FIGURE 6.44. The absolute value of the complex number z a bi is 0_0605.qxd /5/05 0:45 AM Page 470 470 Chapter 6 Additioal Topics i Trigoometry 6.5 Trigoometric Form of a Complex Number What you should lear Plot complex umbers i the complex plae ad fid absolute values

More information

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

More information

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee

More information

Math C067 Sampling Distributions

Math C067 Sampling Distributions Math C067 Samplig Distributios Sample Mea ad Sample Proportio Richard Beigel Some time betwee April 16, 2007 ad April 16, 2007 Examples of Samplig A pollster may try to estimate the proportio of voters

More information

Section 1.6: Proof by Mathematical Induction

Section 1.6: Proof by Mathematical Induction Sectio.6 Proof by Iductio Sectio.6: Proof by Mathematical Iductio Purpose of Sectio: To itroduce the Priciple of Mathematical Iductio, both weak ad the strog versios, ad show how certai types of theorems

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

1.3 Binomial Coefficients

1.3 Binomial Coefficients 18 CHAPTER 1. COUNTING 1. Biomial Coefficiets I this sectio, we will explore various properties of biomial coefficiets. Pascal s Triagle Table 1 cotais the values of the biomial coefficiets ( ) for 0to

More information

Overview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals

Overview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals Overview Estimatig the Value of a Parameter Usig Cofidece Itervals We apply the results about the sample mea the problem of estimatio Estimatio is the process of usig sample data estimate the value of

More information

A RANDOM PERMUTATION MODEL ARISING IN CHEMISTRY

A RANDOM PERMUTATION MODEL ARISING IN CHEMISTRY J. Appl. Prob. 45, 060 070 2008 Prited i Eglad Applied Probability Trust 2008 A RANDOM PERMUTATION MODEL ARISING IN CHEMISTRY MARK BROWN, The City College of New York EROL A. PEKÖZ, Bosto Uiversity SHELDON

More information

Entropy of bi-capacities

Entropy of bi-capacities Etropy of bi-capacities Iva Kojadiovic LINA CNRS FRE 2729 Site école polytechique de l uiv. de Nates Rue Christia Pauc 44306 Nates, Frace iva.kojadiovic@uiv-ates.fr Jea-Luc Marichal Applied Mathematics

More information

Chapter 6: CPU Scheduling. Previous Lectures. Basic Concepts. Histogram of CPU-burst Times. CPU Scheduler. Alternating Sequence of CPU And I/O Bursts

Chapter 6: CPU Scheduling. Previous Lectures. Basic Concepts. Histogram of CPU-burst Times. CPU Scheduler. Alternating Sequence of CPU And I/O Bursts Multithreadig Memory Layout Kerel vs User threads Represetatio i OS Previous Lectures Differece betwee thread ad process Thread schedulig Mappig betwee user ad kerel threads Multithreadig i Java Thread

More information

6.042/18.062J Mathematics for Computer Science. Recurrences

6.042/18.062J Mathematics for Computer Science. Recurrences 6.04/8.06J Mathematics for Computer Sciece Srii Devadas ad Eric Lehma March 7, 00 Lecture Notes Recurreces Recursio breakig a object dow ito smaller objects of the same type is a major theme i mathematics

More information

Totally Corrective Boosting Algorithms that Maximize the Margin

Totally Corrective Boosting Algorithms that Maximize the Margin Mafred K. Warmuth mafred@cse.ucsc.edu Ju Liao liaoju@cse.ucsc.edu Uiversity of Califoria at Sata Cruz, Sata Cruz, CA 95064, USA Guar Rätsch Guar.Raetsch@tuebige.mpg.de Friedrich Miescher Laboratory of

More information

Notes on exponential generating functions and structures.

Notes on exponential generating functions and structures. Notes o expoetial geeratig fuctios ad structures. 1. The cocept of a structure. Cosider the followig coutig problems: (1) to fid for each the umber of partitios of a -elemet set, (2) to fid for each the

More information

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means)

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means) CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:

More information

Universal coding for classes of sources

Universal coding for classes of sources Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric

More information

NUMBERS COMMON TO TWO POLYGONAL SEQUENCES

NUMBERS COMMON TO TWO POLYGONAL SEQUENCES NUMBERS COMMON TO TWO POLYGONAL SEQUENCES DIANNE SMITH LUCAS Chia Lake, Califoria a iteger, The polygoal sequece (or sequeces of polygoal umbers) of order r (where r is r > 3) may be defied recursively

More information

1. C. The formula for the confidence interval for a population mean is: x t, which was

1. C. The formula for the confidence interval for a population mean is: x t, which was s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

More information

Standard Errors and Confidence Intervals

Standard Errors and Confidence Intervals Stadard Errors ad Cofidece Itervals Itroductio I the documet Data Descriptio, Populatios ad the Normal Distributio a sample had bee obtaied from the populatio of heights of 5-year-old boys. If we assume

More information

Plug-in martingales for testing exchangeability on-line

Plug-in martingales for testing exchangeability on-line Plug-i martigales for testig exchageability o-lie Valetia Fedorova, Alex Gammerma, Ilia Nouretdiov, ad Vladimir Vovk Computer Learig Research Cetre Royal Holloway, Uiversity of Lodo, UK {valetia,ilia,alex,vovk}@cs.rhul.ac.uk

More information

x(x 1)(x 2)... (x k + 1) = [x] k n+m 1

x(x 1)(x 2)... (x k + 1) = [x] k n+m 1 1 Coutig mappigs For every real x ad positive iteger k, let [x] k deote the fallig factorial ad x(x 1)(x 2)... (x k + 1) ( ) x = [x] k k k!, ( ) k = 1. 0 I the sequel, X = {x 1,..., x m }, Y = {y 1,...,

More information

Lecture 4: Cheeger s Inequality

Lecture 4: Cheeger s Inequality Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular

More information

Arithmetic Sequences and Partial Sums. Arithmetic Sequences. Definition of Arithmetic Sequence. Example 1. 7, 11, 15, 19,..., 4n 3,...

Arithmetic Sequences and Partial Sums. Arithmetic Sequences. Definition of Arithmetic Sequence. Example 1. 7, 11, 15, 19,..., 4n 3,... 3330_090.qxd 1/5/05 11:9 AM Page 653 Sectio 9. Arithmetic Sequeces ad Partial Sums 653 9. Arithmetic Sequeces ad Partial Sums What you should lear Recogize,write, ad fid the th terms of arithmetic sequeces.

More information

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

AQA STATISTICS 1 REVISION NOTES

AQA STATISTICS 1 REVISION NOTES AQA STATISTICS 1 REVISION NOTES AVERAGES AND MEASURES OF SPREAD www.mathsbox.org.uk Mode : the most commo or most popular data value the oly average that ca be used for qualitative data ot suitable if

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

The Euler Totient, the Möbius and the Divisor Functions

The Euler Totient, the Möbius and the Divisor Functions The Euler Totiet, the Möbius ad the Divisor Fuctios Rosica Dieva July 29, 2005 Mout Holyoke College South Hadley, MA 01075 1 Ackowledgemets This work was supported by the Mout Holyoke College fellowship

More information