MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this to the Abracadabra problem, which calculates the expected time ecessary for a radomly typed sequece of letters to occur. Cotets 1. Itroductio 1 2. Measure Spaces 1 3. Radom Variables, Idepedece, ad Expectatio 3 4. Martigales 5 5. The Abracadabra Problem 8 Ackowledgmets 10 Refereces 10 1. Itroductio While probability ca be studied without utilizig measure theory, takig the measure-theoretic approach to probability provides sigificatly more geerality. I sectio 2, we begi the paper by costructig geeral measure spaces, without cosiderig their uses i probability. I sectio 3, we provide measure-theoretic defiitios for familiar probability topics such as radom variables ad expectatio as well as prove several theorems which are ecessary for later proofs. Sectio 4 will provide the defiitio of martigales ad develop martigales i order to prove Doob s Optioal-Stoppig Theorem which is istrumetal i solvig the Abracadabra problem. Sectio 5 will coclude the paper with a solutio of exercise 10.6 i Probability with Martigales. This problem cocers the expected time it takes for a mokey to type the letters ABRACADABRA i that order. This paper closely follows David Williams Probability with Martigales [1] ad may of the proofs preseted i this paper ca be foud i his book. 2. Measure Spaces Defiitio 2.1. A collectio Σ 0 of subsets of S is called a algebra o S if (1) S Σ 0 (2) F Σ 0 F c Σ 0 (3) F, G Σ 0 F G Σ 0 Date: DEADLINE AUGUST 26, 2011. 1

2 TURNER SMITH Corollary 2.2. By defiitio, = S c Σ 0 ad F, G Σ 0 F G = (F c G c ) c Σ 0 These imply that a algebra o S is a family of subsets of S stable uder fiitely may set operatios. Defiitio 2.3. A collectio Σ of subsets of S is called a σ - algebra o S if Σ is a algebra o S such that wheever F Σ ( N), the F Σ. Additioally, this implies that F = ( F) c c Σ. Thus, a σ-algebra o S is a family of subsets of S stable uder ay coutable collectio of set operatios. A σ-algebra ca be thought of as cotaiig all the kow iformatio about a give sample space. This will be importat i Sectio 4 i our discussio of martigales. Not all sets are measurable ad the σ-algebra is the collectio of sets over which a measure is defied. Perhaps the easiest way to thik about σ-algebras is to picture a σ-algebra F as correspodig to collectios of yes or o questios with every set i F as a outcome i Ω for which there is a aswer to the yes or o questio. For example, let Ω be possible arrival times for a bus. The, oe set i F could be all the outcomes where the bus arrives after 10:30am today. If oe ca ask if the bus arrives after 10:30am the it is obvious that the complemet ca be asked as well, Did the bus arrive before or at 10:30am today? Now we ca combie multiple questios with or s or ad s, which is the equivalet of takig uios ad itersectios, to get aother yes or o questio. Defiitio 2.4. Let C be a class of subsets of S. The σ(c), the σ-algebra geerated by C, is the smallest σ-algebra Σ o S such that C Σ. Alteratively, this is the itersectio of all σ-algebras o S which have C as a subclass. Defiitio 2.5. The Borel σ-algebra, deoted by B, is the σ-algebra geerated by the family of ope subsets i R. Because the complemet of ay ope set is closed, a Borel set is thus ay that ca be writte with ay coutable combiatio of the set operatios uio ad itersectio of closed ad ope sets. Next we eed to defie a measure. This will require defiitios of coutably additive ad measure spaces. Defiitio 2.6. Let S be a set ad let Σ 0 be a algebra o S. The a oegative set fuctio µ 0 : Σ 0 [0, ] is called coutably additive if µ 0 ( ) = 0 ad wheever (F : N) is a sequece of disjoit sets i Σ 0 with uio F = F Σ 0, the µ 0 (F ) = µ 0 (F ). Defiitio 2.7. A pair (S, Σ), where S is a set ad Σ is a σ-algebra o S, is a measurable space. A elemet of Σ is called a Σ-measurable subset of S.

MARTINGALES AND A BASIC APPLICATION 3 Defiitio 2.8. Let (S, Σ) be a measurable space, so that Σ is a σ-algebra o S. A map µ : Σ [0, ] is called a measure o (S, Σ) if µ is coutably additive. The triple (S, Σ, µ) is the called a measure space. Essetially, a measure assigs a real umber to all of the subsets of S. This umber is most simply viewed as the volume of the set. Measures ca be chose arbitrarily, but they must be coutably additive. This prevets a measure made of two sets beig combied from beig smaller tha either of the two idividual sets measures. Measure spaces traslate ito probability through the probability triple geerally deoted (Ω, F, P). Of course, these symbols have some additioal ituitio ad/or meaig whe relatig to probability. Defiitio 2.9. The set Ω is called the sample space ad cosists of elemets ω which are called sample poits. Similarly, the σ-algebra F is called the family of evets, where a evet F is a elemet of F, or, alteratively, a F-measurable subset of Ω. Defiitio 2.10. A measure P is called a probability measure if P(Ω) = 1. 3. Radom Variables, Idepedece, ad Expectatio Radom variables are a cocept which someoe who has studied probability at ay level would be familiar with. Naturally they occur i the measure-theoretic approach to probability as well. I order to preset the defiitio of a radom variable, we first must defie Σ-measurable fuctios. Defiitio 3.1. Let (S, Σ) be a measurable space, so that Σ is a σ-algebra o S. Suppose that h : S. For A R, defie h 1 (A) := {s S : h(s) A}. The h is called Σ-measurable if h 1 : B Σ, that is, h 1 (A) Σ, A B. We write mσ for the the class of Σ-measurable fuctios o S, ad (mσ) + for the class of o-egative elemets i mσ. Now we ca defie what a radom variable is. Defiitio 3.2. Let (Ω, F) be our (sample space, family of evet). A radom variable is a elemet of mf. Thus, where X is the radom variable. X : ω R, X 1 : B F, Defiitio 3.3. Sub-σ-algebras F 1, F 2,..., F are idepedet if, for all evets F ik F i, P(F 1 F 2... F ) = Σ i=1p(f i ). Defiitio 3.4. Radom variables X 1, X 2,...,X are idepedet if their relevat σ-algebras (σ(x 1 ), σ(x 2 ),..., σ(x )) are idepedet. For idepedece, the familiar otio which does ot ivolve σ-algebras is sufficiet for our purposes so it is also preseted here.

4 TURNER SMITH Defiitio 3.5. Evets E 1, E 2,..., E are idepedet if ad oly if wheever N ad i 1,..., i are distict, the P(E i1... E i ) = P(E ik ). Next, we require a defiitio for expectatio. Defiitio 3.6. For a radom variable X L 1 = L 1 (Ω, F, P), we defie the expectatio E(X) of X by E(X) := XdP = X(ω)P(dω). Ω Ω k=1 Ad lastly, we preset the defiitio of coditioal expectatio. Defiitio 3.7. Let (Ω, F, P) be a probability triple, ad X a radom variable with E( X ) <. Let G be a sub-σ-algebra of F. The there exists a radom variable Y such that (1) Y is G measurable. (2) E( Y ) < (3) for every set G i G, we have Y dp = XdP, G G. G G A radom variable Y with properties (1), (2), ad (3) is called a a versio of the coditioal expectatio E(X G) of X give G, ad we write Y = E(X G), almost surely. [1, Theorem 9.2] Next we will preset the Mootoe-Covergece Theorem which will begi a chai of proofs required to prove the Domiated-Covergece Theorem ad the Bouded Covergece Theorem which are ecessary for Sectio 4. Theorem 3.8. If f is a sequece of elemets of (mσ) + such that f f, the µ(f ) µ(f) Proof. This is the Mootoe Covergece Theorem ad the proof ca be foud i the Appedix of [1]. Next we preset the Fatou Lemma. Theorem 3.9. For a sequece (f ) i (mσ) +, Proof. We have µ(lim if f ) lim if µ(f ). (3.10) lim if f = lim g k, whereg k := if f. k For k, we have f g k, so that µ(f ) µ(g k ), where µ(g k ) if k µ(f ); ad by combiig this with a applicatio of MON to (*), we obtai µ(lim if f ) = lim µ(g k ) lim if k µ(f ) =: lim if µ(f ). k k

MARTINGALES AND A BASIC APPLICATION 5 Ad ow the Reverse Fatou Lemma. Theorem 3.11. If (f ) is a sequece i (mσ) + such that for some g i (mσ) +, we have f g,, ad µ(g) <, the Proof. Apply Fatou Lemma to (g f ). µ(lim sup f ) lim sup µ(f ). We ow preset the two theorems which are used i Sectio 4 for the proof of Doob s Optioal-Stoppig Theorem. The first is called the Domiated-Covergece Theorem. Theorem 3.12. If X (ω) Y (ω) for all, ω ad X X poitwise almost surely, where E(Y) <, the so that E( X X ) 0. E(X ) E(X) Proof. We have f f 2g, where µ(2g) <, so by the reverse Fatou Lemma, Sice lim sup µ( f f ) µ(lim sup f f ) = µ(0) = 0. µ(f ) µ(f) = µ(f f) µ( f f ) The secod theorem is the Bouded Covergece Theorem. Theorem 3.13. Let {X } be a sequece of radom variables, ad let X be a radom variable. Suppose that X X i probability ad that for some K i [0, ), we have for every ad ω The X (ω) K E( X X ) 0. Proof. This is a direct cosequece of the Domiated-Covergece Theorem ad ca be obtaied by takig Y (ω) = K, for all ω. 4. Martigales Now we take (Ω, F, P) to be the probability triple which we are referrig to. We will ow preset the defiitio of martigales after first providig the ecessary defiitios of filtratios ad adapted processes. Defiitio 4.1. Now istead of usig probability triples as before, we will take a filtered space. (Ω, F, F, P) {F : 0} is called a filtratio. A filtratio is a icreasig family of sub-σalgebras of F such that F F 1... F ad F is defied as F := σ( F ) F

6 TURNER SMITH Each filtratio ca ituitively be thought of as the iformatio available about the evets i a sample space after time. Note that as icreases, more iformatio is available ad oe of the previous iformatio is lost. Defiitio 4.2. A process X = (X : 0) is called adapted to the filtratio F if for each, X is F -measurable. This is essetially statig that the value X (ω) is kow at time. Each of the X (ω) depeds oly o the iformatio we have up to ad icludig time, that is they do ot deped o iformatio i the future. Now we provide the defiitio of a martigale. Defiitio 4.3. A process X is called a martigale if (1) X is adapted (2) E( X ) <, (3) E[X F 1 ] = X 1, almost surely ( 1). Defiitio 4.4. A supermartigale is defied the same as a martigale oly with (3) replaced by E[X F 1 ] X 1. Defiitio 4.5. Similarily, A submartigale is defied the same as a martigale oly with (3) replaced by E[X F 1 ] X 1. It is importat to ote is that a supermartigale decreases o average as the expected value of X give all previous iformatio is less tha X 1. Likewise, a submartigale icreases o average. Addtioally, if X is a supermartigale, ote that X is a submartigale, ad that X is a martigale if ad oly if it is both a supermartigale ad a submartigale. It is also readily apparet how martigales ad supermartigales ca be thought of as fair ad ufair games, respectively. If you cosider X X 1 to be your et wiigs per uit stake i game, with 1. If X is a martigale, you ca see that E[X F 1 ] = X 1 E[X F 1 ] X 1 = 0 E[X F 1 ] E[X 1 F 1 ] = 0 E[X X 1 F 1 ] = 0. The secod arrow follows as E[X 1 F 1 ] = X 1, sice X is adapted to F 1 ad thus you kow exactly what X 1 is by kowig F 1. This fits the defiitio of a fair game as the expected et wiigs is zero per game for all games. By usig similar method, you ca tell that whe X is a supermartigale, E[X X 1 F 1 ] 0. This is a ufair game, as your expected et wiigs are egative after each game. The submartigale case is similar, but less iterestig as very few people are cocered by a game where they wi moey o average. Additioally this provides us with a ew way to defie martigales as all of the arrows ca be proved i reverse. Next we will defie previsible process, which is essetially the mathematical expressio of a particular gamblig strategy.

MARTINGALES AND A BASIC APPLICATION 7 Defiitio 4.6. Process C = (C : N) is called previsible if C is F 1 measurable for 1. Each C represets your particular stake i game. You are able to use the history of your previous bets up to time 1 to chage C. Thus, C (X X 1 ) are your wiigs o game ad your total wiigs up to time are Y = C k (X k X k 1 ) =: (C X). 1 k Clearly (C X) 0 = 0 as there are o wiigs if o games are played. Additioally, we ca obtai the wiigs for game as follows, Y Y 1 = C (X X 1 ). Now we defie stoppig time as it is importat to our defiitio of Doob s Optioal-Stoppig Theorem. Defiitio 4.7. A map T : Ω 0, 1, 2,..., is called a stoppig time if, or {T } = {ω : T (ω) } F, {T = } = {ω : T (ω) = } F,. For a stoppig time T, it is possible to decide whether {T } has occurred based o filtratio F, meaig that the evet {T } is F -measurable. For example, if a perso gambles util the either play te games or ru out of moey, this is a stoppig time. However, a perso gamblig util they wi more moey tha they ever will is ot, as that is ot kowable with the kow iformatio at time. Now we must prove a theorem which is itegral to our proof of Doob s Optioal- Stoppig Theorem. Before that, there is a bit of otatio which may be ew. For x, y R, x y := mi(x, y). Theorem 4.8. If X is a supermartigale ad T is a stoppig time, the the stopped process X T = (X T : Z + ) is a supermartigale, so that i particular, E(X T ) = E(X 0 ),. Note that this theorem does ot say aythig about (4.9) E(X T ) = E(X 0 ) Now we ca preset a proof of Doob s Optioal-Stoppig Theorem. Theorem 4.10. (a) Let T be a stoppig time. Let X be a supermartigale. The X T is itegrable ad E(X T ) E(X 0 ) i each of the followig situatios: (1) T is bouded (for some N i N, T (ω) N, ω), (2) X is bouded (for some K i R +, X (ω) K for every ad every ω) (3) E(T ) <, ad, for some K i R +, X (ω) X 1 (ω) K, (, ω) (b) If ay of the coditios 1-3 hold ad X is a martigale, the E(X T ) = E(X 0 ).

8 TURNER SMITH Proof. First we will prove (a). We kow that X T is itegrable ad (4.11) E(X T X 0 ) 0 because of Theorem 4.8. To prove (1), take = N. For (2), let i (4.11) usig Theorem 3.7. For (3), we have T X T X 0 = (X k X k 1 KT k=1 ad E(KT ) <, so that Theorem 3.6 ca be used to allow i (4.11) which provides the ecessary aswer. Next we prove (b). By applyig (a) to X ad ( X) you will get two iequalities i opposite directios. This implies equality. Corollary 4.12. Suppose that M is a martigale, the icremets M M 1 of which are bouded by some costat K 1. Suppose that C is a previsible process bouded by some costat K 2, ad that T is a stoppig time such that E(T ) <. The, E(C M) T = 0. I essece, this corollary states that, assumig you have o kowledge of future evets, you caot beat a fair game. Lemma 4.13. Suppose that T is a stoppig time such that for some N i N ad some ɛ > 0, we have, for every i N: The E(T ) <. P(T + N F ) > ɛ, almostsurely. 5. The Abracadabra Problem To coclude the paper, I will ow preset a problem which ca be solved with the material preseted above. A mokey types letters at radom, oe per each uit of time, producig a ifiite sequece of idetically idepedet radom letters. If the mokey is equally likely to type ay of the 26 letters, how log o average will it take him to produce the sequece ABRACADABRA Now it may iitially appear difficult to solve this problem, but the use of martigale simplifies the problem greatly. Costruct a sceario where before each time = 1, 2,..., m aother gambler arrives. This gambler bets $1 that the th letter will be A. If he loses, the he leaves. If he wis, the he receives $26 which he the bets all of o the evet that the ( + 1)th letter will be B. If he loses, he leaves. If he wis, the he bets his etire wiigs of $26 2 o the evet that the (+2)th letter will be R. This process cotiues util the etire ABRACADABRA sequece has occurred. Let T be the first time by which the mokey has produced the sequece ABRACADABRA. We are tryig to fid E(T ).

MARTINGALES AND A BASIC APPLICATION 9 Let C j be the bet of the jth gambler at time. Defie A to be the th letter of the sequece. 0, if < j C j 1, if = j = 26 k, ifa j,..., A j+k 1 were correct ad = j + k 0, otherwise It is clear that each C j is a previsible process as it is determied oly by iformatio up to the ( 1)th bet. Now if we defie M j to be the martigale of payoffs after bets for j gamblers, the it will simplify the solutio to the problem. To show that M j is a martigale, we must show that (1) M j is adapted, (2) E( M j ) <,, (3) E[M j F 1 ] = M j 1, 1 To show (1), simply otice that M j is determied by the evet A, ad whether the letter typed at time is correct. For (2), ote that M j is always positive ad is bouded above by 26. As E[ M ] j = E[M] j < 26 <, (2) is satisfied. There are two cases for (3). First whe the gambler loses before time, the E[M F j 1 ] = 0 = M j 1. If the gambler wis the first ( 1) times, the E[M j F 1 ] = 26 1/26 + 0 25/26 = 26 1 = M j 1. We ca ow apply Doob s Optioal-Stoppig Theorem because M j is a martigale. All we must do ow is show that oe of its three coditios are satisfied. For simplicity we will choose (3), (5.1) E(T ) <, ad, for some K i R +, X (ω) X 1 (ω) K, (, ω). First refer to (4.13). Let N = 11 ad ɛ = (1/26) N. It is clear that whatever is, there is always a (1/26) 11 chace that ABRACADABRA will occur i the ext 11 letters. Thus, the coditio holds with N = 11 ad E(T ) <. Now we must show the secod part of the coditio of (5.1). First defie X := M j = M. j j=1 The secod equality holds because after the stoppig time, the + 1 gamblers have ot begu bettig so all terms after are zero. It is useful to view X as the cumulative wiigs of every gambler up to ad icludig time. X is a martigale as it is simply the sum of expectatios ad M j is a martigale. This ca be easily show from the defitios ad ca be worked as a exercise if a rigourous proof is eeded to covice the reader. Notice that, X X 1 26 11 + 26 4 + 26. This is because X X 1 deotes the maximum payoff at time. To fid the maximum, assume that the mokey has typed everythig correctly ad fid the maximum amout of moey that ca be wo after a sigle uit of time. Sice each gambler wis icrease the more correct bets they get i a row, it is easy to see that the first gambler has wo $26 11 at time 11 if he started wiig at the first A. There ca be o more wiig gamblers util the 4th A because if the 1st gambler j=1

10 TURNER SMITH wis $26 11 the the 2d, 3rd,..., 7th gamblers all must lose. The gambler who started wiig at the 4th A ca wi a maximum of $26 4 because there are four more letters that the mokey ca type correctly. Lastly, the 11th gambler ca also wi $26 because the last letter is a A. By meetig the requiremets for Doob s Optioal-Stoppig Theorem, we ca utilize its coclusio: E(X T ) = E(X 0 ) E(X 0 ) = 0 because othig happes at time 0. E(X T ) is the cumulative wiigs of all the gamblers after the mokey types ABRACADABRA correctly. As show earlier, the wiigs will equal 26 11 + 26 4 + 26. However, i calculatig these wiigs, we eglected to iclude the moey that the gamblers lose with each subsequet letter. As each gambler arrives, he bets $1 so after time T, there have bee $T dollars lost because of these $1 bets. Thus, E(X T ) = E(26 11 + 26 4 + 26 T ) = 26 11 + 26 4 + 26 E(T ) = 0 E(T ) = 26 11 + 26 4 + 26. The key reaso for the solutio beig possible this way is the fact that E(T ) appears i the calculatio for E(X T ). This was possible due to the way we defied the problem, with each gambler bettig $1 upo their arrival. This is a useful trick to remember as this method works for computig E(T ) of ay patter i a radom sequece of symbols. For example E(T ) = 6 4 + 6 3 + 6 2 + 6 1 for the evet where someoe rolls four 6 s i a row o a fair six-sided die. Ackowledgmets. It is a pleasure to thak my metor, Ilya Gehktma, for his kowledgable isights ad his flexible schedulig. Refereces [1] Michael Williams. Probability with Martigales. Cambridge Uiversity Press. 1991.