The Sample Complexity of Exploration in the Multi-Armed Bandit Problem

Size: px
Start display at page:

Download "The Sample Complexity of Exploration in the Multi-Armed Bandit Problem"

Transcription

1 Joura of Machie Learig Research 5 004) Submitted 1/04; Pubished 6/04 The Sampe Compexity of Exporatio i the Muti-Armed Badit Probem Shie Maor Joh N. Tsitsikis Laboratory for Iformatio ad Decisio Systems Massachusetts Istitute of Techoogy Cambridge, MA 0139, USA Editors: Kristi Beett ad Nicoò Cesa-Biachi Abstract We cosider the muti-armed badit probem uder the PAC probaby approximatey correct ) mode. It was show by Eve-Dar et a. 00) that give arms, a tota of O /ε )og1/δ) ) trias suffices i order to fid a ε-optima arm with probabiity at east 1 δ. We estabish a matchig ower boud o the expected umber of trias uder ay sampig poicy. We furthermore geeraize the ower boud, ad show a expicit depedece o the ukow) statistics of the arms. We aso provide a simiar boud withi a Bayesia settig. The case where the statistics of the arms are kow but the idetities of the arms are ot, is aso discussed. For this case, we provide a ower boud of Θ 1/ε ) + og1/δ)) ) o the expected umber of trias, as we as a sampig poicy with a matchig upper boud. If istead of the expected umber of trias, we cosider the maximum over a sampe paths) umber of trias, we estabish a matchig upper ad ower boud of the form Θ /ε )og1/δ) ). Fiay, we derive ower bouds o the expected regret, i the spirit of Lai ad Robbis. 1. Itroductio The muti-armed badit probem is a cassica probem i decisio theory. There is a umber of aterative arms, each with a stochastic reward whose probabiity distributio is iitiay ukow. We try these arms i some order, which may deped o the sequece of rewards that have bee observed so far. A commo objective i this cotext is to fid a poicy for choosig the ext arm to be tried, uder which the sum of the expected rewards comes as cose as possibe to the idea reward, i.e., the expected reward that woud be obtaied if we were to try the best arm at a times. Oe of the attractive features of the muti-armed badit probem is that despite its simpicity, it ecompasses may importat decisio theoretic issues, such as the tradeoff betwee exporatio ad expoitatio. The muti-armed badit probem has bee widey studied i a variety of setups. The probem was first cosidered i the 50 s, i the semia work of Robbis 195), which derives poicies that asymptoticay attai a average reward that coverges i the imit to the reward of the best arm. The muti-armed badit probem was ater studied i discouted, Bayesia, Markovia, expected reward, ad adversaria setups. See Berry ad Fristedt 1985) for a review of the cassica resuts o the muti-armed badit probem. c 004 Shie Maor ad Joh Tsitsikis.

2 MANNOR AND TSITSIKLIS Lower bouds for differet variats of the muti-armed badit have bee studied by severa authors. For the expected regret mode, where the regret is defied as the differece betwee the idea reward if the best arm were kow) ad the reward uder a oie poicy, the semia work of Lai ad Robbis 1985) provides asymptoticay tight bouds i terms of the Kuback-Leiber divergece betwee the distributios of the rewards of the differet arms. These bouds grow ogarithmicay with the umber of steps. The adversaria muti-armed badit probem i.e., without ay probabiistic assumptios) was cosidered i Auer et a. 1995, 00b), where it was show that the expected regret grows proportioay to the square root of the umber of steps. Of reated iterest is the work of Kukari ad Lugosi 000) which shows that for ay specific time t, oe ca choose the reward distributios so that the expected regret is iear i t. The focus of this paper is the cassica muti-armed badit probem, but rather tha ookig at the expected regret, we are cocered with PAC-type bouds o the umber of steps eeded to idetify a ear-optima arm. I particuar, we are iterested i the expected umber of steps that are required i order to idetify with high probabiity at east 1 δ) a arm whose expected reward is withi ε from the expected reward of the best arm. This aturay abstracts the case where oe must evetuay commit to oe specific arm, ad quatifies the amout of exporatio ecessary. This is i cotrast to most of the resuts for the muti-armed badit probem, where the mai aim is to maximize the expected cumuative reward whie both exporig ad expoitig. I Eve-Dar et a. 00), a poicy, caed the media eimiatio agorithm, was provided which requires O/ε )og1/δ)) trias, ad which fids a ε-optima arm with probabiity at east 1 δ. A matchig ower boud was aso derived i Eve-Dar et a. 00), but it oy appied to the case where δ > 1/, ad therefore did ot capture the case where high cofidece sma δ) is desired. I this paper, we derive a matchig ower boud which aso appies whe δ > 0 is arbitrariy sma. Our mai resut ca be viewed as a geeraizatio of a O1/ε )og1/δ)) ower boud provided i Athoy ad Bartett 1999), ad Cheroff 197), for the case of two badits. The proof i Athoy ad Bartett 1999) is based o a hypothesis iterchage argumet, ad reies criticay o the fact there are oy two uderyig hypotheses. Furthermore, it is imited to oadaptive poicies, for which the umber of trias is fixed a priori. The techique we use is based o a ikeihood ratio argumet ad a tight martigae boud, ad appies to geera poicies. A differet type of ower boud was derived i Auer et a. 00b) for the expected regret i a adversaria setup. The bouds derived there ca aso be used to derive a ower boud for our probem, but do ot appear to be tight eough to capture the og1/δ) depedece o δ. Our work aso provides fudameta ower bouds i the cotext of sequetia aaysis see, e.g., Cheroff, 197; Jeiso et a., 198; Siegmud, 1985). I the aguage of Siegmud 1985), we provide a ower boud o the expected egth of a sequetia sampig poicy uder ay adaptive aocatio scheme. For the case of two arms, it was show i Siegmud 1985) p. 148) that if oe restricts to sampig poicies that oy take ito accout the empirica average rewards from the differet arms, the the probems of iferece ad arm seectio ca be treated separatey. As a cosequece, ad uder this restrictio, Siegmud 1985) shows that a optima aocatio caot be much better tha a uiform oe. Our resuts are differet i a umber of ways. First, we cosider mutipe hypotheses mutipe arms). Secod, we aow the aocatio rue to be competey geera ad to deped o the whoe history. Third, uike most of the sequetia aaysis iterature see, e.g., Jeiso et a., 198), we do ot restrict ourseves to the imitig case where the probabiity of error coverges to zero. Fiay, we cosider fiite time bouds, rather tha asymptotic oes. We further commet that 64

3 EXPLORATION IN MULTI-ARMED BANDITS our resuts exted those of Jeiso et a. 198), i that we cosider the case where the reward is ot Gaussia. Paper Outie The paper is orgaized as foows. I Sectio, we set up our framework, ad sice we are maiy iterested i ower bouds, we restrict to the specia case where each arm is a coi, i.e., the rewards are Beroui radom variabes, but with ukow parameters biases ). I Sectio 3, we provide a O/ε )og1/δ)) ower boud o the expected umber of trias uder ay poicy that fids a ε-optima coi with probabiity at east 1 δ. I Sectio 4, we provide a refied ower boud that depeds expicity o the specific though ukow) biases of the cois. This ower boud has the same og1/δ) depedece o δ; furthermore, every coi roughy cotributes a factor iversey proportioa to the square differece betwee its bias ad the bias of a best coi, but o more that 1/ε. I Sectio 5, we derive a ower boud simiar to the oe i Sectio 3, but withi a Bayesia settig, uder a prior distributio o the set of biases of the differet cois. I Sectio 6 we provide a boud o the expected regret which is simiar i spirit to the boud i Lai ad Robbis 1985). The costats i our bouds are sighty worse tha the oes i Lai ad Robbis 1985), but the differet derivatio, which iks the PAC mode to regret bouds, may be of idepedet iterest. Our boud hods for ay fiite time, as opposed to the asymptotic resut provided i Lai ad Robbis 1985). The case where the coi biases are kow i advace, but the idetities of the cois are ot, is discussed i Sectio 7. We provide a poicy that fids a ε-optima coi with probabiity at east 1 δ, uder which the expected umber of trias is O 1/ε ) + og1/δ)) ). We show that this boud is tight up to a mutipicative costat. If istead of the expected umber of trias, we cosider the maximum over a sampe paths) umber of trias, we estabish a matchig upper ad ower bouds of the form Θ/ε )og1/δ)). Fiay, Sectio 8 cotais some brief cocudig remarks.. Probem Defiitio The exporatio probem for muti-armed badits is defied as foows. We are give arms. Each arm is associated with a sequece of ideticay distributed Beroui i.e., takig vaues i 0, 1}) radom variabes Xk, k = 1,,..., with ukow mea. Here, Xk correspods to the reward obtaied the kth time that arm is tried. We assume that the radom variabes Xk, for = 1,...,, k = 1,,..., are idepedet, ad we defie p = p 1,..., p ). Give that we restrict to the Beroui case, we wi use i the seque the term coi istead of arm. A poicy is a mappig that give a history, chooses a particuar coi to be tried ext, or seects a particuar coi ad stops. We aow a poicy to use radomizatio whe choosig the ext coi to be tried or whe makig a fia seectio. However, we oy cosider poicies that are guarateed to stop with probabiity 1, for every possibe vector p. Otherwise, the expected umber of steps woud be ifiite.) Give a particuar poicy, we et P p be the correspodig probabiity measure o the atura probabiity space for this mode). This probabiity space captures both the radomess i the cois accordig to the vector p), as we as ay additioa radomizatio carried out by the poicy. We itroduce the foowig radom variabes, which are we defied, except possiby o the set of measure zero where the poicy does ot stop. We et T be the tota umber of times that 65

4 MANNOR AND TSITSIKLIS coi is tried, ad et T = T 1 + +T be the tota umber of trias. We aso et I be the coi which is seected whe the poicy decides to stop. We say that a poicy is ε,δ)-correct if ) P p p I > max ε 1 δ, for every p [0,1]. It was show i Eve-Dar et a. 00) that there exist costats c 1 ad c such that for every, ε > 0, ad δ > 0, there exists a ε,δ)-correct poicy uder which E p [T ] c 1 ε og c δ, p [0,1]. A matchig ower boud was aso estabished i Eve-Dar et a. 00), but oy for arge vaues of δ, amey, for δ > 1/. I cotrast, we aim at derivig bouds that capture the depedece of the sampe-compexity o δ, as δ becomes sma. 3. A Lower Boud o the Sampe Compexity We start with our cetra resut, which ca be viewed as a extesio of Lemma 5.1 from Athoy ad Bartett 1999), as we as a specia case of Theorem 5. We preset it here because it admits a simper proof, but aso because parts of the proof wi be used ater. Throughout the rest of the paper, og wi stad for the atura ogarithm. Theorem 1 There exist positive costats c 1, c, ε 0, ad δ 0, such that for every, ε 0,ε 0 ), ad δ 0,δ 0 ), ad for every ε,δ)-correct poicy, there exists some p [0,1] such that E p [T ] c 1 ε og c δ. I particuar, ε 0 ad δ 0 ca be take equa to 1/8 ad e 4 /4, respectivey. Proof Let us cosider a muti-armed badit probem with + 1 cois, which we umber from 0 to. We cosider a fiite set of + 1 possibe parameter vectors p, which we wi refer to as hypotheses. Uder ay oe of the hypotheses, coi 0 has a kow bias p 0 = 1+ε)/. Uder oe hypothesis, deoted by H 0, a the cois other tha zero have a bias of 1/, H 0 : p 0 = 1 + ε, p i = 1, for i 0, which makes coi 0 the best coi. Furthermore, for = 1,...,, there is a hypothesis H : p 0 = 1 + ε, = 1 + ε, p i = 1, for i 0,, which makes coi the best coi. We defie ε 0 = 1/8 ad δ 0 = e 4 /4. From ow o, we fix some ε 0,ε 0 ) ad δ 0,δ 0 ), ad a poicy, which we assume to be ε/,δ)-correct. If H 0 is true, the poicy must have probabiity at east 1 δ of evetuay stoppig ad seectig coi 0. If H is true, for some 0, the poicy must have probabiity at east 1 δ of evetuay stoppig ad seectig coi. We deote by E ad P the expectatio ad probabiity, respectivey, uder hypothesis H. 66

5 EXPLORATION IN MULTI-ARMED BANDITS We defie t by t = 1 cε og 1 4δ = 1 cε og 1 θ, 1) where θ = 4δ, ad where c is a absoute costat whose vaue wi be specified ater. 1 Note that θ < e 4 ad ε < 1/4. Reca that T stads for the umber of times that coi is tried. We assume that for some coi 0, we have E 0 [T ] t. We wi evetuay show that uder this assumptio, the probabiity of seectig H 0 uder H exceeds δ, ad vioates ε/,δ)-correctess. It wi the foow that we must have E 0 [T ] > t for a 0. Without oss of geeraity, we ca ad wi assume that the above coditio hods for = 1, so that E 0 [T 1 ] t. We wi ow itroduce some specia evets A ad C uder which various radom variabes of iterest do ot deviate sigificaty from their expected vaues. We defie ad obtai from which it foows that A = T 1 4t }, t E 0 [T 1 ] 4t P 0 T 1 > 4t ) = 4t 1 P 0 T 1 4t )), P 0 A) 3/4. We defie K t = X X t 1, which is the umber of uit rewards heads ) if the first coi is tried a tota of t ot ecessariy cosecutive) times. We et C be the evet defied by C = max 1 t 4t Kt 1 t < t og1/θ) We ow estabish two emmas that wi be used i the seque. Lemma We have P 0 C) > 3/4. Proof We wi prove a more geera resut: we assume that coi i has bias p i uder hypothesis H, defie Kt i as the umber of uit rewards heads ) if coi i is tested for t ot ecessariy cosecutive) times, ad et K i C i = max t p i t < } t og1/θ). 1 t 4t First, ote that Kt i p i t is a P -martigae i the cotext of Theorem 1, p i = 1/ is the bias of coi i = 1 uder hypothesis H 0 ). Usig Komogorov s iequaity Coroary 7.66, i p. 44 of Ross, 1983), the probabiity of the compemet of C i ca be bouded as foows: K i P max t p i t ) t og1/θ) 1 t 4t Sice E [K i 4t 4p it ) ] = 4p i 1 p i )t, we obtai }. E [K4t i 4p it ) ] t. og1/θ) P C i ) 1 4p i1 p i ) og1/θ) > 3 4, ) where the ast iequaity foows because θ < e 4 ad 4p i 1 p i ) I this ad subsequet proofs, ad i order to avoid repeated use of trucatio symbos, we treat t as if it were iteger.. The proof for a geera p i wi be usefu ater. 67

6 MANNOR AND TSITSIKLIS Lemma 3 If 0 x 1/ ad y 0, the where d = x) y e dxy, Proof A straightforward cacuatio shows that og1 x) + dx 0 for 0 x 1/. Therefore, yog1 x) + dx) 0 for every y 0. Rearragig ad expoetiatig, eads to 1 x) y e dxy. We ow et B be the evet that I = 0, i.e., that the poicy evetuay seects coi 0. Sice the poicy is ε/,δ)-correct for δ < e 4 /4 < 1/4, we have P 0 B) > 3/4. We have aready show that P 0 A) 3/4 ad P 0 C) > 3/4. Let S be the evet that A, B, ad C occur, that is S = A B C. We the have P 0 S) > 1/4. Lemma 4 If E 0 [T 1 ] t ad c 100, the P 1 B) > δ. Proof We et W be the history of the process the sequece of cois chose at each time, ad the sequece of observed coi rewards) uti the poicy termiates. We defie the ikeihood fuctio L by ettig L w) = P W = w), for every possibe history w. Note that this fuctio ca be used to defie a radom variabe L W). We aso et K be a shorthad otatio for K T1, the tota umber of uit rewards heads ) obtaied from coi 1. Give the history up to time t 1, the coi choice at time t has the same probabiity distributio uder either hypothesis H 0 ad H 1 ; simiary, the coi reward at time t has the same probabiity distributio, uder either hypothesis, uess the chose coi was coi 1. For this reaso, the ikeihood ratio L 1 W)/L 0 W) is give by L 1 W) L 0 W) = 1 + ε)k 1 ε)t 1 K 1 )T 1 = 1 + ε) K 1 ε) K 1 ε) T 1 K = 1 4ε ) K 1 ε) T 1 K. 3) We wi ow proceed to ower boud the terms i the right-had side of Eq. 3) whe evet S occurs. If evet S has occurred, the A has occurred, ad we have K T 1 4t, so that 1 4ε ) K 1 4ε ) 4t = 1 4ε ) 4/cε ))og1/θ) e 16d/c)og1/θ) = θ 16d/c. We have used here Lemma 3, which appies because 4ε < 4/4 < 1/. Simiary, if evet S has occurred, the A C has occurred, which impies, T 1 K t og1/θ) = /ε c)og1/θ), 68

7 EXPLORATION IN MULTI-ARMED BANDITS where the equaity above made use of the defiitio of t. Therefore, 1 ε) T 1 K 1 ε) /ε c)og1/θ) e 4d/ c)og1/θ) = θ 4d/ c. Substitutig the above i Eq. 3), we obtai L 1 W) L 0 W) c) θ16d/c)+4d/. By pickig c arge eough c = 100 suffices), we obtai that L 1 W)/L 0 W) is arger tha θ = 4δ wheever the evet S occurs. More precisey, we have L 1 W) L 0 W) 1 S 4δ1 S, where 1 S is the idicator fuctio of the evet S. The, [ ] L1 W) P 1 B) P 1 S) = E 1 [1 S ] = E 0 L 0 W) 1 S E 0 [4δ1 S ] = 4δP 0 S) > δ, where we used the fact that P 0 S) > 1/4. To summarize, we have show that whe c 100, if E 0 [T 1 ] 1/cε )og1/4δ)), the P 1 B) > δ. Therefore, if we have a ε/,δ)-correct poicy, we must have E 0 [T ] > 1/cε )og1/4δ)), for every > 0. Equivaety, if we have a ε,δ)-correct poicy, we must have E 0 [T ] > /4cε ))og1/4δ)), which is of the desired form. 4. A Lower Boud o the Sampe Compexity - Geera Probabiities I Theorem 1, we worked with a particuar ufavorabe vector p the oe correspodig to hypothesis H 0 ), uder which a ot of exporatio is ecessary. This eaves ope the possibiity that for other, more favorabe choices of p, ess exporatio might suffice. I this sectio, we refie Theorem 1 by deveopig a ower boud that expicity depeds o the actua though ukow) vector p. Of course, for ay give vector p, there is a optima poicy, which seects the best coi without ay exporatio: e.g., if p 1 for a, the poicy that immediatey seects coi 1 is optima. However, such a poicy wi ot be ε,δ)-correct for a possibe vectors p. We start with a ower boud that appies whe a coi biases p i ie i the rage [0,1/]. We wi ater use a reductio techique to exted the resut to a geeric rage of biases. I the rest of the paper, we use the otatioa covetio x) + = max0,x}. Theorem 5 Fix some p 0,1/). There exists a positive costat δ 0, ad a positive costat c 1 that depeds oy o p, such that for every ε 0,1/), every δ 0,δ 0 ), every p [0,1/], ad every ε,δ)-correct poicy, we have } Mp,ε) 1) + 1 E p [T ] c 1 ε + p ) og 1 8δ, 69 Np,ε)

8 MANNOR AND TSITSIKLIS where p = max i p i, Mp,ε) = : > p ε, ad > p, ad ε + p } 1 +, 4) 1/ ad Np,ε) = : p ε, ad > p, ad ε + p } ) 1/ I particuar, δ 0 ca be take equa to e 8 /8. Remarks: a) The ower boud ivoves two sets of cois whose biases are ot too far from the best bias p. The first set Mp,ε) cotais cois that are withi ε from the best ad woud therefore be egitimate seectios. I the presece of mutipe such cois, a certai amout of exporatio is eeded to obtai the required cofidece that oe of these cois is sigificaty better tha the others. The secod set Np,ε) cotais cois whose bias is more tha ε away from p ; they come ito the ower boud because agai some exporatio is eeded i order to obtai the required cofidece that oe of these cois is sigificaty better tha the best coi i Mp,ε). b) The expressio ε + p )/1 + 1/) i Eqs. 4) ad 5) ca be repaced by ε + p )/ α) for ay positive costat α, by chagig some of the costats i the proof. c) This resut actuay provides a famiy of ower bouds, oe for every possibe choice of p. A tighter boud ca be obtaied by optimizig the choice of p, whie aso takig ito accout the depedece of the costat c 1 o p. This is ot hard the depedece of c 1 o p is described i Remark 7), but does ot provide ay ew isights. Proof Let us fix δ 0 = e 8 /8, some p 0,1/), ε 0,1/), δ 0,δ 0 ), a ε,δ)-correct poicy, ad some p [0,1/]. Without oss of geeraity, we assume that p = p 1. Let us deote the true ukow) bias of each coi by q i. We cosider the foowig hypotheses: H 0 : q i = p i, for i = 1,...,, ad for = 1,...,, H : q = p 1 + ε, q i = p i, for i. If hypothesis H is true, the poicy must seect coi. We wi boud from beow the expected umber of times the cois i the sets Np,ε) ad Mp,ε) must be tried, whe hypothesis H 0 is true. As i Sectio 3, we use E ad P to deote the expectatio ad probabiity, respectivey, uder the poicy beig cosidered ad uder hypothesis H. We defie θ = 8δ, ad ote that θ < e 8. Let t = 1 cε og 1 θ, 1 cp 1 ) og 1 θ, 630 if Mp,ε), if Np,ε),

9 EXPLORATION IN MULTI-ARMED BANDITS where c is a costat that oy depeds o p, ad whose vaue wi be chose ater. Reca that T stads for the tota umber of times that coi is tried. We defie the evet A = T 4t }. As i the proof of Theorem 1, if E 0 [T ] t, the P 0A ) 3/4. We defie Kt = X1 + + X t, which is the umber of uit rewards heads ) if the -th coi is tried a tota of t ot ecessariy cosecutive) times. We et C be the evet defied by C = max 1 t 4t Simiar to Lemma, ad sice θ = 8δ < e 8, we have 3 } Kt t < t og1/θ). P 0 C ) > 7/8. Let B be the evet I = }, i.e., that the poicy evetuay seects coi, ad et B c compemet. Sice the poicy is ε,δ)-correct with δ < δ 0 < 1/, we must have be its P 0 B c ) > 1/, Np,ε). We aso have Mp,ε) P 0 B ) 1, so that the iequaity P 0 B ) > 1/ ca hod for at most oe eemet of Mp,ε). Equivaety, the iequaity P 0 B c ) 1/ ca hod for at most oe eemet of Mp,ε). Let M 0 p,ε) = Mp,ε) ad P 0 B c ) > 1 }. It foows that M 0 p,ε) Mp,ε) 1) +. The foowig emma is a aaog of Lemma 4. Lemma 6 Suppose that M 0 p,ε) Np,ε) ad that E 0 [T ] t. If the costat c i the defiitio of t is chose arge eough possiby depedig o p), the P B c ) > δ. Proof Fix some M 0 p,ε) Np,ε). For future referece, we ote that the defiitios of Mp,ε) ad Np,ε) icude the coditio ε+ p )/1+ 1/). Recaig that p = p 1, 1/, ad usig the defiitio = p 1 0, some easy agebra eads to the coditios ε + 1 ε ) We defie the evet S by S = A B c C. Sice P 0 A ) 3/4, P 0 B c ) > 1/, ad P 0C ) > 7/8, we have P 0 S ) > 1 8, M 0p,ε) Np,ε). 3. The derivatio is idetica to Lemma except for Eq. ), where oe shoud repace the assumptio that θ < e 4 with the stricter assumptio that θ < e 8 used here. 631

10 MANNOR AND TSITSIKLIS As i the proof of Lemma 4, we defie the ikeihood fuctio L by ettig L w) = P W = w), for every possibe history w, ad use agai L W) to defie the correspodig radom variabe. Let K be a shorthad otatio for K T, the tota umber of uit rewards heads ) obtaied from coi. We have L W) L 0 W) = p 1 + ε) K 1 p 1 ε) T K p K 1 ) T K p1 = + ε ) K 1 p1 ε 1 1 = 1 + ε + ) K 1 ε + ) T K, 1 ) T K where we have used the defiitio = p 1. It foows that L W) L 0 W) = = = 1 + ε + ) K 1 ε + ) ) ε + K 1 1 ε + ) K 1 ε + ) ) ε + K 1 1 ε + ) K 1 ε + 1 ) K 1 ε + 1 ) K 1 ε + 1 ) T K ) K1 p )/ 1 ε + 1 ) T K ) p T K)/. 7) We wi ow proceed to ower boud the right-had side of Eq. 7) for histories uder which evet S occurs. If evet S has occurred, the A has occurred, ad we have K T 4t, so that for every Nε, p), we have ) ) ε + K 1 = a b ) ) ε + 4t 1 ) ) ε + 4/c )og1/θ) 1 exp exp = θ 16d/p c. ε/ ) + 1 d 4 c d 16 cp og1/θ) ) } og1/θ) I step a), we have used Lemma 3 which appies because of Eq. 6); i step b), we used the fact ε/ 1, which hods because Nε, p). } 63

11 EXPLORATION IN MULTI-ARMED BANDITS Simiary, for Mε, p), we have ) ) ε + K 1 = a b ) ) ε + 4t 1 ) ) ε + 4/cε )og1/θ) 1 exp exp = θ 16d/p c. 1 + /ε) d 4 c d 16 cp og1/θ) ) } og1/θ) I step a), we have agai used Lemma 3; i step b), we used the fact /ε 1, which hods because Mε, p). We ow boud the product of the secod ad third terms i Eq. 7). If b 1, the the mappig y 1 y) b is covex for y [0,1]. Thus, 1 y) b 1 by, which impies that 1 ε + ) 1 p )/ 1 ε + ), 1 so that the product of the secod ad third terms ca be ower bouded by 1 ε + ) K 1 ε + ) K1 p )/ 1 ε + ) K 1 ε + ) K = 1. 1 We sti eed to boud the fourth term of Eq. 7). We start with the case where Np,ε). We have 1 ε + ) p T K)/ a 1 ε b = c d e 1 ε + 1 } ) 1/p ) t og1/θ) 8) ) 1/p c )og1/θ) exp d } ε + og1/θ) c 1 ) } d exp og1/θ) c1 p ) exp 4d } og1/θ) cp = θ 4d/ c). Here, a) hods because we are assumig that the evets A ad C occurred; b) uses the defiitio of t for Np,ε); c) foows from Eq. 6) ad Lemma 3; d) foows because > ε; ad e) hods because 0 1/, which impies that 1/1 ). 9) 10) 633

12 MANNOR AND TSITSIKLIS Cosider ow the case where M 0 p,ε). Equatio 8) hods for the same reasos as whe Np,ε). The oy differece from the above cacuatio is i step b), where t shoud be repaced with 1/cε )og1/θ). The, the right-had side i Eq. 9) becomes exp d } ε + og1/θ). c ε1 ) For M 0 p,ε), we have ε, which impies that ε + )/ε, which the eads to the same expressio as i Eq. 10). The rest of the derivatio is idetica. Summarizig the above, we have show that if M 0 p,ε) Np,ε), ad evet S has occurred, the L W) L 0 W) θ4d/ c)+16d/p c). For M 0 p,ε) Np,ε), we have p <. We ca choose c arge eough so that L W)/L 0 W) θ = 8δ; the vaue of c depeds oy o the costat p. Simiar to the proof of Theorem 1, we have L W) L 0 W) 1 S 8δ1 S, where 1 S is the idicator fuctio of the evet S. It foows that [ ] P B c ) P L W) S ) = E [1 S ] = E 0 L 0 W) 1 S E 0 [8δ1 S ] = 8δP 0 S ) > δ, where the ast iequaity reies o the aready estabished fact P 0 S ) > 1/8. Sice the poicy is ε,δ)-correct, we must have P B c ) δ, for every. Lemma 6 the impies that E 0 [T ] > t for every M 0p,ε) Np,ε). We sum over a M 0 p,ε) Np,ε), use the defiitio of t, together with the fact M 0p,ε) Mp,ε) 1) +, to cocude the proof of the theorem. Remark 7 A cose examiatio of the proof reveas that the depedece of c 1 o p is captured by a requiremet of the form c 1 c p, for some absoute costat c. This suggests that there is a tradeoff i the choice of p. By choosig a arge p, the costat c 1 is made arger, but the sets M ad N become smaer, ad vice versa. The precedig resut may give the impressio that the sampe compexity is high oy whe the p i are bouded by 1/. The ext resut shows that simiar ower bouds hod with a differet costat) wheever the p i ca be assumed to be bouded away from 1. However, the ower boud becomes weaker i.e., the costat c 1 is smaer) whe the upper boud o the p i approaches 1. I fact, the depedece of a ower boud o ε caot be Θ1/ε ) whe max i p i = 1. To see this, cosider the foowig poicy π. Try each coi O1/ε)og/δ)) times. If oe of the cois aways resuted i heads, seect it. Otherwise, use some ε,δ)-correct poicy π. It ca be show that the poicy π is ε,δ)-correct for every p [0,1] ), ad that if max i p i = 1, the E p [T ] = O/ε)og/δ)). 634

13 EXPLORATION IN MULTI-ARMED BANDITS Theorem 8 Fix a iteger s, ad some p 0,1/). There exists a positive costat c 1 that depeds oy o p such that for every ε 0, s+) ), every δ 0,e 8 /8), every p [0,1 s ], ad every ε,δ)-correct poicy, we have E p [T ] c 1 sη M p,εη) 1) + ε + N p,ηε) } 1 p ) og 1 8δ, where p = max i p i, η = s+1 /s, p is the vector with compoets p i = 1 1 p i ) 1/s for i = 1,,...,), ad M ad N are as defied i Theorem 5. Proof Let us fix s, p 0,1/), ε 0, s+) ), ad δ 0,e 8 /8). Suppose that we have a ε,δ)-correct poicy π whose expected time to termiatio is E p [T ], wheever the vector of coi biases happes to be p. We wi use the poicy π to costruct a ew poicy π such that ) P p p I > max p i ηε 1 δ, p [0,1/) + ηε] ; i we wi the say that π is ηε,δ)-correct o [0,1/)+ηε] ). Fiay, we wi use the ower bouds from Theorem 5, appied to π, to obtai a ower boud o the sampe compexity of π. The ew poicy π is specified as foows. Ru the origia poicy π. Wheever π chooses to try a certai coi i oce, poicy π tries coi i for s cosecutive times. Poicy π the feeds π with 0 if a s trias resuted i 0, ad feeds π with 1 otherwise. If p is the true vector of coi biases faced by poicy π, ad if poicy π chooses to sampe coi i, the poicy π sees a outcome which equas 1 with probabiity p i = 1 1 p i ) s. Let us defie two mappigs f,g : [0,1] [0,1], which are iverses of each other, by f p i ) = 1 1 p i ) 1/s, g p i ) = 1 1 p i ) s, ad with a sight abuse of otatio, et f p) = f p 1 ),..., f p )), ad simiary for g p). With our costructio, whe poicy π is faced with a bias vector p, it evoves i a idetica maer as the poicy π faced with a bias vector p = g p). But uder poicy π, there are s trias associated with every tria uder poicy π, which impies that T = st T is the umber of trias uder poicy π) ad therefore E π p[ T ] = se π g p) [T ], E π f p) [ T ] = se π p[t ], 11) where the superscript i the expectatio operator idicates the poicy beig used. We wi ow determie the correctess guaratees of poicy π. We first eed some agebraic preimiaries. Let us fix some p [0,1/)+ηε] ad a correspodig vector p, reated by p = f p) ad p = g p). Let aso p = max i p i ad p = max i p i. Usig the defiitio η = s+1 /s ad the assumptio ε < s+), we have p 1/) + 1/s), from which it foows that p s ) s = 1 1 s 1 1 s ) s 1 1 s 1 4 = 1 s+). The derivative f of f is mootoicay icreasig o [0,1). Therefore, f p ) f 1 s+) ) = 1 s+)) 1/s) 1 1 = s s s+)1 s)/s = 1 s s+1 /s) 1 s s+1 = η. 635

14 MANNOR AND TSITSIKLIS Thus, the derivative of the iverse mappig g satisfies g p ) 1 η, which impies, usig the cocavity of g, that g p ηε) g p ) g p )εη g p ) ε. Let I be the coi idex fiay seected by poicy π whe faced with p, which is the same as the idex chose by π whe faced with p. We have the superscript i the probabiity idicates the poicy beig used) P π p p I p ηε) = P π p g p I ) g p ηε)) P π p g p I ) g p ) ε) = P π p p I p ε) 1 δ, where the ast iequaity foows because poicy π was assumed to be ε,δ)-correct. We have therefore estabished that π is ηε,δ)-correct o [0,1/) + ηε]. We ow appy Theorem 5, with ηε istead of ε. Eve though that theorem is stated for a poicy which is ε,δ)-correct for a possibe p, the proof oy requires the poicy to be ε,δ)-correct for p [0,1/) + ε]. This gives a ower boud o E π p [ T ] which, usig Eq. 11), trasates to the caimed ower boud o E π p[t ]. This ower boud appies wheever p = g p), for some p [0,1/], ad therefore wheever p [0,1 s ]. 5. The Bayesia Settig There is aother variat of the probem which is of iterest. I this variat, the parameters p i associated with each arm are ot ukow costats, but radom variabes described by a give prior. I this case, there is a sige uderyig probabiity measure which we deote by P, ad which is the average of the measures P p over the prior distributio of p. We aso use E to deote the expectatio with respect to P. We the defie a poicy to be ε,δ)-correct, for a particuar prior ad associated measure P, if We the have the foowig resut. P ) p I > max p i ε 1 δ. i Theorem 9 There exist positive costats c 1, c, ε 0, ad δ 0, such that for every ad ε 0,ε 0 ), there exists a prior for the -badit probem such that for every δ 0,δ 0 ), ad ε,δ)-correct poicy for this prior, we have E[T ] c 1 ε og c δ. I particuar, ε 0 ad δ 0 ca be take equa to 1/8 ad e 4 /1, respectivey. 636

15 EXPLORATION IN MULTI-ARMED BANDITS Proof Let ε 0 = 1/8 ad δ 0 = e 4 /1, ad et us fix ε 0,ε 0 ) ad δ 0,δ 0 ). Cosider the hypotheses H 0,...,H, itroduced i the proof of Theorem 1. Let the prior probabiity of H 0 be 1/, ad the prior probabiity of H be 1/, for = 1,...,. Fix a ε/,δ)-correct poicy with respect to this prior, ad ote that it satisfies E[T ] 1 E 0[T ] 1 =1 E 0 [T ]. 1) Sice the poicy is ε/,δ)-correct, we have Pp I > max ε/)) 1 δ. As i the proof of Theorem 5, et B be the evet that the poicy evetuay seects coi. We have 1 P 0B 0 ) + 1 P B ) 1 δ, which impies that 1 =1 =1 P B 0 ) δ. 13) Let G be the set of hypotheses 0 uder which the probabiity of seectig coi 0 is at most 3δ, i.e., G = : 1, P B 0 ) 3δ}. From Eq. 13), we obtai 1 G )3δ < δ, which impies that G > /3. Foowig the same argumet as i the proof of Lemma 4, we obtai that there exists a costat c such that if δ 0,e 4 /4) ad E 0 [T ] 1/cε )og1/4δ ), the P B 0 ) > δ. By takig δ = 3δ ad requirig that δ 0,e 4 /1), we see that the iequaity E 0 [T ] 1/cε )og1/1δ) impies that P B 0 ) > 3δ here, c is the same costat as i Lemma 4). But for every G we have P B 0 ) 3δ, ad therefore E 0 [T ] 1/cε )og1/1δ). The, Eq. 1) impies that E[T ] 1 E 0 [T ] G 1 G cε og 1 1δ c 1 ε og c δ, where we have used the fact G > /3 i the ast iequaity. To cocude, we have show that there exists costats c 1 ad c ad a prior for a probem with + 1 cois, such that ay ε/,δ)-correct poicy satisfies E[T ] c 1 /ε )ogc /δ). The resut foows by takig a arger costat c 1 to accout for havig + 1 ad ot cois, ad ε istead of ε/). 6. Regret Bouds I this sectio we cosider ower bouds o the regret of ay poicy, ad show that oe ca derive the Θogt) regret boud of Lai ad Robbis 1985) usig the techiques i this paper. The resuts of Lai ad Robbis 1985) are asymptotic as t, whereas ours dea with fiite times t. Our ower boud has simiar depedece i t as the upper bouds give by Auer et a. 00a) for some 637

16 MANNOR AND TSITSIKLIS atura sampig agorithms. As i Lai ad Robbis 1985) ad Auer et a. 00a), we aso show that whe t is arge, the regret depeds ieary o the umber of cois. Give a poicy, et S t be the tota umber of uit rewards heads ) obtaied i the first t time steps. The regret by time t is deoted by R t, ad is defied by R t = t max i p i S t. Note that the regret is a radom variabe that depeds o the resuts of the coi tosses as we as of the radomizatio carried out by the poicy. Theorem 10 There exist positive costats c 1,c,c 3,c 4, ad a costat c 5, such that for every, ad for every poicy, there exists some p [0,1] such that for a t 1, E p [R t ] mic 1 t, c + c 3 t, c 4 ogt og + c 5 )}. 14) The iequaity 14) suggests that there are essetiay two regimes for the expected regret. Whe is arge compared to t, the expected regret is iear i t. Whe t is arge compared to, the regret behaves ike ogt, but depeds ieary o. Proof We wi prove a stroger resut, by cosiderig the regret i a Bayesia settig. By provig that the expectatio with respect to the prior is ower bouded by the right-had side i Eq. 14), it wi foow that the boud aso hods for at east oe of the hypotheses. Cosider the same sceario as i Theorem 1, where we have +1 cois ad +1 hypotheses H 0,H 1,...,H. The prior assigs a probabiity of 1/ to H 0, ad a probabiity of 1/ to each of the hypotheses H 1,H,...,H. Simiar to Theorem 1, we wi use the otatio E ad P to deote expectatio ad probabiity whe the th hypothesis is true, ad E to deote expectatio with respect to the prior. Let us fix t for the rest of the proof. We defie T as the umber of times coi is tried i the first t time steps. The expected regret whe H 0 is true is E 0 [R t ] = ε =1 ad the expected regret whe H = 1,...,) is true is so that the expected Bayesia) regret is E[R t ] = 1 ε E 0 [T ], E [R t ] = ε E [T 0 ] + ε E [T i ], i 0, =1 E 0 [T ] + ε 1 =1 E [T 0 ] + ε Let D be the evet that coi 0 is tried at east t/ times, i.e., D = T 0 t/}. =1 i 0, E [T i ]. 15) We cosider separatey the two cases P 0 D) < 3/4 ad P 0 D) 3/4. Suppose first that P 0 D) < 3/4. I that case, E 0 [T 0 ] < 7t/8, so that =1 E 0[T ] t/8. Substitutig i Eq. 15), we obtai E[R t ] εt/3. This gives the first term i the right-had side of Eq. 14), with c 1 = ε/3. 638

17 EXPLORATION IN MULTI-ARMED BANDITS We assume from ow o that P 0 D) 3/4. Rearragig Eq. 15), ad omittig the third term, we have E[R t ] ε 4 E 0 [T ] + 1 ) E [T 0 ]. Sice E [T 0 ] t/)p D), we have E[R t ] ε 4 For every 0, et us defie δ by =1 =1 E 0 [T ] + t ) P D). 16) E 0 [T ] = 1 cε og 1 4δ. Such a δ exists because of the mootoicity of the mappig x og1/x).) Let δ 0 = e 4 /4. If δ < δ 0, we argue exacty as i Lemma 4, except that the evet B i that emma is repaced by evet D. Sice P 0 D) 3/4, the same proof appies ad shows that P D) δ, so that E 0 [T ] + t P D) 1 cε og 1 4δ + t δ. If o the other had, δ δ 0, the E 0 [T ] 1/cε )og1/4δ 0 ), which impies by the earier aaogy with Lemma 4) that P D) δ 0, ad E 0 [T ] + t P D) 1 cε og 1 4δ + t δ 0. Usig the above bouds i Eq. 16), we obtai E[R t ] ε 4 =1 1 cε og 1 + hδ ) t ), 17) 4δ where hδ) = δ if δ < δ 0, ad hδ) = δ 0 otherwise. We ca ow view the δ as free parameters, ad cocude that E[R t ] is ower bouded by the miimum of the right-had side of Eq. 17), over a δ. Whe optimizig, a the δ wi be set to the same vaue. The miimizig vaue ca be δ 0, i which case we have E[R t ] 4cε og 1 ε + δ 0 4δ 0 8 t. Otherwise, the miimizig vaue is δ = /ctε, i which case we have 1 E[R t ] 16cε + 1 ) 4cε ogcε /) + 1 4cε og1/) + 4cε ogt. Thus, the theorem hods with c = 1/4cε)og1/4δ 0 ), c 3 = δ 0 ε/8, c 4 = 1/4cε, ad c 5 = 1/4) + ogcε /). 639

18 MANNOR AND TSITSIKLIS 7. Permutatios We ow cosider the case where the coi biases p i are kow up to a permutatio. More specificay, we are give a vector q [0,1], ad we are tod that the true vector p of coi biases is of the form p = q σ, where σ is a ukow permutatio of the set 1,...,}, ad where q σ stads for permutig the compoets of the vector q accordig to σ, i.e., q σ) = q σ). We say that a poicy is q,ε,δ)-correct if the coi I evetuay seected satisfies ) P q σ p I > maxq ε 1 δ, for every permutatio σ of the set 1,...,}. We start with a O + og1/δ))/ε ) upper boud o the expected umber of trias, which is sigificaty smaer tha the boud obtaied whe the coi biases are competey ukow cf. Sectios 3 ad 4). We aso provide a ower boud which is withi a costat factor of our upper boud. We the cosider a differet measure of sampe compexity: istead of the expected umber of trias, we cosider the maximum over a sampe paths) umber of trias. We show that for every q,ε,δ)-correct poicy, there is a Θ/ε )og1/δ)) ower boud o the maximum umber of trias. We ote that i the media eimiatio agorithm of Eve-Dar et a. 00), the egth of a sampe paths is the same ad withi a costat factor from our ower boud. Hece our boud is agai tight. We therefore see that for the permutatio case, the sampe compexity depeds criticay o whether our criterio ivoves the expected or maximum umber of trias. This is i cotrast to the geera case cosidered i Sectio 3: the ower boud i that sectio appies uder both criteria, as does the matchig upper boud from Eve-Dar et a. 00). 7.1 A Upper Boud o the Expected Number of Trias Suppose we are give a vector q [0,1], ad we are tod that the true vector p of coi biases is a permutatio of q. The poicy i Tabe 1 takes as iput the accuracy ε, the cofidece parameter δ, ad the vector q. I fact the poicy oy eeds to kow the bias of the best coi, which we deote by q = max q. The poicy aso uses a additioa parameter δ 0,1/]. The foowig theorem estabishes the correctess of the poicy, ad provides a upper boud o the expected umber of trias. Theorem 11 For every δ 0,1/], ε 0,1), ad δ 0,1), the poicy i Tabe 1 is guarateed to termiate after a fiite umber of steps, with probabiity 1, ad is q, ε,δ)-correct. For every permutatio σ, the expected umber of trias satisfies E q σ [T ] 1 ε c 1 + c og 1 ), δ for some positive costats c 1 ad c that deped oy o δ. Proof We start with a usefu cacuatio. Suppose that at iteratio k, the media eimiatio agorithm seects a coi I k whose true bias is p Ik. The, usig the Hoeffdig iequaity, we have P ˆp k p Ik ε/3) exp ε/3) m k } δ k. 18) 640

19 EXPLORATION IN MULTI-ARMED BANDITS Iput: Accuracy ad cofidece parameters ε 0,1) ad δ 0,1); the bias of the best coi q. Parameter: δ 1/. 0. k = 1; 1. Ru the media eimiatio agorithm to fid a coi I k whose bias is withi ε/3 of q, with probabiity at east 1 δ.. Try coi I k for m k = 9/ε )og k /δ) times. Let ˆp k be the fractio of these trias that resut i heads. 3. If ˆp k q ε/3 decare that coi I k is a ε-optima coi ad termiate. 4. Set k := k + 1 ad go back to Step 1. Tabe 1: A poicy for fidig a ε-optima coi whe the bias of the best coi is kow. Let K be the umber of iteratios uti the poicy termiates. Give that K > k 1 i.e., the poicy did ot termiate i the first k 1 iteratios), there is probabiity at east 1 δ 1/ that p Ik q ε/3), i which case, from Eq. 18), there is probabiity at east 1 δ/ k ) 1/ that ˆp k q ε/3). Thus, PK > k K > k 1) 1 η, with η = 1/4. Cosequety, the probabiity that the poicy does ot termiate by the kth iteratio, PK > k), is bouded by 1 η) k. Thus, the probabiity that the poicy ever termiates is bouded above by 3/4) k for a k, ad is therefore 0. We ow boud the expected umber of trias. Let c be such that the umber of trias i oe executio of the media eimiatio agorithm is bouded by c/ε )og1/δ ). The, the umber of trias, tk), durig the kth iteratio is bouded by c/ε )og1/δ ) + m k. It foows that the expected tota umber of trias uder our poicy is bouded by k)tk) k=1pk 1 ) ε 1 η) k 1 cog1/δ ) + 9/)og k /δ) + 1 k=1 = 1 ε 1 η) k 1 cog1/δ ) + 9/)og1/δ) + 9k/)og + 1 ) k=1 1 ε c 1 + c og1/δ)), for some positive costats c 1 ad c. We fiay argue that the poicy is q,ε,δ)-correct. For the poicy to seect a coi I with bias p I q ε, it must be that at some iteratio k, a coi I k with p Ik q ε was obtaied, but ˆp k came out arger tha q ε/3. From Eq. 18), for ay fixed k, the probabiity of this occurrig is bouded by δ/ k. By the uio boud, the probabiity that p I q ε is bouded by k=1 δ/k = δ. Remark 1 The kowedge of q turs out to be sigificat: it eabes the poicy to termiate as soo as there is high cofidece that a coi has bee foud whose bias is arger tha q ε, without havig to check the other cois. A poicy of this type woud ot work for the hypotheses 641

20 MANNOR AND TSITSIKLIS cosidered i the proofs of Theorems 1 ad 5: uder those hypotheses, the vaue of q is ot a priori kow. We ote that Theorem 11 disagrees with a ower boud i a preimiary versio Maor ad Tsitsikis, 003) of this paper. It turs out that the atter ower boud is oy vaid uder a additioa restrictio o the set of poicies, which wi be the subject of Sectio A Lower Boud We ow prove that the upper boud i Theorem 11 is tight, withi a costat. Theorem 13 There exist positive costats c 1, c, ε 0, ad δ 1, such that for every ad ε 0,ε 0 ), there exists some q [0,1], such that for every δ 0,δ 1 ) ad every q,ε,δ)-correct poicy, there exists some permutatio σ such that E q σ [T ] 1 ε c 1 + c og 1 ). δ Proof Let ε 0 = 1/4 ad et δ 1 = δ 0 /5, where δ 0 is the same costat as i Theorem 5. Let us fix some ad ε 0,ε 0 ). We wi estabish the caimed ower boud for q = ε, 0.5 ε,...,0.5 ε), 19) ad for every δ 0,δ 1 ). I fact, it is sufficiet to estabish a ower boud of the form c /ε )og1/δ) ad a ower boud of the form c 1 /ε. We start with the former. Part I. Let us cosider the foowig three hypothesis testig probems. For each probem, we are iterested i a δ-correct poicy, i.e., a poicy whose probabiity of error is ess tha δ uder ay hypothesis. We wi show that a δ-correct poicy for the first probem ca be used to costruct a δ-correct poicy for the third probem, with the same sampe compexity, ad the appy Theorem 5 to obtai a ower boud. Π 1 : We have two cois ad the bias vector is either 0.5 ε, ε) or ε, 0.5 ε). We wish to determie the best coi. This is a specia case of our permutatio probem, with =. Π : We have a sige coi whose bias is either 0.5 ε or ε, ad we wish to determie the bias of the coi. 4 Π 3 : We have two cois ad the bias vector ca be 0.5, 0.5 ε), 0.5+ε, 0.5 ε), or 0.5,0.5+ε). We wish to determie the best coi. Cosider a δ-correct poicy for probem Π 1 except that the coi outcomes are ecoded as foows. Wheever coi 1 is tried, record the outcome uchaged; wheever coi is tried, record the opposite of the outcome i.e., record a 0 outcome as a 1, ad vice versa). Uder the first hypothesis i probem Π 1, every tria o matter which coi was tried) has probabiity 0.5 ε of beig equa to 1, ad uder the secod hypothesis has probabiity ε of beig equa to 1. With this ecodig, it is apparet that the iformatio provided by a tria of either coi i probem Π 1 is the same as the 4. A ower boud for this probem was provided i Lemma 5.1 from Athoy ad Bartett 1999). However, that boud is oy estabished for poicies with a a priori fixed umber of trias, whereas our poicies aow the umber of trias to be determied adaptivey, based o observed outcomes. 64

Asymptotic Growth of Functions

Asymptotic Growth of Functions CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

Space-Efficient Estimation of Statistics over Sub-Sampled Streams

Space-Efficient Estimation of Statistics over Sub-Sampled Streams Noame mauscript No. wi be iserted by the editor Space-Efficiet Estimatio of Statistics over Sub-Samped Streams Adrew McGregor A. Pava Srikata Tirthapura David Woodruff the date of receipt ad acceptace

More information

Department of Computer Science, University of Otago

Department of Computer Science, University of Otago Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

Modified Line Search Method for Global Optimization

Modified Line Search Method for Global Optimization Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

CHAPTER FIVE Network Hydraulics

CHAPTER FIVE Network Hydraulics . ETWOR YDRAULICSE CATER IVE Network ydrauics The fudameta reatioships of coservatio of mass ad eergy mathematicay describe the fow ad pressure distributio withi a pipe etwork uder steady state coditios.

More information

Properties of MLE: consistency, asymptotic normality. Fisher information.

Properties of MLE: consistency, asymptotic normality. Fisher information. Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

More information

5 Boolean Decision Trees (February 11)

5 Boolean Decision Trees (February 11) 5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

More information

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

More information

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13 EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may

More information

Irreducible polynomials with consecutive zero coefficients

Irreducible polynomials with consecutive zero coefficients Irreducible polyomials with cosecutive zero coefficiets Theodoulos Garefalakis Departmet of Mathematics, Uiversity of Crete, 71409 Heraklio, Greece Abstract Let q be a prime power. We cosider the problem

More information

MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

More information

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the

More information

Infinite Sequences and Series

Infinite Sequences and Series CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...

More information

Incremental calculation of weighted mean and variance

Incremental calculation of weighted mean and variance Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

More information

THE ABRACADABRA PROBLEM

THE ABRACADABRA PROBLEM THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected

More information

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee

More information

Chair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics

Chair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics Chair for Network Architectures ad Services Istitute of Iformatics TU Müche Prof. Carle Network Security Chapter 2 Basics 2.4 Radom Number Geeratio for Cryptographic Protocols Motivatio It is crucial to

More information

4.1 Sigma Notation and Riemann Sums

4.1 Sigma Notation and Riemann Sums 0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

More information

Section 11.3: The Integral Test

Section 11.3: The Integral Test Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

More information

Confidence Intervals for One Mean

Confidence Intervals for One Mean Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

Lesson 15 ANOVA (analysis of variance)

Lesson 15 ANOVA (analysis of variance) Outlie Variability -betwee group variability -withi group variability -total variability -F-ratio Computatio -sums of squares (betwee/withi/total -degrees of freedom (betwee/withi/total -mea square (betwee/withi

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

Entropy of bi-capacities

Entropy of bi-capacities Etropy of bi-capacities Iva Kojadiovic LINA CNRS FRE 2729 Site école polytechique de l uiv. de Nates Rue Christia Pauc 44306 Nates, Frace iva.kojadiovic@uiv-ates.fr Jea-Luc Marichal Applied Mathematics

More information

A Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length

A Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length Joural o Satisfiability, Boolea Modelig ad Computatio 1 2005) 49-60 A Faster Clause-Shorteig Algorithm for SAT with No Restrictio o Clause Legth Evgey Datsi Alexader Wolpert Departmet of Computer Sciece

More information

1. C. The formula for the confidence interval for a population mean is: x t, which was

1. C. The formula for the confidence interval for a population mean is: x t, which was s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

More information

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or p-series (the Compariso Test), but of

More information

A short note on quantile and expectile estimation in unequal probability samples

A short note on quantile and expectile estimation in unequal probability samples Cataogue o. 2-00-X ISS 492-092 Survey Methodoogy A short ote o quatie ad expectie estimatio i uequa probabiity sampes by Lida Schuze Watrup ad Göra Kauerma eease date: Jue 22, 206 How to obtai more iformatio

More information

The Stable Marriage Problem

The Stable Marriage Problem The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

More information

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009) 18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

3 Basic Definitions of Probability Theory

3 Basic Definitions of Probability Theory 3 Basic Defiitios of Probability Theory 3defprob.tex: Feb 10, 2003 Classical probability Frequecy probability axiomatic probability Historical developemet: Classical Frequecy Axiomatic The Axiomatic defiitio

More information

Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis

Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

Sequences II. Chapter 3. 3.1 Convergent Sequences

Sequences II. Chapter 3. 3.1 Convergent Sequences Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,

More information

arxiv:1012.1336v2 [cs.cc] 8 Dec 2010

arxiv:1012.1336v2 [cs.cc] 8 Dec 2010 Uary Subset-Sum is i Logspace arxiv:1012.1336v2 [cs.cc] 8 Dec 2010 1 Itroductio Daiel M. Kae December 9, 2010 I this paper we cosider the Uary Subset-Sum problem which is defied as follows: Give itegers

More information

Learning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr.

Learning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr. Algorithms ad Data Structures Algorithm efficiecy Learig outcomes Able to carry out simple asymptotic aalysisof algorithms Prof. Dr. Qi Xi 2 Time Complexity Aalysis How fast is the algorithm? Code the

More information

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,

More information

Sequences and Series

Sequences and Series CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

More information

CHAPTER 3 DIGITAL CODING OF SIGNALS

CHAPTER 3 DIGITAL CODING OF SIGNALS CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity

More information

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

More information

Theorems About Power Series

Theorems About Power Series Physics 6A Witer 20 Theorems About Power Series Cosider a power series, f(x) = a x, () where the a are real coefficiets ad x is a real variable. There exists a real o-egative umber R, called the radius

More information

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed. This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

Determining the sample size

Determining the sample size Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors

More information

Estimating Probability Distributions by Observing Betting Practices

Estimating Probability Distributions by Observing Betting Practices 5th Iteratioal Symposium o Imprecise Probability: Theories ad Applicatios, Prague, Czech Republic, 007 Estimatig Probability Distributios by Observig Bettig Practices Dr C Lych Natioal Uiversity of Irelad,

More information

Universal coding for classes of sources

Universal coding for classes of sources Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric

More information

9.8: THE POWER OF A TEST

9.8: THE POWER OF A TEST 9.8: The Power of a Test CD9-1 9.8: THE POWER OF A TEST I the iitial discussio of statistical hypothesis testig, the two types of risks that are take whe decisios are made about populatio parameters based

More information

The Power of Free Branching in a General Model of Backtracking and Dynamic Programming Algorithms

The Power of Free Branching in a General Model of Backtracking and Dynamic Programming Algorithms The Power of Free Brachig i a Geeral Model of Backtrackig ad Dyamic Programmig Algorithms SASHKA DAVIS IDA/Ceter for Computig Scieces Bowie, MD sashka.davis@gmail.com RUSSELL IMPAGLIAZZO Dept. of Computer

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

Class Meeting # 16: The Fourier Transform on R n

Class Meeting # 16: The Fourier Transform on R n MATH 18.152 COUSE NOTES - CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,

More information

Plug-in martingales for testing exchangeability on-line

Plug-in martingales for testing exchangeability on-line Plug-i martigales for testig exchageability o-lie Valetia Fedorova, Alex Gammerma, Ilia Nouretdiov, ad Vladimir Vovk Computer Learig Research Cetre Royal Holloway, Uiversity of Lodo, UK {valetia,ilia,alex,vovk}@cs.rhul.ac.uk

More information

Recursion and Recurrences

Recursion and Recurrences Chapter 5 Recursio ad Recurreces 5.1 Growth Rates of Solutios to Recurreces Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer. Cosider, for example,

More information

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

More information

Chapter 5 O A Cojecture Of Erdíos Proceedigs NCUR VIII è1994è, Vol II, pp 794í798 Jeærey F Gold Departmet of Mathematics, Departmet of Physics Uiversity of Utah Do H Tucker Departmet of Mathematics Uiversity

More information

CHAPTER 3 THE TIME VALUE OF MONEY

CHAPTER 3 THE TIME VALUE OF MONEY CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all

More information

Perfect Packing Theorems and the Average-Case Behavior of Optimal and Online Bin Packing

Perfect Packing Theorems and the Average-Case Behavior of Optimal and Online Bin Packing SIAM REVIEW Vol. 44, No. 1, pp. 95 108 c 2002 Society for Idustrial ad Applied Mathematics Perfect Packig Theorems ad the Average-Case Behavior of Optimal ad Olie Bi Packig E. G. Coffma, Jr. C. Courcoubetis

More information

Sum and Product Rules. Combinatorics. Some Subtler Examples

Sum and Product Rules. Combinatorics. Some Subtler Examples Combiatorics Sum ad Product Rules Problem: How to cout without coutig. How do you figure out how may thigs there are with a certai property without actually eumeratig all of them. Sometimes this requires

More information

THE HEIGHT OF q-binary SEARCH TREES

THE HEIGHT OF q-binary SEARCH TREES THE HEIGHT OF q-binary SEARCH TREES MICHAEL DRMOTA AND HELMUT PRODINGER Abstract. q biary search trees are obtaied from words, equipped with the geometric distributio istead of permutatios. The average

More information

Lecture 2: Karger s Min Cut Algorithm

Lecture 2: Karger s Min Cut Algorithm priceto uiv. F 3 cos 5: Advaced Algorithm Desig Lecture : Karger s Mi Cut Algorithm Lecturer: Sajeev Arora Scribe:Sajeev Today s topic is simple but gorgeous: Karger s mi cut algorithm ad its extesio.

More information

NEW HIGH PERFORMANCE COMPUTATIONAL METHODS FOR MORTGAGES AND ANNUITIES. Yuri Shestopaloff,

NEW HIGH PERFORMANCE COMPUTATIONAL METHODS FOR MORTGAGES AND ANNUITIES. Yuri Shestopaloff, NEW HIGH PERFORMNCE COMPUTTIONL METHODS FOR MORTGGES ND NNUITIES Yuri Shestopaloff, Geerally, mortgage ad auity equatios do ot have aalytical solutios for ukow iterest rate, which has to be foud usig umerical

More information

Supervised Rank Aggregation

Supervised Rank Aggregation Sessio: Search Quaity ad Precisio Supervised Rak Aggregatio Yu-Tig Liu,*, Tie-Ya Liu, Tao Qi,3*, Zhi-Mig Ma 4, ad Hag Li Microsoft Research Asia 4F, Sigma Ceter, No. 49, Zhichu Road, Haidia District, Beijig,

More information

THIN SEQUENCES AND THE GRAM MATRIX PAMELA GORKIN, JOHN E. MCCARTHY, SANDRA POTT, AND BRETT D. WICK

THIN SEQUENCES AND THE GRAM MATRIX PAMELA GORKIN, JOHN E. MCCARTHY, SANDRA POTT, AND BRETT D. WICK THIN SEQUENCES AND THE GRAM MATRIX PAMELA GORKIN, JOHN E MCCARTHY, SANDRA POTT, AND BRETT D WICK Abstract We provide a ew proof of Volberg s Theorem characterizig thi iterpolatig sequeces as those for

More information

Soving Recurrence Relations

Soving Recurrence Relations Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

More information

Engineering 323 Beautiful Homework Set 3 1 of 7 Kuszmar Problem 2.51

Engineering 323 Beautiful Homework Set 3 1 of 7 Kuszmar Problem 2.51 Egieerig 33 eautiful Homewor et 3 of 7 Kuszmar roblem.5.5 large departmet store sells sport shirts i three sizes small, medium, ad large, three patters plaid, prit, ad stripe, ad two sleeve legths log

More information

Chapter 5: Inner Product Spaces

Chapter 5: Inner Product Spaces Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples

More information

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio

More information

Statistical inference: example 1. Inferential Statistics

Statistical inference: example 1. Inferential Statistics Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

More information

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means)

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means) CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:

More information

Basic Elements of Arithmetic Sequences and Series

Basic Elements of Arithmetic Sequences and Series MA40S PRE-CALCULUS UNIT G GEOMETRIC SEQUENCES CLASS NOTES (COMPLETED NO NEED TO COPY NOTES FROM OVERHEAD) Basic Elemets of Arithmetic Sequeces ad Series Objective: To establish basic elemets of arithmetic

More information

Amendments to employer debt Regulations

Amendments to employer debt Regulations March 2008 Pesios Legal Alert Amedmets to employer debt Regulatios The Govermet has at last issued Regulatios which will amed the law as to employer debts uder s75 Pesios Act 1995. The amedig Regulatios

More information

Optimal Strategies from Random Walks

Optimal Strategies from Random Walks Optimal Strategies from Radom Walks Jacob Aberethy Divisio of Computer Sciece UC Berkeley jake@csberkeleyedu Mafred K Warmuth Departmet of Computer Sciece UC Sata Cruz mafred@cseucscedu Joel Yelli Divisio

More information

A Recursive Formula for Moments of a Binomial Distribution

A Recursive Formula for Moments of a Binomial Distribution A Recursive Formula for Momets of a Biomial Distributio Árpád Béyi beyi@mathumassedu, Uiversity of Massachusetts, Amherst, MA 01003 ad Saverio M Maago smmaago@psavymil Naval Postgraduate School, Moterey,

More information

Linear Algebra II. 4 Determinants. Notes 4 1st November Definition of determinant

Linear Algebra II. 4 Determinants. Notes 4 1st November Definition of determinant MTH6140 Liear Algebra II Notes 4 1st November 2010 4 Determiats The determiat is a fuctio defied o square matrices; its value is a scalar. It has some very importat properties: perhaps most importat is

More information

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process A example of o-queched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which

More information

x : X bar Mean (i.e. Average) of a sample

x : X bar Mean (i.e. Average) of a sample A quick referece for symbols ad formulas covered i COGS14: MEAN OF SAMPLE: x = x i x : X bar Mea (i.e. Average) of a sample x i : X sub i This stads for each idividual value you have i your sample. For

More information

Annuities Under Random Rates of Interest II By Abraham Zaks. Technion I.I.T. Haifa ISRAEL and Haifa University Haifa ISRAEL.

Annuities Under Random Rates of Interest II By Abraham Zaks. Technion I.I.T. Haifa ISRAEL and Haifa University Haifa ISRAEL. Auities Uder Radom Rates of Iterest II By Abraham Zas Techio I.I.T. Haifa ISRAEL ad Haifa Uiversity Haifa ISRAEL Departmet of Mathematics, Techio - Israel Istitute of Techology, 3000, Haifa, Israel I memory

More information

Research Article Sign Data Derivative Recovery

Research Article Sign Data Derivative Recovery Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

INFINITE SERIES KEITH CONRAD

INFINITE SERIES KEITH CONRAD INFINITE SERIES KEITH CONRAD. Itroductio The two basic cocepts of calculus, differetiatio ad itegratio, are defied i terms of limits (Newto quotiets ad Riema sums). I additio to these is a third fudametal

More information

Permutations, the Parity Theorem, and Determinants

Permutations, the Parity Theorem, and Determinants 1 Permutatios, the Parity Theorem, ad Determiats Joh A. Guber Departmet of Electrical ad Computer Egieerig Uiversity of Wiscosi Madiso Cotets 1 What is a Permutatio 1 2 Cycles 2 2.1 Traspositios 4 3 Orbits

More information

Maximum Likelihood Estimators.

Maximum Likelihood Estimators. Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

A RANDOM PERMUTATION MODEL ARISING IN CHEMISTRY

A RANDOM PERMUTATION MODEL ARISING IN CHEMISTRY J. Appl. Prob. 45, 060 070 2008 Prited i Eglad Applied Probability Trust 2008 A RANDOM PERMUTATION MODEL ARISING IN CHEMISTRY MARK BROWN, The City College of New York EROL A. PEKÖZ, Bosto Uiversity SHELDON

More information

Notes on Hypothesis Testing

Notes on Hypothesis Testing Probability & Statistics Grishpa Notes o Hypothesis Testig A radom sample X = X 1,..., X is observed, with joit pmf/pdf f θ x 1,..., x. The values x = x 1,..., x of X lie i some sample space X. The parameter

More information

Trackless online algorithms for the server problem

Trackless online algorithms for the server problem Iformatio Processig Letters 74 (2000) 73 79 Trackless olie algorithms for the server problem Wolfgag W. Bei,LawreceL.Larmore 1 Departmet of Computer Sciece, Uiversity of Nevada, Las Vegas, NV 89154, USA

More information

A Combined Continuous/Binary Genetic Algorithm for Microstrip Antenna Design

A Combined Continuous/Binary Genetic Algorithm for Microstrip Antenna Design A Combied Cotiuous/Biary Geetic Algorithm for Microstrip Atea Desig Rady L. Haupt The Pesylvaia State Uiversity Applied Research Laboratory P. O. Box 30 State College, PA 16804-0030 haupt@ieee.org Abstract:

More information