Review Horse Race Gambling and Side Information Dependent horse races and the entropy rate. Gambling. Besma Smida. ES250: Lecture 9.

Transcription

1 Gambling Besma Smida ES250: Lecture 9 Fall B. Smida (ES250) Gambling Fall / 23

2 Today s outline Review of Huffman Code and Arithmetic Coding Horse Race Gambling and Side Information Dependent horse races and the entropy rate B. Smida (ES250) Gambling Fall / 23

3 Existence of a particular optimal code Lemma: Existence of a particular optimal code For any distribution, there exists an optimal instantaneous code (with minimum expected length) that satisfies the following properties: 1 The lengths are ordered inversely with the probabilities (i.e., if p j > p k, then l j l k ). 2 The two longest codewords have the same length. 3 Two of the longest codewords differ only in the last bit and correspond to the two least likely symbols. B. Smida (ES250) Gambling Fall / 23

4 Huffman Code Huffman Code A code which satisfies the properties stated in the above lemma is called an Huffman code. It can be obtained by repeatedly merging the last two symbols, assigning to them the last codeword minus the last bit, and reordering the symbols in order to have non-increasing probabilities. Theorem: Optimality of Huffman coding Huffman coding is optimal; that is, if C is a Huffman code and C is any other uniquely decodable code, then L(C ) L(C ). B. Smida (ES250) Gambling Fall / 23

5 Huffman: Good and Bad Good Bad Shortest possible symbol code H D (X) L H H D (X) + 1 Redundancy of up to 1 bit per symbol Expensive if H(X) is small Less sf you use a block of N symbols Redundancy equals 0 iff distribution is D-adic Must recompute entire code if any symbol probability changes A block of N symbols needs X N pre-calculated probabilities B. Smida (ES250) Gambling Fall / 23

6 Arithmetic Coding Based on Shannon-Fano-Elias code. Let X = {0, 1}. Let us use the above idea. if x n = (x 1, x 2,..., x n ) X n. We need to calculate F(x n ) and l(x n ) = log 1 p(x n ) + 1 B. Smida (ES250) Gambling Fall / 23

7 Computing F(x n ) Note that if we have computed F(x n ) for every x n, it is easy to compute F(x n+1 ) for every x n+1. In fact, { F(x n+1 ) = F(x n F(x, x n+1 ) = n ) + p(x n, 0) if x n+1 = 1 F(x n ) if x n+1 = 0 Thus encoding can be done sequentially. This procedure is efficient provided that p(x n ) can be efficiently computed (iid and Markov). B. Smida (ES250) Gambling Fall / 23

8 Horse Race We assume m horses run in a race. Let the i-th horse win with probability p i. If horse i wins, the payoff is for 1. The gambler distributes all his wealth across the horses b i 0 and i b i = 1. At the end of the race, the gambler will have multiplied his wealth by a factor b i and this will happen with probability p i. Then the resulting wealth relative is S(X) = b(x)o(x), with probability p(x). Repeat gambles n times. Let S n be the gambler s wealth after n races, then S n = n S(X i ) i=1 B. Smida (ES250) Gambling Fall / 23

9 Doubling Rate Definition: doubling rate The doubling rate of a horse race is Theorem: W(b,p) = E[logS(X)] = m p k log b k o k Let the race outcomes X 1, X 2, be i.i.d. p(x). Then the wealth of the gambler using betting strategy b grows exponentially at rate W(b,p); that is, S n. = 2 nw(b,p) k=1 B. Smida (ES250) Gambling Fall / 23

10 Proof Let S n be the gambler s wealth after n races, then Hence, S n = n S(X i ). i=1 1 n logs n = 1 n n log S(X i ). i=1 Then, by the weak law of larger numbers, Thus, S n. = 2 nw(b,p). 1 n log S n E(log S(X)), B. Smida (ES250) Gambling Fall / 23

11 Optimum doubling rate Definition: The optimum doubling rate W (p) is the maximum doubling rate over all choices of the portfolio b: W (p) = maxw(b,p) = b Theorem: proportional gambling is log-optimal the optimal doubling rate is given by max b:b i 0, i bi=1 m p i log b i i=1 W (p) = p i log H(p) and is achieved by the proportional gambling scheme b = p. B. Smida (ES250) Gambling Fall / 23

12 Proof Goal: Maximize W(b,p) as a function of b subject to the constraint bi = 1. We change the base of the logarithm constraints and use the Lagrange multiplier J(b) = p i ln b i + λ b i, and J b i = pi b i + λ for i = 1, 2,...,m. If you set J b i = 0, we have b i = pi λ. Substituting this in the constraint b i = 1 yields b i = p i B. Smida (ES250) Gambling Fall / 23

13 Proof We rewrite the function W(b,p) as: W(b,p) = p i log b i = p i log b i p i p i = p i log H(p) D(p b) p i log H(p) with equality iff p = b (i.e., the gambler bets on each horse in proportion to its probability of winning). B. Smida (ES250) Gambling Fall / 23

14 Example 1 The case of 3 horses with probability to win p 1 = 1 2 and p 2 = p 3 = 1 4. We also assume uniform fair odds (3 for 1). Then the optimum bet is proportional betting (b 1 = p 1, b 2 = p 2, and b 3 = p 3 ). The optimal doubling rate is W (p) = p i log H(p) = log 3 H( 1 2, 1 4, 1 4 ) = 0.085, and the resulting wealth grows tnfinity S n. = 2 n0.085 = (1.06) n. If we put all the money on the first horse, then the probability that we do not go broke in n races is 1 2n. Since this probability goes to zero with n, we go broke with probability 1. if b(1, 0, 0), then W(b) = and S n. = 2 nw = 0. B. Smida (ES250) Gambling Fall / 23

15 Example 2 The case of fair odds with respect to some distribution (i.e. 1 = 1). Let r i = 1, then we can write the doubling rate as: W(b,p) = p i log b i = p i log b i p i p i r i = D(p r) D(p b) B. Smida (ES250) Gambling Fall / 23

16 Conservation theorem Theorem: Conservation theorem For uniform fair odds, W (p) + H(p) = log m Thus, the sum of the doubling rate and the entropy rate is a constant. Proof: We assume uniform fair odds : = 1 m. Hence the optimum doubling rate is W (p) = D(p 1 ) = log m H(p). m B. Smida (ES250) Gambling Fall / 23

17 The optimum strategy with cash option The optimum strategy depends on the odds and will not necessarily have the simple form of proportional gambling. There are three cases: 1 Fair odds with respect to some distribution: 1 = 1. By betting b i = 1, one achieves S(X) = 1. Proportional betting is optimal. 2 Superfair odds: 1 < 1. By choosing b i = c 1, where c = 1/ 1, one has S(X) = 1/ 1 > 1 with probability 1. In this case, the optimum strategy is proportional betting. 3 Subfair odds: 1 > 1. Proportional gambling is no longer log-optimal. The optimal strategy for this case can be found using water-filling interpretation with Kuhn-Tucker conditions. B. Smida (ES250) Gambling Fall / 23

18 Sketch of Proof (Relative Entropy) We want to maximize the expected log return W(b,p) = m p i log(b 0 + b i ) i=1 Let try express W(b,p) as a sum of relative entropies: W(b,p) = = = m p i log(b 0 + b i ) = i=1 m p i log i=1 ( b0 + b i p i p i 1 ) ( m b0 ) o p i log i + b i 1 i=1 m p i log p i + log K D(p r). i=1 where K = ( b0 + b i ) = b 0 ( 1 1) + 1 and r i = b 0 oi +b i K. B. Smida (ES250) Gambling Fall / 23

19 Sketch of Proof (Relative Entropy) 1 Fair odds with respect to some distribution: 1 = 1. K = 1, In this case, setting b 0 = 0 and b i = p i would imply that r i = p i and hence D(p r) = 0. Proportional betting is optimal. 2 Superfair odds: 1 < 1. Examining the expression of K, we see that K is maximized for b 0 = 0. In this case, setting b 0 = 0 and b i = p i would imply that D(p r) = 0. In this case, the optimum strategy is proportional betting. 3 Subfair odds: 1 > 1. The argument breaks down. Looking at the expression of for K, we see that it is maximized for b 0 = 1. However, we cannot simultaneously minimize D(p r). The optimal strategy for this case can be found using water-filling interpretation with Kuhn-Tucker conditions. B. Smida (ES250) Gambling Fall / 23

20 Gambling and Side Information Definition: The increase W is defined as: W = W (X Y ) W (X), where W (X) = max b(x) W (X Y ) = max b(x y) p(x) log b(x)o(x) x p(x, y)log b(x y)o(x) x,y Theorem: The increase W in doubling rate due to side information Y for a horse race X is W = I(X; Y ) B. Smida (ES250) Gambling Fall / 23

21 Proof With side information, the maximum value of W (X Y ) with side information Y is achieved by conditionally proportional gambling [i.e. b (x y) = p(x y)]. Thus W (X Y ) = max E[log S] = max b(x y) b(x y) = x,y p(x, y)log b(x y)o(x) x,y p(x, y)log p(x y)o(x) = x p(x)log o(x) H(X Y ). Without side information, the optimal doubling rate is W (X) = max p(x)log b(x)o(x) = p(x)log o(x) H(X). b(x) x x Thus, the increase in doubling rate due to the presence of side side information Y is W = W (X Y ) W (X) = I(X; Y ). B. Smida (ES250) Gambling Fall / 23

22 Dependent horse races and the entropy rate For a stochastic process of the sequence {X k }: The optimal doubling rate for uniform fair odds (m for 1) is, W (X k X k 1, X k 2,, X 1 ) = logm H(X k X k 1, X k 2,, X 1 ), which is achieved by b (x k x k 1,, x 1 ) = p(x k x k 1,, x 1 ). and the exponent in the growth rate is, 1 n E[log S n] = logm H(X 1,, X n ) n B. Smida (ES250) Gambling Fall / 23

23 Stationary process For stationary process with entropy rate H(X), the limit yields 1 lim n n E log S n + H(X) = log m Again, we have the results that the entropy plus the doubling rate is a constant. B. Smida (ES250) Gambling Fall / 23