Lecture 4: Dynamic Programming

Transcription

1 Lecture 4: Dynamic Programming (3 units) Outline Shortest paths Principle of optimality 0-1 knapsack problem Integer knapsack problem Uncapacitated lot-sizing 1 / 21

2 Shortest paths Consider a digraph G = (V, A) with nonnegative arc distance c ij. Problem: find the shortest path from s to t. s p t Figure: s-t shortest path 2 / 21

3 Naive approach: Find all path from s to t; Evaluate the length of each s t path Find the minimum length The number of s t path increases exponentially. Infeasible for large graph Observation: If the s t shortest path passes by node p, the subpaths (s, p) and (p, t) are shortest path from s to p and from p to t respectively. d(v)=length of shortest path from s to v. Then d(v) = min {d(i) + c i V iv }. (v) 3 / 21

4 For i s, the complexity of calculating d(v) is O(m). Suppose V = n and A = m. Order the nodes such that i < j for all (i, j) A. D k (j)=length of shortest path from s to j containing at most k arcs. Then D k (j) = min{d k 1 (j), min [D k 1(i) + c ij ]}. i V (j) Increasing k from 1 to n 1, we obtain the shortest path. Complexity of the algorithm: O(mn). 4 / 21

5 Example: Find the shortest path from A to J. A B C E F H I 3 4 J D 5 G 3 Figure: Shortest path from A to J 5 / 21

6 Forward dynamic programming. Let j be a node at stage k, D k (j) be the shortest distance from A to j. D k (j) = min{d k 1 (i) + c ij i stage k 1}, k = 0, 1, 2, 3, 4, f = D 4 (J). D 0 (A) = 0. D 1 (B) = 2, D 2 (C) = 4, D 2 (D) = 2. D 2 (E) = min j V (E)(D 1 (j)+c je ) = min{2+7, 4+3, 2+4} = 6. D 2 (F ) = min j V (F )(D 1 (j)+c jf ) = min{2+4, 4+2, 2+1} = 3. D 2 (G) = min j V (G)(D 1 (j) + c jg ) = min{2 + 6, 4 + 4, 2 + 5} = 7. D 3 (H) = min j V (H)(D 2 (j) + c jh ) = min{6 + 1, 3 + 6, 7 + 3} = 7. D 3 (I ) = min j V (I )(D 2 (j)+c ji ) = min{6+4, 3+3, 7+3} = 6. D 4 (J) = min j V (J)(D 3 (j) + c jj ) = min{7 + 3, 6 + 4} = / 21

7 Starting from D 4 (J), we have: D 4 (J) = D 3 (H) + C HJ = D 2 (E) + c EH + c HJ = D 1 (D) + c DE + c EH + c HJ = c AD + c DE + c EH + c HJ ; or D 4 (J) = D 3 (I ) + c IJ = D 2 (F ) + c FI + c IJ = D 1 (D) + c DF + c FI + c IJ = c AD + c DF + c FI + c IJ. We can backtrack the Shortest path: (1) J H E D A. (2) J I F D A. How does it compare with total enumeration? = 71 > = 28 7 / 21

8 Backward dynamic programming. Let D k (j) be the shortest distance from node j at stage k to the destination J. D k (j) = min{d k+1 (i) + c ji i stage k + 1}, k = 4, 3, 2, 1, 0. f = D 0 (A), D 4 (J) = 0. (Exercise: Try to solve the example by backward DP.) 8 / 21

9 Principle of optimality Richard Bellman (1952) introduced the principle of optimality and dynamic programming. Richard Bellman on the Birth of Dynamic Programming. Stuart Dreyfus. Operations Research, Vol. 50, No. 1, 48-51, 2002 Figure: Richard Bellman ( ) 9 / 21

10 Principle of optimality: Given an optimal sequence of decisions or choices, each subsequence must also be optimal. Principle of optimality applies to multi-stage decision making problem A large number of optimization problems satisfy this principle. It applies to a problem (not an algorithm) States: nodes for which values need to be calculated (j) Stages: steps which define the ordering 10 / 21

11 0-1 Knapsack Problem 0-1 Knapsack Problem f = max{c T x a T x b, x {0, 1} n }. Define 1 k n as stage, 0 λ b as state Optimal value function: k k f k (λ) = max{ c j x j a j x j λ, x {0, 1} k } f = f n (b). Recursive equation: j=1 j=1 f k (λ) = max{f k 1 (λ), c k + f k 1 (λ a k )}. Initial conditions: f 0 (λ) = 0, or f 1 (λ) = 0 if 0 λ a 1 and f 1 (λ) = max(c 1, 0) for λ a 1. How to find the optimal solution? Let p k (λ) = 0 if f k (λ) = f k 1 (λ), or 1 otherwise. Complexity: O(nb). 11 / 21

12 An example of 0-1 KP max 10x 1 + 7x x x 4 s.t. 2x 1 + x 2 + 6x 3 + 5x 4 7, x {0, 1} 4 f 1 f 2 f 3 f 4 p 1 p 2 p 3 p 4 λ = Thus f 4 (7) = 34. p 4 (7) = 1, so x 4 = 1, p 3 (7 5) = p 3 (2) = p 2 (2) = 0, so x 3 = x 2 = 0, p 1(2) = 1, so x 1 = / 21

13 Integer Knapsack Problem Consider the integer knapsack problem: f = max{ c j x j j=1 a j x j b, x Z n +}, j=1 where c j > 0, a j > 0, j = 1,..., n. Define r r g r (λ) = max{ c j x j a j x j b, x Z r +}. Then z = g n (b). Recursive equation: g r (λ) = j=1 j=1 max {c r t + g r 1 (λ ta r )}. t=0,1,..., λ/a r For r = 1,..., n and λ = 1,..., b, calculating g r (λ) needs at most O(b), so the complexity of computing g n (b) is O(nb 2 ). 13 / 21

14 Question: Can we do better? We have the following two cases for xr : (i) xr = 0, g r (λ) = g r 1 (λ); (ii) xr 1, then xr = 1 + t with t 0 integer (x1,..., x r 1, t) is optimal to the knapsack problem with r stages and λ a r resource g r (λ) = c r + g r (λ a r ). Thus Complexity: O(nb). g r (λ) = max{g r 1 (λ), c r + g r (λ a r )}. 14 / 21

15 An example of integer knapsack problem max 7x 1 + 9x 2 + 2x x 4 s.t. 3x 1 + 4x 2 + x 3 + 7x 4 10, x Z 4 +. g 1 g 2 g 3 g 4 λ = f = g 4 (10) = 23 with x = (2, 1, 0, 0) T. 15 / 21

16 Uncapacitated lot-sizing Uncapacitated lot-sizing problem is to determine a minimum cost production and inventory holding schedule for a product so as to satisfy its demand over a finite discrete-time planning horizon. 16 / 21

17 f t : setup (fixed) cost of production in period t; p t : the unit production cost in period t; h t : the unit storage cost in period t; d t : the demand in period t; x t : the production in period t; s t : the stock at the end of period t, s 0 = 0, s n = 0. Figure: Uncapacitated lot-sizing network (n = 4) 17 / 21

18 ULS can be viewed as a fixed-charge network flow problem in which one must choose which arcs (0, t) to open (which are the production periods), and then find the minimum cost flow through the network. Integer program model: min p t x t + t=1 h t s t + t=1 f t y t t=1 s.t. s t 1 + x t = d t + s t, t = 1,..., n, x t My t, t = 1,..., n, s 0 = 0, s n = 0, s t, x t 0, y t {0, 1}, t = 1,..., n. where M > 0 is a sufficiently large number. 18 / 21

19 Basic properties: There exists an optimal solution with st 1 x t = 0. There exists an optimal solution such that if xt > 0, then x t = t+k i=t d i for some k. Denote d it = t j=i d j. Since s t = t i=1 x i t i=1 d i, the objective function can be expressed as p t x t + t=1 h t s t + t=1 f t y t = t=1 c t x t + t=1 f t y t t=1 h t d 1t, where c t = p t + n i=t h i. Since n t=1 h td 1t is a constant, it can be removed from the objective. t=1 19 / 21

20 Let H(k) be the minimum cost of a solution for period 1,..., k. If t k is the last period in which production occurs, then x t = k i=t d i, and the cost is H(t 1) + f t + c t d tk. Forward recursion (Wagner-Whitin algorithm): with H(0) = 0. H(k) = min 1 t k {H(t 1) + f t + c t d tk }, Calculating H(k) for k = 1, 2,..., n, we obtain the optimal value H(n). The complexity of calculating H(n) is O(n 2 ). 20 / 21

21 Example: n = 4, d = (2, 4, 5, 1), p = (3, 3, 3, 3), h = (1, 2, 1, 1), f = (12, 20, 16, 8). Then c = (8, 7, 5, 4). H(1) = H(0) + f 1 + c 1 d 11 = = 28. H(2) = { H(0) + f1 + c min 1 d 12 = = 60 H(1) + f 2 + c 2 d 22 = = 76 = 60. H(3) = min H(0) + f 1 + c 1 d 13 = = 100 H(1) + f 2 + c 2 d 23 = = 111 H(2) + f 3 + c 3 d 33 = = 101 = 100. H(4) = H(0) + f 1 + c 1 d 14 = = 108 H(1) + f min 2 + c 2 d 24 = = 118 H(2) + f 3 + c 3 d 34 = = 106 H(3) + f 4 + c 4 d 44 = = 112 = 106. So the optimal solution is: y = (1, 0, 1, 0), x = (6, 0, 6, 0). 21 / 21