SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 6: More Examples on DP and Shortest Path Problem Instructor: Shiqian Ma February 17, 2014 Suggested Reading: Bertsekas Lecture Slides on Dynamic Programming; Section 2.3 of Chapter 2 of Bertsekas, Dynamic Programming and Optimal Control: Volume I (3rd Edition), Athena Scientific, 2005. 1 The Three-Jug Puzzle An 8-gallon jug is filled with fluid. Given two empty 5- and 3-gallon jugs, we want to derive the 8 gallons of fluid into two equal parts using the three jugs. No other measuring devices are allowed. What is the smallest number of pourings needed to achieve this result? The solution process can be systemarized by representing the problem as a shortest path problem. A node is defined to represent the amount of fluid in the 8-, 5-, and 3-gallon jugs, respectively. This means that the network starts with node (8,0,0) and terminates with the desired solution node (4,4,0). A new node is generated from the current node by pouring fluid from one jug into another. Figure 1 shows different routs that lead from start node (8,0,0) to end node (4,4,0). The arc between two successive nodes represents a single pouring, and hence can be assumed to have a length of 1 unit. The problem reduces to determining the shortest path between node (8,0,0) to node (4,4,0). The optimal solution, given by the bottom path in Figure 1, requires 7 pourings. Figure 1: Three-jug puzzle as a shortest path problem 2 0/1 Knapsack Problem We want to select items from a collection of n items to put into a backpack. The total capacity of the backpack is W. The volumes of the n items are w 1,..., w n, and their values are v 1,..., v n. Decide which items should be put into the backpack to maximize the total value. 1
A binary integer programming formulation is: max s.t. n v i x i i=1 n w i x i W i=1 x i = {0, 1}, i = 1,..., n. Note that there are 2 n possible solutions to be investigated. A DP formulation: Stage i is represented by item i The state at stage i is the volume limit assigned to stages (items) i, i + 1,..., n. Define m[i, w] recursively as m[i, w] = m[i 1, w], m[i, w] = max{m[i 1, w], m[i 1, w w i ] + v i }, if w i w. if w i > w (the new item is more than the current volume limit) Note that m[i, w] is the optimal value using items 1,..., i, when the volume limit is w. The terminal value is m[0, w] = 0 for 0 w W, which corresponds to the situation that no item can be selected. The solution we want to compute is m[n, W ]. To do this efficiently we can use a table to store previous computations. Example: Let W = 10, and i 1 2 3 4 v i 10 40 30 50 w i 5 4 6 3 Then table based on the DP recursive formula is V [i, w] 0 1 2 3 4 5 6 7 8 9 10 i = 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 10 10 10 10 10 10 2 0 0 0 0 40 40 40 40 40 50 50 3 0 0 0 0 40 40 40 40 40 50 70 4 0 0 0 50 50 50 50 90 90 90 90 Clearly, total complexity is O(nW ). A shortest path formulation. Find the longest path in Figure 2. 3 Tower of Hanoi (Resource: http : //en.wikipedia.org/wiki/t ower of Hanoi) This puzzle of Tower of Hanoi consists of three rods, and a number of disks of different sizes which can slide onto any rod. The puzzle starts with the disks in a neat stack in ascending order of size on one rod, the smallest at the top, thus making a conical shape. The objective of the puzzle is to move the entire stack to another rod, obeying the following simple rules: 2
Figure 2: Knapsack problem as a shortest path problem Only one disk can be moved at a time. Each move consists of taking the upper disk from one of the stacks and placing it on top of another stack i.e. a disk can only be moved if it is the uppermost disk on a stack. No disk may be placed on top of a smaller disk. A key to solving this puzzle is to recognize that it can be solved by breaking the problem down into a collection of smaller problems and further breaking those problems down into even smaller problems until a solution is reached. For example: label the rods A, B, C. let n be the total number of disks number the discs from 1 (smallest, topmost) to n (largest, bottommost) 3
Figure 3: The Tower of Hanoi Puzzle To move n disks from rod A to rod C: move n 1 disks from A to B. This leaves disk n alone on rod A move disk n from A to C move n 1 disks from B to C so they sit on disk n The above is a recursive algorithm, to carry out steps 1 and 3, apply the same algorithm again for n 1. Let S(n) be the minimum number of moves. The DP recursive formula is S(n) = 2S(n 1) + 1. It can be shown that S(n) = 2 n 1, an exponential number! Figure 4: The Tower of Hanoi as a shortest path problem: 1 disk 4
Figure 5: The Tower of Hanoi as a shortest path problem: 2 disks Figure 6: The Tower of Hanoi as a shortest path problem: 3 disks 5