Integer Programming: Large Scale Methods Part I (Chapter 16)

Transcription

1 Integer Programming: Large Scale Methods Part I (Chapter 16) University of Chicago Booth School of Business Kipp Martin November 9, / 55

2 Outline List of Files Key Concepts Column Generation/Decomposition 2 / 55

3 List of Files tsp gms.txt tsp7.txt tsp8.txt tspsolution7.txt 3 / 55

4 A Review of Key Concepts the quality of the formulation is what counts, not the size quality usually measured by the lower bound if we can read the problem into memory we can solve the linear relaxation (of course it may be a poor relaxation) many interesting applications have too many constraints or variables to read into memory if there are too many variables we generate variables as needed we generate variables as needed by a pricing algorithm if there are too many constraints we generate constraints as needed we generate constraints as needed by a separation algorithm 4 / 55

5 Motivation: The mixed integer linear program under consideration is: min c x (MIP) s.t. Ax b Bx d x 0 x j Z, j I One solution approach is to solve the LP relaxation within branch and bound min c x (LP) s.t. Ax b Bx d x 0 5 / 55

6 Motivation: Solving the LP relaxation may not be a good idea? Why? What is one of our major themes? Γ = {x Bx d, x 0, x j Z, j I } 6 / 55

7 Motivation: As an alternative, we try to define A and B appropriately and solve: min c x (conv(mip)) s.t. Ax b x conv(γ) How to break the constraints into A and B takes a fair amount of skill and practice. We want the characterization of conv(γ) to be an easy hard problem. If it is too easy there may not be a big win in terms of better lower bounds. 7 / 55

8 Motivation: x x 1 8 / 55

9 Motivation: There are three approaches to actually solving min c x (conv(mip)) s.t. Ax b x conv(γ) Approach 1: Characterize conv(γ) by the inequalities that define the polyhedron. This is the cutting plane approach. Approach 2: Characterize conv(γ) through inverse projection. Often called decomposition or column generation. Approach 3: Lagrangian Relaxation 9 / 55

10 Motivation: Mixed Integer Finite Basis Theorem: If Γ = {x R n Bx d, x 0, x j Z, j I }, then conv(γ) is a polyhedron and there exist x 1,..., x r Γ such that conv(γ) = {x R n x = q z i x i + i=1 r i=q+1 z i x i, q z i = 1, z i 0, i = 1,..., r} (1) i=1 10 / 55

11 Motivation: Apply finite basis theorem to Γ min c x (conv(mip)) s.t. Ax b x conv(γ) min c x (conv(mipfb)) s.t. Ax b q z i x i + r z i x i = x i=1 i=q+1 q z i = 1 i=1 z i 0, i = 1,..., r 11 / 55

12 Motivation: Substitute out the x variables and solve the convex hull relaxation: min q r z i c x i + z i c x i i=1 i=q+1 q z i Ax i + i=1 r i=q+1 z i Ax i b q z i = 1 i=1 z i 0 i = 1,..., q 12 / 55

13 Motivation: The formulation in the z variables is often called an extended variable formulation. Using the finite basis theorem is just one way to get an extended variable formulation. Think back to our lot sizing example we gave an extended variable formulation without using the finite basis theorem. Finding good extended variable formulations is an important research area. 13 / 55

14 Motivation: Important: If life were fair the solution to the convex hull relaxation will binary (0/1) z i. Why do I want the z i binary? Unfortunately, what do we know about life? So what do we do if we have fractional z i? Branch of course! Next slide please. 14 / 55

15 Motivation: Oh no, no, no: It is time for me to do what I do best and what I do best is worry! By this point you know me pretty well. What am I worried about? I think this has homework potential. 15 / 55

16 Motivation: Yet Another Problem: Too many z i. So here are the dragons to sleigh: Too many z i. Fractional z i. My worry from the previous page. 16 / 55

17 Consider a graph G = (V, E) with vertex set V and undirected edges E, with a weight (distance or time or cost) c e for each e V. The objective is find a tour of minimum weight where a tour starts with vertex 1, visits every other vertex exactly one, and returns to vertex 1. If i V, let δ(i) denote the set of edges adjacent to i. The TSP problem is min c e x e (2) e E e δ(i) e E(S) x e = 2, i V (3) x e S 1, S V \{1} (4) x e = n (5) e E x e {0, 1}, e E (6) 17 / 55

18 Here is a tour / 55

19 Another way to write the symmetric TSP is: min e E c e x e x Γ 1 x Γ 2 where Γ 1 is the set of all edge sets that satisfy e δ(i) x e = 2, i V Thus Γ 1 is the set of all edge sets where the degree of every vertex in the resulting graphs is exactly two. We call these constraints the degree two constraints. 19 / 55

20 Here is an example of an edge set that satisfies the degree two constraints / 55

21 Γ 2 is the set of all edge sets that satisfy x e = 2 e δ(1) x e S 1, S V \{1} e E(S) x e = n e E A set of edges that satisfy these constraints is called a 1-tree. 21 / 55

22 A 1-tree is a graph where: Vertex one has exactly two edges incident to it There is a tree defined on vertices {2,..., n} there is path from every vertex to every other vertex in the set {2,..., n} there are exactly n 2 edges that lie in vertex set {2,..., n} 22 / 55

23 Here is an example of a 1-tree / 55

24 Here is an example of a 1-tree / 55

25 A tour is therefore an edge set that satisfies the degree two constraints and is a 1-tree. So our problem is: min e E c e x e x Γ 1 x Γ 2 25 / 55

26 Alternative Formulation: We now give another formulation for this problem where we define a variable z j for every 1-tree in Γ 2. The index j is used to index the 1-trees. Corresponding to the j th index, there is a set of edges E j that correspond to the 1-tree. The cost, f j, associated with the j th 1-tree is: f j = e E j c e The number of edges incident to node i in the j th tree is d ij = 1 e (E j δ(i)) 26 / 55

27 Example Problem: We illustrate with a five-city problem (taken from the Wolsey book) / 55

28 Here is a one-tree that we index with j = 1: / 55

29 When j = 1: E 1 = {(1, 2), (2, 3), (3, 4), (4, 5), (1, 5)} f j = e E 1 c e = = 28 d 11 = 2, d 21 = 2, d 31 = 2, d 41 = 2, d 51 = 2 29 / 55

30 Here is a one-tree we index with j = 2: / 55

31 When j = 2: E 2 = {(1, 2), (1, 3), (2, 3), (3, 4), (4, 5)} f j = e E 1 c e = = 25 d 12 = 2, d 22 = 2, d 32 = 3, d 42 = 2, d 52 = 1 31 / 55

32 Alternative Formulation (continued): s.t. min J f j z j (7) j=1 J d ij z j = 2, i V (8) j=1 J z j = 1 (9) j=1 z j {0, 1}, j = 1,..., J (10) Constraints (9) and (10) taken together force the selection of exactly one 1-tree. Constraint (8) forces the degree of every vertex in the selected 1-tree to be 2 (i.e. we have a tour) and the objective forces the selection of the minimum cost 1-tree of degree / 55

33 Apply finite basis theorem to Γ min c x (conv(mip)) s.t. Ax b x conv(γ) min c x (conv(mipfb)) s.t. Ax b q z i x i + r z i x i = x i=1 i=q+1 q z i = 1 i=1 z i 0, i = 1,..., r 33 / 55

34 In the TSP model what corresponds to: 1. the Ax b constraints 2. the x conv(γ) constraints 34 / 55

35 Substitute out the x variables and solve the convex hull relaxation: min q r z i c x i + z i c x i i=1 i=q+1 q z i Ax i + i=1 r i=q+1 z i Ax i b q z i = 1 i=1 z i 0 i = 1,..., q 35 / 55

36 Alternative Formulation (continued): How large is this new formulation? How many rows? How many columns? 36 / 55

37 Formulation Comparison: Original Model Reformulated Model Variables O(n 2 ) Argh! Constraints Argh! n + 1 Once again we have fourth and long and time to bring in the punter. 37 / 55

38 Here is how we punt: Step 1: Solve (naturally) a relaxation drop the binary constraints in (10) and replace with nonnegative. This is our master problem. Step 2: We simply have too many columns to deal with. We solve the problem over a small (very small) subset of columns. This is the restricted master. 38 / 55

39 Example Restricted Master with seven 1-tree solutions: min 28z z z z z z z 7 s.t. 2z 1 + 2z 2 + 2z 3 + 2z 4 + 2z 5 + 2z 6 + 2z 7 = 2 2z 1 + 2z 2 + 2z 3 + 1z 4 + 1z 5 + 2z 6 + 3z 7 = 2 2z 1 + 3z 2 + 2z 3 + 3z 4 + 2z 5 + 3z 6 + z 7 = 2 2z 1 + 2z 2 + 3z 3 + 3z 4 + 3z 5 + z 6 + z 7 = 2 2z 1 + z 2 + z 3 + z 4 + 2z 5 + 2z 6 + 3z 7 = 2 z 1 + z 2 + z 3 + z 4 + z 5 + z 6 + z 7 = 1 z i 0 z = 22.25, z = (0, 0,.25, 0,.25,.25,.25) u = (14.875, 0, 4.5, 0.25, 1), π = 0 39 / 55

40 The transformation matrix for the seven 1-trees: z 1 z 2 z 3 z 4 z 5 z 6 z 7 x x x x x x x 25 1 x x x / 55

41 Pricing: At each step of the process, we solve the LP relaxation of the restricted master. Here is the key problem How do we calculate the reduced costs of the columns NOT IN THE MASTER? Stated another way we want to find the 1-tree with minimum reduced cost not yet in the master. We do the same thing we did for cuts solve a subproblem that finds the 1-tree with minimum reduced cost and we add that to the formulation to get a new restricted master. 41 / 55

42 Pricing Continued: Let u be a vector of dual values associated with the degree two constraints, constraint set (8). There is one dual variable for each vertex (constraint (8)) in the graph. If E j is the set of vertices in a 1-tree the reduced cost is n f j u l d lj π l=1 Rewriting based on the definition of f j and d ij gives c e u 1 1 u 2 1 u n 1 π e E j e δ(1) e δ(2) e δ(n) 42 / 55

43 Pricing Continued: Each edge e E j is incident to exactly two vertices e i and e k. Therefore, the reduced cost for variable z j is e E j (c e u ei u ek ) π To illustrate let s calculate the reduced cost for z 2. In the picture on the next slide we show the corresponding 1-tree with the corresponding values of c e u ei u ek. 43 / 55

44 Pricing Continued: 1-tree for z For example, = c 12 u 1 u 2 = / 55

45 Pricing Continued: 1-tree for z 2 with updated costs E 2 = {(1, 2), (1, 3), (2, 3), (3, 4), (4, 5)} e E 2 (c e u ei u ek ) π = = / 55

46 Pricing Continued: We just calculated the reduced cost for a column in the restricted master. But what about all of the columns NOT in the restricted master. We solve the following subproblem. Subproblem: min e E(c e u ei u ek )x e π x Γ 2 46 / 55

47 Pricing Continued: Using the updated costs, c e u ei u ek, the minimum 1-tree is found as follows: Step 1: Find the two edges adjacent to vertex 1 with minimum cost. Add those to E j. Step 2: Find the minimum cost tree on V \{1}. Apply the greedy algorithm. Sort the edges in ascending order by cost Loop over the sorted edges and add an edge if it does not create a cycle, keep adding edges until a tree results. 47 / 55

48 Pricing Continued: The current dual solution on the degree two constraints is u = (14.875, 0, 4.5, 0.25, 1) which gives the following values for c e u ei u ek / 55

49 Pricing Continued: The numbers we are using come from tsp.gams and including the file tsp7.txt. We have the following display statements: DISPLAY z.l # optimal objective value DISPLAY lambda.l; # optimal primal variables DISPLAY degree2.m; # degree 2 dual values DISPLAY convexity.m; # convexity dual value DISPLAY edgecost; # updated edge costs I put in code to calculate the updated edge costs. 49 / 55

50 Pricing Continued: Find the minimum cost 1-tree based on the updated costs. Step 1: The two edges adjacent to vertex 1 with minimum cost are edges (1, 4) and (1, 5). Add these to the 1-tree. Step 2: Form the tree. Add edge (3, 5) first, it is the cheapest. Next add edge (2, 4). The next cheapest is edge (2, 5). This does not result in cycle. We are now done. The minimum reduced cost is (in this solution π = 0) = / 55

51 The One Tree Generated by the Subproblem: / 55

52 We found a new 1-tree with reduced cost Give this new 1-tree index 8 the corresponding variable is z 8. Add this to the restricted master. See the file tsp8.txt for the GAMS parameters. The new restricted master (solved with GAMS using tsp8.txt) appears next. 52 / 55

53 The Restricted Master After One Iteration: min 28z z z z z z z z 8 s.t. 2z 1 + 2z 2 + 2z 3 + 2z 4 + 2z 5 + 2z 6 + 2z 7 + 2z 8 = 2 2z 1 + 2z 2 + 2z 3 + 1z 4 + 1z 5 + 2z 6 + 3z 7 + 2z 8 = 2 2z 1 + 3z 2 + 2z 3 + 3z 4 + 2z 5 + 3z 6 + z 7 + z 8 = 2 2z 1 + 2z 2 + 3z 3 + 3z 4 + 3z 5 + z 6 + z 7 + 2z 8 = 2 2z 1 + z 2 + z 3 + z 4 + 2z 5 + 2z 6 + 3z 7 + 3z 3 = 2 z 1 + z 2 + z 3 + z 4 + z 5 + z 6 + z 7 + z 8 = 1 z i 0 z = 20.33, z = (0, 0, 1 3, 0, 0, 1 3, 0, 1 3 ) u = (10.833,.333, 1.667, 0.667, 0), π = 0 53 / 55

54 The transformation matrix: z 1 z 2 z 3 z 4 z 5 z 6 z 7 z 8 x x x x x x x x x x / 55

55 What is next? Keep adding columns until there are no columns (1-trees) with a negative reduced left to add. Uh oh, I am worried: What if there are fractional solutions? Life is not fair, but we already know this. If there are fractional z variables do branch, cut, AND price. Uh oh, I am STILL worried: Even if we do find the optimal solution in the z i variables in branch and bound, will we have an optimal solution to the original problem? 55 / 55