Duality for Mixed Integer Linear Programming

Transcription

1 Duality for Mixed Integer Linear Programming TED RALPHS MENAL GÜZELSOY ISE Department Lab Lehigh University University of Newcastle, Newcastle, Australia 13 May 009 Thanks: Work supported in part by the National Science Foundation Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

2 Outline 1 Duality Theory Introduction IP Duality The Subadditive Dual Formulation Properties 3 Constructing Dual Functions The Value Function Gomory s Procedure Branch-and-Bound Method Branch-and-Cut Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February 009 / 56

3 What is Duality? It is difficult to give a general definition of mathematical duality, though mathematics is replete with various notions of it. Set Theory and Logic (De Morgan Laws) Geomety (Pascal s Theorem & Brianchon s Theorem) Combinatorics (Graph Coloring) The duality we are interested in is a sort of functional duality. We define a generic optimization problem to be a mapping f : X R, where X is the set of possible inputs and f(x) is the result. Duality may then be defined as a method of transforming a given primal problem to an associated dual problem such that the dual problem yields a bound on the primal problem, and applying a related transformation to the dual produces the primal again. In many case, we would also like to require that the dual bound be close to the primal result for a specific input of interest. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

4 Duality in Mathematical Programming In mathematical programming, the input is the problem data (e.g., the constraint matrix, right-hand side, and cost vector for a linear program). We view the primal and the dual as parametric problems, but some data is held constant. Uses of the Dual in Mathematical Programing If the dual is easier to evaluate, we can use it to obtain a bound on the primal optimal value. We can also use the dual to perform sensitivity analysis on the parameterized primal input data. Finally, we can also use the dual to warm start solution procedure based on evaluation of the dual. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

5 Duality in Integer Programming We will initially be interested in the mixed integer linear program (MILP) instance z IP = min x S cx, (P) where, c R n, S = {x Z r + R n r + Ax = b} with A Q m n, b R m. We call this instance the base primal instance. To construct a dual, we need a parameterized version of this instance. For reasons that will become clear, the most relevant parameterization is of the right-hand side. The value function (or primal function) of the base primal instance (P) is z(d) = min x S(d) cx, where for a given d R m, S(d) = {x Z r + R n r + Ax = d}. We let z(d) = if d Ω = {d R m S(d) = }. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

6 Example: Value Function z IP = min 1 x 1 + x 3 + x 4 s.t x 1 3 x + x 3 x 4 = b and x 1, x Z +, x 3, x 4 R +. z(d) d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

7 Dual Functions A dual function F : R m R is one that satisfies F(d) z(d) for all d R m. How to select such a function? We choose may choose one that is easy to construct/evaluate and/or for which F(b) z(b). This results in the base dual instance where Υ m {f f : R m R} z D = max {F(b) : F(d) z(d), d R m, F Υ m } We call F strong for this instance if F is a feasible dual function and F (b) = z(b). This dual instance always has a solution F that is strong if the value function is bounded and Υ m {f f : R m R}. Why? Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

8 The LP Relaxation Dual Function It is easy to obtain a feasible dual function for any MILP. Consider the value function of the LP relaxation of the primal problem: F LP (d) = max {vd : va c}. v Rm By linear programming duality theory, we have F LP (d) z(d) for all d R m. Of course, F LP is not necessarily strong. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

9 Example: LP Dual Function F LP (d) = min s.t which can be written explicitly as { F LP (d) = vd, 0 v 1, and v R, 0, d 0 1 d, d > 0. 3 z(d) F LP(d) d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

10 The Subadditive Dual By considering that F(d) z(d), d R m F(d) cx, x S(d), d R m F(Ax) cx, x Z n +, the generalized dual problem can be rewritten as z D = max {F(b) : F(Ax) cx, x Z r + R n r +, F Υ m }. Can we further restrict Υ m and still guarantee a strong dual solution? The class of linear functions? NO! The class of convex functions? NO! The class of sudadditive functions? YES! Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

11 The Subadditive Dual Let a function F be defined over a domain V. Then F is subadditive if F(v 1 ) + F(v ) F(v 1 + v ) v 1, v, v 1 + v V. Note that the value function z is subadditive over Ω. Why? If Υ m Γ m {F is subadditive F : R m R, F(0) = 0}, we can rewrite the dual problem above as the subadditive dual z D = max F(b) F(a j ) c j j = 1,..., r, F(a j ) c j j = r + 1,..., n, and F Γ m, where the function F is defined by F(d) = lim sup δ 0 + F(δd) δ d R m. Here, F is the upper d-directional derivative of F at zero. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

12 Example: Upper D-directional Derivative The upper d-directional derivative can be interpreted as the slope of the value function in direction d at 0. For the example, we have 3 z(d) z(d) d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

13 Weak Duality Weak Duality Theorem Let x be a feasible solution to the primal problem and let F be a feasible solution to the subadditive dual. Then, F(b) cx. Proof. Corollary For the primal problem and its subadditive dual: 1 If the primal problem (resp., the dual) is unbounded then the dual problem (resp., the primal) is infeasible. If the primal problem (resp., the dual) is infeasible, then the dual problem (resp., the primal) is infeasible or unbounded. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

14 Strong Duality Strong Duality Theorem If the primal problem (resp., the dual) has a finite optimum, then so does the subadditive dual problem (resp., the primal) and they are equal. Outline of the Proof. Show that the value function z or an extension to z is a feasible dual function. Note that z satisfies the dual constraints. Ω R m : z Γ m. Ω R m : z e Γ m with z e (d) = z(d) d Ω and z e (d) < d R m. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

15 Example: Subadditive Dual For our IP instance, the subadditive dual problem is max F(b) F(1) 1 F( 3 ) 0 F(1) F( 1) 1 F Γ 1.. and we have the following feasible dual functions: 1 F 1 (d) = d is an optimal dual function for b {0, 1,,...}. F (d) = 0 is an optimal function for b {..., 3, 3, 0}. d d d 4, d 3 3 F 3 (d) = max{ 1 b {[0, 1 4 ] [1, 5 4 ] [, 9 4 ]...}. d d d 4 } is an optimal function for 4 F 4 (d) = max{ 3 d 3 d 3 d 3 3 d, 3 4 d d 3 3 d d } is an optimal function for b {... [ 7, 3] [, 3 ] [ 1, 0]} Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

16 Example: Feasible Dual Functions 3 z(d) F(d) d Notice how different dual solutions are optimal for some right-hand sides and not for others. Only the value function is optimal for all right-hand sides. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

17 Farkas Lemma (Pure Integer) For the primal problem, exactly one of the following holds: 1 S There is an F Γ m with F(a j ) 0, j = 1,..., n, and F(b) < 0. Proof. Let c = 0 and apply strong duality theorem to subadditive dual. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

18 Complementary Slackness (Pure Integer) For a given right-hand side b, let x and F be feasible solutions to the primal and the subadditive dual problems, respectively. Then x and F are optimal solutions if and only if 1 x j (c j F (a j )) = 0, j = 1,..., n and F (b) = n j=1 F (a j )x j. Proof. For an optimal pair we have F (b) = F (Ax ) = n F (a j )xj = cx. j=1 Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

19 Example: Pure Integer Case If we require integrality of all variables in our previous example, then the value function becomes z(d) d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

20 Constructing Dual Functions Explicit construction The Value Function Generating Functions Relaxations Lagrangian Relaxation Quadratic Lagrangian Relaxation Corrected Linear Dual Functions Primal Solution Algorithms Cutting Plane Method Branch-and-Bound Method Branch-and-Cut Method Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

21 Properties of the Value Function It is subadditive over Ω. It is piecewise polyhedral. For an ILP, it can be obtained by a finite number of limited operations on elements of the RHS: (i) rational multiplication (ii) nonnegative combination (iii) rounding (iv) taking the minimum Chvátal fcns. Gomory fcns. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

22 The Value Function for MILPs There is a one-to-one correspondence between ILP instances and Gomory functions. The Jeroslow Formula shows that the value function of a MILP can also be computed by taking the minimum of different values of a single Gomory function with a correction term to account for the continuous variables. The value function of the earlier example is { 3 max d 3, z(d) = min { 3 max d 3, d d } + 3 d + d, } d, Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February 009 / 56

23 Jeroslow Formula Let the set E consist of the index sets of dual feasible bases of the linear program min{ 1 M c Cx C : 1 M A Cx C = b, x 0} where M Z + such that for any E E, MA 1 E aj Z m for all j I. Theorem (Jeroslow Formula) There is a g G m such that z(d) = min E E g( d E ) + v E(d d E ) d R m with S(d), where for E E, d E = A E A 1 E d and v E is the corresponding basic feasible solution. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

24 The Single Constraint Case Let us now consider an MILP with a single constraint for the purposes of illustration: min cx, x S (P1) c R n, S = {x Z r + R n r + a x = b} with a Q n, b R. The value function of (P) is z(d) = min x S(d) cx, where for a given d R, S(d) = {x Z r + R n r + a x = d}. Assumptions: Let I = {1,...,r}, C = {r + 1,...,n}, N = I C. z(0) = 0 = z : R R {+ }, N + = {i N a i > 0} and N = {i N a i < 0}, r < n, that is, C 1 = z : R R. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

25 Example min 3x x + 3x 3 + 6x 4 + 7x 5 + 5x 6 s.t 6x 1 + 5x 4x 3 + x 4 7x 5 + x 6 = b and x 1, x, x 3 Z +, x 4, x 5, x 6 R +. (SP) z d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

26 Example (cont d) η = 1, ζ = 3 4, ηc = 3 and ζ C = 1: 18 z F U F L ζ C η C ζ η d {η = η C } {z(d) = F U (d) = F L (d) d R + } {ζ = ζ C } {z(d) = F U (d) = F L (d) d R } Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

27 Observations Consider d + U, d U, d+ L, d L : d L d L d L d U d + d + d + 3d + 4d + U L L L L d The relation between F U and the linear segments of z: {η C,ζ C } Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

28 Redundant Variables Let T C be such that t + T if and only if η C < and η C = c t + a t + t T if and only if ζ C > and ζ C = c t a t. and define and similarly, ν(d) = min s.t. c I x I + c T x T a I x I + a T x T = d x I Z I +, x T R T + Then ν(d) = z(d) for all d R. The variables in C\T are redundant. z can be represented with at most continuous variables. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

29 Example min x 1 3/4x + 3/4x 3 s.t 5/4x 1 x + 1/x 3 = b, x 1, x Z +, x 3 R +. η c = 3/, ζ C = For each discontinuous point d i, we have d i (5/4y i 1 y i ) = 0 and each linear segment has the slope of η C = 3/. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

30 Jeroslow Formula Let M Z + be such that for any t T, Maj a t Z for all j I. Then there is a Gomory function g such that z(d) = min t T {g( d t ) + c t a t (d d t )}, d t = a t M Md a t, d R Such a Gomory function can be obtained from the value function of a related PILP. For t T, setting ω t (d) = g( d t ) + c t a t (d d t ) d R, we can write z(d) = min t T ω t(d) d R Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

31 Piecewise Linearity and Continuity For t T, ω t is piecewise linear with finitely many linear segments on any closed interval and each of those linear segments has a slope of η C if t = t + or ζ C if t = t. ω t + is continuous from the right, ω t is continuous from the left. ω t + and ω t are both lower-semicontinuous. Theorem z is piecewise-linear with finitely many linear segments on any closed interval and each of those linear segments has a slope of η C or ζ C. (Meyer 1975) z is lower-semicontinuous. η C < if and only if z is continuous from the right. ζ C > if and only if z is continuous from the left. Both η C and ζ C are finite if and only if z is continuous everywhere. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

32 Maximal Subadditive Extension Let f : [0, h] R, h > 0 be subadditive and f(0) = 0. The maximal subadditive extension of f from [0, h] to R + is f(d) if d [0, h] f S (d) = inf f(ρ) if d > h, C C(d) ρ C C(d) is the set of all finite collections {ρ 1,..., ρ R} such that ρ i [0, h], i = 1,..., R and P R i=1 ρi = d. Each collection {ρ 1,..., ρ R} is called an h-partition of d. We can also extend a subadditive function f : [h, 0] R, h < 0 to R similarly. (Bruckner 1960) f S is subadditive and if g is any other subadditive extension of f from [0, h] to R +, then g f S (maximality). Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

33 Extending the Value Function Lemma Suppose we use z itself as the seed function. Observe that we can change the inf to min : Let the function f : [0, h] R be defined by f(d) = z(d) d [0, h]. Then, z(d) if d [0, h] f S (d) = min z(ρ) if d > h. C C(d) ρ C For any h > 0, z(d) f S (d) d R +. Observe that for d R +, f S (d) z(d) while h. Is there an h < such that f S (d) = z(d) d R +? Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

34 Extending the Value Function (cont.) Yes! For large enough h, maximal extension produces the value function itself. Theorem Let d r = max{a i i N} and d l = min{a i i N} and let the functions f r and f l be the maximal subadditive extensions of z from the intervals [0, d r ] and [d l, 0] to R + and R, respectively. Let { fr (d) d R F(d) = + f l (d) d R then, z = F. Outline of the Proof. z F : By construction. z F : Using MILP duality, F is dual feasible. In other words, the value function is completely encoded by the breakpoints in [d l, d r ] and slopes. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

35 General Procedure We will construct the value function in two steps Construct the value function on [d l, d r]. Extend the value function to the entire real line from [d l, d r]. Assumptions We assume η C < and ζ C <. We construct the value function over R + only. These assmuptions are only needed to simplify the presentation. Constructing the Value Function on [0, d r ] If both η C and ζ C are finite, the value function is continuous and the slopes of the linear segments alternate between η C and ζ C. For d 1, d [0, d r], if z(d 1) and z(d ) are connected by a line with slope η C or ζ C, then z is linear over [d 1, d ] with the respective slope (subadditivity). With these observations, we can formulate a finite algorithm to evaluate z in [d l, d r]. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

36 Example (cont d) d r = 6: η C ζ C Figure: Evaluating z in [0, 6] Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

37 Example (cont d) d r = 6: η C ζ C Figure: Evaluating z in [0, 6] Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

38 Example (cont d) d r = 6: η C ζ C Figure: Evaluating z in [0, 6] Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

39 Example (cont d) d r = 6: η C ζ C Figure: Evaluating z in [0, 6] Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

40 Example (cont d) d r = 6: η C ζ C Figure: Evaluating z in [0, 6] Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

41 Example (cont d) Extending the value function of (SP) from [0, 1] to [0, 18] z F U F L Υ d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

42 Example (cont d) Extending the value function of (SP) from [0, 1] to [0, 18] z F U F L Υ d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

43 Example (cont d) Extending the value function of (SP) from [0, 18] to [0, 4] z F U F L Υ d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

44 Example (cont d) Extending the value function of (SP) from [0, 18] to [0, 4] z F U F L Υ d Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

45 General Case For E E, setting we can write ω E (d) = g( d E ) + v E (d d E ) d R m with S(d), z(d) = min E E ω E(d) d R m with S(d). Many of our previous results can be extended to general case in the obvious way. Similarly, we can use maximal subadditive extensions to construct the value function. However, an obvious combinatorial explosion occurs. Therefore, we consider using single row relaxations to get a subadditive approximation. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

46 Gomory s Procedure There is a Chvátal function that is optimal to the subadditive dual of an ILP with RHS b Ω IP and z IP (b) >. The procedure: In iteration k, we solve the following LP z IP (b) k 1 = min s.t. cx Ax = b n j=1 f i (a j )x j f i (b) i = 1,..., k 1 x 0 The k th cut, k > 1, is dependent on the RHS and written as: m k 1 f k (d) = λ k 1 i d i + λ k 1 m+i f i (d) where λ k 1 = (λ k 1 1,...,λ k 1 m+k 1 ) 0 i=1 i=1 Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

47 Gomory s Procedure (cont.) Assume that b Ω IP, z IP (b) > and the algorithm terminates after k + 1 iterations. If u k is the optimal dual solution to the LP in the final iteration, then F k (d) = m k u k i d i + u k m+if i (d), i=1 i=1 is a Chvátal function with F k (b) = z IP (b) and furthermore, it is optimal to the subadditive ILP dual problem. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

48 Gomory s Procedure (cont.) Example: Consider an integer program with two constraints. Suppose that the following is the first constraint: x 1 + x + x 3 x 4 = 3 At the first iteration, Gomory s procedure can be used to derive the valid inequality / x 1 + / x + 1/ x 3 + 1/ x 4 3/ by scaling the constraint with λ = 1/. In other words, we obtain x 1 x + x 3. Then the corresponding function is: F 1 (d) = d 1 / + 0d = d 1 / By continuing Gomory s procedure, one can then build up an optimal dual function in this way. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

49 Branch-and-Bound Method Then, Assume that the primal problem is solved to optimality. Let T be the set of leaf nodes of the search tree. Thus, we ve solved the LP relaxation of the following problem at node t T z t (b) = min cx s.t x S t (b), where S t (b) = {Ax = b, x l t, x u t, x Z n } and u t, l t Z r are the branching bounds applied to the integer variables. Let (v t, v t, v t ) be the dual feasible solution used to prune node t, if t is feasibly pruned a dual feasible solution (that can be obtained from it parent) to node t, if t is infeasibly pruned F BB (d) = min t T {vt d + v t l t v t u t } is an optimal solution to the generalized dual problem. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

50 Branch-and-Cut Method It is thus easy to get a strong dual function from branch-and-bound. Note, however, that it s not subadditive in general. To obtain a subaditive function, we can include the variable bounds explicitly as constraints, but then the function may not be strong. For branch-and-cut, we have to take care of the cuts. Case 1: Do we know the subadditive representation of each cut? Case : Do we know the RHS dependency of each cut? Case 3: Otherwise, we can use some proximity results or the variable bounds. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

51 Case 1 If we know the subadditive representation of each cut: At a node t, we solve the LP relaxation of the following problem z t (b) = min cx s.t Ax b x l t x g t H t x h t x Z r + R n r + where g t, l t R r are the branching bounds applied to the integer variables and H t x h t is the set of added cuts in the form Fk(a t k j )x j + F k(a t k j )x j Fk(σ t k (b)) k = 1,...,ν(t), j I j N\I ν(t): the number of cuts generated so far, a k j, j = 1,..., n: the columns of the problem that the kth cut is constructed from, σ k (b): is the mapping of b to the RHS of the corresponding problem. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

52 Case 1 Let T be the set of leaf nodes, u t, u t, u t and w t be the dual feasible solution used to prune t T. Then, ν(t) F(d) = min t T {ut d + u t l t u t g t + w t kfk(σ t k (d))} is an optimal dual function, that is, z(b) = F(b). Again, we obtain a subaddite function if the variables are bounded. However, we may not know the subadditive representation of each cut. k=1 Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

53 Case If we know the RHS dependencies of each cut: We know for Gomory fractional cuts. Knapsack cuts Mixed-integer Gomory cuts? Then, we do the same analysis as before. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

54 Case 3 In the absence of subadditive representations or RHS dependencies: For each node t T, let ĥt be such that n { l ĥ t k = h t t kjˆx j with ˆx j = j if h t kj 0, k = 1,...,ν(t) otherwise j=1 where h t kj is the kth entry of column h t j. Furthermore, define n { l h t = h t t j x j with x j = j if w t h t j 0 otherwise Then the function j=1 F(d) = min t T {ut d + u t l t u t g t + max{w t ht, w t ĥ t }} g t j g t j. is a feasible dual solution and hence F(b) yields a lower bound. This approach is the easiest way and can be used for bounded MILPs (binary programs), however it is unlikely to get an effective dual feasible solution for general MILPs. Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56

55 Conclusions It is possible to generalize the duality concepts that are familiar to us from linear programming. However, it is extremely difficult to make any of it practical. There are some isolated cases where this theory has been applied in practice, but they are far and few between. We have a number of projects aimed in this direction, so stay tuned... Ralphs and Güzelsoy (Lehigh University) Duality for MILP 18 February / 56