Week 10 1 Integer Linear Programg So far we considered LP s with continuous variables. However sometimes decisions are intrinsically restricted to integer numbers. E.g., in the long term planning, decision variables that model acquisition of a number of machines for a future production process, or ordering of airplanes, cars etc., are bound to be integer. Therefore an important class of optimization problems is so-called mixed integer linear programg problems (MILP): c T x + d T y Ax + Dy = b x, y 0 x integer. Without continuous variables y it is called Integer LP (ILP). Ignoring the integrality restriction gives the so-called LP-relaxation. Solving the LP-relaxation and rounding is usually not too bad in case the optimal solution has high numbers. But in case the x-values of the optimal LP-solution are between 0 and 1 then rounding may make a significant difference and indeed lead to unacceptable infeasibilities. This is all the more important since integer variables that are restricted to have value either 0 or 1, so-called binary variables, enhance the modelling power of ILP enormously. 1.1 Modelling with binary variables Binary variables are used to represent a choice: take an action or not, process a job on a particular machine or not, select an item in your knapsack or not. Facility location m clients are to be served from facilities to be established on any of n candidate locations c j is the cost of opening a facility at location j, j = 1,..., n d ij is the cost of serving client i from facility j facilities have unlimited capacity w.r.t. serving clients select locations for facilities and assign each client to a facility opened such as to imize total cost We introduce binary variables y j, which gets value 1 if a facility is opened on location j and value 0 otherwise, j = 1,..., n; 1
x ij, which gets value 1 if client i is served from a facility at location j, and 0 otherwise, i = 1,..., m, j = 1,..., n The ILP-formulation n j=1 c jy j + m n i=1 j=1 d ijx ij n j=1 x ij = 1, x ij y j, x ij, y j {0, 1}. i, i, j, (1) The second set of constraints prohibit service from a facility that is not open. We can apply a trick with a large enough number M to get to a more compact ILP-formulation of the facility location problem: n j=1 c jy j + m n i=1 j=1 d ijx ij n j=1 x ij = 1, m i=1 x ij My j, x ij y j {0, 1}. Again, the second constraints prohibit service from a facility that is not open, and if the facility is open then M is chosen so big that there is no restriction on the number of clients to be serve from it. Notice that M = m suffices here. We could be tempted to conclude that the more compact formulation is the better of the two. However, we will exhibit a criterion for the strength of an ILP-formulation, under which, in fact, formulation (1) is stronger. Clearly, both formulations have the same optimal solution with value that we call Z IP. For formulations (1) and (2) the LP-relaxation gives an optimal solution with value, respectively, Z F L1 and Z F L2, both less than or equal to Z IP. A closer look at the formulations reveals that Z F L2 Z F L1 Z IP. Notice that, in the LP-relaxation, any solution that satisfies x ij y j, i, also satisfies m i=1 x ij my j, but not necessarily the other way round (think of a solution with value y j = 1/2). Thus the feasible region of the LP-relaxation of (2) contains the feasible region of the LP-relaxation of (1), whereas both contain the convex hull of the feasible (integer) points of (1) and (2). It is easy to see that all extreme points of the convex hull of a set of integer points are integer. This means that if we had the formulation in terms of linear inequalities of the convex hull of the feasible points of the facility location problem, i.e., the formulation whose LP-relaxation has only integer extreme points, then we could simply solve the facility location problem by Simplex or any other LP-method. i, j, (2) 2
In Chapter 11 we will see solution methods that are based on solutions for the LP-relaxations and for these methods the closer the feasible region of the LPrelaxation is to the convex hull of the integer feasible solutions the better. In this context we call formulation (1) stronger than formulation (2) since the feasible region of the LP-relaxation of (1) the feasible region of the LP-relaxation of (1). We do not qualify one polyhedron as stronger than another if it is not fully contained in the other one. Disjunctive constraints The trick with big M is also applicable to model the following situation. Suppose that at least one of two constraints a T 1 x b 1, a T 2 x b 2 has to be satisfied. Introduce binary variable y and a large enough number M and set a T 1 x b 1 My, a T 2 x b 2 M(1 y) y {0, 1}. If M is large enough then y = 1 implies that the first restriction is always satisfied, and y = 0 that the second restriction is always satisfied. This can be generalised as is asked in Exercise 10.1. Making ILP-formulations of problems is a craft, in which some people are very good. Please read for yourself the examples for job sequencing with set-up times and for the allocation of fellowships and make Exercise 10.4. Minimum spanning tree Minimum Spanning Tree (MST): Given a graph G = (V, E) and (integer) weights on the edges c : E Z, find a spanning tree of edges of G with imum total weight of its edges. We skipped this problem in Section 7.10. It is extremely easily solved by a greedy method that orders the edges according to non-decreasing weight and selects them in this order unless the next edge in the order makes a cycle with the previously selected edges, in which case it is not selected. However, from the viewpoint of learning about strength of a ILP-formulation we will study two ILP-formulations of the MST which at first sight look equally good. In both formulations we use decision variables x e getting value 1 if they 3
are in the tree and 0 otherwise. In the first formulations we use notation E(S) = {{i, j} i S, j S}, for all edges entirely inside a subset S of the nodes. e E c ex e e E x e = n 1, S V, S, V e E(S) x e S 1, S V, S, V x e {0, 1}, e E. (3) The first constraint ensures the right number of edges in the solution and the second group of constraints ensures that there are no cycles (they have become known in the literature under the name subtour eliation constraints. In the LP-relaxation we replace the constraints x e {0, 1} by 0 x e 1, and call its feasible P sub. In the second formulation we use notation δ(s) = {{i, j} i S, j / S} for the cutset of subset of nodes S, i.e., the edges that have one vertex in S and the other outside S. Clearly a spanning tree must have at least one edge in the cutset of any subset of the nodes. Indeed this leads to the following formulation: e E c ex e e E x e = n 1, S V, S, V e δ(s) x (4) e 1, S V, S, V x e {0, 1}, e E. Also here, in the LP-relaxation we replace the constraints x e {0, 1} by 0 x e 1, and call its feasible P cut. Theorem 1.1 P sub P cut Proof. Proof as in the book. The example in the book shows that P cut can have fractional extreme points. It shows a fractional solution that satisfies all constraints of the LP-relaxation and has a value that is superior to the optimal integer value. The point does not satisfy all constraints of the LP-relaxation of formulation (3), since it contains a cycle with sum of x-values 2 i.o. the required 3. In fact it can be shown that P sub is exactly equal to the convex hull of all spanning trees. Hence, solving the LP-relaxation of (3) solves the problem MST. The only obstacle is that the LP-relaxation has 2 n 1 constraints. However, the separation problem is solvable by solving a -cut problem and therefore the LP-relaxation can be solved in polynomial time through the ellipsoid method. Of course, one would never do so in practice, given a pure MSTproblem instance, but it could be a nice alternative in case of MST with some side constraints. 4
Perfect matching Given a graph G = (V, E), a matching is a subset of the edges that are vertex disjoint (no two edges are incident to the same vertex). Given a graph with an even number of vertices, a perfect matching is a matching such that the edges cover all vertices, i.e., each vertex is coupled to one other vertex. Given graph G = (V, E) and weights on edges c : E Z, find a perfect matching of imum total weight. The {0, 1}-LP formulation is simply, using the notation of δ introduced before, e E c ex e e δ(v) x e = 1, v V x e {0, 1}, e E. This allows to set values 1 2 on all edges of odd cycles, while in fact at least one vertex on an odd cycle should have an edge to some vertex outside the cycle. This observation can be captured in constraints. For each odd size subset S V, x e 1. e δ(s) Adding these constraints to the ones given before and relaxing the integrality constraints defines the polytope of all perfect matchings as you can read in the book Theory of linear and integer programg by Alexander Schrijver. This is not an easy proof! To prove that the convex hull of all perfect matchings is contained in the polytope is rather straightforward. But to prove the converse is far from trivial. Again, a formulation with an exponential number of constraints, which can be separated in polynomial time. In this case, the standard solution method is a primal-dual method based on the extended formulation. Please read for yourself the two alternative formulation of the Travelling Salesman Problem and make Exercise 10.14. Example of problematic ILP-formulation That it is craftmanship I recently encountered in a Bachelor s project at the VU. It concerns a problem from biology, specifically from phylogeny. We are to construct the phylogeny tree of a set of species (or subspecies) from data that are given in the form of rooted binary trees on three leaves, as in Fig. 1. The unique rooted triplet (triplet for short) on a leaf set {x, y, z} in which the parent of x and y is a proper descendant of the lowest common ancestor of x and z is denoted by xy z (which is identical to yx z). The triplet in the figure is xy z. We say that a directed binary tree containing, amongst others, leaves x, y and z is consistent with triplet xy z (and conversely xy z is consistent with the tree) if it contains a subdivision of xy z, i.e., the rooted binary tree xy z can be 5
x y z Figure 1: One of the three possible triplets on the leaves x, y and z. Note that, we adopt the convention to direct all arcs downwards, away from the root. obtained from the tree by a series of contractions (of edges) and deletions (of vertices). Clearly the triplets xy z and xz y cannot be consistent with one tree. But also the triplets xy z, xw z and zw y cannot be consistent with one tree. Due to measurement errors the triplets are not all perfect, hence inconsistencies may occur. The optimization problem then emerges: Max Phylogenetic Tree: Given a set of triplets on n species, find the binary tree that contains all n species as its leaves and that is consistent with a maximum number of triplets. It is remarkably hard to find a correct ILP-formulation for this problem. In fact the only one we have is indirect, in the sense that it gives the set of consistent triplets in an optimal tree, without giving the tree. Given a set of triplets known to allow for a consistent tree, such a tree can be constructed by a recursive algorithm in O(n log n) time. The ILP-formulation itself is based on a theorem that has been proved by various authors: Theorem 1.2 A binary tree is not consistent with a triplet set if and only if there is a subset of at most four leaves such that the triplets on these four leaves are not all consistent with the tree. Thus, we only have to make sure that for every subset of four leaves the triplets that we select can be made consistent. For the ILP-formulation ( ) we introduce a n variable t ab c for each possible triplet (there are 3 of them), getting value 3 1 meaning that the triplet is chosen to be consistent and 0 otherwise. Then for each set a, b, c we add the restriction t ab c + t ac b + t bc a = 1, and for each ordered set a, b, c, d we add the restriction t ab c + t ac d t bc d 1 enforcing that is the first two triplets are consistent then the third one also needs to be consistent. Given input triplet set T we set the objective t ab c. ab c T 6
There are some remarkable features. The number of constraints is growing in the O(n 4 ) which means that problems with more than 40 species become problematic, just by the size of the input, for a desktop version of CPLEX. It is worth investigating how much redundancy there is i the formulation and diish the number of constraints. Or if the separation problem can be solved more efficiently than checking all O(n 4 ) constraints, i.e., without needing to specify all of them. Secondly, in all cases from practice that we had the optimal solution of the LP-relaxation appeared to be integer-valued. The polytope certainly has not only integral extreme points, since then we would have shown that P = NP. But maybe it has the {0, 1, 1 2 }-property? Material of Week 10 from [B& T] Chapter 10 Exercises of Week 10 10.1, 10.4, 10.11, 10.13, 10.14 and if you like 10.8 Next time in two weeks: April 20 Chapter 11 7