3.3 Easy ILP problems and totally unimodular matrices Consider a generic ILP problem expressed in standard form where A Z m n with n m, and b Z m. min{c t x : Ax = b, x Z n +} (1) P(b) = {x R n : Ax = b, x 0 } is the polyhedron of the feasible solutions of the linear relaxation of (1). Assumption: A is of full rank (since n m, rank(a)=m), i.e, there are no redundant constraints. In general, optimal solutions of the linear relaxation are fractional and far away from optimal solutions of the ILP problem (1). If all the extreme points of P(b) are integer, the formulation is ideal and the ILP problem can be solved by just solving its linear relaxation. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 1 / 14
According to Linear Programming theory: Any linear program min{c t x : Ax = b, x 0} with a finite optimal solution has an optimal extreme point. To each extreme point of P(b) corresponds (at least) one basic feasible solution x = (x B, x N ) = (B 1 b, 0), where B is a basis of A, i.e., an m m non singular submatrix of A. Consider any basis B. By partitioning the columns of A in basic and non basic columns, the system Ax = b, x 0 can be written as and can be expressed in canonical form: Bx B + Nx N = b with x B 0 and x N 0, x B = B 1 b B 1 Nx N with x B 0 and x N 0, which emphasizes the basic feasible solution (x B, x N ) = (B 1 b, 0). Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 2 / 14
Observation: If an optimal basis B has det(b) = ±1, the optimal solution of the linear relaxation of (1) is integer and hence is also an optimal solution for the ILP problem (1). Proof: We know that x B = B 1 b and B 1 = 1 det(b) C t, where C is the matrix of cofactors α ij = ( 1) i+j det(b ij ) with B ij the squared submatrix obtained from B by deleting the i-th row and the j-th column. If all the components of B are integer, all the cofactors α ij are integer. If det(b) = ±1, B 1 is integer and therefore, since b is integer, also the basic feasible solution (x B, x N ) = (B 1 b, 0) is integer. Since B 1 b is integer also if det(b) = 2 and all components of b are even integers, det(b) = ±1 is only a sufficient condition for the integrality of (x B, x N ) = (B 1 b, 0). Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 3 / 14
3.3.1 Totally unimodular matrices and optimal integer solutions Definition: An m n matrix A with integer coefficients is totally unimodular (TU) if every squared submatrix has a determinant 1, 0 or 1. Clearly, if A is TU, a ij { 1, 0, 1} for all i and j. Examples of matrices that are TU and not TU: Recall: The Laplace expansion of the determinant of an m m matrix B along any row i, with 1 i m, is given by det(b) = m j=1 b ijα ij, where α ij = ( 1) i+j det(b ij ) are the cofactors of B. Analogously the expansion along any column j, with 1 j m, is det(b) = m i=1 a ijα ij. Proposition: A is TU if and only if A t is TU. A is TU if and only if (A I m) is TU. The matrix A obtained from A by permuting and/or changing the sign of some columns and/or rows is TU if and only if A is TU. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 4 / 14
Theorem 1: Consider an m n TU matrix A and an integer vector b such that P(b) = {x R n : Ax = b, x 0} =, then all the extreme points (vertices) of the polyhedron P(b) are integer. Proof: As for the previous observation. The result also holds for ILP problems in inequality form. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 5 / 14
Corollary: Consider an m n TU matrix A and an integer vector b such that P(b) = {x R n : Ax b, x 0} =, then all the extreme points of the polyhedron P(b) are integer. Proof: Let x be any arbitrary extreme point of P(b). First we show that ( x, s) with s := A x b is an extreme of P (b) := {(x, s) R n+m : Ax s = b, (x, s) 0}. If it were false, there would exist two distinct points (x 1, s 1 ) and (x 2, s 2 ) of P (b) such that ( x, s) = α(x 1, s 1 ) + (1 α)(x 2, s 2 ) for some α with 0 < α < 1. Since s 1 = Ax 1 b 0 and s 2 = Ax 2 b 0, x 1 and x 2 belong to P(b). Moreover, (x 1, s 1 ) (x 2, s 2 ) would imply x 1 x 2 and hence x = αx 1 + (1 α)x 2 could not be a vertex of P(b). Since A is TU, also (A I m) is TU. According to Theorem 1 for P (b), ( x, s) is integer, hence also x is integer. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 6 / 14
Sufficient conditions for a matrix to be TU Proposition: An m n matrix A with integer coefficients is TU if i) a ij { 1, 0, +1} for all i and j, ii) each column of A contains at most two nonzero coefficients, iii) the set I of the indices of the rows of A can be partitioned into two subsets I 1 and I 2 such that, for each column j with two nonzero coefficients, we have i I 1 a ij i I 2 a ij = 0. N.B.: If a column has two nonzero coefficients of the same (different) sign, their rows must belong to different subsets (same subset) of indices I 1 and I 2. Proof: Suppose A is not TU, and let Q be a smallest squared submatrix of A among those with det(q) / { 1, 0, 1}. Q cannot contain a column with a single nonzero coefficient, otherwise Q would not be minimal. Thus, each column of Q contains exactly two nonzero coefficients. According to (iii), i I 1 a ij i I 2 a ij = 0 and hence, since a ij = 0 for every column j of Q and every row i not in Q, we would have det(q) = 0, which is a contradiction. Examples of TU matrices (not) satisfying these conditions: Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 7 / 14
Characterization of TU matrices Theorem 2: An m n matrix A with integer coefficients is TU if and only if each subset I I = {1,..., m} of indices of the rows of A can be partitioned into two subsets I 1 and I 2 such that the sum of the rows in I 1 minus the sum of the rows in I 2 is a vector with all components in { 1, 0, 1}, that is i I 1 a ij i I 2 a ij 1 for every column j. From the ILP point of view, when the matrix A is TU it suffices to solve the linear relaxation. In some sense also the converse is true: Proposition: The linear program min{c t x : Ax = b, x R n +} has an optimal integer solution for any integer vector b for which it admits a finite optimal solution if and only if A is TU. It can be shown that, given A and a basis B with det(b) 1, there always exists a LP min{c t x : Ax = b, x R n +}, for appropriate c and b, with a fractional optimal solution. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 8 / 14
3.3.2 Examples of natural formulations that are ideal 1) Assignment problem Given n jobs and n machines with costs c ij for all i, j {1,..., n}, decide which job to assign to which machine so as to minimize the total cost to complete all the jobs. ILP formulation: min n i=1 n j=1 c ijx ij n i=1 x ij = 1 j (2) n j=1 x ij = 1 i (3) x ij {0, 1} i, j where x ij = 1 if i-th job is assigned to j-th machine, with 1 i, j n. N.B.: In the linear relaxation, x ij 0 i, j suffice, because (2)-(3) imply x ij 1 i, j. Property: The matrix of constraints (2)-(3) is TU. In the sufficient conditions for TU, just take I 1 = {1,..., n} and I 2 = {n + 1,..., 2n}. Consequence: Each extreme point of the feasible region of the linear relaxation is integer, and the formulation is ideal. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 9 / 14
2) Transportation problem Suppose there is a single type of product. Given m production plants (1 i m) n clients (1 j n) c ij = transportation cost of one unit of product from plant i to client j p i = maximum amount that can be produced (capacity) at plant i d j = demand of client j q ij = maximum amount that can be transported from plant i to client j determine a transportation plan so as to minimize total transportation costs while satisfying all client demands and all the plant capacity constraints. Assumption: m i=1 p i n j=1 d j (to guarantee feasibility) Without loss of generality, we can assume that m i=1 p i = n j=1 d j. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 10 / 14
Natural ILP formulation: Decision variables: x ij = amount of product transported from plant i to client j, with 1 i m and 1 j n min m i=1 n j=1 c ijx ij n j=1 x ij p i i (4) m i=1 x ij d j j (5) x ij q ij i, j (6) x ij 0 integer i, j Property: The matrix of constraints (4)-(6) is TU. Proof: Put the problem in consistent inequality form by multiplying all constraints (4) and (6) by 1. Since the matrix of the demand and capacity constraints is TU, we just apply the Corollary. Consequence: If all p i, d j and q ij are integer, every extreme point of the feasible region of the linear relaxation is integer and, hence, the formulation is ideal. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 11 / 14
3) Minimum cost flow problem Given a directed graph G = (V, A) with a capacity u ij and a unit cost c ij for each arc (i, j) A, and a demand/availability b i for each node i V (b i < 0 for sources, b i > 0 for destinations, and i V b i = 0), determine a feasible flow of minimum total cost which satisfies all the demands. Natutal ILP formulation: min (i,j) A c ijx ij (h,i) δ (i) x hi (i,j) δ + (i) x ij = b i i V (7) x ij u ij (i, j) A (8) x ij 0 integer (i, j) A where δ + (i) = {(i, j) A : j V } and δ (i) = {(h, i) A : h V }. Property: The matrix of constraints (7)-(8) is TU. Proof: The n A node-arc incidence matrix of the directed graph is TU since it contains exactly one 1 and one 1 per column (take I 1 = I and I 2 = ). The matrix obtained by juxtaposing to A the identity matrix I A A of the capacity constraints (8) is also TU. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 12 / 14
Consequence: If the demands/availabilities b i and the capacities u ij are integer, every extreme point is integer and, thus, the formulation is ideal. Exercise: Show that the following well-known combinatorial optimization problems are special cases of the Min cost flow problem. - Shortest path: Given a directed graph G = (V, A) with a cost c ij associated to each arc (i, j) A, and two prescribed nodes s and t, determine a minimum cost path from s to t. - Maximum flow: Given a directed graph G = (V, A) with a capacity u ij associated to each arc (i, j) A, a source s and a sink t, determine a feasible flow of maximum value from s to t. Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 13 / 14
Rounding optimal solutions of the linear relaxation In general, when the constraint matrix of an ILP is not TU, the optimal solution x PL of its linear relaxation is fractional. As a first attempt to derive a sub-optimal solution of the ILP, we may try to round each fractional component of x PL to the closest integer value. However the rounded solution are often infeasible for the ILP, even if the rounded solution is feasible for the ILP, the error with respect to an optimal ILP solution may be arbitrarily large. In general, by rounding x PL we obtain a good approximation of the optimal solution only when the components of x PL have large values. Illustration of the different cases: Edoardo Amaldi (PoliMI) Optimization A.Y. 2014-15 14 / 14