Lecture 11: 0-1 Quadratic Program and Lower Bounds

Transcription

1 Lecture : - Quadratic Program and Lower Bounds (3 units) Outline Problem formulations Reformulation: Linearization & continuous relaxation Branch & Bound Method framework Simple bounds, LP bound and semidefinite relaxation Variable fixation / 25

2 Problem formulation Standard form with - variables: (-QP) min f (x) = x {,} n 2 x T Qx + c T x where Q is an n n symmetric matrix and c R n. Homogenous form: Binary variables: (-QP h ) (BQP) min x T Qx. x {,} n min x T Qx + c T x. x {,} n Transformation: x i = 2 (y i + ). Homogenous form with binary variables: (BQP h ) min x T Qx, x {,} n (BQP) ( (BQP h ) with ) Q := 2 ct, x := (±, x c Q T ) {, } n+. 2 / 25

3 Max-Cut problem Consider a graph G = (E, V ) with vertex set V = {,..., n} and edge set E = {ij i < j n}. For every edge ij E, there is an associated weight w ij. Cut: For a given set S V, a cut δ(s) is the set of all edges with one endpoint in S and the other in V \ S, and the weight of cut δ(s) is ij δ(s) w ij. Max-Cut: find a cut δ(s) with maximum weight. Binary quadratic problem: (Max-Cut) max 2 i<j n s.t. x {, } n. w ij ( x i x j ) x {, } n S = {i V x i = } and V \ S = {i V x i = }. 3 / 25

4 Linearization Method - Quadratic problem: (P) n min Q(x) = c i x i + x {,} n i= i<j n q ij x i x j. For x i, x j {, }, y ij = x i x j iff y ij = max{x i + x j, }, y ij {, }, or y ij = min{x i, x j }, y ij {, }. 4 / 25

5 (P) is equivalent to the following - linear integer program: n min c i x i + q ij y ij + q ij y ij x,y i= (i,j) I + (i,j) I s.t. y ij x i, y ij x j, (i, j) I (q ij < ) y ij x i + x j, (i, j) I + (q ij ), x i {, }, i =,..., n, y ij {, }, i < j n. A polynomially solvable case: q ij : n c i x i + q ij y ij min x,y i= i<j n s.t. y ij x i, i < j n y ij x j, i < j n x i, x j, y ij {, }, i < j n. The constraint matrix is totally unimodular! Can we use transformation: z i = x i for q ij >? 5 / 25

6 Continuous relaxation Consider the continuous relaxation of (P): ( P) n min Q(x) = c i x i + x [,] n i= i<j n q ij x i x j. Then, at least one of the optimal solutions of ( P) is located at an extreme point of [, ] n. Therefore v(p) = v( P). Unfortunately, the objective function of ( P) is nonconvex and nonconcave. Define n Q p (x) = c i x i + x T Qx px T x + pe T x. i= Q p (x) = Q(x) for x {, } n. For large p, Q p (x) is a concave function. Thus, (P) is equivalent to the concave minimization problem: (P c ) min x [,] n Q p(x) 6 / 25

7 Branch and Bound Framework Computing lower bound; Branching on x i = or x i = ; Fixing variable by certain optimality condition. 7 / 25

8 Basic lower bounding methods Simple lower bounds Continuous relaxation LP relaxation Lagrangian relaxation & SDP relaxation 8 / 25

9 Simple bounds Lower bound. Let Q(x) = n c i x i + 2 x T Qx = i= n c i x i + 2 x T Qx. i= An obvious lower bound of Q(x) over {, } n is: LB s = 2 n n min(q ij, ) + min(c i + 2 q ii, ). i= j i i= Lower bound 2. An improved simple lower bound is derived by noting that: since x, if Qx a, then 2 x T Qx 2 at x. Let Q i denote the ith row of Q. Then a i = min Q i x = min(q ij, ). x {,} n j i 9 / 25

10 So = min x {,} n 2 x T Qx + c T x min x {,} n( 2 a + c)t x n min{c i + 2 q ii + min(q ij, ), } 2 i= = LB 2 s. j i It is easy to show that LB 2 s is better than LB s, i.e., LB 2 s LB s. / 25

11 Continuous relaxation Since Q(x) = 2 x T (Q + diag(u))x + (c 2 u)t x takes the same value on {, } n as Q(x), it is natural to compute a lower bound via solving the continuous relaxation: ( P) β(u) = min x [,] n 2 x T (Q + diag(u))x + (c 2 u)t x. Some observations: If ui s are large enough, then Q = Q + diag(u) will be diagonally dominant (thus positive definite). u = λ min e is an obvious choice to make Q positive semidefinite (but not necessarily the best one). The optimal solution to ( P) tends to ( 2, 2,..., 2 )T as u i s are increased. / 25

12 Consider a small example of (P) where ( ) ( 3 Q =, c = 3 ). For this example, we have x = (, ) T with Q(x ) =. The two simple bounds for this problem are: LB s = 3 and LB 2 s =. The eigenvalues of Q is ( 2, 4). 2 / 25

13 Figure: The figure of Q(x) over [, ] 2, which is nonconvex 3 / 25

14 Figure: u = λ min e, x u = (.864, ) T, β(u) = / 25

15 Figure: u = 2e, x u = (.577,.5538) T, β(u) = / 25

16 Figure: u = e, x u = (.53,.5) T, β(u) = / 25

17 Another way of choosing u is to find a u such that β(u ) = max{β(u) (Q diag(u)), u R n }. The above problem is equivalent to a semidefinite quadratic program which can be solved efficiently (polynomially). 7 / 25

18 LP relaxation The continuous relaxation of the - linearized problem is a linear program (y ij = x i x j ): min x,y i<j n q ij y ij + n (c i + 2 q ii)x i i= s.t. y ij x i, y ij x j, i < j n, q ij <, x i + x j y ij, i < j n, q ij >, x i, i =,..., n, y ij, i < j n. 8 / 25

19 SDP relaxation Consider Q(x) = 2 x T Qx + c T. The Lagrangian dual is: v(d) = max λ R n d(λ) = max λ R n x R n{ 2 x T [Q + 2diag(λ)]x + c T x e T λ} = max τ (λ,τ) R n+ s.t. 2 x T [Q + 2diag(λ)]x + c T x e T λ τ x R n. 9 / 25

20 2 x T [Q + 2diag(λ)]x + c T x e T λ τ, x R n ( (x T Q + 2diag(λ) c, t) c T 2τ 2e T λ (t, x) R n+, ) ( x t ) ( Q + 2diag(λ) c c T 2τ 2e T λ ), 2 / 25

21 Variable fixation Let x denote the optimal solution of (P). Let l i = c i + 2 q ii + j i min(, q ij ), u i = c i + 2 q ii + j i max(, q ij ). (i) If l i > then x i = ; (ii) If u i <, then x i =. 2 / 25

22 Fixing variable by outerbox Equivalent perturbed problem: (P µ ) min x {,} n f µ(x) = 2 x T (Q + µi )x + (c 2 µe)t x, where µ, I is the n n identity matrix and e = (,..., ) T. The shape of contour changes when µ changes. The properties of (P µ ), e.g., the conditional number, change when µ changes. 22 / 25

23 (,) (,) (,) (,) x 2 µ= µ=5 µ=3 2 µ= x (µ) 3 4 µ= x 23 / 25

24 Ellipsoid contour: Let x {, } n. E µ = {x f µ (x) = f (x ). Let the minimum box that contains ellipsoid E µ be B = [a, b], where a = x s, b = x + s. Then Let x be an optimal solution to (P). Then (i) If b i < for some i, then x i = ; (ii) If a i > for some i, then x i =. 24 / 25

25 .5 (,) (,).5 (,) (,).5 x (µ) Eµ(ṽ) Bµ(ṽ) / 25