Mixed integer programming methods for supply chain optimization

Transcription

1 Mixed integer programming methods for supply chain optimization C h r i s t o s T. M a r a v e l i a s Chemical and Biological Engineering University of Wisconsin, Madison, WI 53706, USA July 9-9, 0, Angra dos Reis, RJ, Brazil

2 Outline. Supply Chain Management. Linear Programming 3. Basic Mixed integer Linear Programming 4. Advanced Mixed integer Linear Programming 5. Software Overview 6. Combinatorial Optimization 7. Chemical Production Scheduling 8. Integration of Production Planning and Scheduling

3 The Chemical Supply Chain Supply chain (SC): network of facilities and distribution options for the procurement of materials; transformation of materials into intermediate and finished products; and distribution of these finished products to customers. Suppliers Manufacturing Facilities Traditional PSE is concerned with the development of methods for the design, operation, and control of chemical systems. Warehouses/ dist. centers Retailers Customers Chemical industry changes: New products multiproduct (and batch?) processes Holistic view of operations Tayur FOCAPO 003: Inventories in SCs: ~ $0 trillion (0% of US GDP) Same customer satisfaction levels with 50% inventory The goal in supply chain management is to coordinate materials, information and financial flows to fulfill customer demands while improving competitiveness of the supply chain as a whole; in coordinating these flows various decisions have to be made.

4 Supply Chain Planning Matrix (Meyr et al., 00) procurement production distribution sales Long term Strategic Planning Medium term Short term Purchasing & Material Requirements Planning Master Planning Production Planning Scheduling Distribution Planning Transport Planning Demand Planning Demand Fulfillment Integration across time scales and functions Many new interesting problems Power industrial gases steel SC Oil industrial gases chemicals SC under power constraints Power Utilities Refinery Industrial Gases Chemicals Customers

5 Supply Chain Planning Matrix (Meyr et al., 00) procurement production distribution sales Long term Strategic Planning Medium term Short term Purchasing & Material Requirements Planning Master Planning Production Planning Scheduling Distribution Planning Transport Planning Demand Planning Demand Fulfillment Integration across time scales and functions Many new interesting problems Power industrial gases steel SC Oil industrial gases chemicals SC under power constraints Biomass to fuels and chemicals Corn Sugarcane Bagasse Corn Stover Switch grass Wood Wood waste Corn Grain Fractionation Catalyst Steam Acid Enzymes Hydrolysis Fast pyrolysis Liquefaction Hydrolysis Sugars Bio oils (Sugars, Acids) & Lignin Lignin C4 C5 C6 Fermentation Dehydration Dehydration Demethoxylation Hydrodeoxygenation Succinic acid Fumaric acid Malic acid Furfural Levulinic acid FDCA BTX Phenols &BTX Zeolite upgrading Aromatics

6 Supply Chain Planning Matrix (Meyr et al., 00) procurement production distribution sales Long term Strategic Planning Medium term Short term Purchasing & Material Requirements Planning Master Planning Production Planning Scheduling Distribution Planning Transport Planning Demand Planning Demand Fulfillment Integration across time scales and functions Many new interesting problems Power industrial gases steel SC Oil industrial gases chemicals SC under power constraints Biomass to fuels and chemicals Integration of production planning and scheduling Exxon Mobil: % reduction in operating costs, 0% inventory reduction (Shobrys & White, 00) DuPont: Capital tied up in inventory reduced from $65 to $90 million (Shobrys & White, 00)

8 Linear Programming Standard optimization problem: max f(x) n - number of variables s.t. g(x) = 0 m - number of equality constraints h(x) 0 l number of inequalities x X m < n If functions f, g, h are linear and variables x are continuous, we have a Linear Programming (LP) Model. Notes:. Free z R variables can be replaced by two nonnegative variables x, y 0: z = x y. Inequalities can be transformed to equalities with slack variables x + x 0 x + x + s = 0, s 0 3. Minimization problems can be expressed as maximization problems: min f(x) max f(x) Thus, we can use the standard LP form: (LP) max z = c T x s.t. Ax = b x 0 where x: (n+l)-vector; A:(m+l) (n+l) matrix; c: (n+l)-vector; b: (m+l)-vector

9 LP Geometry A process is used for the production of products P and P from raw materials A and B according to: A + B 3P () A + B 3P () If the maximum availability of raw materials A and B is 0 kg/hr, the price of both chemicals is $00/kg, and the demand for products P and P is and 3 kg/hr, respectively, what is the optimal production mix? The problem can be represented as the following network (the labels refer to the streams, not species): A B S A S B S A Reaction S A S B S B Reaction S S P P If F P and F P are the amounts of products P and P, respectively, the optimization problem (P) can be stated as follows: F P 30 F P + F P 30 max Z st. F F = 00F P P 0 F + F + F P P P P, + 00F P 0 F P 3 (P) 0 F P 3 0 F P F P + F P F P

10 LP Geometry max 0x +0x s.t x + x 30 x +x 30 x x 3 x, x 0 Definition: max 0x +0x s.t x + x + s = 30 x +x + s = 30 x + s = 3 x + s 4 = 3 x, x, s, s, s 3, s 4 0 A basic solution to an (LP) is determined by fixing (n-m) variables to 0, and solving the (m m) system to obtain the remaining. Variables fixed at zero are non-basic variables. Variables obtained by solving the equalities are basic. Definition: A basic feasible solution to an (LP) is a basic solution that satisfies nonnegativity constraints. Definition: x F is an extreme point iff x = α x + (-α) x and α (0,), x, x F implies x = x. i.e. x can not be obtained by linear combination of x, x F: x x. The basic feasible points of an LP are the extreme point of its feasible region. Let A = [B N], Ax = Bx B + Nx N = b If x N = 0 x B = B - b - B - Nx N z = c B x B + (-c B T B - N + c N T )x N x + s 4 =3 0 x B x = where B: m m full-rank, N: m (n-m), x B : m-vector and x N : (n-m)-vector xn Basic point Objective function Theorem : A point x in F is an extreme point iff x B = B b 0, x N = 0 Theorem : The optimal solution x * of (LP) lies at an extreme point of F. Theorem 3: Point x * is optimum if Reduced Cost RC = c BT B N + c NT 0 30 x x + x + s = 30 0 Basic solution x + s 3 = x + x + s = x Basic feasible solution (extreme point)

11 Simplex Method Main idea of Simplex: If current basis non optimal, swap a non basic variable for a basic one until ( c BT B N + c NT ) 0 T we start at x =x =0 0 M M 0M M N B x B = B b z = c B x B + ( c T B B N + c T N )x N x N = [x, x ] = [0, 0] x B = [s, s, s 3, s 4 ] = [30, 30,, 3] z = 0 + 0x + 0x x (0,3) x N = [s, s ] = [0, 0] x B = [x, s 3, x, s 4 ] = [0,, 0, 3] z = 00 (0/3)s (0/3)s (4,3) (0,0) (6,) (0,) x max 0x +0x s.t x + x + s = 30 x +x + s = 30 x + s 3 = x + s 4 = 3 x, x, s, s, s 3, s 4 0 x N = [s 3, s ] = [0, 0] x B = [x, s, x, s 4 ] = [6, 6,, 7] z = s 3 0s x N = [s 3, x ] = [0, 0] x B = [s, s, x, s 4 ] = [6, 8,, 3] z = 0 + 0x 0s 3 Simplex algorithm developed by G. Dantzig (947) Very popular due to wide range of applications formulated using LP Considered as one of the most important algorithms in the 0 th century Not polynomial but outperforms polynomial algorithms (Ellipsoid, Khachiyan Khachiyan, 97)

12 Simplex Method Remarks Degeneracy A basic solution is degenerate if a basic variable is equal to zero. In this case, the Simplex method may go through many iterations without increasing the value of the objective function, and the same basis is encountered again (Cycling). To avoid cycling, several rules have been developed; e.g. Bland s rule. Initialization It is not always easy to find an initial basis. For instance, if after adding the slack variable we have some components of b that are negative. Infeasible In this case we apply Simplex in two phases. Phase I: To each equality aij x j = b j i with b i <0, subtract an artificial variable u i, add constraint u i 0 and solve: min Σu i s.t. Ax = b -u u 0 The original LP has a feasible solution iff the auxiliary LP has an optimal solution with objective value equal to zero. Phase II: Use the last basis of phase I as a starting basis for Phase II, after you delete artificial variables u i and change objective function. Termination It is not guaranteed that a feasible LP has an optimal solution. The problem can be unbounded. The simplex method detects such programs and returns a proof of unboundedness. When checking to determine the leaving variable, no constraint becomes binding.

13 Duality I Given a primal LP problem: min c T x (LP-P) s.t. Ax = b x 0 the corresponding dual LP problem is formulated as: min b T y (LP-D) s.t. A T y c How do we construct (LP-D) from (LP-P)? For each constraint of (LP-P), there is a variable in (LP-D) For each variable of (LP-P), there is a constraint in (LP-D) If (LP-P) is min problem, the dual is max. If the constraints of (LP-P) are equalities (inequalities) the variables of (LP-D) are unconstrained (constrained) If the variables of (LP-P) are free (non-negative), the constraints of (LP-D) are equalities (inequalities) Non-negative variables result in inequalities in max problems Non-negative variables result in inequalities in min problems The dual of an LP can be used to better study and even solve faster an LP

14 Duality II Let F P and F D be the feasible region of (LP-P) and (LP-D), respectively, we introduce the following convention: If F P = inf x = + x F P c T If F D = sup y = y F D b T Theorem : For any (LP): T T inf c x sup b y [Proof] x F P y F D Theorem : If either F P 0 or F D 0, then T T inf c x = sup b y [Proof] x F P y F if one of these problems is solvable, then so is the other and D min c x F P T x = max b y F D T y Primal Dual Optimal Infeasible Unbounded Optimal - - Infeasible - Unbounded - -

15 Duality III Thus, we can solve (LP-D) instead of (LP-P) if the first is easier. Computationally, the number of constraints is more important than the no of variables If the (m n) primal (LP-P) has m >> n, then it may be easier to solve the dual (LP-D) Furthermore, the dual solution provides us with sensitivity information The dual variable corresponding to a primal constraint, indicates whether this constraint is active; and by how much Dual values = shadow prices = marginal values Sign of dual variables Why equality constraints, have unrestricted dual variables? What does the sign say about the RHS? Nonnegative variables in (LP-P) result in inequalities in max problems inequalities in min problems

16 Duality IV Re-optimization Let assume that we have solved to optimality (LP-P) to obtain z * = c T x* = b T y * What happens if we add a new constraint a m+ x = b m+ to (LP-P)? (LP-P) may become infeasible. What happens to (LP-D)? We add a variable; if we set y m+ = 0, we still have a feasible solution We can use the existing basis to continue. Dual Simplex Method In general, a feasible non-optimal point of (LP-P) corresponds to an infeasible point of (LP-D) [Why???]. In the simplex algorithm, we try to reduce infeasibility of the dual problem. Dual simplex method Commercial solvers use there three powerful LP algorithms: Primal Simplex Dual Simplex Barrier method Different methods perform better in different problems Commercial solvers detect certain problem structures & automatically choose appropriate method If we know the structure we can also choose the method

18 MIP in Chemical Engineering Many decisions are discrete in nature: Process Design: Selection of equipment units (e.g. distillation vs. extraction) Superstructure Optimization Selection of equipment sizes Matching in heat exchanger networks (hot cold pairs) Process operations: Decision to expand/retrofit a process Split of customer orders to batches Assignment of batches to equipment units Sequencing of batches in the same unit Process Control: Selection of linearized model for MPC On/off control action Applications: Systems Biology Gene addition and/or knock out

19 Air Discrete Decisions in Process Design Example: What is the best way to produce Vinyl Chloride Monomer? Oxygen Ethylene Chlorine Direct Chlorination Oxychlorination Purge Flash Hydrogen Chloride Water Hydrogen Chloride Ethylene Dichloride Low P High P Turkay & Grossmann, 998 Reactions: Direct chlorination C H 4 + Cl C H 4 Cl Oxychlorination C H 4 + HCl + /O C H 4 Cl + H O Pyrolisis C H 4 Cl C H 3 Cl + HCl Separation: HCl/C H 3 Cl/C H 4 Cl mixture. Separate HCl/C H 3 Cl C H 4 Cl. Separate HCl C H 3 Cl Vinyl Chloride. Separate HCl C H 3 Cl/C H 4 Cl. Separate C H 3 Cl C H 4 Cl Ethylene Dichloride Optimization Decisions: Distillation sequence (discrete), sizing (discrete or continuous), operating conditions (continuous) Discrete variables are also used to: Represent logic conditions Enforce set of constraints (often including only continuous variables)

20 Discrete Decisions in Process Operations Example: Scheduling of a multi-product, multi-stage plant Products st stage nd stage 3 rd stage (Batches) Reaction Separation Separation Optimization Decisions: Number of batches Assignment of batches to units Sequencing of batches in a unit Starting time (continuous)

21 Basic Integer Programming Problems Knapsack Problem We are given n items to be packed in a knapsack Each item i {,, n} has a value p i and a weight w i The capacity of the knapsack is W The goal is to choose the items that fit in the knapsack with the max value We define: X i = if item i is chosen n Items... Knapsack Objective: max z = p i X i i, j (K.) i w X i i W (K.) Capacity constraint X i {0, } i If we had multiple resources j {,,..m} (e.g. weight, volume, etc.) with capacities W j we would replace (K.) with (K.3): i w ij X i W j j (K.3)

22 Basic Integer Programming Problems Assignment Problem We are given n jobs and n machines (classes/instructors, routes/buses) Each job i {,, n} has to be assigned to a machine j {,, n} The cost of assignment is c ij The objective is to find the assignment with the minimum total cost We define: X ij = if job i is assigned to machine j n Jobs... Machines... n Objective: min z = c ij X ij i, j (A.) X ij = i (A.) Every job is assigned j X ij = j (A.3) Exactly one job per machine i X ij { 0,} i, j If solved as LP, there is always an optimal solution with integral values Studied by Carl Gustav Jacobi (890) Solved effectively by the Hungarian method (Kuhn, 955) Improved by Munkres (957)

23 Basic Integer Programming Problems Traveling Salesman Problem There are n cities to be visited exactly once by a traveling salesman Cities The distance/cost from city i to city j is c ij The objective is to find the sequence of cities that yield the minimum total distance/cost We define: X ij = if we visit city j after city i Objective: min z = c ij X ij i, j (TSP.) X ij = j (TSP.) There is an incoming arc for every city j i X ij = i (TSP.3) There is an outgoing arc for every city i j X ij { 0,} i, j Exercise Are constraints (TSP.) & (TSP.3) enough? What type of solution can we get? How can we improve our model?

24 Basic Integer Programming Problems Set Covering Problem We are given n tasks to be covered by m<n machines (flights/crew) Each machine j {, m} can carry out a subset of tasks I j I = {,, n} The compatibility is represented via parameter: α ij = if i I j The cost of a machine is c j The objective is to find the set of machines that covers all tasks and has the minimum cost n Jobs... Machines... m We define: X j = if machine j is chosen Objective: min z = c j X j j (SC.) aij X j i (SC.) Every task is covered j X j {0, } j

25 Basic Mixed integer Programming Problems Facility Location Problem I There are n (existing + new) facilities (plants) and m customers We want to decide how to satisfy customer demand What new facilities to build, how much to produce and ship The capacity (cost of setting up) of facility i {,, n} is α i (f i ) The cost of shipping material from i to j is c ij The demand of customer j {,, m} is d j Facilities a i, f i... c ij Customers d j... We define: X i = if we set up (install) facility i Y ij = amount shipped from i to j Objective: min z c Y (FLI.) Minimize set-up and shipping costs = fi X i + i i j Y Y ij ij d a i j i, j X i ij j X i { 0,} i, Yij 0 i, ij i (FLI.) More realistic problem: There is also production cost p i, transportation set-up cost t ij, and transportation capacity s ij The problem has to be solved over multiple planning periods The network involves multiple products j (FLI.3) n Customer demand satisfaction Facility capacity constraint m

26 Basic Mixed integer Programming Problems Facility Location Problem II The capacity (cost of setting up) of facility i {,, n} is α i (f i ) The production cost in facility i is p i The setup (variable) cost of shipping material from i to j is t ij (c ij ) The capacity of transportation link (i j) is s ij The demand of customer j {,, m} is d j Facilities a i, f i... c ij Customers d j... We define: X i = if we set up (install) facility i Y ij = amount shipped from i to j Z ij = if transportation link (i j) is set-up n s ij, t ij m Objective: min z = fi X i + pi Yij + tijzij + i i j Y Y ij ij d a i j X i i j j i i, j i, j c ij Y ij (FLII.) (FLII.) (FLII.3) Minimize total cost Customer demand satisfaction Facility capacity constraint Y s Z i ij ij ij, j (FLII.4) Transportation capacity and set-up X i, Zij {0,} i, j Yij 0 i, j

27 Modeling with 0 Variables. Select at least one item from a set j = {,,, m} OR operator: y y y m y j. Select exactly one item Exclusive OR: y y y m y j = 3. If select k then select l Implication: k l y l y k 4. If and only if Equivalence: k l y k = y l 5. Disjunctions f (x) 0 OR f (x) 0 y + y = f (x) M ( - y ) f (x) M ( - y ) where M, M are sufficiently large numbers 6. Discontinuous Functions/ Domains Cost = α + bx L x U 0 x = 0 New variable y: If y = 0 x = 0 If y = L x U Formulation: c = αy + bx Ly x Uy y {0,} j j Cost α L U x

28 Modeling with Disjunctions We often want to enforce different conditions in different cases Examples: Selection of reactor type Selection of an additional flash tank Grossmann & co workers, 998 CO, CO, H 3-A) 3-B) Methanol Synthesis Isothermal Reactor Methanol Synthesis Adiabatic Reactor Flash Drum 4-A) Flash Drum x x x k OUT w j k IN k Process stream variables of stream Variables of streams feeding unit k Variables of streams leaving unit k Operational parameters of unit k j Methanol Water Distillation column 4-B) Mathematical model of unit k : h g k k k k k ( w, xin, xout ) k k k ( w, x, x ) C k IN = f k OUT ( S k = 0 0 ) To find the optimal design we want to be able to handle the following logic constraints: h g k k k k k ( w, xin, xout ) k k k ( w, x, x ) C k = y IN f k k y k = 0 w = 0, C k OUT 0 x = 0, IN x k ( S ) k k k OUT = 0 = 0 We have to convert the disjunction into MIP constraints, handled by MIP solvers

29 Reformulations x x y x x y x x ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) = y y y M x y M y M x y M y M x y M y M x y M ( ) ( ) ( ) ( ) y M x y M y M x y M y M x y M y M x y M ( ) ( ) ( ) ( ) y x y y x y y x y y x y : = 7 M Selecting Big M reformulation

30 Reformulations x x y x x y x x = + + = + = ,, y x y y x y y x y y x y y y y M x y M x y M x y M x x x x x x x ( ) ( ) ( ) ( ) + = + = , y x y y x y y x y y x y y M x y M x y M x y M x x x x x x x Convex hull reformulation

31 General Mixed integer Programming Model min s.t. c T x + d T y Ax + By b x 0, y integer or max s.t. c T x Ax b x j integer, j N I = {...p} If N i = (i.e. there are no y variables) Linear Programming (LP) Model If N i = N (i.e. there are no x variables) (Pure) Integer Programming (IP) Model If N i N (i.e. we have both x and y) Mixed-Integer Programming (MIP or MILP) Model A general integer Y {0,,, 3, N} can be modeled via k binary variables Y k : So, the general MIP problem is: Y = k = 0,... N ky k or min s.t. min s.t. c T x + d T y Ax + By b x 0, y {0, } c T x + d T y (x, y) X

32 LP relaxation of a MIP Model General MIP problem (P): min c T x + d T y (P) s.t. Ax + By b x 0, y {0,} If we replace the integrality requirement y {0,} with y [0,] we obtain a relaxation (RP) of (P): min c T x + d T y (RP) s.t. Ax + By b x 0, y [0,] In general, a relaxation of a problem is obtained by: Either removing constraints (i.e. expanding the feasible region) Decreasing the objective function (e.g. removing a nonegative term) min c T x + c T x s.t. A x b A x b (P) x X where x, c, c 0 min c T x + c T x s.t. A x b x X min c T x s.t. A x b A x b x X (RP ) (RP ) z(rp ) z(p) Larger feasible region Better obj. value at the same point

33 Branch and Bound: Basic Ideas LP based method: Integrality constraints, y {0,}, are replaced by y [0, ] max c T x + d T y (P) max c T x + d T y (LRP) s.t. Ax + By = b s.t. Ax + By = b x 0, y {0, } p x 0, y [0, ] p LP problem P(i) is solved at node i, with feasible region F i and objective Z i We know how to solve (LRP) effectively using Simplex (vs. other relaxations) Lang & Doig, 960 Key observations: If a node is infeasible, then all its descendants will also be infeasible A child node will always have a lower (worse) objective function (bounding) Any feasible solution Z i in node i provides a lower bound on the optimal solution Z* Easy to add rows (constraints) to (LRP) (why?) x (x,x ) = (.75, ) max 3x + 4x 4x +3x 0 x {0,,,3}, x {0,} x Z LP = 9.5 x (, )-INT Z LP = 7.0 x x 0 (.5, 0)-INT Z LP = 7.5 x x 3 (, 0.66) Z LP =8.66 x INFEASIBLE (,) Z LP =6 INFEASIBLE

34 Branch and bound: Basic Ingredients. The list L of problems to be solved. Relaxation (RP): upper (& lower?) bounding procedure 3. Branching Rule: replace problem P(i) in node i by problems P(i ), P(i ),, P(i q ): FR(i ) FR(i q ) = FR(i) 4. Node (subproblem) selection rule: how to select a problem from the current subproblem list L 5. Pre-processing and variable fixing procedures (0, 0.8) Z LP = 8.5 (x,x ) = (0.75, 0.3) Z LP = 9.5 x 0 x (, 0.7) Z LP =8.75 On which binary/constraint to branch? x 0 x x 0 x Which active node to evaluate next? (0, 0)-INT Z LP = 7 (0, ) -INT Z LP = 8 (, 0) -INT Z LP =7.5 INFEASIBLE Method first proposed by Lang and Doig (960) Around the same time an alternative method was developed (cutting planes) Brute force branch and bound remained ineffective for many years Cutting planes were not used for many years

35 Branch and bound Algorithm. Initialization L = {P} Z U := +. Termination If L = then If Z U = +, then X = (infeasible problem); STOP If Z U < +, then the solution (x, y) X with Z U = cx + dy is optimal; STOP 3. Node Selection and Solution Using SSR select subproblem (node) P(i) L and let L := L\{P(i)} Compute the optimal LP-value Z i and LP-solution (x i, y i ) for P(i). 4. Pruning If Z i Z U, then GOTO. (P(i) is either infeasible or dominated by the best solution (upper bound) found so far) If Z i < Z U, then If y i is integral (i.e. y i j is integral j N I ), then (a better solution is found) Update upper bound by setting Z U := Z i Update list L by removing dominated subproblems: for each i L: If Z i Z U, then L := L\{P(i )} GOTO. 5. Branching* Select j N I such that y i j is fractional; y j is the branching variable Update list L by adding programs with fixed y j values Set L:= L {P(i0), P(i)} where P(i0) = P(i) {(x,y) R n + R l + y j y i j } P(i) = P(i) {(x, y) n l R + R + y j y i j } GOTO. * Occurs only when P(i) has not been pruned, i.e. when Z i < Z U and yj i is fractional for some j N I min s.t. c T x + d T y (x, y) X

36 Branch and bound Tree Level 0 3 Nodes = 0 = 4 = 8 = = 4 Notes: Level denotes the number of fixed variables Total number of nodes = N = N+ -, where N = no of binary variables [Why?] The goal is to explore much fewer nodes via bounding How do we carry out branching?

37 Branching (0, 0.8) Z LP = 8.5 x 0 (x,x ) = (0.75, 0.3) Z LP = 9.5 x 0 x x x 0 (, 0.7) Z LP =8.75 x On which binary/constraint to branch? Which active node to evaluate next? (0, 0)-INT Z LP = 7 (0, ) -INT Z LP = 8 (, 0) -INT Z LP =7.5 INFEASIBLE Variable Selection Rules. Try to find a good solution fast: Obtain a good lower bound Prune many nodes fast. Choose a variable that results in bad solutions: Prune as many nodes as possible Keep the b&b tree small Node selection rules Try to find good feasible solutions as early as possible In general: The number of active nodes increases if the active node is chosen high-up It takes a long time to improve the upper bound if the active node is chosen from down low

38 Variable Selection Rules x We want balanced branching and tight bounds: choose j N I that yields: max{ min{ j = 0 x j = Z, Z }} Example j INT: Integer feasible INF: Infeasible FRAC: Fractional INT Z U =5 INF INF INT Z U =6 FRAC Z U =8 FRAC Z U =4.5 FRAC Z U =6 FRAC Z U =4 Current best (Z U ) Closed (infeas or Z i > Z U ) Open (Z i <Z U ) Exercise

39 Variable Selection Rules x We want balanced branching and tight bounds: choose j N I that yields: max{ min{ j = 0 x j = Z, Z }} j Example Z U =4 Current best (Z U ) Closed (infeas or Z i > Z U ) Open (Z i <Z U ) (0,,0., 0.4, 0.95,...) Z U =5 Z U =4.5

40 Variable Selection Rules x We want balanced branching and tight bounds: choose j N I that yields: max{ min{ j = 0 x j = Z, Z }} j Example Z U =4 Current best (Z U ) Closed (infeas or Z i > Z U ) Open (Z i <Z U ) y=0 y= (0,,0., 0.4, 0.95,...) Z U =5 Z U =4.5 Z U =4. Z U =6

41 Variable Selection Rules x We want balanced branching and tight bounds: choose j N I that yields: max{ min{ j = 0 x j = Z, Z }} j Example Z U =4 Current best (Z U ) Closed (infeas or Z i > Z U ) Open (Z i <Z U ) y=0 y= (0,,0., 0.4, 0.95,...) Z U =5 Z U =4.5 Z U =5 Z U =5.

42 Node Selection Rules Depth-first Search: Always choose one of the new subproblems just generated; typically the one with the best lower bound Backtrack when node is discarded 0 z= Advantages: Can potentially find a (good) solution fast Re-optimization involves the addition of a single constraint (???) Requires modest storage Disadvantages: Hidden good solutions difficult to find

43 Node Selection Rules Breadth-first Search: Expand all nodes at each level before go to the next level 0 z= Disadvantages: Examine many non-promising nodes Feasible solutions are found late Create very large tree (memory requirements) Not used in practice

44 Node Selection Rules Best-first Search: Always choose the subproblem with the best lower bound 0 z = Advantages: Examines potentially good nodes Uniformly tighten bounds Disadvantages: Re-optimization is more expensive Tree tends to be larger Modern commercial solvers use a combination of depth-first and best-first.

45 Special Branching Schemes Special Ordered Sets (SOS): x i 0 AND x + x + x 3 + x 4 + x 5 = AND only one non-zero, (0, 0.5, 0.5, 0, 0) INFEASIBLE x + x = x 3 + x 4 + x 5 = or (0, 0.5, 0.5, 0, 0) INFEASIBLE x = x = x 3 = x 4 = x 5 = Special Ordered Sets (SOS): x i 0 AND x + x + x 3 + x 4 + x 5 = AND at most two non-zeros AND consecutive (0, 0.5, 0, 0.5, 0) INFEASIBLE x =0, x =0 x 3 =0, x 4 =0

46 Pre processing Preprocessing applies simple logic to reformulate and tighten the LP relaxation For MIP problems is better to spend extra time initially to reduce the possibility of long runs Probing Temporarily set a 0- variable to 0 or and then redo the logical testing. 5x + 3x 4 becomes infeasible when x =0 x = in every feasible solution 5x + 4x + x 3 8 If x = then x =0 and vice versa x + x x + x + x 3 becomes strictly redundant when x =. Can be replaced by x + x + x 3 (0.8, 0) (, 0.75, 0) (0.5, 0, 0) Generate Bounds e.g., If the upper bound of a 0- variable is less than, then it can be fixed to 0. Detect Infeasibilities and Redundant Constraints If L i Ax i U i (for l x u) then Ax i b i is redundant if U i b i and is infeasible for L i < b i.

47 Pre processing In summary preprocessing: Improves bounds Identifies infeasibility and redundant constraints Fixes variables Generates new valid inequalities Tightens existing inequalities (coefficient improvement) Not possible to perform all operations and save all information Computational Effort for preprocessing:. Techniques applied to one row Always worth doing. Saving and preprocessing implication and clique tables 3. Full probing on the matrix Common strategy:. Generate clique and implication inequalities. Store them in separate tables 3. Add them to the linear program only when they are violated by the current LP solution

48 Heuristics Solution Time proportional to tree size Tree is large when there are many active nodes; i.e. nodes with z i < z best = Z U How do we keep the size of the tree small? Try to reduce z i as fast as possible Variable Selection Rule Try to find large z best as fast as possible Heuristics Also for large scale instances: good feasible solutions may be all we can hope for If we know the structure of the problem: Use any known heuristic to provide an initial z best e.g. use Lin-Kernigham heuristic to get a usually very good solution for the TSP If we do not know a heuristic: Use LP-based heuristics: Enumerate 0- vectors in the neighborhood of the current LP solution and check feasibility OCTANE heuristic (Balas et al., 00) Successive rounding heuristics Fractional variables sequentially rounded until an integral solution or infeasibility is detected Equivalent to diving in the b&b tree: branching on a variable and examine only one child

49 Motivation for Branch and Cut IP Feasible Region X LP Relaxation Feasible Region x max 3x + 4x (M) 4x +3x 0 x {0,,,3}, x {0,} x max 3x + 4x (M) 4x +3x 0 x {0,,,3}, x {0,} x x Convex Hull of IP Feasible Region x max 3x + 4x (M) 4x +3x 0 x {0,,,3}, x {0,} x

50 Valid Inequalities and Cutting Planes LP Relaxation Feasible Region x If we have CH(X) we solve just one LP We need facet defining inequalities Valid Inequality Has been done for some problems In general very difficult Try to approximate instead x Find cutting planes Facet Cutting Plane 4x + 3x 0 x x x

51 Original inequality (i th row): Gomory Chvatal Inequalities Σ j a ij x j b i () If all the variables in () are non-negative integers then the GC inequality is given by: Σ j a ij x j b i () A GC inequality can be derived by the LP optimal simplex table from a row with a fractional variable Pure IP s can be solved by successively adding GC inequalities to the optimal LP solution Examples: x + x.5 x + x Cuts-off (0.75, 0.75) x x.5 x.5 Cuts-off (.5, 0).5x + x x + x Not tighter

52 Cover Inequalities Original knapsack inequality: Σ j J a j x j b, x j {0,} () Let C be a subset of J such that Σ j C a j > b. Then a valid cover inequality is given by: Σ j C x j C - x j {0,} () We have to separate cover inequalities Cover inequalities can be made tighter by lifting; i.e. including coefficients for j C. Example: 5x + 5x + 4x 3 8 x + x Cuts-off (, 0.5, 0), (0.5,, 0) x + x 3 Cuts-off (0,, 0.5), (0, 0.5, ) x + x + x 3 Lifted tighter inequality: Gives the Convex Hull

53 Example Revisited x max 3x + 4x 4x +3x 0 x {0,,,3}, x {0,} 4x +3x 0 0 x 3 0 x x Gomory-Chvatal Inequality: 4x + 3x 0 x x.5 x x.5 x Cover Inequalities: 4x + 3x 0 Three units of x is a cover: x Lifted cover inequality: x + x x x x x

54 Branch and Cut Algorithm z LP () z LP () CP() z LP (3) CP() Root Node () () (3) CP(3)

55 Branch and Cut Algorithm z LP (4) Root Node () (5) (7) x () (3) (4) x (6) (8) z LP (5) z LP (6) z LP (7) z LP (8)

56 MIP Computational History: : Dantzig, Fulkerson, S. Johnson: 4 city TSP Solved to optimality using cutting planes and LP 957: Gomory Cutting plane algorithm: A complete solution 960: Land, Doig; 965 Dakin Branch-and-bound (B&B) 97: MPSX/370, Benichou et al. 97: UMPIRE, Forrest, Hirst, Tomlin : Good B&B remained the state-of-the-art in commercial codes, in spite of 973 Padberg 974 Balas (disjunctive programming) 983 Crowder, Johnson, Padberg: PIPX, pure 0/ MIP 987 Van Roy and Wolsey: MPSARX, mixed 0/ MIP Grötschel, Padberg, Rinaldi TSP (0, 666, 39 city models solved)

57 MIP Computational History: 998 Date Linear programming Stable, robust dual simplex Variable/node selection Influenced by traveling salesman problem Primal heuristics 8 different tried at root node; retried based upon success Construction & improvement heuristics Relaxation-induced neighbor search Node pre-solve Fast, incremental bound strengthening (very similar to Constraint Programming) Presolve numerous small ideas Probing in constraints: x j ( u j ) y, y = 0/ x j u j y (for all j) Cutting planes Cliques, disjunctive, flow covers, Gomory, GUB covers, knapsack covers, implied bounds, mix-integer rounding, path cuts, etc. Aggregation

58 Computational Progress Algorithms Primal simplex in 988 versus best(primal,dual,barrier) today,360x Machines 800x Net: Algorithm * Machine ~,900,000x * Bixby, Fenelon, Gu, Rothberg, Wunderling. A New Generation of MIP Codes, IP Conference in Honor of Egon Balas, June , Carnegie Mellon University, Pittsburgh, PA

59 Computational Progress California Unit Commitment 7 Day Model UNITCAL_7: cons, 5755 vars (856 binary) Previous attempts (by model formulator): Day model: 8 hours, no progress 7 Day model: hour to solve initial LP CPLEX 8.0 on.0 GHz Pentium 4 Running defaults... Reduced MIP has 3887 rows, 9989 columns, and 056 nonzeros. MIP emphasis: balance optimality and feasibility Root relaxation solution time = 6.70 sec. Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt Gap e e e Cuts: e Cuts: e Cuts: e Cuts: e Cuts: * e e % * e e % * e e % * e e % * e e % * e e % GUB cover cuts applied: Clique cuts applied: 7 Cover cuts applied: 4 Implied bound cuts applied: 038 Flow cuts applied: 09 Flow path cuts applied: 5 Gomory fractional cuts applied: 5 Integer optimal, tolerance (0.000/e-06) : Objective = e+07 Current MIP best bound = e+07 (gap = 947.7, 0.0%) Solution time = 74.4 sec. Iterations = 7436 Nodes = 3575 (04)

60 A customer model: Computational Progress 44 cons, 5 vars, 67 nzs, maximization 5 general integer variables (and no bounds) Branch-and-cut: Initial integer solution Initial upper bound after.4 days, 3,000,000 B&C nodes, 5.5 Gig tree Integer solution and bound: UNCHANGED What s wrong? Free vars chase each other off to infinity. Maximize x + y + z Subject To x + y z = 0 x free y free x,y integer This problem can be solved in several ways Euclidean reduction on the constraint [Presolve] Removing z=0, objective is integral [Presolve] Locally valid cuts at the nodes However: Branch-and-bound cannot solve!!!

62 How Can We Solve Hard Problems? Traditional Approaches Better models Better solvers Customized methods Decomposition algorithms Problem specific algorithms New and emerging trend Exploiting computational power parallel computing

63 Better Models I The Pigeon Hole problem: There are n+ pigeons and n holes. We want to place all pigeons in the holes but in such way that no pair of pigeons is in the same hole. Infeasible, but how is the branch and bound tree? Let x ij = if pigeon i is in hole j. Assignment of every pigeon in a hole: Σ j x ij = i No pair of pigeons in the same hole: x ij + x kj j, i,k i<k Feasible LP relaxation: x ij =/n LP rel. feasible after branching Total enumeration is needed Alternative Formulation Assignment of every pigeon in a hole: Σ j x ij = i At most one pigeon at each hole: Σ i x ij j Replace n 3 constraints ( var s) with n constraints (n+ var s) INFEASIBLE LP relaxation

64 Better Models II The discrete time, finite horizon, economic lot sizing problem There is a given demand d t for each period t which can be satisfied by production in any period t t. For each period there is a production cost, a holding cost and a setup cost (if production occurs). y t : production in period t s t : inventory in period t x t = if production occurs at t y t D t x t = (Σ t =t..t d t )x t s t = s t- + y t d t t t Optimal LP relaxation: fractional: x t = y t /D t y tt : production in period t for demand at t x t = if production occurs at t Σ t=..t y tt = d t y tt d t x t t t, t t Optimal LP relaxation: LP solution is also MIP solution

65 Stronger Formulations Using Cutting Planes Ideas x + y 3.5, x, y 0, y integral y 4.0 (0, 3.5) x

66 Stronger Formulations Using Cutting Planes Ideas x + y 3.5, x, y 0, y integral y y Mixed-integer Rounding cut: x + y y x

67 How Can We Solve Hard Problems? Traditional Approaches Better models Better solvers Customized methods Decomposition algorithms Problem specific algorithms New and emerging trend Exploiting computational power parallel computing

68 How Can We Guide Search? Guess correct branching binary Heuristic: Choose fractional variable closest to 0.5 Does it always work? z 0 x Branching on x x Branching on x No, try to find the sensitivities for x and x. How???

69 Branching Using Pseudocosts z d u z d u Choose x although x is closer to x * x 0 x * x We want to choose the variable with the maximum min {d j, u j } Degradations d j and u j can be estimated from dual simplex method: = (reduced cost) change Choose the variable with the maximum estimated degradation

70 Strong Branching But the reduced cost (dual value) is valid only for small perturbations Can we do better than this, i.e. can we get a better estimate of the impact of adding x j = 0 and x j =? [0., 0, 0.5, 0.6, 0.8] 0 x 3 = 0 x3 = [0.,, 0, 0.5, 0.] [0.3, 0,, 0.4, 0.6] Choose a subset of fractional binaries For each chosen binary fix to x j = 0 and x j = and start solving resulting LP s. Stop after a fixed number of Simplex iterations Choose the binary that led to poor subproblems [0., 0, 0.6, 0.4, 0.3] [0., 0, 0.6, 0.4, 0.3] [0., 0, 0.6, 0.4, 0.3] [0., 0, 0.6, 0.4, 0.3] x = 0 x = x 3 = 0 x3 = x 4 = 0 x4 = x 5 = 0 x5 =

71 OK, What Can We Do Realistically? Pseudo-costs, strong branching, all variants of node selection rules (depth-first, best-first, etc.), heuristics and other exotic methods have been implemented in existing software Even if we knew how to implement all of them ourselves, it would be difficult to do it more efficiently than their implementation within CPLEX, Xpress, etc. But, knowing what these methods do allows us to use them, or know how/when to use them Main point of this subsection: Before you start developing a new method make sure that you are not re-inventing the wheel Understand the problem and which of the existing methods can help Find corresponding solver options Experiment

72 How Can We Solve Hard Problems? Traditional Approaches Better models Better solvers Customized methods Decomposition algorithms Problem-specific algorithms New and emerging trend Exploiting computational power parallel computing

73 Decomposition Basic Decomposition Idea Algebra x +x +x 3 = 6 x + x + x 3 = 6 3x +4x 3 = 7 x 3 +3x 4 + x 5 = 8 x 4 +x 5 = 8 x = x = x 3 = x 4 = x 5 =3 Optimization M M Underdetermined Many solutions Solve (M) & (M) many times Solution time: N (M) 0 = 04 sec (Μ) & (Μ) = 8 sec Key Issues How do we decompose? Which constraints do we keep/remove? How do we iterate? What type of information from (M) can we use to guide the solution of (M)? How do we pass this information? How do we solve models M and M? Submodels (M) and (M) may be different (e.g. MIP vs. LP) Do we have to use the same algorithm? If not, how do we make them compatible?

74 Lagrangean Relaxation Consider the problem: min z = c T x (P) s.t. Ax b () Dx d () x X where constraints in eq. () are nice, but constraints in eq. () are hard. Our goal is to develop an easy subproblem that does not violate eq. () substantially. min z = c T x+ u(dx-d) (LR) s.t. Ax b () x X, u 0 For every u 0, (LR) is a relaxation of (P) because a) the feasible region of (LR) is greater, and b) the objective value of (P) is always greater than the one of (LR) for all feasible solutions. Key Idea: Try to find the correct values for penalties u. What do the multipliers u look like? If a constraint in Dx d is violated, we would like to increase u. If a constraint in Dx d is violated by a solution of (LR), this constraint would be active in (P). u is the dual value of a violated constraint in (P).

75 Lagrangean Relaxation Implementation min z = c T x+ u(dx-d) s.t. Ax b () x X, u 0 (LR) Select K, ε, m k Set UB = +, LB= - Start with u 0 = 0 k = k+ Solve (LR) Obtain LB Return UB & Solution YES (LR) feasible? Optional Modify (LR) solution to obtain a feasible solution & UB NO YES UB-LB <ε or k=k? NO Update u k : u k+ = max {u k -m k (Dx-d),0}* m k is the step length * Subgradient optimization

76 Lagrangean Relaxation Choice of Dual Consider the problem: min z = c T x (P) s.t. A x b () A x b () A 3 x b 3 (3) x X Which constraints should be dualized? x CH x CH LR x CH LR P P P I II III P x P P I II LR P x II P LR P III P x (a) LP-relaxation of (M3): P=P I P II P III (b) Dualization of A III x= e III : P LR = P I P II (c) Dualization of A I x= e I : P LR = P II P III : feasible regions defined by subsets of constraints; : integer points; : feasible integer points; : convex hull of feasible integer points; : intersection of constraint subsets. Key issues: The strength of the resulting Lagrangian dual bound LB The ease of solution of the Lagrangian problem (LR)

77 Lagrangean Relaxation Remarks min z = c T x s.t. Ax b () Dx d () x X (P) Hard constraints are inequalities min z = c T x+ u(dx-d) s.t. Ax b () x X, u 0 (LR) min z = c T x s.t. Ax b () Dx = d () x X (P) Hard constraints are equalities? min z = c T x+ u(dx-d) s.t. Ax b () x X (LR)

78 Lagrangean Relaxation Example Structure of original model: Independent blocks with linking constraints Independent unit models + flow constraints Multi-period models with inventory constraints Assignment + sequencing constraints min z = c T x s.t. Ax b () D t x d t t=, T () x X Dualize linking constraints min z = c T x+ u(ax-b) s.t. D t x d t t=, T () x X, u 0 Lagrangian relaxation (LR) easy to solve

79 Bender s Decomposition Structure of original model: Independent blocks with linking variables Stochastic programming: here-and-now & wait-and-see decisions Hierarchical decisions How can we decompose the problem? Benders Decomposition max c 0 x + Σ k c k x x A 0 x = b 0 () A k x+ B k y k = b k, k=,,k (). Solve problem master problem (MP): A 0 x = b 0. Fix linking variables x 3. Solve K decomposed problems B k y k = b k A k x 4. Generate cuts and add them to (MP) Cuts are generated from dual information 5. Check convergence; if not go to Bender s decomposition lends itself well to parallelization

80 Consider the problem: Hybrid Methods min z = c T x s.t. Ax e () Bx + Dy f () x X, y Y where constraints in eq. () are hard for mathematical programming but easy if we use a different solution method. (P) 3 4 x y Hybrid Model min cx s.t. Ax e g(x,y) 0 x X, y Y Remove Constraints Add Constraints Relaxed Model (RM) min cx s.t. Ax e x X Subproblem (SM) min cx s.t. Ax e x = x k g(x,y) 0 x X, y Y Lower Bound Upper Bound (Feasible ) min cx (RM) s.t. Ax e x X x = x k g(x k, y) 0 y Y (SM) x X

81 How Can We Solve Hard Problems? Traditional Approaches Better models Better solvers Customized methods Decomposition algorithms Problem-specific algorithms New and emerging trend Exploiting computational power parallel computing

82 What Is Parallel Computing? Parallel Processing Multiple CPUs on a machine Multiple machines distributed globally Grid Computing Parallel Computing Cluster Computing Multiple machines located close by Distributed Computing Multiple machines distributed globally

83 Parallel Integer Programming LP-based branch-and-bound has inherent structure for parallelization Subproblems associated with each node are independent Evaluate nodes simultaneously Fundamental issue for parallelizing branch-and-cut: Management of cuts Minimize cut generation time: do not develop same cuts several times Maximize useful sharing: pass relevant cuts Avoid flooding the communication system with cuts Nodes solved at the master computer Information flow: master slave Subproblem formulation Variable bounds Information flow: slave master New feasible solutions Current lower bound Global cutting planes

84 Condor UW Madison Started in early 80s at CS Dept. Manages 000 workstations and delivers 650 CPU days on a day Gained popularity from around the world Industry, government, academia Matchmaker Problem Solver Agent Resource Shadow Sandbox

85 Implementation Challenges* Theoretical foundations well-known and well documented Many algorithmic and implementation issues need to be resolved Example: Cut Management The addition of cuts results in smaller trees BUT Extra time spent in generating cuts (regardless of whether all cuts are used) Extra time spent on solving LP s at each node Questions: Basic ideas: When to generate cuts and how many? Which cuts (if any) to add to the active linear problem? When to delete previously generated cuts? Generate cuts only every k nodes Perform only k rounds of cut generation Have only limited number of active cuts; keep inactive cuts in the cut pool Basic Management Scheme:. Solve the active linear program. Identify inactive cuts; delete them from the active LP; add them to the cut pool 3. Search the cut pool for violated cuts; add a subset to the active LP and go to 4. Generate violated cuts. Add them to the cut pool and go to * Johnson, E.L. Progress in Linear Programming-Based Algorithms for Integer Programming: An Exposition. INFORMS Journal on Computing, Vol., No, Winter 000

86 Applications Promising Research Areas* Robust models that integrate planning and operations Massively large, multi-period problems Multi-scenario stochastic problems; long solution times may be acceptable Recourse models that correct in real-time planning solutions Have to be solved in minutes Methods Parallel Integer Programming LP-based branch-and-bound is a natural candidate for parallelization Subproblems associated with each node are independent; evaluate nodes simultaneously Fundamental issue for parallelizing branch-and-cut: Management of cuts Integration between heuristic and cutting planes Integration with local search and constraint programming * Johnson, E.L. Progress in Linear Programming-Based Algorithms for Integer Programming: An Exposition. INFORMS Journal on Computing, Vol., No, Winter 000

88 Software Overview Papers usually focus on How_to_Solve_a_Problem How_to_solve_a_problem is only a single step of the overall process Modeling life-cycle. Develop the model. Generate instance(s) of the model 3. Solve the instance(s) 4. Verify the results/model 5. Modify the model Ideally, software has to nurture the entire modeling life cycle Two components of this ideal environment have received the most attention: The modeling module: The solver module: GAMS, AIMMS, Xpress-IVE, OPL Studio CPLEX, XPRESS-MP, OSL The two modules have to communicate with each other MPS format All modeling modules are able to produce an MPS input file and to process an MPS output file All solver modules are able to process an MPS input file and to produce an MPS output file.

89 Modeling Systems Modeling systems provide an interface with a variety of different solvers (algorithms). The user supplies a model (i.e. parameters, variables and constraints) using an easy language. The user-generated input file is compiled to generate a standard optimization file (e.g..lp,.mps) This file is passed to the appropriate solver The solution found by the solver is communicated back to the modeling system, and reported via an output file. A simple diagram for GAMS is shown below. Input file XXX.gms GAMS Output file XXX.lst User Solvers LP Solver NLP Solver MIP Solver MINLP Solver LP Solver NLP Solver MIP Solver MIP Solver 3

90 Solvers There are many different solvers for each problem (LP, NLP, MINLP, etc.) Different solvers are good for different classes of problems; e.g. MINOS vs. CONOPT for NLPs

91 Modeling Systems GAMS: AIMMS: Old language; limited flow control & visualization aids Compatible with many solvers New, user-friendly language; fancy interface Compatible with many solvers (not as many as GAMS) Xpress-IVE: Product of Dash Optimization Solvers of Dash Optimization Provides tools for algorithm customization OPL Studio: Product of ILOG Solvers of ILOG Provides tools for algorithm customization Enhancements Advanced users often want to specify branching rules, customize cut generation routines, etc. To support this functionality software developers provide callable library version of the software

92 Modeling Levels * Irv Lustig, Embedding CPLEX Algorithms Overview

96 Graph Theory A graph G is a pair (V, E) of a set of vertices V (points) and set of edges E (lines) Undirected graph: undirected edges, (i, j) = (j, i) Directed graph: directed edges, (i, j) (j, i) Network: directed graph with capacitated edges If i, j V, then (i, j) E is an edge; i is the tail, j is the head. u d z a b e v w c Graph G = (V, E ) V = {u, v, w, z} E = {(u, v), (u, w), (v, w), (v, z), (w, z)} E = {a, b, c, d, e}

97 Graph Theory A graph G is a pair (V, E) of a set of vertices V (points) and set of edges E (lines) Undirected graph: undirected edges, (i, j) = (j, i) Directed graph: directed edges, (i, j) (j, i) Network: directed graph with capacitated edges If i, j V, then (i, j) E is an edge; i is the tail, j is the head. Representation of Graphs and Networks A vertex i is incident to an edge (i, j) Incidence matrix: Nodes in Rows, edges in columns if a node is incident to an edge In directed graphs, incident matrix has a for the tail and + for the head u ), ( ), ( ), ( ), ( ), ( z w v u z w z v w v w u v u v z w a b c d e (4,) (4,) (3,4) (,3) (,),) ( Digraph G = (V, E ) V = {,, 3, 4} E = {(, ), (, ), (, 3), (3, 4), (4, ), (4, )} 3 4

98 Familiar Combinatorial Optimization Problems Traveling Salesman Problem There are n cities to be visited exactly once by a traveling salesman; the distance/cost from city i to city j is c ij. Find the Hamiltonian of minimum total distance/cost Problem appears in 830s; treated mathematically in 930s (Menger) One of the most intensively studied problem in optimization No effective algorithm exists It was shown it is NP-hard (Karp, 97) Integer programming formulation Subtour elimination: for every V,V : V = V V, V V = ensure there is an edge between V and V. Exponential number of subtour elimination constraints IP formulation very hard x ij = if we visit city j after city i min i, j c ij x ij x ji = i There is an incoming edge j xij = i There is an outgoing edge j Subtour elimination constraints x ij { 0,} i, j

99 Familiar Combinatorial Optimization Problems Matchings Given a graph G = (V, E), a matching M in G is a set of pairwise non-adjacent edges; i.e., no two edges share a common vertex. Find a matching that contains the maximum number of edges. A perfect matching is a matching which matches all vertices Matchings in bipartite graphs are related to the assignment problem Finding a maximum matching in bipartite graphs is easy Finding a maximum matching in general graphs is harder Polynomial algorithm for the general case by Edmonds (965) Jobs Machines Maximum matchings Perfect matching in bipartite graph: solution to assignment problem Integer programming formulation Formulation for (complete) bipartite graphs solved as LP Equivalent to assignment problem, which was solved effectively before IP formulations x ij = if edge (i,j) is selected max i, j j V i V c ij x ij x x ij ij = = i V j V x ij { 0,} i, j

100 Familiar Combinatorial Optimization Problems Matchings Given a graph G = (V, E), a matching M in G is a set of pairwise non-adjacent edges; i.e., no two edges share a common vertex. Find a matching that contains the maximum number of edges. A perfect matching is a matching which matches all vertices Matchings in bipartite graphs are related to the assignment problem Finding a maximum matching in bipartite graphs is easy Finding a maximum matching in general graphs is harder Polynomial algorithm for the general case by Edmonds (965) Jobs Machines Maximum matchings Perfect matching in bipartite graph: solution to assignment problem Integer programming formulation Formulation for (complete) bipartite graphs solved as LP Equivalent to assignment problem, which was solved effectively before IP formulations Odd-set constraints: for vertex subset U V: U > Problem solved as LP if all odd-set constraints are added x e = if edge e is selected max c x e e e x i e e δ ( i) x U e e E ( U ) x e {0, } e U V Odd set constraints

101 Integer Programming and Combinatorial Optimization Hard combinatorial optimization problems have also hard IP formulations Algorithmically well solved combinatorial problems have good IP formulations Network problems are easy also as LP/IP models Have other types of problems similar structures? Can we formulate other problems as network problems? Can we generalize what we learnt from networks to general MIP models? Why network problems are so easy?

102 Polytopes and Integrality Definition. A polyhedron P is the solution set of a finite system of linear inequalities, i.e. P = {x: Ax b}. Definition. A polyhedron P is a polytope (i.e. a bounded polyhedron) if there exist l, u R n such that l x u; i.e. P = {x: Ax b, l x u for l, u R n }. Definition 3. A nonempty polyhedron is integral if its vertices are integral. Remarks Polyhedron Polytope Integral Polytope P = {x: Ax = b} ) A set of inequalities can be converted into a set of equalities by introducing slack variables, and a set of equalities Ax = b can be converted into a set of inequalities ([A, A]T x [b b]t). ) A polyhedron can be described by a set of linear equalities, P = {x: Ax = b }, and a polytope can be described by a set of linear equalities and bounding inequalities, P = {x: Ax = b, l x u}. 3) A popytope can also be described as the convex hull of a finite number of points. Conversely, the convex hull of a finite number of points is a polytope. 4) A vector v of a polyhedron is a vertex (extreme point) if and only if it cannot be written as a linear combination of points in P\{v}. A polytope P is the convex hull of its vertices.

103 Polytopes and Integrality Theorem. The polyhedron P(A, b) = {x: Ax b, x 0} is integral for all integral vectors b Z m if and only if A is totally unimodular (Hoffman & Kruskal, 956). Proposition. If A is TU and vectors b, b, d and d are integral, then P(b, b, d, d ) = {x : b Ax b, d x d} is integral if it is not empty. Definition 4. An integral matrix A is totally unimodular (TU) if the determinant of each square submatrix of A is equal to 0,, or. Proposition. For every non singular submatrix R of a TU matrix, R is integral. Proposition 3. A {0, } matrix is an interval matrix (or has the consecutive s property) if for all rows with a lm = a l m =, l < l, implies a l m = for l < l < l. Interval matrices are TU (Fulkerson & Gross, 965). Proposition 4. If a {0, ±} matrix has no more than two non zero coefficients in each column, and Σ l a lm = 0 if column m has two coefficients, then A is TU (from Ghouila Houri, 96). P = {x: Ax = b} A totally unimodular Proposition 3 Proposition 4... Integral Polytope

104 Networks and Integrality Proposition 4. If a {0, ±} matrix has no more than two non zero coefficients in each column, and Σ l a lm = 0 if column m has two coefficients, then A is TU (from Ghouila Houri, 96). Incidence matrix of networks satisfies this condition Matrix A is used to express flow balance constraints in shortest path, max flow problems 4 3 A = 3 4 (,) (,) (,3) (3,4) (4,) (4,) Very few problems can be formulated using constraints that lead to TU matrices Side constraints destroy the structure Can we develop tighter formulations from combinatorial optimization insight? If yes, can we add these constraints in our formulation immediately?

106 Supply Chain Planning Matrix (Meyr et al., 00) procurement production distribution sales Long term Strategic Planning Medium term Short term Purchasing & Material Requirements Planning Master Planning Production Planning Scheduling Distribution Planning Transport Planning Demand Planning Demand Fulfillment Integration across time scales and functions Many new interesting problems Power industrial gases steel SC Oil industrial gases chemicals SC under power constraints Biomass to fuels and chemicals As chemical engineers, we know the specifics of chemical production Study all problems (time scales) of manufacturing function

107 Manufacturing Function Strategic Planning Network of plants/warehouses/markets What new plants/warehouses to build? What are the interconnections? Investment/retrofit costs Pacific Northwest Southwest Rockies Midwest Northeast South Southeast Manufacturing sites Warehouses Markets Production Planning Network of existing production facilities Where/when is a product produced? Production and inventory profile of products Transportation/holding costs Aggregate plant capacities Scheduling Single site/plant How do we optimally use our resources? Batch-sizes, storage policies, sequencing Utility/changeover costs Production (tn/month) Reactor Reactor Column Production Profiles Heating Pr. A Pr. B Pr. C Reaction Reaction Reaction Reaction 3 Separation Time (months) Time (days)

108 Scheduling Allocation of limited resources to tasks over time - Michael Pinedo Scheduling has been practiced in manufacturing sectors since early 0 th century First scheduling publications in the early 950 Naval Research Logistics Quarterly Closely related to the growth of computers and algorithms Research in 970s Complexity theory Scheduling has widespread applications Airlines industry Shipping industry Railways Manufacturing/production/any Industry Government / non-government Organizations Educational/Research Institutions Sports / Recreation Purely multi-disciplinary Computer Scientists, Operations researchers, mathematicians, business science, etc. Very challenging problem 8 unsolved problem instances in MIPLIB 003 Instance timtab (railway timetabling problem) has seen no optimal solution yet 0% gap even after several days, 3 Binaries, 38 Continuous

109 Single-Stage Scheduling N jobs to be processed in M machines Preliminaries p ij, c ij, s ii Assignment of jobs to machines Sequencing of jobs in the same machine M task task task3 M task4 task5 Resources (e.g. machines) M3 task task4 M4 task task task5 M5 task4 task task Time (min/hr/day)

110 Classes of Scheduling Problems α / β / γ Machine environment Single machine Parallel machines Machines in series Processing characteristics Preemption Release/due times Setup times Objective Makespan Tardiness Cost jobs A d A B C M B D A C D Single machine tardiness jobs A B M M A C Sequence-dependent Setup time C D M M B Parallel machines D makespan

111 Chemical Production Scheduling Jobs Batches (sometimes orders) Operations Stages or Tasks Machines Units Modeling of storage is important (solid, liquid and gas phases) Storage vessels & states Utilities are also important (steam, water, electricity) Utilities (or resources) No Preemption Variable Processing Times Changeover times

112 Classification of Chemical Production Scheduling Problems Multi-stage processes Orders st stage nd stage K th stage (batches) J J J K Finished Orders Sequential structure: Orders := batches i I Stages k K = {,, K} Units j J = J J J K Special Cases: Single-stage: K = Flow-shop: J = = J K = Network (multi-purpose) processes Batch splitting/mixing; recycle streams Utility requirements F S Int S3 ADD S4 F 5% 0% S5 Int 50% S6 95% 90% 50% Links between seemingly independent tasks Need to account for variable no of batches via common time representation P P P3

113 Batch based Approaches Goal: Exploit sequential structure of multi stage processes Approach: Predefine number of batches (lot sizing, batching, task selection) Solve fixed scheduling problem Demand (orders) A B Batching Batches A B... Scheduling Batches i I Stages k K Units j J C C Scheduling Formulations Precedence based Pinto & Grossmann, 995; Lamba & Karimi, 00 X ij = if task i is assigned to unit j Y ii j = if batch i precedes batch i in unit j T ik = Finish time of batch i in stage kv Slot based Hui & Gupta, 000; Mendez et al. 00 X ij = if task i is assigned to unit j Y ijm = if batch i is assigned to slot m of j T ik = Finish time of slot m of unit j Tt ik = Transfer time k k+ Tasks (i) X ij Units (j) Y ii j Tasks (i) X ij Y ii j

114 Network based Approaches Goal: Model multi-purpose batch processes Account for batch splitting/mixing; recycle streams; utility requirements Approach: Use a common time coordinate Express assignment and balance constraints for every time point/period/event Discrete-time Formulations Kondili et al. (993); Pantelides (994) RM T Int T T3 A B Time points/periods t T.0 hr.5 hr Δt = 0.5 hr 3.0 hr Tasks i I Units j J States s S Utilities u U T T T t (hr) W ijt = if task i starts in unit j at time point t B ijt = Batch-size of task i in unit j at time point t S st = Amount of state s at time point t = Consumption of utility u at time t U ut Assignment: Mass Balance: Utility: ST U i I st ut j i T s j K i i T s j pi MAX = ( αui θwijt θ + βui θ Bijt θ ) U u t pi + MIN MAX W j, t tˆ t ij tˆ W = ijtvij Bijt WijtVij i, t, j Ki MAX = STst + ρis Bij, t p is Bijt Ss s, t is ρ t j K i θ = 0 K i u, t Variables defined t T Constraints expressed t T Large MIP models Constant processing times

115 Modeling Problems. Singe-unit problem Find the schedule that yields the minimum total completion time (makespan) Batches i I = {,,, N} Processing time: τ i Release/due times: r i /d i A B C D M.. Single-unit with release/due times Same as, but batches have release/due dates. The objective is the minimization of makespan.. Single-unit and changeover times Same as but there are changeover times between batches The objective is the minimization of makespan.3. Single-unit and changeover costs Same as. but there are changeover costs between batches The objective is the minimization of cost

116 Modeling Problems. Singe-stage (multi-unit) problem N batches have to be processed in a single stage that consists of M parallel units. Find the schedule that yields the minimum total completion time (makespan) Batches i I = {,,, N} Units j J = {,,, M} Processing time: τ ij Processing cost: c ij Release/due times: r i /d i A B C D M M.. Single-stage problem with release/due times Same as, but batches have release/due dates The objective is the minimization of lateness/tardiness.. Single-stage problem with changeover costs and times Same as., but there are changeover costs γ ii j and changeover times δ ii j between batches The objective is the minimization of cost (subject to meeting the release/due times)

117 Modeling Problems 3 3. Multi-stage problem N batches have to be processed in K stages; each stage has parallel units. Find the schedule that yields the minimum total completion time (makespan) Batches i I = {,,, N} Units j J = {,,, M} Stages k K; J k is the set of units in stage k Processing time: τ ij Processing cost: c ij jobs A B C D M M M3 Release/due times: r i /d i 3.. Multi-stage problem with release/due times and changeover times/costs Same as 3, but batches have release/due dates and changeovers Model the minimization of makespan, tardiness, cost and earliness

118 Modeling Problems 4 S h Heat S h Reaction 3h Reaction 30% S3 40% 70% 60% S5 S6 0% h Separation h Reaction 3 90% S4 S7 States Tasks Units Reactor Reactor Column Heating Reaction Reaction Reaction Reaction 3 Separation Inventory Time (h) S S Time (h)

119 Discrete time Formulations Kondili et al. (993); Pantelides (994) T T T t (hr) Fixed time points Constant processing times Scheduling: Modeling Continuous time Formulations Global grid: Pantelides et al.; Reklaitis et al.; Lee et al.; Barbosa Póvoa et al.; Grossmann et al.; Karimi et al. Unit specific grid (events): Floudas et al.; Ierapetritou et al.; Giannelos & Georgiadis T T T (periods) Variable time points Variable processing times Mixed time Formulation Maravelias (006) T T T t (hr) Fixed time points Variable processing times

120 Discrete time Formulations Kondili et al. (993); Pantelides (994) T T T t (hr) Fixed time points Constant processing times Scheduling: Modeling Continuous time Formulations Global grid: Pantelides et al.; Reklaitis et al.; Lee et al.; Barbosa Póvoa et al.; Grossmann et al.; Karimi et al. Unit specific grid (events): Floudas et al.; Ierapetritou et al.; Giannelos & Georgiadis T T T (periods) Variable time points Variable processing times Mixed time Formulation Maravelias (006) T T T t (hr) Fixed time points Variable processing times

121 Are Smaller Formulations Better? RMAB RMC I I ABI CI ABI CI INTABI II INTCI BII CII Continuous processing Five products - three stages Time Horizon: weeks Intermediate due dates, shipment dates Backlogged demand III INTBII INTCII AIII BIII CIII BIII CIII III Demand Shipments Backlogs A B C B C Equipment Unit Gantt Chart I CI ABI CI AB I ABI CI ABI A II CII BII CII BII CII B III AIII BIII AIII BIII CIII AIII C III CIII BIII CIII BIII CIII Idle t (d)

122 Effect of Strong Valid Inequalities Basic STN Formulation (M0) (C) (C) (C3) (C4) (C5) (C6) (M0) (M) (M) (M3) (M4) (M5) (M6) +C +C +C3 +C4 +C5 +C6 # Const.,907,043,043,907,907,975,043 LP-relax,8.8,054.4,66.7,73.,30.5,76.6,060.3 Objective,766.7,766.7,766.7,766.7,766.7,766.7,766.7 CPU-sec LP-gap (%) Nodes 5,689 6,40, ,0,97 Inequalities (C), (C), (C3), (C4) and (C6) appear to be the most effective Combinations of these inequalities are studied further (M0) (M) (M) (M3) (M4) (M6) + C, C, C6 + C, C, C3 + C, C, C4 +C-4 + C3, C4, C6 +C-4, C6 # Const.,35,79,79,47,,383 LP-relax,037.3,964.0, ,046.6,964.0 Objective,766.7,766.7,766.7,766.7,766.7,760.0 CPU-sec LP-gap (%) Nodes Models (M), (M), (M3) & (M6) are studied further using a set of six problems Model (M6) appears to be the most effective

123 Branch and Cut Algorithm: Basic Idea Additional improvements needed for practical applications How can we use remaining valid inequalities? Branch and cut algorithm Advantages:. Do not load unnecessary constraints Smaller incidence matrix Each node is solved faster. Use a larger pool of constraints Tighter formulation after a few rounds of cut generation Combines effective (LP) solution and tight formulation Disadvantage: Very difficult to compete with CPLEX, XPRESS Do not exploit built in cut generation routines Have to write new algorithm to add user cuts Branch on one fractional variable Add violated cuts Find new (fractional) solution Pruned node Open node

124 Branch and cut Algorithm: Implementation Implementation in XPRESS IVE using XPRESS Mosel Do not remove/change pre solve and cut generation routines Add few cuts (C, C3 & C5) at the root node to facilitate heuristics Add violated high priority cuts (C, C4, C6 & C7) at the root node Allow addition of high priority cuts at nodes of intermediate depth Add low priority cuts (C8 C3) later Add all C, C3 & C5 constraints in the formulation Add violated C, C4, C6 and C7 constraints in k rounds at the root node Add violated constraints C, C4, C6 and C7 in nodes of depth k k k k3 Add violated constraints C, C4, C6, C7 and C8 C in nodes of depth k k3 k k4

125 Remarks: Branch and cut Algorithm: General Remarks. Parameters k, k, k3 and k4 depend on the size of the problem Typical values: 5 0 iterations at the root node Start adding high priority cuts after nodes, at depth 0 00 Start low priority cuts at depth 00,000. Algorithm requires fewer number of nodes, but is expensive at each node To reduce overhead: Add cuts every 5 0 nodes Add first 4 violated cuts 3. On the fly tuning necessary

126 Customized Methods Precedence based Pinto & Grossmann, 995; Lamba & Karimi, 00 X ij = if task i is assigned to unit j Y ii j = if batch i precedes batch i in unit j T ik = Finish time of batch i in stage kv Tasks (i) X ij Units (j) Y ii j What is the problem? Sequence-based: T T + τ X M ( Y ) ( i, i' ) IL, k K Tt Tt i' k ik Slot-based: ij j JA( i, k ) ij ii' k i, k Ts jm + M ( U ijm ) i, k >, j J k, ik jm ij ijm ijm m M Ts + τ U M ( U ) i, k < K, j J, m M Sequencing is achieved via big-m constraints Resulting MILP formulations are not strong k j j Compared to manufacturing problems:. Unknown type and number of batches (tasks); unknown assignments of tasks to units. Mixing of intermediates; variable batch-size and processing time There are good algorithms for problems with fixed type and number of tasks and fixed assignments Can we combine these algorithms with MILP methods? Decompose problem in two subproblems. Determine type and number of tasks and assignments of units to tasks. Solve reduced problem with an efficient, problem-specific algorithm

127 Constraint Programming Preliminaries New Modeling and Solution Paradigm Developed in early 90 s in AI Very effective for classes of optimization problems Highly constrained (feasibility) problems Some scheduling problems Special constructs and constraints for classes of problems Constructs: activity X, unary resource Y Constraints: X requires Y (GLOBAL) A B, A B (LOGIC) Highly Expressive Effective local search Search is based on constraint propagation

128 Constraint Programming Basic Idea Problem Find the order at which jobs A, B and C are sequenced on machine M Job Processing Time Release Due A 0 5 B 4 9 C 3 7

129 Constraint Programming Basic Idea Problem Find the order at which jobs A, B and C are sequenced on machine M Job Processing Time Release Due A 0 5 B 4 9 C 3 7 A {,,3} B {,,3} C {,,3} A A A 3 B B B 3 B B B 3 B B B 3 C C 3 C C 3 A B C C A C B

130 Constraint Programming Basic Idea Problem Find the order at which jobs A, B and C are sequenced on machine M Job Processing Time Release Due A 0 5 B 4 9 C 3 7 A must be st C must be nd A {,,3} B {,,3} C {,,3} A B B 3 C 3 A B C C A C B

131 Mathematical vs. Constraint Programming Mathematical Programming Intelligent search strategy but computationally expensive for large problems Computationally effective for optimization problems with many feasible solutions Not effective for feasibility problems and machine sequencing problems Constraint Programming Fast algorithms for special problems Computationally effective for highly constrained, feasibility and machine sequencing problems Not effective for optimization problems with complex structure and many feasible solutions MAIN IDEA Decompose problem into two parts Use MP for high level optimization decisions Use CP for low level sequencing decisions

132 Decomposition Strategy Solve MIP Master Problem max production s.t. RELAXATION Obtain UB Fix no/type of tasks, assignment to units Add integer cuts Solve CP Subproblem max production s.t. ALL CONSTRAINTS w/ fixed no/type of tasks Obtain LB Express problem in an aggregated MP form Use MP to identify potentially good solutions Fix no/type of tasks, assignment of tasks to units Fix no/type of tasks and assignment decisions Problem is highly constrained: suitable for CP If feasible, obtain lower bound Add integer cut and continue until bounds converge Production Z* Upper bound Feasible solution Iterations

133 Example Food (5) Coater () Packer (3) Flour T0 T0 T30 Flakes Loops T T3 Flakes Coated Flakes Coated Loops T T T3 Corn Flakes Frosted Flakes Froot Loops Unlimited Storage Finite Storage Zero Wait Maximize Production MILP STN Model Horizon n 0- Cont. Constr. LP Relax Obj CPU s ,63, * 6, Hybrid MILP/CP # iters Obj CPU s Minimize Makespan MILP STN Model Demand n 0- Cont. Constr. LP Relax Obj CPU s 4 / 5 / ,077 5 / 6 / >36,000 5 / 8 / >36,000 Hybrid MILP/CP # iters Obj CPU s

134 General Hybrid Algorithm S Multipurpose Batch Plant Reaction Heat S 70% Reaction 30% S3 0% Separation 40% 60% Reaction 3 S5 S6 90% S4 S7 i I ( j) B S S S MIN i s s s c Z D ic ic Z ic MS ST j EST MAX B B Z i, c ic O I = S + ρ B ρ B 0 d s C s i c is i ic s FP s INT ic i c is ic j j s Bi R MAX Bic Bi i, c = α + β B i, c MIN B i ic c i O ics i d s ic s FP Task[i,c] requires Unit[j] j, i I(j), c Task[i,c] requires R ic Utility[r] i, c Task[i,c] consumes ρ c is B is State[s] i, c, s Task[i,c] produces ρ p is B ics State[s] i, c, s Task[i,c].end MS i, c Multi-stage: no utilities unlimited storage F F F3 T0 T0 T30 S0 S0 S30 T T T3 S S S3 T T T3 P P P3 i I ( j) B S S MIN i s s c Z ic D ic Z ic MS ST j EST MAX B B Z i, c ic = S0 + Bic d s i c i ic s FP i c j B ic j s B MIN i i O( s) MAX B B i, c c B is ic d s i s FP Task[i,c] requires Unit[j] j, i I(j), c Task[i,c] precedes Task[i,c] (i,i ) A, c Task[i,c].end MS i, c Single-stage Drop stage index k Doj X oj maxo O { do} mino O { ro } j o O ( j) Zoj = j o Preprocessing: Better Cover Cuts Task[o,j].start r o o, j Task[o,,j].end d o o, j Task[o,j] requires Unit[j] o, k, j Z ok = Task[o,k,j].end MS o, k, j Z ok =

135 Integration with Problem specific Algorithms Example: Minimization of cost of multi-stage problem for orders with release and due times N orders have to be processed sequentially in K stages, where each stage consists of M k units. Each order i has release r i and due d i time, and a processing cost c ij and processing time τ ij. The objective is to minimize the sum of processing costs subject to meeting the release and due times. Master Problem: Assignment Subproblem: Sequencing Subproblem is a traditional OR problem (job-shop problem) There are efficient algorithms Use Shifting Bottleneck Procedure (Adams and Balas, 988) to solve the subproblem Computational Results Obj MIP CP MIP/CP: CUT H MIP/SBP CPU s CPU s Iter s/cuts CPU s Iter s/cuts CPU s PD / /4.3 PD /3 0. /0 0. P3D / /3 6.9 P3D /5 0.4 /. P4D / 8.8 /5 7.8 P4D /6.0 6/4 9.8 P5D / /5 7. P5D / 3.7 / 0.4

136 Exploit Parallel Computing Computer manufacturers focus on multi core architectures Computational resources become cheap Algorithms that utilize new architectures or exploit parallelization necessary Modeling languages provide tools for parallel algorithm development Parallel Processing Multiple CPUs on a machine Multiple machines distributed globally Grid Computing Parallel Computing Cluster Computing Multiple machines located close by Distributed Computing Multiple machines distributed globally

137 Parallel Computing & Integer Programming LP based branch and bound has inherent structure for parallelization Subproblems associated with each node are independent Evaluate nodes simultaneously; share cuts and bounds Nodes solved at the master computer Information flow: master slave Subproblem formulation Variable bounds Information flow: slave master New feasible solutions Current lower bound Global cutting planes Fundamental issues for parallelizing branch and cut Generate good subproblems: not too easy or too hard Minimize subproblem (task) and cut generation time Maximize useful sharing: pass bound information relevant cuts

138 How Can We Use Grid Computing? Master-worker Paradigm Master generates and spawns all tasks (subproblems) tasks can be user-defined or generated automatically Workers execute tasks and return results Master collects results Master Subproblems. Independent (e.g. scenarios in stochastic programming) Submission Loop: modelname.solvelink=3; loop(scenario, parameter = spara(scenario); parameter = spara(scenario); solve modelname using mip minimizing objective; h(scenario) = modelname.handle; ); Workers. Solver generated Subtrees resulting from strong branching 3. User generated Manually fix binary variables (Bussieck et al., 007)

139 Optimization on the Grid. Master creates a separate directory for each subproblem Unique handle for each subproblem finished file when task completed trigger file when new incumbent. Condor ships the subproblems to workers; executed on a Sandbox 3. Master collects results using handlecollect Repeatedly checks for completion; collection time can be adjusted 4. Update on incumbent solution using utility condor_chirp If new global incumbent is found, trigger file is created It also puts the new incumbent to the subproblem s directory in Master computer. Separate directory for each subproblem Condor Worker Sandbox Master Worker Sandbox Worker N Sandbox finished file upon completion of subproblem trigger file is created if new incumbent is found condor_chirp utility Fetch: copies trigger file Remove: removes trigger file after copying Put: places new incumbent in directory of master

140 Dynamic Decomposition. Run branch-and-bound on master to generate 400 open nodes. Spawn open nodes to servers 3. Communicate bounds and incumbents every 5 min; wait for hr Integer solution Pruned node Fractional (open) node

141 Dynamic Decomposition. Run branch-and-bound on master to generate 00 open nodes. Spawn open nodes to servers 3. Communicate bounds and incumbents; wait for hr 4. Decompose hard problems dynamically again using strong branching Integer solution Pruned node Fractional (open) node

142 Results Example 0 orders, 3 stages, units/stage; dissimilar units, forbidden assignments and paths Two instances parameterized via demand Instance : ~0 batches Strong Branching to generate 400 subproblems 3 hr of CPU time, hr of wall clock time,905,74 nodes Instance : ~30 batches Strong Branching generated too many subproblems Filled the disk of the submitting machine! Failed in solving this hard problem A better scheme is needed to generate subproblems

143 Exploit Problem Structure Hierarchy of decisions: Z il = if batch l is selected towards order i X (i,l)j = if batch (i,l) is assigned to unit j Y (i,l)(i,l )k = if batch (i,l) preecedes batch (i,l ) in stage k X ilj j JA( i, k ) = Z il i I, l l max i, k K ( i, l, i' l' ) IL, k j X ilj + X i' lj Yili' l' k + Yi ' l ' ilk, Batch Selection Z il Batch assignment X ilj Batch sequencing Y ili l k Jobshop problem Traditional Multi-stage scheduling

144 Domain based Dynamic Decomposition P Master computer Decomposition generates subproblems Fix Z il P P P 3 P M Subproblems solved in workers Collect and updates are used P 4 Π = {P, P,. P M } Master collects unsolved problems; nd -level decomposition by fixing X ilj in one stage P P P 3 Promising Non-promising 4 PM() P m Π = UΠ, m =,,.M where m Π = {P, P, P,...} m m m m 3 Fix X ilj Process repeated as necessary 4 P, 4 P, Promising 4 P,3 4 P,M3(4) Non-promising Few st level subproblems are promising (and hard); reasonable batching nd level decomposition can potentially generate too many subproblems Process can be combined with strong branching m Π3 = UΠ3s, m =,,.M m m m m where, Π 3 = {P,P,P,...} m,s s s, s, s,3

145 Results Example 0 orders, 3 stages, units/stage; dissimilar units, forbidden assignments and paths Two instances parameterized via demand Instance : ~0 batches Decomposition with strong branching 3 hr of CPU time, hr of wall clock time,,905,74 nodes st level decomposition followed by strong branching 7.5 min of wall clock time; 9,60 nodes Instance : ~30 batches Decomposition with strong branching Failed to solve the problem st, nd, 3 rd level decomposition (Z il + stages) followed by strong branching 6 nd level unsolved problems 56,68 3 rd level subproblems 6 3 rd level unsolved subproblems 3,76 subproblems 7.7 days of CPU time; 9 hr of wall clock time;,065,793 nodes

147 Supply Chain Planning Matrix (Meyr et al., 00) procurement production distribution sales Long term Strategic Planning Medium term Short term Purchasing & Material Requirements Planning Master Planning Production Planning Scheduling Distribution Planning Transport Planning Demand Planning Demand Fulfillment Integration across time scales and functions Many new interesting problems Power industrial gases steel SC Oil industrial gases chemicals SC under power constraints Biomass to fuels and chemicals Integration of production planning and scheduling Exxon Mobil: % reduction in operating costs, 0% inventory reduction (Shobrys & White, 00) DuPont: Capital tied up in inventory reduced from $65 to $90 million (Shobrys & White, 00)

148 Problem Statement Given: Planning horizon divided into T uniform or non uniform time periods t {,, T}. Products (items), i I = {A, B, } with demand δ it and holding cost θ i. Resource constraints: unit and utility capacities, raw material availability, etc. Production costs: these may include variable and fixed costs. Determine Production P it and inventory levels I it to minimize cost Manufacturing Facility P P P P P P P i, P i, P it I i,0 I i, I i, I i, t I it δ i, δ i, δ it Warehouse Distribution Center Production planning provides production targets (set points) for each period P P P P P P P P t = 0 t- t

149 Why Integration with Scheduling? RM 50 P 00 (kg/d) 00 P 300 (kg/d) U U INT (ZW) P 3 = 50 (kg/d) U3 A B P B,000 AG: P A +P B,00 P B P B,000 U: P A +P B,00,000,000,000 U3: P B,050 U: P A,400,000,000,000 P A (a) Bottleneck constraint,000,000 P A (b) Unit capacity constraints,000,000 P A (c) True feasible region (via scheduling) Bottleneck and unit capacity (aggregate) constraints insufficient Chemical processes: batch splitting/mixing, recycle streams, utilities, storage constraints Scheduling models needed to provide feasibility and cost information 4

150 z = min Cp + t t Cht Iit = Ii, t + Pit δit, i, t Ch ( P, i I) 0, t = g( P, i I), t f it Cp t t, 0 P it I it it General Production Planning Formulation ( ) = θ I, i i it Batching t Demand (orders) A B C D E Hidden scheduling problem A A A3 B B C D D Batch unit Assignment B A3 A B C D D A I i, t P it δ it P P How much can we produce and at what cost? Satisfy demand? I it At what (inventory) cost? Scheduling Given: A production facility (processing units, storage vessels, utilities) & processing cost (feedstocks, utilities, etc) Production recipes (stoichiometric coefficients, processing times, utility requirements) Production targets Determine the optimal: Number and type of processing tasks (batches and lots) Allocation of resources to processing tasks Sequencing of tasks on processing units Batches E E U U Sequencing & Timing A A A3 C D D B B E

151 Planning Scheduling Integration z = min Cp + t t Cht Iit = Ii, t + Pit δit, i, t Ch t ( ) = θ I, i i it t FP Pkt = Skn, k K, t, n = N Planning Linking Optimization n' n k K n' > n τ j W i kn' j, n Integrated Formulation β S MIN k in Cp W = S t kn B i, n + kn k P β i ρ ik MAX k B W k, n τ kn i + k, n k C ( γ W ε B ) = + k n k kn k kn i ρ ik B kn ζ MAX i k, n Scheduling P, I 0, W {0,}, B, S 0 kt kt in in kt Planning time grid (big-bucket H t ) Weeks-months t=0 H Pi, P i, Planning Scheduling time grid (small-bucket h n ) Hours-days S i, N n=0 N N+ N h n S i, N Scheduling

152 Challenges in Chemical Production Planning Modeling Different production environments Wide range of processing constraints Integration with multiple planning functions Develop general modeling framework Theory Discrete decisions lead to mixed integer programming models Existing hardware and software insufficient Develop fundamental optimization theory Use underlying theory to develop better solution methods Solution Methods Applications: optimize over long horizon optimize multiple systems simultaneously Intractable large scale models Develop general & computationally effective methods Projection based strategy

153 The Modeling Challenge No batch mixing Cleaning in place between batches Cooling utilities Yeast requirements Release time for media Batch processing FERMENTACIÓN Semi continuous processing D. W. DILUTION TANQUES DE GOBIERNO I TANQUES DE GOBIERNO II Continuous processing Litro 500 BPM ¼STD 700 BPM ½STD 600 BPM ½PREMIER 600 BPM ½ T.A., ¼ BOH. 750 BPM Products grouped in families Changeover times & costs between families Orders at due dates Inventory constraints Unit product compatibility Storage constraints Litro 350 BPM REPOSO LATA 00 LPM TANQUES DE GOBIERNO ½T.A EXP TANQUE REPOSO 750 BPM Fermentation Filtering Storage Bottling

154 Modeling Schools of Thought Sequential processing Orders st stage nd stage K th stage (batches) J J J K Finished Orders Operations follow a defined sequence The same batch/lot processed in all stages No batch mixing (blending) or splitting allowed Common in biotech, specialty chemical, and pharmaceutical industries Batch based methods Network processing 40% A RM RM 40% Int 60% 60% Int3 80% Int ImB 0% 0% 90% B Batch splitting/mixing allowed Recycle streams, utility requirements Common in bulk chemicals & oil industries Material based methods RM3

155 Implications of Modeling Diversity Network M 30% 60% Feed intermediate 40% Mixer U Feed Feed 3 R 70% Reactor U R Reactor U3 P P R3 Reactor U4/U5 R4 Reactor U4/U5 Sequential Intermediate Intermediate S Separator U6/U7 S Separator U6/U7 P3 (5 orders) P4 (7 orders) Traditional process representation S 60% Material based S4 {U} S3 70% 30% T T 40% {U, U3} S {U, U3} T3 S5 S6 Batch based U4 U6 U5 U7 Stage Stage S4 S {U} 70% S3 60% 30% T T S5 40% {U, U3} S {U, U3} S6 T3 Sundaramoorthy & Maravelias, 0 {U4,U5} T4 {U4,U5} T6 S7 S9 {U6,U7} T5 {U6,U7} T7 S8 S0 Network task Network state Hybrid state Sequential task Sequential state

156 TANQUES DE GOBIERNO Industrial Application Processing Stages: Fermentation Filtering Storage Bottling Production Type: Batch, continuous, and semi continuous Planning Horizon: 6 weeks Production Environment: Fermentation Filtering Storage Bottling Litro 500 BPM FERMENTACIÓN ¼STD D. W. 700 BPM DILUTION TANQUES DE GOBIERNO I TANQUES DE GOBIERNO II ½STD 600 BPM ½PREMIER 600 BPM ½T.A., ¼BOH. 750 BPM Litro 350 BPM 8 processing units (lines) products families 5 product subfamilies 6 products REPOSO LATA 00 LPM ½T.A EXP TANQUE REPOSO 750 BPM Product families: Products (for bottling) are grouped into families Changeover costs/times between families; setup costs/times between products Products belong to subfamilies Product family major changeover Product family Kopanos et al.,, 0 Product (item) minor changeover (sequence independent)

157 Industrial Application: Executed Schedule

158 Industrial Application: Executed Schedule

159 Industrial Application: Integrated Appoach No memory across planning periods

160 Industrial Application: Integrated Approach No memory across planning periods Minimize setup time & costs

161 Industrial Application: Results Executed Schedule

162 Industrial Application: Results Schedule Found Using Integrated Framework

163 Comparison with Implemented Solution Industrial Application: Results

164 FERMENTACIÓN REPOSO TANQUE REPOSO D. W. TANQUES DE GOBIERNO I TANQUES DE GOBIERNO II TANQUES DE GOBIERNO Litro 500 BPM ¼STD 700 BPM ½STD 600 BPM ½PREMIER 600 BPM ½ T.A., ¼ BOH. 750 BPM Litro 350 BPM LATA 00 LPM ½ T.A EXP 750 BPM Challenges in Chemical Production Planning Modeling Different production environments DILUTION Wide range of processing constraints Integration with multiple planning functions Develop general modeling framework Theory Discrete decisions lead to mixed integer programming models Existing hardware and software insufficient Develop fundamental optimization theory Use underlying theory to develop better solution methods Solution Methods Applications: optimize over long horizon optimize multiple systems simultaneously Intractable large scale models Develop general & computationally effective methods Projection based strategy

165 Polyhedral Background Summary Definition. A polyhedron P is the solution set of a finite system of linear inequalities, i.e. P = {x: Ax b}. Definition. A polyhedron P is a polytope (i.e. a bounded polyhedron) if there exist l, u R n : l x u; i.e. P = {x: Ax b, l x u for l, u Rn}. Remark : A set of inequalities Ax b can be converted into a set of equalities Ax + Is = b, s 0. A polytope can be described by as P = {x: Ax = b, l x u}. Remark : A vector v of a polyhedron is a vertex (extreme point) if and only if it cannot be written as a linear combination of points in P\{v}. A polytope P is the convex hull of its vertices. Definition 3. A nonempty polyhedron is integral if its vertices are integral. Definition 4. An integral matrix A is totally unimodular (TU) if the determinant of each square submatrix of A is equal to 0,, or. Theorem. If A is TU and vectors b, b, d and d are integral, then P(b, b, d, d ) = {x : b Ax b, d x d} is integral if it is not empty (Hoffman & Kruskal, 956). Theorem. If a {0, ±} matrix has no more than two non zero coefficients in each column, and Σ l a lm = 0 if column m has two coefficients, then A is TU (Ghouila Houri, 96). Theorem 3. A {0, } matrix is an interval matrix (or has the consecutive s property) if for all rows with a lm = a l m =, l < l, implies a l m = for l < l < l. Interval matrices are TU (Fulkerson & Gross, 965). Polyhedron P = {x: Ax = b} Polytope P = {x: Ax = b, l x u} Integral Polytope P = {x: Ax = b, l x u} Theorem A totally unimodular Theorem Theorem 3... MIP Formulations

166 Polyhedral Background Summary Definition. A polyhedron P is the solution set of a finite system of linear inequalities, i.e. P = {x: Ax b}. Definition. A polyhedron P is a polytope (i.e. a bounded polyhedron) if there exist l, u R n : l x u; i.e. P = {x: Ax b, l x u for l, u Rn}. Remark : A set of inequalities Ax b can be converted into a set of equalities Ax + Is = b, s 0. A polytope can be described by as P = {x: Ax = b, l x u}. Remark : A vector v of a polyhedron is a vertex (extreme point) if and only if it cannot be written as a linear combination of points in P\{v}. A polytope P is the convex hull of its vertices. Definition 3. A nonempty polyhedron is integral if its vertices are integral. Definition 5. A rational matrix is κ regular if for all its non singular square submatrices R, κr is integral. Theorem 4. If A is an m n integral matrix, then the polyhedron P(A, b, b, d, d) = {x: b Ax b, d x d} is integral for all a,b κz m and l, u κz n, iff A is κ regular (Appa and Kotnyek, 004). Proposition.Let A be a rational matrix and σ and κ positive integers. Then σ A is κ regular iff A is σκ regular. Proposition.Let A be κ regular. Then the following matrices are also κ regular: i) the transpose of A, ii) any submatrix of A, iii) the matrix obtained by multiplying a row or column of A by, iv) the matrix obtained by interchanging two rows or columns of A, v) the matrix obtained by duplicating a row or column of A, vi) the matrix obtained by dividing a row or column of A by a non zero integer. Theorem 4 k regular Polyhedron P = {x: Ax = b} Polytope P = {x: Ax = b, l x u} Integral Polytope P = {x: Ax = b, l x u} Proposition Proposition... MIP Formulations

167 Discrete time MIP Scheduling Model Given are a set of tasks i I, processing units j J, chemicals (states) k K, and a scheduling horizon η. A processing unit j can be used to carry out tasks i I j. The fixed production rate of task i is ρ i. A chemical can be consumed (produced) by multiple tasks i I k- (i I k+ ). Each task consumes and produces one chemical. Chemical k is stored in a dedicated tank with capacity ζ k. The time horizon is divided into periods n N={,,, N} of uniform length Δt = η/n, W in {0, }; it is equal to if task i is processed during period n. B in 0; extend (amount processed) of task i during period n. S sn 0; inventory of chemical k at time point n. i I S S j kn kn Win =, j, n = Sk,n + Bin B β i i I = ρ Δ t, i k+ I k i in + γ kn ζ k, k, n = Sk, n + βiwin βibin + γkn, k, n i I k+ I k i Bin = βi Win, i, n i I N A { w [0,] : A w } K N MB { x = ( s, w) [0, ξ] R+ : A x } A P = = MB P = = Win { 0,}, i, n; Skn [0, ζ k ], k, n w i n { 0,}, s [0, ξ]

168 Polyhedral Results P = P A I P MB A { w [0,] I N : A w } K N MB { x = ( s, w) [0, ξ] R + : A x = γ} A P = = P MB = A P P MB P Proposition 3. Matrix A A is an interval matrix; thus, it is totally unimodular. Corollary. Polytope P A is integral. Let lcm β be the least common multiple of all batchsizes, lcm β = lcm(β i, i I) = lcm(β); Let gcf ε be the greatest common factor of deliveries γ kn and capacities ζ k, gcf ε = gcf(γ kn, k K, n N; ζ k, k K) = gcf(ε). If κ * is the greatest common factor of lcm β and gcf ε, κ * = gcf(lcm β, gcf ε ), then lcm β = κ * κ β and gcf ε = κ * κ ε Proposition 5. Matrix Α is (κ β κ * κ ε ) regular. Proposition 6. Polyhedron P MB is integral if Δt = gcf ε /lcm ρ, where lcm ρ is the least common multiple of production rates ρ i, lcm β = lcm(β i, i I) = lcm(β). Proposition 7. If λ is the least common multiple of the denominators of rational parameters γ, ζ and ρ, then the vertices of P MB are w integral if Δt = Δt* = gcf(λε)/lcm(λρ).

169 Geometric Interpretation RM U ζ = 4, S 0 = W T 4 3 W T INT T ρ = ζ =, S 0 = 0 P A P MB P A P MB U T ρ = 0 P W T a) Δt = 0.5 hr 0 P W T b) Δt = 0.75 hr FP ζ =, S 0 = 0 W T W T gcf(λε) =, lcm(λρ) = Δt * = P A P MB P A P MB 0 W T 0 P 0 0 c) Δt =.0 hr d) Δt =.5 hr P W T

170 Example 3 stages; 5 units; tasks; chemicals Three instances In : week horizon; demands in 0.5 increments Proposition 7 Δt* = 6 hr RM RM U U T T T T INT INT U3 T3 T3 U4 INT3 INT4 T4 T4 T43 T5 T5 U5 A B C B C Model and solution statistics for instances Ex/In-/Δt. Δt (hr) Periods (N) Constraints ,345 Binary var s ,344 Continuous var s LP-relaxation Optimal solution Integrality gap (%)* CPU time (sec) Nodes 03,63 874,6 5 8,968 Optimality gap (%) * Integrality gap: IG = (Z * - Z LP )/Z * Z LP = LP-relaxation; Z * = Optimal solution

171 Example 3 stages; 5 units; tasks; chemicals Three instances In : week horizon; demands in 0.5 increments Proposition 7 Δt* = 6 hr In : week horizon; demands in 0. increments Proposition 7 Δt* =.4 hr U4 U INT T T T T INT3 RM T3 U3 T3 RM INT4 INT T4 T4 T43 T5 T5 U U5 Model and solution statistics for instances Ex/In-/Δt. Δt (hr) Periods (N) Constraints 897,345,793,4,689 Binary var s 896,344,79,40,688 Continuous var s 67 95,33,54,849 LP-relaxation Best/optimal solution Integrality gap* (%) CPU time (sec),00,00, ,00 Nodes 985,50 77,50 699, ,50 Optimality gap (%) Instance not solved to optimality. * If not solved to optimality we calculate a range [IG MIN, IG MAX ]. IG MIN = [Z * (-OG) Z LP ]/ Z * (-OG); IG MAX = (Z * - Z LP )/Z *. where Z LP = LP-relaxation; Z * = Best solution; OG = Optimality gap. A B C B C

172 More Polyhedral Results Production planning with backlogged demand A { w [0,] I N : A w } K N I N = {( s, u, w) [0, ξ ] R + R + : As s + Au u + A w w = γ} A P = = P MB Proposition 8. i) Matrix Α is (κ β κ * )-regular. ii) Polyhedron P MB is integral for Δt + = gcf ε /lcm ρ. iii) The vertices of P MB are w-integral for Δt * = gcf(λε)/lcm(λρ). S kn U kn W in k = A k = B k = C k = A k = B k = C i = T i = T n β T k = RM - - β T β T - -β T β T k = INT - - -β T β T β T β T - -β T k = FP - - -β T β T As Au Aw Production planning with backlogged demand and shipments P MB3 A { w [0,] I N : A w } I N K N K N 3 T = { x = ( s, w, v, u) [0, ξ] R R : A x = [0, γ] } A P = = where A 3 x A = 0 s A 0 w + + R + I I 0 A u Proposition 9. i) Matrix Α 3 is (κ β κ * )-regular. ii) Polyhedron P MB3 is integral if Δt + = gcf ε /lcm ρ. iii) The vertices of P MB3 are w-integral if Δt * = gcf(λε)/lcm(λρ). S kn W in V kn U kn k = A k = B k = C i = T i = T k = A k = B k = C k = A k = B k = C n β T k = RM - β T 3 - β T -β T β T k = INT - -β T β T 3 - -β T β T -β T k = FP - -β T 3 - -β T - - k = RM k = INT k = FP

173 Example 4 stages; units; 37 tasks; 3 chemicals η = 8 weeks (56 days); In : Demand must be satisfied on time Proposition Δt* = (/6) d = 4 hr U4 T4 T4 U T U T43 T RM T U T T T3 T4 RM RM3 U3 T T3 T3 T3 INA INB INCD INA INB T T T3 T4 U3 U3 U33 INC T3 T3 T3 T3 T33 T33 IND IN3A IN3B IN3A IN3B IN3C IN3D U43 T44 U4 T4 T4 T43 T44 T43 T43 T433 T434 U44 U T44 T44 T443 T444 A B A B A B A B C C D D Model and solution statistics for instances of Example : In/Δt. Δt (hr) Periods (N) Constraints 4,48 6,7 8,96 3,44 7,9 Binary var s 5,488 8,3 0,976 6,464,95 Continuous var s 3,37 4,705 6,73 9,409,545 LP-relaxation,576.5,588.9,595.0,60.,604. Best/optimal solution - - -,60. - Integrality gap * (%) n/a n/a n/a 0 n/a CPU time (sec),00,00,00 4.,00 Nodes 94,900 77,000, ,400 Optimality gap * (%) n/a n/a n/a 0 n/a * If no integer feasible solution is found, optimality and integrality gaps cannot be calculated.

174 Example 4 stages; units; 37 tasks; 3 chemicals η = 8 weeks (56 days); In : Demand must be satisfied on time Proposition Δt* = (/6) d = 4 hr In : Shipments/backlogs are introduced Proposition Δt* = (/6) d = 4hr U4 T4 T4 U U T43 T T RM T U T T T3 T4 RM RM3 U3 T T3 T3 T3 INA INB INCD INA INB T T T3 T4 U3 U3 U33 INC T3 T3 T3 T3 T33 T33 IND IN3A IN3B IN3A IN3B IN3C IN3D U43 T44 U4 T4 T4 T43 T44 T43 T43 T433 T434 U44 U T44 T44 T443 T444 Model and solution statistics for instances Ex/In-/Δt. A B A B A B A B C C D D Δt (hr) Periods (N) Constraints 6,9 9,349,469 8,709 4,949 Binary var s 5,880 8,80,760 7,640 3,50 Continuous var s 69 9,349,469 8,709 4,949 LP-relaxation 3,65.3 3, , , ,634.7 Best/optimal solution 3, , ,80.3 3,637. 3,65.5 Integrality gap * (%) CPU time (sec),00,00,00 34.,00 Nodes 55,76 95,98 67,50,350,5 Optimality gap (%) Instance not solved to optimality * If not solved to optimality we calculate a range [IG MIN, IG MAX ]. IG MIN = [Z * (-OG) Z LP ]/ Z * (-OG); IG MAX = (Z * - Z LP )/Z *. where Z LP = LP-relaxation; Z * = Best solution; OG = Optimality gap.

175 Remarks Integrality of P A and P MB is interesting theoretically It cannot be directly used (e.g. in decomposition methods) Can be proved only for subset of problems Our results are useful in practice: Starting point for extended results and special cases Computational performance of commercial solvers is improved Proof of results suggests network structure Study relationship between production facilities and graph theoretic networks

176 Network Structure of Scheduling MIP Model MIP Formulation P = {x: Ax = b} Integral Polytope A: totally unimodular (TU) Theorem. If a {0, ±} matrix has no more than two non zero coefficients in each column, and Σ l a lm = 0 if column m has two coefficients, then A is TU 4 3 (,) (,) (,3) (3,4) (4,) (4,) 3 4 Problem Representation RM RM U U T T T T INT INT U3 T3 T3 INT3 INT4 U4 U5 T4 T4 T43 T5 T5 RM T T INT INT3 T3 T4 T43 T4 RM T T T3 INT INT4 T5 T5 Processing restrictions correspond to known problems in graph theory A B C B C Production Facility A B C B C Digraph 9

177 FERMENTACIÓN REPOSO TANQUE REPOSO D. W. TANQUES DE GOBIERNO I TANQUES DE GOBIERNO II TANQUES DE GOBIERNO Litro 500 BPM ¼STD 700 BPM ½STD 600 BPM ½PREMIER 600 BPM ½ T.A., ¼ BOH. 750 BPM Litro 350 BPM LATA 00 LPM ½ T.A EXP 750 BPM Challenges in Chemical Production Planning Modeling Different production environments DILUTION Wide range of processing constraints Integration with multiple planning functions Develop general modeling framework Theory Discrete decisions lead to mixed integer programming models A P P MB P Existing hardware and software insufficient Develop fundamental optimization theory Use underlying theory to develop better solution methods Solution Methods Applications: optimize over long horizon optimize multiple systems simultaneously Intractable large scale models Develop general & computationally effective methods Projection based strategy

178 RM Example Revisited 50 P 00 (kg/d) 00 P 300 (kg/d) U U INT (ZW) P 3 = 50 (kg/d) U3 A B P B,000 k K FP w l k P kt Π l l L F Cp = f(p A,P B ),000 P A P B.5P A +0.5P B,00 Cp t l γ + P B k K FP w l k P kt l L O P B 0,000,000 P A Process Attainable Region P A Surrogate cost model All necessary production feasibility information in: All necessary production cost information in: Can we systematically develop these constraints? We have to analyze the underlying scheduling model k K Cp FP t w l k P kt l γ + Π k K l FP w l L l k P kt F l L O

179 Projection based Method Projection based method: Develop feasibility/optimality constraints that involve only planning (x) variables Convergence between overestimation (OE) and underestimation (UE)of true convex hull. Choose initial search vectors w = [w, w, w K ] T. Solve scheduling model (M) for each vector to obtain a UE vertex and an OE inequality 3. Run Quickhull to convert UE vertices into inequalities 4. Iterate until MPD stops improving: Find maximum perpendicular distance (MPD) and a new search vector Solve (M) for new search vector; obtain new vertex and new inequality Run Quickhull to update UE 5. Set PAR equal to UECH

180 Rolling horizon Approach (PP) z = min Cp + t t Cht Iit = Ii, t + Pit δit, i, t Ch t k K FP w l k P kt ( ) = θ I, i i it Π l t l L F I i, t P it P P How much can we produce and at what cost? Satisfy demand? I it At what (inventory) cost? Cp t l γ +, 0 P it I it k K FP w l k P kt l L O δ it Model (PP) yields a complete production planning solution: P it and I it If detailed schedules are required, model (PP) can be used in a rolling horizon algorithm (S PP) to provide production targets for (weekly) scheduling models 0) Set θ = ) Solve (PP) for planning periods t {θ, θ +,, T} to obtain targets Pkθ, Pkθ+,, PkT ) Solve scheduling model for period t = θ to meet target Pkθ 3) If production target is met exactly (i.e. SkN = Pkθ, k K FP ), fix Pkθ,, and return to otherwise, fix Pkθ, and re solve (PP) to update targets Pkθ+, Pkθ+,, PkT, 4) Set θ =θ+, and return to

181 Example HT RI RII Sep H S R, R, R3 RM H RM 40% A 40% Int R 60% 0% ImP 60% S Int 80% 90% R R3 B 0% RM3 Kondili et al., 993 P B (kg) 4500 R 68 R 6 R 0 : H t = 0 hr R 68 : H t = 68 hr R 6 : H t = 6 hr 3000 R P A (kg)

182 Example HT RI RII Sep H S R, R, R3 RM H RM 40% A 40% Int R 60% 0% ImP 60% S Int 80% 90% R R3 B 0% RM3 P B (kg) 3000 R 500 : S In MAX = 500 Kg R 000 : S In MAX =,000 Kg R 500 : S In MAX =,500 Kg 000 R R R 500 P A (kg)

183 Example Production Planning 48 week periods U U U3 U4 U3 U4 T, T4 T T3 T5, T6 T7, T9 T8, T0 Papageorgiou and Pantelides, 996 F S S Int S3 T T T3 T4 T5 P F 95% S4 50% 5% 0% S5 Int T7 T8 T9 90% 50% S6 T6 T0 P P3 Projection of scheduling feasible region onto P i 000 P C P B P A 500 0

184 Example : Results 3000 (kg) Inventory Production Inventory (I kt ) and production (P kt ) levels for product A 500 (kg) Demand Shipments Backlog Demand (Dem kt ), shipment (D kt ) and backlog (U kt ) profiles for product A

185 Solutions of (PP) and (S-PP) Example : Results Model (PP) Algorithm (S-PP) Holding cost ($) 574, ,345 Backlog cost ($) 83,933 0, Production cost ($) 408,405 4,90 Total cost ($),67,094,90,475 CPU secs (kg) 500 Production Targets Deviation from Production Targets (kg) Absolute deviation between production targets predicted by (PP) and production amounts obtained by detailed scheduling (S PP) Average total production (P A +P A +P C ):,80.0 kg Average total absolute deviation (PP) (S PP): 4.5 kg 0.5%

186 FERMENTACIÓN REPOSO TANQUE REPOSO D. W. TANQUES DE GOBIERNO I TANQUES DE GOBIERNO II TANQUES DE GOBIERNO Litro 500 BPM ¼STD 700 BPM ½STD 600 BPM ½PREMIER 600 BPM ½ T.A., ¼ BOH. 750 BPM Litro 350 BPM LATA 00 LPM ½ T.A EXP 750 BPM Conclusions General Modeling Framework No decoupling into subproblems better solutions DILUTION Unique problem representation better IT systems Computational advances for wide range of problems Polyhedral & Graph theoretic Results Rigorous treatment of MIP tightness A P P MB P Provides insights into formulation of large scale problems Integration of combinatorial optimization algorithms Projection based Method Compact description of production targets and costs Can be combined with any scheduling formulation Generated once off line; improved on line if necessary Effective production planning approach P B,000,000 FP k K P A P B.5P A +0.5P B,00 P B 0,000,000 P A w P l k kt Π l l L F

187 Questions?