Overview of Mixed-integer Nonlinear Programming

Transcription

1 Overview of Mied-integer Nonlinear Programming Ignacio E. Grossmann Center for Advanced Process Decision-maing Department of Chemical Engineering Carnegie Mellon Universit Pittsburgh PA 15213

2 Outline MINLP models in planning and scheduling Overview MINLP methods Brief reference to Generalized Disjunctive Programming Overview of global optimization of MINLP models Challenges How to develop effective algorithms for nonlinear discrete/continuous? How to improve relaation? How to solve nonconve GDP problems to global optimalit? 2

3 Observation for EWO problems -Most Planning Scheduling Suppl Chain models are linear = > MILP Major reasons: Simplified performance models (fied rates processing times Fied-time horizon minimization maespan problems See recent reviews in area: Mendez C.A. J. Cerdá I. E. Grossmann I. Harjunosi and M. Fahl (2006. State-Of-The-Art Review of Optimization Methods for Short-Term Scheduling of Batch Processes Computers & Chemical Engineering Burard R. & Hatzl J. (2005. Review etensions and computational comparison of MILP formulations for scheduling of batch processes. Computers and Chemical Engineering Kelle J..D. Formulating Production Planning Problems Chemical Engineering Progress Jan. p.43 (2004 Floudas C. A. & Lin X. (2004. Continuous-time versus discrete-time approaches for scheduling of chemical processes: A review. Computers and Chemical Engineering Kallrath J. (2002. Planning and scheduling in the process industr. OR Spectrum

4 When are nonlinearities required? Cclic scheduling problems: infinite horizon (onl nonlinear objective Sahinidis N.V. and I.E. Grossmann "MINLP Model for Cclic Multiproduct Scheduling on Continuous Parallel Lines" Computers and Chemical Engineering (1991. Pinto J. and I.E. Grossmann "Optimal Cclic Scheduling of Multistage Multiproduct Continuous Plants" Computers and Chemical Engineering (1994 Performance models: Jain V. and I.E. Grossmann "Cclic Scheduling and Maintenance of Parallel Process Units with Decaing Performance" AIChE J. 44 pp (1998 Eponential deca furnaces Van den Heever S.A. and I.E. Grossmann "An Iterative Aggregation/Disaggregation Approach for the Solution of a Mied Integer Nonlinear Oilfield Infrastructure Planning Model" I&EC Res (2000. Pressure and production curves reservoir Bizet V.M. N. Juhasz and I.E. Grossmann Optimization Model for the Production and Scheduling of Catalst Changeovers in a Process with Decaing Performance AIChE Journal (2005. Reactor model Flores- Tlacuahuac A. and I.E. Grossmann Simultaneous Cclic Scheduling and Control of a Multiproduct CSTR Ind. Eng. Chem. Res (2006. Dnamic models CSTR reactor Karuppiah R. K. Furman and I.E. Grossmann Global Optimization for Scheduling Refiner Crude Oil Operations in preparation (2007. Crude blending 4

5 5 MINLP f( and g( - assumed to be conve and bounded over X. f( and g( commonl linear in } {01} { } { 0 (.. ( min a A Y b B R X Y X g t s f Z m U L n = = = Mied-Integer Nonlinear Programming Objective Function Inequalit Constraints

6 Generalized Disjunctive Programming (GDP Raman and Grossmann (1994 (Etension Balas 1979 min Z = s.t. r c ( + f 0 ( Objective Function Common Constraints OR operator j J Y j g j c Ω ( Y R j J n Y = = j ( Y c γ j j true 0 R { true false } 1 K Disjunction Constraints Fied Charges Logic Propositions Continuous Variables Boolean Variables Can be transformed into MINLP or solved directl as a GDP See previous wor Sangbum Lee (2002 and Nic Sawaa (2006 6

7 Jobshop Scheduling Problem Processing times τ (hr Jobs/Stages A B C Stage 1 Stage 2 Stage 3 Find sequence times to minimize maespan T t A A B t B t C C T GDP: min MS ( t t 5 ( t t 0 A B B A ( t t 5 ( t t 2 A C C A ( t t 1 ( t t 6 B C C B C = T st T ta + 8 T tb + 5 T t + 6 T t t t A B C 0 (Zero-wait transfer A before B or B before A A before C or C before A B before C or C before B time 7

8 8 min A B C MS T st T t T t T t = (1 6 (1 1 (1 2 (1 5 (1 0 (1 5 = = + = + = MILP: 0 C B A t t t T + CB BC CA AC BA AB CB BC CA AC BA AB CB B C BC C B CA A C AC C A BA A B AB B A M t t M t t M t t M t t M t t M t t Big-M reformulation Parameter M sufficientl large

9 Continuous multistage plants Pinto Grossmann (1994 Cclic schedules (constant demand rates infinite horizon Intermediate storage Given : Product 1 2 NP?????? 晻 Stage 1 Stage 2 Stage M N Products Transition times (sequence dependent Demand rates????????? Determine : PLANNING Amount of products to be produced Inventor levels SCHEDULING Cclic production schedule Sequencing Lengths of production Ccle time Objective : Maimize Profit = + Sales of products - inventor costs - transition costs 9

10 Optimal Trade-offs Optimal length of ccle determined largel b transition and inventor costs $/hr sales of products profit transition costs 100 inventor costs Length ccle (hrs 200 Critical to model properl inventor levels and transition times 10

11 Assumption: MINLP Model (1 Each product is processed in the same sequence at each stage Basic ideas : a. NP products = b. NP time slots at each stage 1 i 0 Binar variable if product i assigned otherwise to slot Stage 1 Stage 2 Stage M = 1 = 2 = NP 晻 = 1 = 2 = NP 晻晻 = 1 = 2 = NP 晻 Binar variables for assignments i = 1 product i assigned to slot 0 otherwise Transition Time Slot Processing time a Assignments of products to slots and vice versa i = 1 i = 1 i i 11

12 MILP model MINLP Model (2 b Definition of transition variables z ij i + j 1 1 i j z ij treated as continuous c Processing rates mass balances and amounts produced Wp W Wp im = γ = γp im Tpp im i m Tpm m + 1Wpim 1 i m = 1... M 1 m m im = α im + d Timing constraints Tpp Tp Tp m m im T U im i 0 = Tppim i = Te m Ts m i m m m } Processing time Ts + 1m = Tem + τ ijmzij + 1 = 1... NP 1 i j 11 = τ ij1z ij i j m Tem+ 1 m = 1... M Ts 1 Te 1 m } Transitions Ts m Tsm+ 1 m = 1... M 1 Tc Tp + τ m i j ijm z ij m Ccle time 12

13 MINLP Model (3 Inventories: Case 1 Case 2 Tsm Tpm Inventor level Tem I1m Tsm+1 I2m Tpm+1 Tem+1 stage m stage m+1 Inventor level Tsm Tpm Tem Tpm+1 Tsm+1 I1m I2m Tem+1 I0m Tpm Tpm+1 I3m time I0m I3m Tsm+1 - Tsm Tem+1 - Tem time e Inventor levels for intermediates I1 { Tp Ts Ts } + I0 m = = min + 1 M m γ m m m m I2 m = ( γ m α m+1 γ m+1 ma{ 0Te m Ts m+1 }+ I1 m m = 1... M 1 I3 m =α m+1 γ m +1 min{ Te m+1 Te m Tp m+1 }+ I2 m m = 1... M 1 m Can be reformulated as 0-1 linear inequalities 0 I1 m Ima m m = 1... M 1 0 I2 m Ima m m = 1... M 1 0 I3 m Ima m m = 1... M 1 Ima m = Ip im m i I Ip im U imi 0 i m = 1... M 13

14 MINLP Model (4 f Demand constraints Wp im d Tc i i Note: Linear g Objective function : Maimize Profit Note: Nonlinear (divide b Tc Profit = i p i Wp im Tc INCOME i j i m Ctr ij z ij Tc Cinv im Ip im Tc TRANSITION COST INTERMEDIATE INVENTORY COST MINLP 1 Cinvf i γp im Wp im FINAL Tpp 2 Tc im INVENTORY COST i 14

15 Solution Algorithms for MINLP Branch and Bound method (BB Ravindran and Gupta (1985 Leffer and Fletcher (2001 Branch and cut: Stubbs and Mehrotra (1999 Generalized Benders Decomposition (GBD Geoffrion (1972 Outer-Approimation (OA Duran & Grossmann (1986 Yuan et al. (1988 Fletcher & Leffer (1994 LP/NLP based Branch and Bound Quesada and Grossmann (1992 Etended Cutting Plane (ECP Westerlund and Pettersson (

16 16 Basic NLP subproblems a NLP Relaation Lower bound FU i i FL i i R j LB I i I i Y X J j g s t f Z = β α (NLP1 0 (.. ( min b NLP Fied Upper bound X J j g s t f Z j U = 0 (.. ( min (NLP2 c Feasibilit subproblem for fied. 1 (.. min R u X J j u g s t u j (NLPF Infinit-norm u > 0 => infeasible

17 17 Cutting plane MILP master (Duran and Grossmann 1986 Based on solution of K subproblems ( =1...K Lower Bound M-MIP Y X K J j g g f f st Z T j j T K L = + + = ( ( ( ( min α α Notes: a Point ( =1...K normall from NLP2 b Linearizations accumulated as iterations K increase c Non-decreasing sequence lower bounds

18 Linearizations and Cutting Planes f( Conve Objective Supporting hperplanes X 1 X 2 X Underestimate Objective Function 2 1 Conve Feasible Region 2 1 Overestimate Feasible Region 18

19 Branch and Bound NLP1: Tree Enumeration min Z LB = LB s.t. g j (<0 f( min Z = f( X Y R i < α i i I FL α i > β i i I FU β j J s.. t g ( 0 j J j X Y R i i FL i I i i FU i I Successive solution of NLP1 subproblems Advantage: Tight formulation ma require one NLP1 (I FL =I FU = Disadvantage: Potentiall man NLP subproblems Convergence global optimum: Uniqueness solution NLP1 (sufficient condition Less stringent than other methods 19

20 Outer-Approimation Alternate solution of NLP and MIP problems: NLP2 Upper bound M-MIP Lower bound NLP2: min min Z U Z= f U = ( f( s ts.t. gg ( j.. j ( <0 0 j J j J X X min L K M-MIP: min Z L = α Z K = α s.t α > f + f T T st α f ( + f ( g + g j T =1..K T ( ( < 0 j J g j + gj 0 j J X Y X Y α R 1 = 1... K Propert. Triviall converges in one iteration if f( and g( are linear - If infeasible NLP solution of feasibilit NLP-F required to guarantee convergence. 20

21 MIP Master problem need not be solved to optimalit Find new +1 such that predicted objetive lies below current upper bound UBK : st.. (M-MIPF min Z K =0 α K L min Z = 0 s.t. α K α < UB K ε α UB α > f ε + f T T α f( + f( g + g T =1..K < 0 T gj( + gj( 0 j J X Y α R 1 X Y = 1.. K Remar. M-MIPF will tend to increase number of iterations 21

22 Generalized Benders Decomposition Benders (1962 Geoffrion (1972 Particular case of Outer-Approimation as applied to (P1 1. Consider Outer-Approimation at ( α > f + f T g + g j T < 0 j J (1 2. Obtain linear combination of (1 using Karush-Kuhn- Tucer multipliers μ and eliminating variables α > f + f T (2 + μ T g + g T Lagrangian cut Remar. Cut for infeasible subproblems can be derived in a similar wa. λ T g + g T < 0 22

23 Generalized Benders Decomposition Alternate solution of NLP and MIP problems: NLP2 Upper bound M-GBD Lower bound NLP2: min UZ U = f( min s. t. minz g ( s.t. gj ( j <0 Z K L X = α = f ( X j J j J M-GBD: min Z L K = α 0 ( T st α f ( + s.t. α > f f ( + f T + μ T T g + g T T + ( μ g( + g ( KFS ( KFS λ T g + g T < 0 KIS T ( [ ( ] Y α R 1 T λ g( + g ( 0 KIS 1 X α R Propert 1. If problem (P1 has zero integralit gap Generalized Benders Decomposition converges in one iteration when optimal ( are found. Sahinidis Grossmann (1991 => Also applies to Outer-Approimation 23

24 Etended Cutting Plane (1995 Westerlund and Pettersson (1995 Evaluate No NLP! M-MIP' Add linearization most violated constraint to M-MIP Remars. J = { ˆj arg{ ma g j J j ( - Can also add full set of linearizations for M-MIP - Successive M-MIP's produce non-decreasing sequence lower bounds - Simultaneousl optimize with M-MIP = > Convergence ma be slow } 24

25 LP/NLP Based Branch and Bound Quesada and Grossmann (1992 (a..a Branch & Cut; Hbrid Integrate NLP and M-MIP problems M-MIP NLP2 Solve NLPs at selected nodes of tree from master MILP and add cutting planes M-MIP LP1 LP2 LP3 LP4 LP5 = > Integer Solve NLP and update bounds open nodes Remar. Fewer number branch and bound nodes for LP subproblems Ma increase number of NLP subproblems 25

26 Numerical Eample min Z = s.t. ( < > (1-2 < (1-1 > > 0 (MIP-EX > > 1 0 < 1 < 4 0 < 2 < = 0 1 Optimum solution: 1=0 2 = 1 3 = 0 1 = 1 2 = 1 Z =

27 Starting point 1 = 2 = 3 = 1. Objective function Upper bound OA Lower bound OA Upper bound GBD Lower bound GBD Summar of Computational Results Iterations Method Subproblems Master problems (LP's solved BB 5 (NLP1 OA 3 (NLP2 3 (M-MIP (19 LP's GBD 4 (NLP2 4 (M-GBD (10 LP's ECP - 5 (M-MIP (18 LP's 27

28 28 Mied-Integer Quadratic Programming Fletcher and Leffer (1994 Quadratic-Approimation ma not provide valid bounds for conve function f( Conve function q( Quadratic approimation Add quadratic objective to feasibilit M-MIPF Proceed as OA M-MIQP: min Z K = α K K 2 K K s.t. α < UB K ε α > f + f T g + g T < 0 =1..K X Y α R 1 Remar. Faster convergence if problems nonlinear in s.t. α < UB K ε ( ( ( ( K g g f f T T = + + α + = T K L Z ( 2 1 min 2 α X Y α R 1

29 Effects of Nonconveities 1. NLP supbroblems ma have local optima 2. MILP master ma cut-off global optimum Multiple minima Cut off! Global optimum Objective Handling of Nonconveities Rigorous approach (global optimization: Replace nonconve terms b underestimtors/conve envelopes Solve conve MINLP within spatial branch and bound 2. Heuristic approach: Assume lower bound valid Add slacs to linearizations MILP Search until no improvement in NLP 29

30 Handling nonlinear equations h( = 0 1. In branch and bound no special provision-simpl add to NLPs 2. In GBD no special provision- cancels in Lagrangian cut 3. In OA equalit relaation T T h( 0 [ ] 1 if λ > 0 T = t t = 1if λ < 0 ii ii i 0 if λ = 0 Lower bounds ma not be valid i i Rigorous if eqtn relaes as h( 0 h( is conve 30

31 MIP-Master Augmented Penalt Viswanathan and Grossmann 1990 Slacs: p q with weights w s. t. min Z K K = α + w p p + wq q Σ=1 s.t. α > f + f T α f ( + f ( T h T T h < p T ( p g + g T < q g g T ( + ( q Σ Σ i i B 1 =1...K i B i N X Y α R 1 p q > 0 (M-APER =1..K = 1... K If conve MINLP then slacs tae value of zero => reduces to OA/ER Basis DICOPT (nonconve version 1. Solve relaed MINLP 2. Iterate between MIP-APER and NLP subproblem until no improvement in NLP 31

32 Mied-integer Nonlinear Programming MINLP Codes: SBB Bussiec Drud (2003 (B&B MINLP-BB (AMPLFletcher and Leffer (1999(B&B-SQP Bonmin (COIN-OR Bonami et al (2006 (B&B OA Hbrid FilMINT Linderoth and Leffer (2006 (Hbrid-MINTO-FilterSQP DICOPT (GAMS Viswanathan and Grossman (1990 (OA AOA (AIMSS (OA αecp Westerlund and Peterssson (1996 (ECP MINOPT Schweiger and Floudas (1998 (GBD OA Global MINLP code: BARON Sahinidis et al. (1998 (Global Optimization MIQP codes: CPLEX-MIQP ILOG (Branch and bound cuts MIQPBB (Fletcher Leffer

33 Continuous multistage plants Cclic schedules (constant demand rates infinite horizon Intermediate storage Given : Product 1 2 NP?????? 晻 Stage 1 Stage 2 Stage M N Products Transition times (sequence dependent Demand rates????????? Determine : PLANNING Amount of products to be produced Inventor levels SCHEDULING Cclic production schedule Sequencing Lengths of production Ccle time Objective : Maimize Profit = + Sales of products - inventor costs - transition costs 33

34 Eample 3 stages 8 products 3 MINLP model 448 binar 0-1 variables 2050 continuous variables 3010 constraints DICOPT (CONOPT/CPLEX: 38.2 secs Optimal solution Profit = $6609/h stage 1 A C BEFHD G stage 2 stage 3 ccle time = 675 hrs Intermediate storage (ton Stages Intermediate storage (ton Stages 2-3 time (h time (h 34

35 Bonmin (COIN-OR Bonami Biegler Conn Cornuejols Grossmann Laird Lee Lodi Margot Sawaa Wächter Single computational framewor (C++ that implements: - NLP based branch and bound (Gupta & Ravindran Outer-Approimation (Duran & Grossmann LP/NLP based branch and bound (Quesada & Grossmann 1994 a Branch and bound scheme b At each node LP or NLP subproblems can be solved NLP solver: IPOPT MIP solver: CLP c Various algorithms activated depending on what subproblem is solved at given node I-OA Outer-approimation I-BB Branch and bound I-Hb Hbrid LP/NLP based B&B (etensions 35

36 Conve MINLP Test Problems Computational performance: Comparison of 3 variants with DICOPT and SBB Largest: vars 2720 cont constr. DICOPT solves 20 of the 38 problems the fastest and in less than 3minutes I-Hb solves the most problem in the time limit I-Hb alwas dominate I-BB and I-OA 36

37 Min Q = c del + del + Cost + st.. del ij i j ij ( ij ij i(( i i ( i i i j i i j N i< j del i j N i < j ij j i del i j N i< j ij i j del i j N i< j ij j i Z ij Z ij Z ij Z ij i j N i< j i + Li/2 j Lj /2 j + Lj /2 i Li/2 i+ Hi/2 j Hj /2 j + Hj /2 ihi/2 i i i UB 1 i 1 i 2 i i N LB i N i UB 2 i Safet Laout Problem MIQP Sawaa (2006 Determine placement a set of rectangles with fied width and length such that the Euclidean distance between their center point and a pre-defined safet point is minimized. i N LB i N del del Z Z Z { True False} i j N i < j ij ij R + Z ij ij ij ij 37

38 Numerical results MIQP Small instance: 5 rectangles vars 31 cont. vars. 91 constr. SBB (CONOPT DICOPT (CONOPT/CPLEX CPLEX-MIQP 281 nodes 2.4 sec 19 major iterations 5.2 sec 18 nodes 0.06 secs Larger instance: 10 rectangles vars 111 cont. vars. 406 constr. Bonmin-BB (IPOT nodes sec Bonmin-Hbrid (Cbc IPOPT 6548 nodes (563 NLPs sec CPLEX-MIQP 1093 nodes 1.6 secs 38

39 Global Optimization Algorithms Most algorithms are based on spatial branch and bound method (Horst & Tu 1996 Nonconve NLP/MINLP αbb (Adjiman Androulais & Floudas 1997; 2000 BARON (Branch and Reduce (Roo & Sahinidis 1995 Tawarmalani and Sahinidis (2002 OA for nonconve MINLP (Kesavan Allgor Gatze Barton 2004 Branch and Contract (Zamora & Grossmann 1999 Nonconve GDP Two-level Branch and Bound (Lee & Grossmann

40 Nonconve MINLP min Z s. t. = f ( g( 0 X Y f ( g( nonconve => Conve MINLP (relaation min Z s. t. = fˆ( gˆ( 0 X Y => Lower bound fˆ ( gˆ( conve underestimators conve envelopes 40

41 41 ( U L L L U L U L g g g g + = ( ( ( ˆ Concave function g( L U g( Conve envelope: secant

42 42 L U L U U L U L U U U U il L i L L w w w w Conve envelopes bilinear term U L U L w = Bilinear terms w= McCormic (1976 under/over estimators Underestimators Overestimators For other conve envelopes/underestimators see: Tawarmalani M. and N. V. Sahinidis Conveification and Global Optimization in Continuous and Mied-Integer Nonlinear Programming: Theor Algorithms Software and Applications Vol. 65 Nonconve Optimization And Its Applications series Kluwer Academic Publishers Dordrecht 2002

43 Spatial Branch and Bound Method 1. Generate subproblems b branching on continuous variables (subregions Conve Relaation => => 2. Compute lower bound from conve MINLP (relaation 3. Compute upper bound from local solution to nonconve MINLP Continue until tolerance of bounds within tolerance Guaranteed to converge to global optimum given a certain tolerance between lower and upper bounds good upper bound generation cutting planes 43

44 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima Objective 44

45 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima Objective Lower bound 45

46 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima Objective Lower bound LB 46

47 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima Objective Lower bound UB = Upper bound LB 47

48 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective Lower bound UB = Upper bound LB 48

49 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective UB = Upper bound 49

50 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective UB = Upper bound 50

51 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective LB UB = Upper bound LB > UB 51

52 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective LB UB = Upper bound LB > UB 52

53 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective UB = Upper bound LB > UB 53

54 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective UB = Upper bound LB > UB 54

55 Eample spatial branch and bound Global optimum search Branch and bound tree Multiple minima LB < UB Objective LB UB = Upper bound LB > UB LB < UB 55

56 Recent development in BARON Sahinidis (2005 Cutting planes: Supporting hperplanes (outer-approimations of conve functions 56

57 Application Global MINLP in Planning/Scheduling Currentl limited due to problem size => Need special purpose methods Eample: Karuppiah Furman Grossmann (2007 Scheduling Refiner Crude Oil Operations Crude Suppl Streams Storage Tans Charging Tans Crude Distillation Units Source non-conveities: bilinearities in mass balances 57

58 58 1. Decompose using Lagrangean Decomposition 2. Globall optimize each subsstem with BARON 3. Derive Lagrangean cutting planes Iterate on Lagrange multipliers ( ( ( ( ( ( 1 1 * v u r w s z T n n T n n n n n n n λ λ λ λ

59 Numerical Results Size MINLP Problems Eample Number of Binar Variables Original MINLP model (P Number of Continuous Variables Number of Constraints 3 Suppl streams 3 Storage tans 3 Charging tans 2 Distillation units 3 Suppl streams 3 Storage tans 3 Charging tans 2 Distillation units 3 Suppl streams 6 Storage tans 4 Charging tans 3 Distillation units Results for Proposed Algorithm Root Node Eample Lower bound [obtained b solving relaation (RP ] (z RP Upper bound [on solving (P-NLP using BARON ] (z P-NLP Relaation gap (% Total time taen for one iteration [1] of algorithm (CPUsecs Local optimum (using DICOPT [1] Total time includes time for generating a pool of cuts updating Lagrange multipliers solving the relaation (RP using CPLEX and solving (P-NLP using BARON 59

60 Conclusions 1. MINLP Optimization Not widespread in planning/scheduling but increasing interest Significant progress has been made More software is available: commercial open-source MINLP problems of significant size can be solved Conve case: rigorous global optimalit Modeling efficienc and robustness still issues 2. Global Optimization Nonconve MINLP Conve MINLP used as a basis Ke: spatial branch and bound Main issue is scaling Special purpose techniques ma be required Open-source: Wor is under wa CMU-IBM: Margot Belotti (Tepper Practical approach: ignore nonconveities or use heuristics 60