Solving Mixed Integer Nonlinear Chemical Engineering Problems via Simulated Annealing Approach

Y. ÖZÇELIK and Z. ÖZÇELIK, Solving Mied Integer Nonlinear Chemical, Chem. Biochem. Eng. Q. 8 (4) 9 5 (004) 9 Solving Mied Integer Nonlinear Chemical Engineering Problems via Simulated Annealing Approach Y. Özçelik and Z. Özçelik Ege Universit, Engineering Facult, Chemical Engineering Department, Bornova Izmir, urke, el: 90 88 40 00 / 488; Fa: 90 88 76 00; E mail: ozcelik@eng.ege.edu.tr; zozcelik@bornova.ege.edu.tr Original scientific paper Received: September 5, 00 Accepted: Ma 4, 004 he optimization models of chemical engineering processes are generall nonlinear and mied integer, nonlinear in structure, and contain several equalit and inequalit constraints and bounds on variables. Mied Integer Nonlinear programming problems (MINLP) can be solved using either gradient methods or stochastic methods. Gradient methods need to separate the problem to Mied Integer Linear Programming (MILP) and Nonlinear Programming (NLP) problems and some special formulations where the continuit or conveit has to be imposed. In this work, an algorithm (SARAN) that was based on a simulated annealing algorithm of Corana was developed to solve MINLP problems. hen the algorithm and some alternatives for the basic steps of a simulated annealing were tested for 00 sequences of pseudo random numbers using MINLP test problems. Finall the results were compared with the results of the M SIMPSA for small and medium scaled problems. Ke words: Mied Integer Nonlinear Programming, Nonlinear Optimization, Simulated Annealing Introduction here are two main approaches to solve NLP and MINLP problems. Gradient Approach Stochastic Approach Various gradient algorithms have been described in the literature. he common propert of deterministic algorithms,4 is to separate the original MINLP problems into sub problems. On the other hand stochastic methods do not necessaril separate the problem into sub problems. However, various problem independent heuristics related to search interval compression and epansion and to shifting strategies are required for their effectiveness. Also these methods require the application of successive relaations, which ma substantiall increase the effort in identifing feasible regions and obtaining the global optimum. he simulated annealing algorithm is also a stochastic method based on stochastic generation of solution vectors and emplo similarities between the phsical processes of annealing and the optimisation problems. he phsical process of annealing he aim of the process is to find the atomic configuration that minimizes internal energ. For a given configuration, a random move is carried out b randoml picking a molecule and moving it in a random direction for a random distance. he new configuration is then accepted or rejected according to an acceptance criterion, based on the Boltzman function, (): p E k e B () where E is the change in the energ of the configurations, k B is the Boltzman constant, and is the temperature of the sstem. he acceptance of a new configuration depends on E and temperature. is reduced to reach a lower energ state. he low temperature is not a sufficient condition to find ground states of matter. he cooling process should be accomplished slowl, otherwise the resulting crstal will have man defects or the substance ma form a glass with no crstalline order. Optimisation b simulated annealing In simulated annealing, the value of the objective function is analogous to the energ of the sstem and the aim is to minimize the value of the objective function, where the values of the continuous and discrete variables represent a particular configuration of the sstem. he behaviour of the sstem,

0 Y. ÖZÇELIK and Z. ÖZÇELIK, Solving Mied Integer Nonlinear Chemical, Chem. Biochem. Eng. Q. 8 (4) 9 5 (004) subject to such a neighbourhood move is determined from the value of the two successive values of the objective function. here are two important approaches in the application of the simulated annealing to the continuous variables. he hbrid of random search and acceptance rejection criteria of simulated annealing algorithms. he hbrid of simulated annealing algorithm and nonlinear simple algorithm Both of these two approaches have the following critical steps. Estimation of the initial temperature of the sstem. Acceptance of points generated. Cooling Schedule ermination of the constant temperature process and termination of the cooling process Dealing with constraints Simulated annealing algorithms can be developed using alternative schemes for these critical steps. Initial value of temperature An initial value of the annealing temperature for a phsical process of annealing and optimization algorithms are based on annealing as an effective parameter for the optimisation path. he value of the initial temperature for a phsical process depends on the properties of the sstem and can be estimated eperimentall. Analogousl, the initial value of the temperature for an optimisation problem is estimated b the observation of the behaviour of the problem using a specified time, determined eperimentall. he alternative schemes are available to estimate the initial temperature. Some of them were used in this work. Prosenjit and Diwekar 5 proposed an temperature value that satisfies the following inequalit () as the initial temperature: F k e B 08. () he vector F is obtained b evaluating the change in the objective function value for a large number (e.g., 00) of neighbourhood moves. his procedure has been applied to the optimal design problem of heat echangers. Aarst and Korst 6 and Aarst and Van Laarhoven 7 proposed an temperature value that satisfies the following equalit () as the initial temperature. f i m 095. m e m m () m and m are the number of successful and unsuccessful moves, respectivel. f + is the average increase in the objective function value for m. he total number of moves to determine an initial temperature is proposed as 00 N. For eample, (0 5 ) ma be proposed as a starting point of the temperature to determine the initial temperature. Acceptance criteria wo tpes of acceptance criteria have been proposed in literature. Metropolis algorithm accepts all downhill moves. It accepts uphill moves with a probabilit p which depends on temperature and the change of the objective function values evaluated in successive iterations (4). Metropolis accepts all the moves with a probabilit of at high temperatures. F 0 F 0 p F (4) pe he method of a nonequilibrium Metropolis accepts uphill moves with a probabilit p which depends on temperature and the change of the objective function values evaluated in successive iterations similar to the Metropolis algorithm. Additionall, the inner loop of the annealing algorithm is terminated and cooling is applied when a downhill move occurs in the objective function. he Glauber algorithm accepts all moves at a high temperature with 0.5 probabilities (5). If the temperature is reduced, the probabilit acceptance of downhill moves remains constant at in the Metropolis algorithm. he value of probabilit increases from 0.5 to in the Glauber algorithm. On the other hand, the acceptance probabilit of an uphill move decreases from 0.5 to 0 in the Glauber algorithm, Annealing schedule e p e F F (5) wo tpes of annealing schedules have been proposed. he eponential tpe cooling schedule, 8 where a constant multiplicative factor is emploed, can be used to obtain a new temperature (6).

Y. ÖZÇELIK and Z. ÖZÇELIK, Solving Mied Integer Nonlinear Chemical, Chem. Biochem. Eng. Q. 8 (4) 9 5 (004) k k (6) is a temperature decrement factor which lies in the range of 0. Aarst and Van Laarhoven 7 proposed an alternating cooling schedule. (7) k k ln( ) k ( ) k (7) ( k ) is the standard deviation of the objective function at k. is the speed parameter ling on the range of 0. he value of varies depending on the equilibrium or the nonequilibrium application of Simulated Annealing with an order of 0 (0.0, 0). ermination criteria for constant temperature step and cooling step he constant number of moves or nonequilibrium comes together when constant number of moving approach can be used for the constant temperature step. In this approach, if the new function value is lower than the previous one, temperature is reduced without reaching equilibrium; otherwise the approach of constant number of moves is applied. he process of cooling can be terminated using the following criteria he constant number of temperature levels k (8) df ( ) (9) d F ( ) C can t be improved for several temperature levels. Dealing with constraints he Cardosa Approach. All infeasible points are penalized assuming a ver large positive value for a minimization problem or a ver large negative value, otherwise. 9 he method is an infeasible path method. his means that infeasible points ma be replaced b the new infeasible points, ecept that the are now centred on the best verte and obe all bound constraints. A feasible path method ma be used. his means, that feasible points ma be replaced b feasible points, and the objective function is evaluated onl for the feasible points. his approach ma lead to inaccurate results. Procedure and results In this work, an algorithm based on the algorithm of Corona, was developed. he algorithm of Corona was used to solve unconstrained problems. his algorithm was modified to solve constrained MINLP problems, and MINLP problems were used to test the criteria that were used in the basic steps of the algorithm. hen these problems were utilised to compare the performance of the algorithm with the algorithm of M SIMPSA for MINLP problems. he 9 test problems of Cardosa et al and test problems of Duran 4 and the initial values for these problems, are given below.. (M. F. Cardosa et al. and is also given in Kocis and Grossmann, Floudas et al. 0 and Roo and Sahinidis ). here is a local optimum at {,, Z} = {0,.8,.6} and nonconveities arise in the first constraint. he global optimum is at (,, Z} = {,0.5,} Minimize Z 5. 0 6. 0 L0 LU { 0,}. (M. F. Cardosa et al. and is also given in Kocis and Grossmann and Salcedo et al. ). he global optimum is at (,,, Z} = {,.75, 0.75,.4}. Minimize Z e ( ) 0 0 L (., 05 0) U (. 40, ) LU { 0,}. (M. F. Cardosa et al. and is also given in Floudas et al. 0 ). here are nonconveities because of the first constraint. he global optimum is at (,,, Z} = {, 0.9494,.,.07654}. Minimize Z07. 5( 05. ) 08. e ( 0. ) 0. 0. 0. 0 L (., 0 554. ) U (, ) LU { 0,} 4.(M. F. Cardosa et al. and is also given in Kocis and Grossmann, Floudas et al. 0, Salcedo and Roo and Sahinidis ). here are nonconveities because of the equalit constraints. he global optimum is at (,,, Z} = {0,,, 7.66780}

Y. ÖZÇELIK and Z. ÖZÇELIK, Solving Mied Integer Nonlinear Chemical, Chem. Biochem. Eng. Q. 8 (4) 9 5 (004) Minimize Z 5. 05. 5. 5. 5. 6. 0. 0 0 L {, 00) L { 0,} 5. (M. F. Cardosa et al. and is also given in Kocis and Grossmann, Diwekar et al. and Diwekar and Rubin 4 ). It is related to the selection among two candidate reactors for minimizing the cost of producing a desired product. he global optimum is at (,,,, Z} = {0,,, 7.66780} Minimize Z75. 55. 765 4 5 05. 09. e ) 04. 08. e ) 670 4 5 0 0 0 0 0 0 0 6 0 0 7 L ( 0000000 ; ; ; ; ; ; ) U ( 0; 0; 40; 0; 0; 0; 0) LU { 0,} 6. (M. F. Cardosa et al. and is also given in Salcedo ). his represents a quadratic capital budgeting problem. It has four binar variables and features bilinear terms in the objective function. he global optimum is at (,,, 4, Z} = {0, 0,,, 6} Minimize Z( 4 )( 564 ) 4 0 { 0,} 4 4 6 7 7. (M. F. Cardosa et al. and was also given in Yuan et al. 5, Floudas et al. 0, Salcedo and Roo and Sahinidis ). It has nonlinearities in, both, the continuous and binar variables. he global optimum is at (,,, 4,,,, Z} = {,, 0,, 0., 0.8,.907878, 4.57958} Minimize Z( ) ( ) ( ) log( 4) ( ) ( ) ( ) 50 55. 0. 0 8. 0 5. 0 4. 0 640. 45. 0 464. 0 L ( 000, ; ) U (.;., 8 5.) LU { 0,} 4 8. (M. F. Cardosa et al. and is also given in Berman and Ashrafi 6 ). he global optimum is at (,,, 4, 5, 6, 7, 8, Z} = {0,,,, 0,,, 0, 0.964} Minimize Z 0 4 5 60 7 80 00 4 5 6 7 8 0. 0. 5 05. 005. 4 0. 5 05. 6 00. 006. 8 7 L ( 000, ; ) U (;; ) LU { 0,} 8 9. (M. F. Cardosa et al. and is also given in Wong 7 ). he global optimum is at (,,, Z} = {78, 7, 7, 7.4} Maimize Z557854085689. 7. 99 40794. 4 aaaa4 5a5a6a7a8 90

Y. ÖZÇELIK and Z. ÖZÇELIK, Solving Mied Integer Nonlinear Chemical, Chem. Biochem. Eng. Q. 8 (4) 9 5 (004) 6 a9 a0 a a 0 L ( 7, 7, 7) U ( 454545,, ) L,, U L ( 78, 78) U ( 0, 0) L U 0. (Marco A. Duran and Ignacio E. Grossmann 4 ). It is related to a snthesizing process. It is the one of simultaneousl determining the optimal structure and operating parameters for a chemical process. Minimize Z56807 078ln( ) 9. ln( ) 0 08. ln( ) 096. ln( ) 08. 0 ln( ). ln( ) 0 0 0 0 0 L ( 000,, ) U ( 000,, ) LU 0 ( 0,,) properties of simulated annealing. aking this risk into account the proposed algorithm can be handled as an alternative approach to solve constrained MINLP problems. he basic steps of the algorithm that was shown schematicall in figure are eplained below:. (Marco A. Duran and Ignacio E. Grossmann 4 ). his is related to a snthesizing process. Minimaze Z58604 65764758055 80455564075859e e ( /. ) 65log( 4 ) 90log( 5) 80log( ) 0 6 4790 04045. 5. 6. 80 06. 06.. 0 08. 0 5 6 8 4 040. e 0 0 4 e /. 0 0 0 0 7 08. 08. 0 0 0 0 5 6 4 4 7 9 6 0 0 0 0 5 6 6 7 4080 4 50 (. 5 ). 8 0 05 ( 5. ) (/.) L ( 000000000 ; ; ; ; ; ; ; ; ) 5 U ( ; ; ; ; ; ; ; ; ) LU Fig. he flowsheet of the proposed algorithm Discussion M SIMPSA is a hbrid of the simulated annealing and nonlinear simple and has a potential risk to reach the global optimum. For, the simple moves ma actuall destro the global convergence. he initial value of the pseudo temperature: he initial value was determined b using the method of Aarst and Korst. 6 he performance of this method was compared with the method of Prosenjit and Diwekar, 5 and the calculated ratios between the initial pseudo temperatures, the number of functions evaluated, and the error % com-

4 Y. ÖZÇELIK and Z. ÖZÇELIK, Solving Mied Integer Nonlinear Chemical, Chem. Biochem. Eng. Q. 8 (4) 9 5 (004) pared with the values of the objective function in global minimum, are given in table. he initial values of the pseudo temperatures calculated b the method of Aarst and Korst 6 were greater than the others. herefore, this method causes the evaluations of more functions. If onl these parameters are considered, the method of Prosenjit and Diwekar 5 has an advantage over the method of Aarst and Korst. 6 But the low initial values of pseudo temperatures caused to converge to a local minimum (Problem 6). able he ratios for initial temperatures and number of functions evaluated and errors compared with the values of the objective function in global minimum Error Problem ia / ip N A /N P e A /% Error e P /%.00. 0.05 0 95.6.46 0.054 0.054 7. 0.49 0.864 0.59078 4 484997.00 0.0006 0.0006 5 04.9.54.84 4.987 6 90.9.5 0.008565 0.695 7 00.06.00 0 0 8 6.09.56 0.4076.64 9 60.99.70 0.00466 0.05458 0 00.9.00 0.88589 0.88589 5..87 0.0048 0.056 ferred to decide the acceptance of the new point generated. he nonequilibrium Metropolis algorithm prevented the computational burden of the algorithm for test problems. Although the nonequilibrium Metropolis algorithm can lead to local optima this effect has not been observed for the test problems in this work. 4. Dealing with constraints: he constrained problems are transformed into the unconstrained ones using the following penalizing scheme, 8 F(, M) f ( ) m n M [ma( 0, g,( ))] h j ( ) i j () M is a positive penalt parameter. he value of the penalt parameter can ideall affect the convergence. M = 00 gave good results for the test problems. he algorithm is an infeasible path method and uses all of the perturbed function values. 5. ermination criteria: he constant number of moves and constant number of step adjustment approaches, that were used, depend on the number of variables used for constant temperature search. he number of moves = 5 * the number of variables he number of step adjustment = 5 * the number of variables Otherwise, the relation (9) was used to terminate the cooling process. he method of Corona was used to adjust the value of the step reduction at the end of inner loop. 6. Annealing scheduling: he method of Aarst and Van Laarhoven 6 (7) was used to reduce the pseudo temperature of the problem.. Generation of a new point: In the second step of the algorithm, the test points were generated using the following relations. r e i i v mh h i LB IN( usr v) (0) i is the current optimum for the continuous variables, r and u are random numbers in the intervals05.,. 05and 0,, respectivel. e h is the vector of the h th coordinate direction and v mh is the component of the step vector (v m ) along the same direction. LB and sr is the vector of the lower bounds and search region for discrete variables.. he acceptance of the point generated: he method of nonequilibrium Metropolis was pre- Conclusions An algorithm was proposed for MINLP problems. It was based on the algorithm of Corona that can solve onl unconstrained continuous problems. he nonequilibrium Metropolis method 9 and a penalizing scheme were also adapted to prevent computational burden and to improve the reliabilit of the algorithm, respectivel. he proposed algorithm was applied to solve several MINLP test functions published in literature. he performance of the algorithm is comparable with the performance of the M SIMPSA for all ill conditioned problems. he average number of function evaluations for 00 runs, and success percentages for eleven ill-conditioned MINLP test problems, are given in table.

Y. ÖZÇELIK and Z. ÖZÇELIK, Solving Mied Integer Nonlinear Chemical, Chem. Biochem. Eng. Q. 8 (4) 9 5 (004) 5 able he fraction of successes and the average number of function evolutions (he success fractions (%) were given for feasible path and infeasible path alternatives of M SIMPSA) Problem he value for speed parameter was a constant of 0.0 for all problems. his speed value provides convergence to problems 5, 7, 9 to a local minimum in 0, 0 and 6 times respectivel per 00 runs. he convergence times were increased to 9, 00, 00 for these problems reducing the value of speed parameter to 0.0007, 0.00 and 0.0006, respectivel. he computer program developed in this work, besides running the proposed algorithm based on mentioned criteria, is open in the user interface to tr different criteria for the basic steps and compare with the structure given in this paper. Nomenclature M Simpsa Saran N F success N F success 68 99 00 567 00 4440 8 00 887 00 804 0 00 090 00 4577 00 58 00 5 495 00 00 5 90 6 4477 00 568 00 7 675 60 97 406 70 8 546 00 498 00 9 956 87 95 674 94 0 5756 546 00 849 8554 95 e A Accurac of convergence to the global minimum for Aarst and Korst 5 criteria, % e D Accurac of convergence to the global minimum for Diwekar method, % e NEM Accurac of convergence to the global minimum for nonequilibrium method, % MINLP Mied Integer NonLinear Programming N Number of variables N A he number of function evaluations in Aarst and Korst 5 procedure in determining initial annealing temperature N Aarst he number of function evaluations in Aarst and Korst 5 procedure in reducing annealing temperature. N CR he number of function evaluations in constant region reduction method N D he number of function evaluations in the method of Diwekar to determine initial values of annealing temperatures N F N Feasible N Infeasible he number of function evaluations he number of function evaluations in the feasible path procedure he number of function evaluations in the infeasible path a procedure N KirkPatrick he number of function evaluations in the method of Kirkpatrick in reducing annealing temperatures N M he number of function evaluations in the method of Metropolis in accepting the points generated N NEM he number of function evaluations in the method of nonequilibrium Metropolis in accepting the points generated NESA Nonequilibrium Simulated Annealing NLP Nonlinear Programming i A he initial value of Pseudo annealing temperature when the method of Aarst and Korst 6 was applied i D he initial value of Pseudo annealing temperature when the method of Diwekar was applied E he change in the energ of the sstem F he change in the value of objective function References. Corona, A. et al., ACM ransactions on Mathematical Software, () (987) 6.. Margarida F. Cardosa et al., Computers and Chem. Engng., () (997) 49.. Kocis, G. R., Grossmann, I. E., Ind. Engng. Chem. Res., 7 (988) 407. 4. Marco A. Duran, Ignacio E. Grossmann, Mathematical Programming 6 (986) 7. 5. Prosenjit D. Chaudhuri, Urmila M. Diwekar, Ind. Engng. Chem. Res., 6 (997) 685. 6. Aarst E., Korst, Simulated Annealing and Boltzmann Machines-a Stochastic Approach to Combinatorial Optimisation and Neural Computers. Wile, New York, 989. 7. Aarst E. H. L., Van Laarhoven P. J. M., (985), Philips J. Res., 40 (985) 9. 8. S. Kirkpatrick et al., Science, 0 (4598) 67. 9. Margarida F. Cardosa et al., Optimisation Computers Chem. Engng. 0 (9) (996) 065. 0. Floudas et al., Computers and. Chem. Engng. (0) (989) 7.. Roo, H. S., Sahinidis. N. V., Computers and Chemical. Engineering, 9 (5) 55.. Salcedo, R. L., Ind. Engng. Chem. Res., () 6.. Diwekar, U. M. et al., Ind. Engng. Chem. Res. (99). 4. Diwekar, U. M., Rubin, E. S., Ind. Engng. Chem. Res. (9) (99) 006. 5. Yuan, X. et al., Une Methode D optimization Nonlineaire en Variables Mites Pour la Conception de ProcPdes, RAIRO oper. Res. 989. 6. Berman, O., Ashrafi, N., IEEE rans. Soft. Eng. 9 (99) 9. 7. Wong, J., COED J. 0 (990) 9. 8. Yiping W., Luke E.K., Computers and. Chem. Engng. 6 (00) 45.