Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o Abstract This paper itroduces a modified versio of the well kow global optimizatio techique amed lie search method. The modificatios refer to the way i which the directio ad the steps are determied. The modified lie search techique (MLS) is applied for some global optimizatio problems. Fuctios havig a high umber of dimesios are cosidered (50 i this case). Results obtaied by the proposed method o a set of well kow bechmarks are compared to the results obtaied by the stadard lie search method, geetic algorithms ad differetial evolutio. Numerical results show the effectiveess of the proposed approach while compared to the other techiques.. Itroductio Global optimizatio is still a challegig domai ad still a huge amout of work is published every year. The stadard mathematical techiques have bee improved, modified ad hybridized so that their performace is improved. I this paper, we propose a modificatio for oe of the stadard mathematical techiques for global optimizatio: lie search. This techique is very simple ad it has several variats. We propose here a ew way of choosig the values of its parameters, amely the directio ad step. Istead of usig some sophisticated ad time cosumig techiques to set the values of these parameters, we applied a radom method. We also cosider more tha o iitial (startig) poit. A detailed descriptio of the origial lie search techique ad the proposed modificatio is preseted i Sectio 2. I order to illustrate the performace of the modified approach we perform some umerical experimets by cosiderig several fuctios havig 50 dimesios. Some comparisos with some well kow techiques for optimizatio (such as geetic algorithms ad differetial evolutio which are shortly described i Sectio 3) are performed i Sectio 4. Coclusios are provided towards the ed. 2. Modified Lie Search (MLS) The origial lie search geeral method ca be described as follows: a search directio p ad a step s are determied at each iteratio k so that the followig coditios are fulfilled: - the directio p k (directio p at iteratio k) is a descet directio, i.e. p, g 0 if g 0 k k where g deotes the gradiet; k - the step s k is calculated so that f(x k + p k s k ) < f(x k ) There are several ways to calculate adequate values for s k (like backtrackig, etc). Readers are advised to cosult [3] for more details. Fidig the right value for s k ca be sometimes difficult. Figure illustrates few situatios cosiderig differet values for p k ad s k for optimizig the fuctio f(x) = x 2 for 0 iteratios. It is observed that for smaller values of s k the fuctio coverges very slowly while for greater values it ca eve miss the optimum. Takig ito accout of this problem, we propose a very simple modificatio of the stadard lie search method as give below: (i) Istead of computig (usig differet other methods) adequate values for s k ad p k we are simply geeratig at radom the values of these parameters at each iteratio. The values of these variables vary betwee the rage [-, ]. Also, the value of p k is modified at each iteratio by p k = p k (-) k+. (ii) Istead of cosiderig a sigle startig poit, a set of several radomly geerated poits are cosidered over the search space. The lie search procedure is applied from each of these poits. Proceedigs of the First Asia Iteratioal Coferece o Modellig & Simulatio (AMS'07)
Figure. Example of lie search method for the fuctio f(x)=x 2 cosiderig: (a) p k =(-) k+ ad s k =2+3/2 k+ ; (b) p k =- ad s k =/2 k+ ; (c) p k =- ad s k =3/2 k+ ; (d) p k =- ad s k =5/2 k+ We preferred this way of fidig a adequate value for s k due to the fact that at each iteratio the purpose is to improve the value of the fuctio by optimizig the ewly obtaied poit. Sice sometimes it ca be time cosumig to fid the right value for s k, we applied the radom procedure to geerate aother step util the value of the fuctio i the ewly obtaied poit is improved. This way, we esure that we are movig i a better positio which ca help i fidig the global optimum poit. The modified lie search method (pseudo code) is described below: Begi Geerate N poits i, i=,, N over the search space. k:=; Repeat For i= to N do repeat p k :=radom; if odd(i) the p k :=(-) p k ; s k :=radom; util f( i + p k s k )<f( i ); k:=k+; for all i i := i + p k s k Util coditio Prit the best solutio. Ed The MLS may be ru for a specified umber of iteratios or whe the best solutio is foud. The algorithm may be also termiated if the solutios foud are close to the optimal value with the kow optimal value. I Figure 2, we illustrate how the MLS works for 0 iteratios. 3. Techiques Used for Comparisos The results obtaied by MLS are compared with the results obtaied by lie search ad Geetic Algorithms for all the cosidered test fuctios. The obtaied results are also compared with Differetial Evolutio but oly for two of the cosidered test fuctios [5]. Proceedigs of the First Asia Iteratioal Coferece o Modellig & Simulatio (AMS'07)
Ed Step 3.. Evaluate idividuals from P(t); Step 3.2. Selectio o P(t). Let P (t) be the set of selected idividuals. Step 3.3. Crossover o P (t); Step 3.4. Mutatio o P (t); Step 3.5. Survival o P (t); Step 3.6. t=t+; P(t)=P (t-) Util t=number_of_geeratios 3.2 Differetial Evolutio Figure 2. Example of the LMS behaviour after 0 iteratios with radom p k ad s k. 3. Geetic Algorithms Geetic algorithms (GA) cosider a populatio of chromosomes (idividuals) ecodig potetial solutios to a give problem [2]. Each chromosome represets a poit i the search space. The idividuals i the populatio the go through a process of simulated evolutio. The search progress is obtaied by modificatio of the chromosome populatio. The most importat search operator is traditioally cosidered to be recombiatio (crossover). Radom mutatio of ewly geerated offsprig iduces variability i the populatio prevetig the premature covergece. A fitess fuctio is used to measure the quality of each idividual. The selectio for crossover is based o the fitess value. A probabilistic selectio operator esures the 'fittest' idividuals the highest probability to produce offsprig. Oe iteratio of the algorithm is referred to as a geeratio. The basic GA is very geeric ad there are may aspects that ca be implemeted differetly accordig to the problem (example, represetatio of solutio (chromosomes), type of ecodig, selectio strategy, type of crossover ad mutatio operators, etc.). I practice, GA's are implemeted by havig arrays of bits or characters to represet the chromosomes The basic geetic algorithm is described below: DE is a populatio based, stochastic fuctio miimizer. A populatio of solutio vectors is successively updated by the additio, subtractio, ad compoet swappig, util the populatio coverges to the optimum. V i = x r +F(x r2 - x r3 ). The algorithm starts with NP radomly chose solutio vectors. For each i (,,NP) a mutat vector is formed: Where r, r2, ad r3 are three mutually distict radomly draw idices from (, NP), ad also distict from i, ad 0<F<=2 Mutatio ad recombiatio are the operators used to improve the quality of solutios. 3. Experimet Setup ad Results We performed several experimets by cosiderig well kow test fuctios. I order to illustrate the performace of the algorithms used, we cosider a high umber of dimesios (50 i our case) because all these algorithms were tested for a small umber of dimesios ad the coclusio is that they all work pretty well. 3.. Test fuctios used There are several test fuctios for global optimizatio available i the literature. We used four test fuctios which is foud i [] [6] ad [7]. Although the objective fuctios are build i a way that the optimal solutios are kow, the optimizatio problems caot be trivially solved by search procedures that do ot exploit the special structure associated with each fuctio [4]. begi Step. Set t= 0; Step 2. Radomly iitialize the populatio P(t); Step 3. Repeat Proceedigs of the First Asia Iteratioal Coferece o Modellig & Simulatio (AMS'07)
Figure 3. Covergece toward the optimum solutios of the algorithms MLS, GA ad LS: (a) Sphere test fuctio; (b) Dixo ad Price test fuctio; (c) Ackley test fuctio; (d) Griewak test fuctio. The followig test fuctios were cosidered: Sphere fuctio (f ) 2 x i f(x)= i= Number of dimesios: ; Rage of iitial poits: - 0 xi 0 for i=...; Global miimum: x* = (0, 0,...,0), f(x*) = 0 Dixo ad Price fuctio (f 2 ) f(x)= 2 2 i( 2xi xi ) + ( x + i= Number of dimesios: ; Rage of iitial poits: - ) 0 xi 0 for i=...; Global miimum: z=2 i-, f(x*) = 0 2 x i z z = 2, Proceedigs of the First Asia Iteratioal Coferece o Modellig & Simulatio (AMS'07)
Fuctio Algorithm No of dimesios No of iitial poits (for MLS) ad populatio size for GA No of iteratios Optimum foud Actual optimum MLS 50 500 20,000 2.483 0 f f 2 f 3 f 4 GA 50 500 20,000 7.99 0 DE 50 500 20,000 93.77 0 LS 50 500 20,000 9.68 0 MLS 50 500 30,000 76.222 0 GA 50 500 30,000 29.78 0 DE LS 50 500 30,000 28.3 0 MLS 50 500 30,000 2.425 0 GA 50 500 30,000 3.530 0 DE 50 500 30,000 8.7 0 LS 50 500 30,000 6.43 0 MLS 50 500 30,000.0006 0 GA 50 500 30,000.00 0 DE LS 50 500 30,000.03 0 Table. Parameters used ad results obtaied by the cosidered techiques for all the four test fuctios. Ackley fuctio (f 3 ) 2 0.2 x i e i = i = f(x)=20 + e 20 - Number of dimesios: ; Rage of iitial poits: - 5.2 xi 5.2 for i=...; Global miimum: x* = (0, 0,...,0), f(x*) = 0 Griewak fuctio (f 4 ) f(x)= 2 xi 4000 i= i= e xi cos + i cos(2π ) Number of dimesios: ; Rage of iitial poits: - 0 xi 0 for i=...; Global miimum: x* = (0, 0,...,0), f(x*) = 0 x i 3.. Results ad discussios Table depicts the values of the parameters used for each techique ad the results obtaied for the four test fuctios. I Figure 3, the covergece of the test fuctios towards the optimum poit is depicted. Comparisos betwee MLS, GA ad LS are performed. As evidet from Table ad from Figure 3, MLS obtaied the best results for all the test fuctios (except for Dixo ad Price fuctio where the stadard LS performed well). Also, there is a big differece betwee results obtaied by MLS ad the results obtaied by the other techiques used (example: GA ad DE). Proceedigs of the First Asia Iteratioal Coferece o Modellig & Simulatio (AMS'07)
4. Coclusios I this paper, we proposed a modified versio of a well kow mathematical techique used for global optimizatio: lie search. The modified versio uses radom geerated values for directio ad step. Some umerical experimets were performed usig popular optimizatio fuctios ivolvig 50 dimesios. Comparisos with stadard lie search, geetic algorithms ad differetial evolutio were performed. Empirical results illustrate that the modified lie search algorithm performs better tha the other cosidered techiques ad better that the stadard lie search for three of the four test fuctios cosidered. The proposed approach ca be exteded for other classes of optimizatio problems ad for high dimesio problems. Refereces [] Floudas, C.A., Pardalos, P.M. A collectio of test problems for costrait global optimizatio algorithms, Spriger-Verlag, Berli Heidelberg, 990. [2] Goldberg DE (989), Geetic algorithms i search, optimizatio ad machie learig. Addiso Wesley, Readig, MA. [3] Gould, N., A itroductio to algorithms for cotiuous optimizatio, Oxford Uiversity Computig Laboratory Notes, 2006. [4] Lagua, M., Marti, R. Experimetal testig of advaced scatter search desigs for global optimizatio of multimodal fuctios, Joural of Global Optimizatio, 33, pp235-255, 2005 [5] Stor, R, O the usage of differetial evolutio for fuctio optimizatio. I: Bieial coferece of the North America fuzzy iformatio processig society, pp 59-523, 996. [6] www.cyberiad.et/realbech.htm (accessed o Ja 24, 2007) [7] www.solo.cma.uiv.ie.ac.at/»eum/glopt/my problems.html (accessed o Ja 24, 2007) Proceedigs of the First Asia Iteratioal Coferece o Modellig & Simulatio (AMS'07)