1 Computers and Structures xxx (4) xxx xxx Globalzed Nelder Mead method f engneerng optmzaton Marco A. Luersen a,b, *, Rodolphe Le Rche c a CNRS UMR 638/LMR, INSA de Rouen, Lab. de Mecanque, Avenue de lõunverste, St Etenne du Rouvray 768, France b Mechancal Department, CEFET-PR, Av. e de embro, Curtba, 83F9, Brazl c CNRS UMR 546/SMS, Ecole des Mnes de Sant Etenne, 58 Cours Faurel, 43, Sant Etenne, France Receved December ; accepted 4 March 4 Abstract One of the fundamental dffcultes n engneerng desgn s the multplcty of local solutons. Ths has trggered great effts to develop global search algthms. Globalty, however, often has a prohbtvely hgh numercal cost f real problems. A fxed cost local search, whch sequentally becomes global s developed. Globalzaton s acheved by probablstc restart. A spatal probablty of startng a local search s bult based on past searches. An mproved Nelder Mead algthm makes the local optmzer. It accounts f varable bounds. It s addtonally made me robust by rentalzng degenerated smplexes. The resultng method, called Globalzed Bounded Nelder Mead (GBNM) algthm, s partcularly adapted to tackle multmodal, dscontnuous optmzaton problems, f whch t s uncertan that a global optmzaton can be affded. Dfferent strateges f restartng the local search are dscussed. Numercal experments are gven on analytcal test functons and composte lamnate desgn problems. The GBNM method compares favably to an evolutonary algthm, both n terms of numercal cost and accuracy. Ó 4 Cvl-Comp Ltd. and Elsever Ltd. All rghts reserved. Keywds: Global optmzaton; Nelder Mead; Composte lamnate desgn. Introducton Complex engneerng optmzaton problems are characterzed by calculaton ntensve system smulatons, dffcultes n estmatng senstvtes (when they exst), the exstence of desgn constrants, and a multplcty of local solutons. Acknowledgng the last pont, much research has been devoted to global optmzaton (e.g., [,]). The * Crespondng auth. Address: CNRS UMR 638/LMR, INSA de Rouen, Lab. de Mecanque, Avenue de lõunverste, St Etenne du Rouvray 768, France. Tel.: ; fax: E-mal address: (M.A. Luersen). hgh numercal cost of global optmzers has been at the gn of subsequent effts to speed up the search ether by addng problem specfc knowledge to the search, by mxng effcent, local algthms wth global algthms. There are many ways n whch local and global searches can cooperate. The smplest strategy s to lnk the searches n seres, meanng that, frstly, a global optmzaton of lmted cost s executed, the soluton of whch s refned by a local search. An example of the seral hybrd s gven n  where smulated annealng, the global optmzer, s coupled wth a sequental quadratc programmng and a Nelder Mead algthm. A large number of parallel local-global searches have been proposed [,4,5] and analyzed [6,7]. In these cases, teratons of global and local algthms are ntertwned /$ - see front matter Ó 4 Cvl-Comp Ltd. and Elsever Ltd. All rghts reserved. do:.6/j.compstruc.4.3.7
2 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx One can further classfy parallel hybrds nto those where the local searches converge, and those where local searches may be prematurely stopped. Memetc genetc algthms  and multstart methods (e.g., determnstc restart n , random restarts n ) are examples of the fmer. The latter are usually based on clusterng steps, where local searches approachng already expled regons of the desgn space are abandoned [6,]. When consderng a real engneerng optmzaton problem, a common stuaton s that the affdable total number of analyses s lmted, that the presence of spurous local mnma s unknown, and that t s uncertan f t wll be possble to complete as few as two local searches. Nevertheless, achevng global results remans an objectve of the optmzer. Ths typcally occurs when dealng wth an unknown functon of less than varables, f whch one s wllng to wat f about evaluatons of the objectve functon. In such a case, a local-global method based on restart s the safest strategy because t can termnate n a sht tme (the length of a sngle local search). The method descrbed n ths artcle, the Globalzed Bounded Nelder Mead algthm (GBNM) s meant to be a black-box local-global approach to real optmzaton problems. A restart procedure that uses an adaptve probablty densty keeps a memy of past local searches. Lmts on varables are taken nto account through projecton. Fnally, GBNM can be appled to dscontnuous (no gradent nfmaton needed), non-convex functons, snce the local searches are based on a varant of the Nelder Mead algthm . Improvements to the Nelder Mead algthm consst of smplex degeneracy detecton and handlng through rentalzaton. Ths paper s structured as follows. The GBNM algthm s descrbed n Secton, and Secton 3 repts numercal experments on analytcal functons and composte lamnated plate desgn problems. In partcular, dfferent strateges f restartng the mproved Nelder Mead search are numercally dscussed. The GBNM algthm s also compared to a steady-state evolutonary algthm .. Globalzaton of a local search by probablstc restart Local optmzers can make up a global search when repeatedly started from dfferent ponts. The smplest restart methods ntalze the search ether from a regular grd of ponts, from randomly chosen ponts. In the frst case, one needs to know how many restarts wll be perfmed to calculate the sze of the mesh. In the other case, knowledge of past searches s not used, so that the same local optma may be found several tmes, costng vast unnecessary efft. In the current wk, the number of restarts s unknown befehand because a maxmum number of analyses s mposed and the cost of each local search s unknown. A grd method cannot be appled here. Also, a memy of prevous local searches s kept by buldng a spacal probablty densty of startng a search... Probablstc restart The probablty, p(x), of havng sampled a pont x s descrbed here by a Gaussan Parzen-wndows approach . Ths method can be consdered as a smoothed verson of the hstograms technques, the hstograms beng centered at selected sampled ponts. The probablty p(x) s wrtten, pðxþ ¼ N X N ¼ p ðxþ; ðþ where N s the number of ponts already sampled, and p s the nmal multdmensonal probablty densty functon, p ðxþ ¼ ðpþ n ðdetðrþþ exp ðx x Þ T R ðx x Þ ; ðþ n s the dmenson (number of varables) and R the covarance matrx, R ¼ 6 4 r... r n : ð3þ The varances, r j, are estmated by the relaton, ; r j ¼ a xmax j x mn j ð4þ where a s a postve parameter that controls the length of the Gaussans, and x max j and x mn j are the bounds n the jth drecton. te that, n der to keep the method as smple and cost effectve as possble, the varances are kept constant. Ths strategy would have a cost n terms of total number of analyses. The probablty densty s such that R pðxþdx ¼, but snce a bounded doman X s consdered, a bounded probablty ~pðxþ s ntroduced, ~pðxþ ¼ pðxþ Z M ; M ¼ pðxþ dx; ð5þ X so that R ~pðxþdx ¼. X The probablty densty of samplng a new pont, /(x), s a probablty densty of not havng sampled x befe. F ts estmaton we adopt the followng assumpton: only the hghest pont x H of ~pðxþ has a null probablty of beng sampled at the next teraton. So, the probablty /(x) s calculated as,
3 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 3 H p( xh) p p( x ) Xmn H p( x ) H ~pðxþ /ðxþ ¼R ; H ¼ max ðh ~pðxþþdx ~pðxþ: ð6þ xx X Fg. llustrates p(x), ~pðxþ and H ~pðxþ, n a undmensonal doman. The maxmzaton of / s not perfmed exactly, frstly because of ts numercal cost, and secondly, as wll be seen n Secton 3., because t would be detrmental to the search. Instead, N r ponts are chosen randomly and the pont that maxmzes / s selected to ntate the next search. te that, n der to maxmze /, t s necessary to calculate nether M (5) n H (6): the maxmum of / s the mnmum of p, sop only s calculated. Three parameters nfluence the probablty densty p and, consequently, the startng ponts: the ponts that are kept f the probablty calculaton, p; the number of random ponts used to maxmze /, N r ; and the Gaussans length parameter, a. Ther settng s dscussed n the numercal results (Secton 3.). The probablstc restart procedure can be appled to any local optmzer. In ths case, an mproved Nelder Mead algthm s proposed... An mproved Nelder Mead search x H Xmax Fg.. Probablty densty functons. Ω The gnal Nelder Mead algthm  and the strategy f boundng varables are summarzed n Appendx A. The GBNM algthm dffers from the Nelder Mead method because of a set of restart optons. The purpose of the restarts s twofold. Frstly, probablstc restarts based on the densty p (Eq. ()) am at repeatng local searches untl a fxed total cost, C max has been reached. The probablty of havng located a global optmum ncreases wth the number of probablstc restarts. Ths s the globalzed aspect of the method. In the current mplementaton of probablstc restart, the sze of the new smplex, a (defned n Eq. (A.)), s a unfm random varable taken between % and % of the smallest doman dmenson. Secondly, restarts are used to check and mprove convergence of the algthm. The two-restart schemes that are convergence related ntalze a new smplex from the current best vertex. The small and large test restarts use a small and large smplex of szes a s and a l, respectvely (see Eq. (A.)). Convergence of the local Nelder Mead searches s estmated through three crtera, the small, flat degenerate smplex tests. The smplex s small f! max k¼;...;nþ X n k¼ x k x max x mn < e s ; ð7þ where x k s the th component of the kth edge, x mn and x max are the bounds n the th drecton, and e s s a termnaton tolerance. The smplex s flat f jf H f L j < e s ; ð8þ where f H and f L are the hghest and lowest objectve functons n the smplex, and e s s a tolerance value. The smplex s degenerated f t has collapsed nto a subspace of the search doman. Ths s the most common symptom of a faled Nelder Mead search  because the method cannot escape the subspace. Me precsely, a smplex s called degenerated here f t s nether small, n touches a varable bound, and one of the two followng condtons s satsfed: mn k¼;n kek k max k¼;n kek k < e s3 det½eš Q ke k k < e s4; k ð9þ where e k s the kth edge, e s the edge matrx, kæk represents the Eucldean nm, and e s3 and e s4 are small postve constants. The lnkng of the three restarts and three convergence tests n the GBNM algthm s shown n Fg.. A memy of past convergence locatons s kept, thus preventng unnecessarly spendng computatons on already analyzed ponts (thrd test, T3, n the flow chart of Fg. ). When the smplex s flat, a probablstc restart s perfmed (T4). A smplex whch s degenerated nduces a large test teraton (T8). When the optmalty of the convergence pont s unsure, such as a convergence on a varable bound where the smplex has degenerated (T6), a small test, that stands f an optmalty check, s perfmed. If the small smplex returns to the same convergence pont, t s consdered to be a local optmum. It should be remembered that the Kuhn and Tucker condtons of mathematcal programmng are not applcable to the present non-dfferentable framewk. The tolerances f small and degenerated smplces, e s and [e s3,e s4 ], respectvely, may be dffcult to tune, so that a smplex whch s becomng small may be tagged as degenerated befe. Thus, f a degeneraton s detected twce consecutvely at the same pont, the pont s taken as a possble optmum, and a probablstc restart s called. Smlarly, f a degeneraton s detected after a small test, ths pont s also saved as a possble optmum, and a large test s dered.
4 4 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx INITIALIZATION restart = PROBABILISTIC Save ntal pont To calculate the probablty densty Nelder-Mead teraton wth projecton (cf. Fg. 5) Save ntal pont RESTART To calculate the probablty densty PROBABILISTIC LARGE TEST SMALL TEST T. of evaluatons > Cmax T Converge The smplex s: small flat degenerated Save the smplex best pont END restart = PROBABILISTIC T3 Pont already known as a local optmum restart = PROBABILISTIC T4 Smplex flat T5 restart = PROBABILISTIC Save local optmum (SMALL LARGE TEST and return to the same pont) (not SMALL TEST and pont not on the bounds and small smplex) T6 restart = SMALL TEST Possble local optmum LARGE TEST PROBABILISTIC and not return to the same pont and pont on the bounds restart = LARGE TEST Save the smplex best pont (possble local optmum) T7 SMALL TEST and not return to the same pont and not on the bounds T8 restart = LARGE TEST t SMALL TEST and ponts not on the bounds and smplex not small and smplex degenerated Degeneracy case restart = SMALL TEST Fg.. Restarts and convergence tests lnkng n GBNM. Once the GBNM algthm termnates, the lst of possble local (eventually global) optma makes the results of the search. In practce, the calculaton of many local global optma s a beneft of the method n comparson wth global optmzers that provde a sngle soluton (e.g., evolutonary algthms). Fnally, t
5 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 5 should be noted that, n der to fnd all the local optma, the number of restarts should be larger equal to the number of local optma. Typcally, me restars are requred because searches started at dfferent ponts may converge to the same local soluton. 3. Numercal results In Secton 3., the choce of GBNM parameters s dscussed. Results on an analytcal functon are gven n Secton 3.and composte lamnate desgn problems are addressed n Sectons 3.3 and 3.4. The GBNM method s compared to an evolutonary algthm (EA). The evolutonary algthm  has a steady-state structure  wth real encodng, contnuous crossover, and Gaussan mutaton of varance r mut ¼ðx max x mn Þ =6. F far comparsons, the parameters of the EA chosen f each test are the ones that perfm best n ndependent trals among all combnatons of populaton szes ( 5), mutaton probabltes (.5.) and crossover probabltes (.4.5). 3.. GBNM parameters choce 3... The Gaussans length parameter, a In ths wk, a s set to., whch means that one standard devaton away from the Gaussan mean pont covers about % of the doman Ponts kept f the probablty calculaton Three strateges have been compared n terms of probablty of not fndng at least one local mnmum, P nfm : the x Õs used n Eqs. () and () are () the startng ponts, () the startng and local convergence ponts, () all the ponts sampled durng the search. One should remember that local convergence ponts are never duplcated (test T3 on Fg. ). From prelmnary tests, t has been observed that strategy () s memy and tme consumng, wth degraded perfmance wth respect to strateges () and (). F these reasons, t has not been consdered f further testng. Strateges () and () are tested wth N r varyng from to, on three functons, 8 f ðx ; x Þ¼ þ :ðx x Þ þð x Þ þ ð x Þ þ snð:5x Þ snð:7x x Þ x ; x ½; 5Š; >< f ðx ; x Þ¼ 4 :x þ 3 x4 x þ x x þð 4 þ 4x Þx x ; x ½ 3; 3Š; f 3 ðx ; x Þ¼ x 5: x 4p þ 5 x p 6 þ cosðx Þþ 8p >: x ½ 5; Š; x ½; 5Š; ðþ f has 4 local mnma, f 6, and f 3 3 (see Fg. 3). f s known as the Sx Humps Camel Back functon and f 3 as the BrannÕs rcos functon. F the frst and the second strateges, results after 5 analyss and based on ndependents runs are presented n Tables and, respectvely. The second strategy perfms best, ndependently of N r. It shows that the startng and local convergence ponts effcently summarze the topology of the basns of attracton. Ths scheme s chosen to update p Number of random ponts, N r If N r s equal to, the rentalzaton s random. If N r s large, the ntal ponts are dstrbuted based on the ntal and convergence ponts of past searches, whch nduces a regular grd-lke pattern. tng N r to a small number, larger than, gves a based-random rentalzaton. It should be seen as a compromse between the grd and the random strateges. Optmum value of N r depends on the test functon: f the basns of attracton are regularly dstrbuted, restarts followng a regular pattern (.e., N r large) are optmal, and vce versa. From the tests results on the three multmodal functons presented n Table, the optmal strategy s N r large f f and f 3, whle t s N r =f f. N r = s chosen as a compromse f general functon optmzaton. 3.. GrewankÕs functon mnmzaton Consder the mnmzaton of the GrewankÕs test functon, F ðx ;...; x n Þ¼ X n x Yn x cos pff ; 4n ¼ ¼ x ½ ; Š; ðþ where the dmenson s n=and the global mnmum s. at x =., =,n. Ths functon has many local mnma. Fg. 4 shows t n the one-dmensonal case (n=), x[,]. Table 3 compares GBNM and the best pont n the populaton of an evolutonary algthm (EA) at,, 5 and, functon calls. Table 3 averages ndependent runs where the startng pont of the GBNM s randomly selected. The followng crteron s used to consder the EA has convergedd to the global mnmum: n k^x x k < ; ðþ where ^x s the best pont found, and x * the global mnmum. The man observaton s that the GBNM method fnds, on average, better objectve functon values, wth a hgher probablty of fndng the global mnmum than the EA does. The advantage of the GBNM method s substantal at a low number of analyses, and slowly decreases as numercal cost grows.
6 6 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 5 Functon f 5 Functon f3 4 3 x x (a) x -5 5 (c) x.5 Functon f.5 x (b) x Fg. 3. Test functons: contour plots Bucklng load maxmzaton Composte lamnates are made of stacked layers where each layer has ented fbers melted n an sotropc matrx (see sketch n Fg. 5). The desgn problems addressed here am at fndng the optmal entaton of the fbers wthn each layer, h, where h s a contnuous varable bounded by and 9. The plates are analyzed Table Probablty P nfm of not fndng at least one of the local mnma (C max =5 analyses, runs, only the startng ponts are kept f the probablty densty calculaton) N r f f f 3 (random restart) Table Probablty P nfm of not fndng at least one of the local mnma (C max =5 analyses, runs, startng and convergence ponts are kept f the probablty densty calculaton) N r f f f 3 (random restart)
7 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 7 F(x) x Fg. 4. GrewankÕs test functon (n=). usng the classcal lamnaton they and an elastc lnear bucklng model (see ). Consder a smply suppted carbon-epoxy square plate, subjected to n-plane compressve loads N x =N y, as shown n Fg. 5. The plate s balanced and symmetrc and has 3layers, each of whch are.5 mm thck. The elastc materal propertes of the layers are E =5 GPa, E =5 GPa, G =5 GPa, m =.35. The lamnate s desgned to maxmze ts bucklng load (assocated to the most crtcal bucklng mode). Snce the plate s balanced and symmetrc, there are eght contnuous desgn varables, the ply entatons, whch are bounded between and 9. Ths problem has a unque mnmum, all ples ented at 45. te also that the outermost ples have me nfluence on the bucklng behav than the nnermost ples. Table 4 compares GBNM and the evolutonary algthm (EA), showng the stackng sequences found, after 3, 5 and analyses based on ndependents runs. At 3 evaluatons, the bucklng load of the desgns proposed by GBNM and EA are equvalent. One notces that the frst local search of GBNM has not always converged at that pont. A steady dfference between GBNM and EA desgns at 3 analyses can be seen: GBNM, whch s by nature a me ented search, converges faster on the me senstve outerples than EA does, and vce versa on the nnerples. From 5 evaluatons on, GBNM converges me accurately to the optmum than EA does Composte plate longtudnal stffness maxmzaton A 6-ply balanced and symmetrc plate, made of glass-epoxy, s to be desgned by maxmzng the longtudnal stffness E x (see [7, p. 47]). The elastc propretes f the glass-epoxy layers are E =45 GPa, E = GPa, Table 3 Comparson of GBNM (N r =) and the best of an EA populaton on GrewankÕs functon, runs (average ± standard devaton) analyses analyses 5 analyses, analyses Mnmum functon value Probablty of fndng the global mnmum Mnmum functon value Probablty of fndng the global mnmum Mnmum functon value Probablty of fndng the global mnmum Mnmum functon value Probablty of fndng the global mnmum GBNM 9.3±6.79 /.56±.499 /.947±.74 5/.98±.4 3/ EA ±5.537 / 4.86±.9 /.9±.96 /.57±. 9/ θ = fber entaton N y N x a a Fg. 5. Smply suppted rectangular plate subjected to n plane loads.
8 8 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx Table 4 Bucklng load maxmzaton, runs, N r = (stackng sequence: average ± standard devaton) 3 analyses GBNM [±45.4 ±44.97 ±45.±45.38 ±45.38 ±44.97 ±43.8 ±49.65] s Std. Dev. ±.47 ±.54 ±.83 ±4.6 ±4.37 ±.53 ±7.46 ±3.47 EA [±45.9 ±44.9 ±45.3 ±44.55 ±44.78 ±45. ±45.6 ±44.85] s Std. Dev. ±.75 ±.96 ±.67 ±3. ±3.66 ±5.9 ±8.8 ± analyses GBNM [±45. ±45.3 ±45. ±45.8 ±45.4 ±44.97 ±45.5 ±45.8] s Std. Dev. ±.7 ±. ±.39 ±.4 ±.9 ±.46 ±.9 ±4. EA [±45.3 ±44.95 ±44.99 ±44.95 ±44.8 ±45. ±44.7 ±46.45] s Std. Dev. ±.9 ±.8 ±.6 ±.9 ±. ±3. ±4.49 ±.3 analyses GBNM [±45. ±45. ±45. ±45. ±44.99 ±45. ±44.98 ±45.] s Std. Dev. ±.±.±.3 ±.5 ±.4 ±.6 ±.5 ±.44 EA [±44.96 ±44.98 ±44.96 ±45.7 ±44.99 ±44.9±45.3 ±45.7] s Std. Dev. ±.6 ±.6 ±.7 ±.95 ±.±.7 ±.7 ±4.95 G =4.5 GPa and m =.3. The plate s balanced and symmetrc, so there are four fber entatons to be found. Ths problem presents 6 local mnma whch are all the combnatons of the and 9 entatons. The global maxmum has all ples ented at. Ths example shows that local solutons exst f smple composte lamnate desgn problems. Table 5 gves the average number of local optma found after analyses based on GBNM runs as a functon of N r. The frst startng pont s chosen randomly. One observes that the average number of optma found grows wth N r. Ths s expected snce the optma are regularly dstrbuted n the doman. Meover, wthn a budget of functon evaluatons, the global optmum s always found because ts basn of attracton s the largest one. Table 5 GBNM E x maxmzaton, analyses, runs N r Average number of optma found Standard devaton Acknowledgment The frst auth would lke to express hs thanks to the Brazlan fundng agency CNPq f the fnancal suppt durng ths research. 4. Concludng remarks A local/global optmzaton method based on probablstc restart has been presented. Local searches are perfmed by an mproved Nelder Mead algthm where desgn varables can be bounded and some search falure cases prevented. The method, called Globalzed and Bounded Nelder Mead search, does not need senstvtes and constructvely uses computer resources up to a gven lmt. It yelds a lst of canddate local optma, whch contan, wth an ncreasng probablty n terms of computer tme, global solutons. The GBNM method s smple n ts prncples, and the afementonned features make t partcularly useful n an engneerng desgn context. It has been found to compare favably to an evolutonary optmzaton algthm n terms of convergence speed, accuracy of results and ablty to fnd local optma. Appendx A. A Nelder Mead algthm wth bounded varables The Nelder Mead method  s the most popular drect search method f mnmzng unconstraned real functons. It s based on the comparson of functon values at the n+ vertces x of a smplex. A smplex of sze a s ntalzed at x based on the rule (see ), x ¼ x þ pe þ Xn qe k ; ¼ ; n; ða:þ k¼ k6¼ where e are the unt base vects and p ¼ a p n ffffff pffffffffffffffffffffff n þ þ n ; q ¼ a p n ffff pffffffffffffffffffffff n þ : ða:þ
9 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 9 Smplex ntalzaton P : smplex pont f : objectve functon value at P Ph : smplex pont where the objectve functon assumes ts hghest value Ps : smplex pont where the objectve functon assumes ts second hghest value Determne Ph, Ps, Pl, Pm fh, fs, fl Pl : smplex pont where the objectve functon assumes ts lowest value Pm : centrod smplex pont (not consderng Ph) r : reflecton coeffcent =. Reflecton: Pr = Pm + r (Pm-Ph) If Pr s out of the doman: Projecton on bounds β: contracton coeffcent = / γ : expanson coeffcent = fr < fl fr <= fs fr < fh new Nelder Mead teraton Expanson: Pe = Pm + γ (Pr-Pm) If Pe s out of the doman: Projecton on bounds fe < fr Replace Ph by Pr Contracton : Pc = Pm + β(ph-pm) fc > fh Replace Ph by Pe Replace Ph by Pr Replace all P s by (P + Pl)/ Replace Ph by Pc Converged end Fg. 6. Nelder Mead algthm wth bounded varables. The smplex vertces are changed through reflecton, expanson and contracton operatons n der to fnd an mprovng pont (see Fg. 6). The algthm termnates when the vertces functon values become smlar, whch s measured wth the nequalty, vffffffffffffffffffffffffffffffffffffffffffffffffffff ux t nþ ðf f Þ < e; X nþ f ¼ f ; ða:3þ n n þ ¼ where e s a small postve scalar. The cumulatve effect of the operatons on the smplex s, roughly speakng, to stretch the shape along the descent drectons, and to zoom around local optma. Two comments on the propertes of the algthm are added. Frstly, the Nelder Mead algthm may fal to converge to a local optmum, whch happens n partcular when the smplex collapses nto a subspace. Secondly, the method may escape a regon that would be a basn of attracton f a pontwse descent search f the smplex s large enough. ¼ Ultmately, as the sze of the smplex decreases, the algthm becomes local. The gnal Nelder Mead algthm was conceved f unbounded doman problems. Wth bounded varables, the ponts can leave the doman after ether the reflecton the expanson operaton. It s straghtfward to account f varables bounds by projecton, ( f ðx < x mn Þ; x ¼ x mn ; ða:4þ f ðx > x max Þ; x ¼ x max : The flowchart of the Nelder Mead method shown n Fg. 6 dffers from the gnal method only n the ntalzaton (Eq. (A.)) and n the bounded varables. An mptant sde effect of accountng f the bounded varables through projecton s that t tends to make the smplex collapse nto the subspace of the saturated varables. A specfc convergence test, based on a small smplex rentalzaton at the pont of convergence s then requred (see small test n Secton.).
10 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx References  Törn AA, Zlnskas A. Global optmzaton. Berln: Sprnger-Verlag; 989.  Bäck T. Evolutonary algthms n they and practce. Oxfd Unversty Press; 996.  Shang Y, Wan Y, Fromherz MPJ, Crawfd L. Toward adaptve cooperaton between global and local solvers f contnuous constrant problems. n: CPÕ Wkshop on cooperatve Solvers n Constrants Programmng, Pahos, Cyprus, December.  Durand N, Allot J-M. A Combned Nelder Mead smplex and genetc algthm. Avalable from:  Okamoto M, naka T, Ocha S, Tomnaga D. nlnear numercal optmzaton wth use of a hybrd genetc algthm ncpatng the modfed Powell method. Appl Math Comput 998;9:63 7.  Törn AA. A search-clusterng approach to global optmzaton. Towards Global Optm 978;:49 6.  Goldberg DE, Voessner S. Optmzng global local search hybrds. In: GECCO 99 Genetc and Evolutonary Computaton Conference, Orlando, FL, USA, 999. p. 8.  Moscato P. On Evoluton, search, optmzaton, genetc algthms and martal arts: towards memetc algthms. Caltech Concurrent Computaton Program, C3P Rept 86, 989.  Barhen J, Protopopescu V, Rester D. TRUST: A determnstc algthm f global constraned optmzaton. Scence 997;76(May):94 7.  Hu X, Shonkwller R, Sprull MC. Random restarts n global optmzaton. Techncal Rept, School of Mathematcs, Gega Insttute of Technology, Atlanta, January 994.  Hckernell FJ, Yuan Y-X. A smple multstart algthm f global optmzaton. OR Trans 997;:.  Nelder JA, Mead R. A smplex f functon mnmzaton. Comput J 965;7:38 3.  Duda OR, Hart PE, Stk DG. Pattern classfcaton. nd ed.. New Yk: John Wley & Sons;.  Wrght MH. Drect search methods: one scned, now respectable. In: Dundee Bennal Conference n Numercal Analyss, Harlow, UK, 996. p  Syswerda G. A study of reproducton n generatonal and steady state genetc algthms. In: Rawlns GJE, edt. Foundatons of genetc algthms. San Mateo, CA: Mgan Kaufmann; 99.  Berthelot J-M. Composte materals mechancal behav and structural analyss. Mechancal engneerng seres. - Sprnger; 999.  Haftka RT, Gürdal Z. Elements of structural optmzaton. 3rd rev. and expanded ed.. Kluwer Academc Publshers; 99.