1 Computers and Structures xxx (4) xxx xxx Globalzed Nelder Mead method f engneerng optmzaton Marco A. Luersen a,b, *, Rodolphe Le Rche c a CNRS UMR 638/LMR, INSA de Rouen, Lab. de Mecanque, Avenue de lõunverste, St Etenne du Rouvray 768, France b Mechancal Department, CEFET-PR, Av. e de embro, Curtba, 83F9, Brazl c CNRS UMR 546/SMS, Ecole des Mnes de Sant Etenne, 58 Cours Faurel, 43, Sant Etenne, France Receved December ; accepted 4 March 4 Abstract One of the fundamental dffcultes n engneerng desgn s the multplcty of local solutons. Ths has trggered great effts to develop global search algthms. Globalty, however, often has a prohbtvely hgh numercal cost f real problems. A fxed cost local search, whch sequentally becomes global s developed. Globalzaton s acheved by probablstc restart. A spatal probablty of startng a local search s bult based on past searches. An mproved Nelder Mead algthm makes the local optmzer. It accounts f varable bounds. It s addtonally made me robust by rentalzng degenerated smplexes. The resultng method, called Globalzed Bounded Nelder Mead (GBNM) algthm, s partcularly adapted to tackle multmodal, dscontnuous optmzaton problems, f whch t s uncertan that a global optmzaton can be affded. Dfferent strateges f restartng the local search are dscussed. Numercal experments are gven on analytcal test functons and composte lamnate desgn problems. The GBNM method compares favably to an evolutonary algthm, both n terms of numercal cost and accuracy. Ó 4 Cvl-Comp Ltd. and Elsever Ltd. All rghts reserved. Keywds: Global optmzaton; Nelder Mead; Composte lamnate desgn. Introducton Complex engneerng optmzaton problems are characterzed by calculaton ntensve system smulatons, dffcultes n estmatng senstvtes (when they exst), the exstence of desgn constrants, and a multplcty of local solutons. Acknowledgng the last pont, much research has been devoted to global optmzaton (e.g., [,]). The * Crespondng auth. Address: CNRS UMR 638/LMR, INSA de Rouen, Lab. de Mecanque, Avenue de lõunverste, St Etenne du Rouvray 768, France. Tel.: ; fax: E-mal address: (M.A. Luersen). hgh numercal cost of global optmzers has been at the gn of subsequent effts to speed up the search ether by addng problem specfc knowledge to the search, by mxng effcent, local algthms wth global algthms. There are many ways n whch local and global searches can cooperate. The smplest strategy s to lnk the searches n seres, meanng that, frstly, a global optmzaton of lmted cost s executed, the soluton of whch s refned by a local search. An example of the seral hybrd s gven n  where smulated annealng, the global optmzer, s coupled wth a sequental quadratc programmng and a Nelder Mead algthm. A large number of parallel local-global searches have been proposed [,4,5] and analyzed [6,7]. In these cases, teratons of global and local algthms are ntertwned /$ - see front matter Ó 4 Cvl-Comp Ltd. and Elsever Ltd. All rghts reserved. do:.6/j.compstruc.4.3.7
2 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx One can further classfy parallel hybrds nto those where the local searches converge, and those where local searches may be prematurely stopped. Memetc genetc algthms  and multstart methods (e.g., determnstc restart n , random restarts n ) are examples of the fmer. The latter are usually based on clusterng steps, where local searches approachng already expled regons of the desgn space are abandoned [6,]. When consderng a real engneerng optmzaton problem, a common stuaton s that the affdable total number of analyses s lmted, that the presence of spurous local mnma s unknown, and that t s uncertan f t wll be possble to complete as few as two local searches. Nevertheless, achevng global results remans an objectve of the optmzer. Ths typcally occurs when dealng wth an unknown functon of less than varables, f whch one s wllng to wat f about evaluatons of the objectve functon. In such a case, a local-global method based on restart s the safest strategy because t can termnate n a sht tme (the length of a sngle local search). The method descrbed n ths artcle, the Globalzed Bounded Nelder Mead algthm (GBNM) s meant to be a black-box local-global approach to real optmzaton problems. A restart procedure that uses an adaptve probablty densty keeps a memy of past local searches. Lmts on varables are taken nto account through projecton. Fnally, GBNM can be appled to dscontnuous (no gradent nfmaton needed), non-convex functons, snce the local searches are based on a varant of the Nelder Mead algthm . Improvements to the Nelder Mead algthm consst of smplex degeneracy detecton and handlng through rentalzaton. Ths paper s structured as follows. The GBNM algthm s descrbed n Secton, and Secton 3 repts numercal experments on analytcal functons and composte lamnated plate desgn problems. In partcular, dfferent strateges f restartng the mproved Nelder Mead search are numercally dscussed. The GBNM algthm s also compared to a steady-state evolutonary algthm .. Globalzaton of a local search by probablstc restart Local optmzers can make up a global search when repeatedly started from dfferent ponts. The smplest restart methods ntalze the search ether from a regular grd of ponts, from randomly chosen ponts. In the frst case, one needs to know how many restarts wll be perfmed to calculate the sze of the mesh. In the other case, knowledge of past searches s not used, so that the same local optma may be found several tmes, costng vast unnecessary efft. In the current wk, the number of restarts s unknown befehand because a maxmum number of analyses s mposed and the cost of each local search s unknown. A grd method cannot be appled here. Also, a memy of prevous local searches s kept by buldng a spacal probablty densty of startng a search... Probablstc restart The probablty, p(x), of havng sampled a pont x s descrbed here by a Gaussan Parzen-wndows approach . Ths method can be consdered as a smoothed verson of the hstograms technques, the hstograms beng centered at selected sampled ponts. The probablty p(x) s wrtten, pðxþ ¼ N X N ¼ p ðxþ; ðþ where N s the number of ponts already sampled, and p s the nmal multdmensonal probablty densty functon, p ðxþ ¼ ðpþ n ðdetðrþþ exp ðx x Þ T R ðx x Þ ; ðþ n s the dmenson (number of varables) and R the covarance matrx, R ¼ 6 4 r... r n : ð3þ The varances, r j, are estmated by the relaton, ; r j ¼ a xmax j x mn j ð4þ where a s a postve parameter that controls the length of the Gaussans, and x max j and x mn j are the bounds n the jth drecton. te that, n der to keep the method as smple and cost effectve as possble, the varances are kept constant. Ths strategy would have a cost n terms of total number of analyses. The probablty densty s such that R pðxþdx ¼, but snce a bounded doman X s consdered, a bounded probablty ~pðxþ s ntroduced, ~pðxþ ¼ pðxþ Z M ; M ¼ pðxþ dx; ð5þ X so that R ~pðxþdx ¼. X The probablty densty of samplng a new pont, /(x), s a probablty densty of not havng sampled x befe. F ts estmaton we adopt the followng assumpton: only the hghest pont x H of ~pðxþ has a null probablty of beng sampled at the next teraton. So, the probablty /(x) s calculated as,
3 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 3 H p( xh) p p( x ) Xmn H p( x ) H ~pðxþ /ðxþ ¼R ; H ¼ max ðh ~pðxþþdx ~pðxþ: ð6þ xx X Fg. llustrates p(x), ~pðxþ and H ~pðxþ, n a undmensonal doman. The maxmzaton of / s not perfmed exactly, frstly because of ts numercal cost, and secondly, as wll be seen n Secton 3., because t would be detrmental to the search. Instead, N r ponts are chosen randomly and the pont that maxmzes / s selected to ntate the next search. te that, n der to maxmze /, t s necessary to calculate nether M (5) n H (6): the maxmum of / s the mnmum of p, sop only s calculated. Three parameters nfluence the probablty densty p and, consequently, the startng ponts: the ponts that are kept f the probablty calculaton, p; the number of random ponts used to maxmze /, N r ; and the Gaussans length parameter, a. Ther settng s dscussed n the numercal results (Secton 3.). The probablstc restart procedure can be appled to any local optmzer. In ths case, an mproved Nelder Mead algthm s proposed... An mproved Nelder Mead search x H Xmax Fg.. Probablty densty functons. Ω The gnal Nelder Mead algthm  and the strategy f boundng varables are summarzed n Appendx A. The GBNM algthm dffers from the Nelder Mead method because of a set of restart optons. The purpose of the restarts s twofold. Frstly, probablstc restarts based on the densty p (Eq. ()) am at repeatng local searches untl a fxed total cost, C max has been reached. The probablty of havng located a global optmum ncreases wth the number of probablstc restarts. Ths s the globalzed aspect of the method. In the current mplementaton of probablstc restart, the sze of the new smplex, a (defned n Eq. (A.)), s a unfm random varable taken between % and % of the smallest doman dmenson. Secondly, restarts are used to check and mprove convergence of the algthm. The two-restart schemes that are convergence related ntalze a new smplex from the current best vertex. The small and large test restarts use a small and large smplex of szes a s and a l, respectvely (see Eq. (A.)). Convergence of the local Nelder Mead searches s estmated through three crtera, the small, flat degenerate smplex tests. The smplex s small f! max k¼;...;nþ X n k¼ x k x max x mn < e s ; ð7þ where x k s the th component of the kth edge, x mn and x max are the bounds n the th drecton, and e s s a termnaton tolerance. The smplex s flat f jf H f L j < e s ; ð8þ where f H and f L are the hghest and lowest objectve functons n the smplex, and e s s a tolerance value. The smplex s degenerated f t has collapsed nto a subspace of the search doman. Ths s the most common symptom of a faled Nelder Mead search  because the method cannot escape the subspace. Me precsely, a smplex s called degenerated here f t s nether small, n touches a varable bound, and one of the two followng condtons s satsfed: mn k¼;n kek k max k¼;n kek k < e s3 det½eš Q ke k k < e s4; k ð9þ where e k s the kth edge, e s the edge matrx, kæk represents the Eucldean nm, and e s3 and e s4 are small postve constants. The lnkng of the three restarts and three convergence tests n the GBNM algthm s shown n Fg.. A memy of past convergence locatons s kept, thus preventng unnecessarly spendng computatons on already analyzed ponts (thrd test, T3, n the flow chart of Fg. ). When the smplex s flat, a probablstc restart s perfmed (T4). A smplex whch s degenerated nduces a large test teraton (T8). When the optmalty of the convergence pont s unsure, such as a convergence on a varable bound where the smplex has degenerated (T6), a small test, that stands f an optmalty check, s perfmed. If the small smplex returns to the same convergence pont, t s consdered to be a local optmum. It should be remembered that the Kuhn and Tucker condtons of mathematcal programmng are not applcable to the present non-dfferentable framewk. The tolerances f small and degenerated smplces, e s and [e s3,e s4 ], respectvely, may be dffcult to tune, so that a smplex whch s becomng small may be tagged as degenerated befe. Thus, f a degeneraton s detected twce consecutvely at the same pont, the pont s taken as a possble optmum, and a probablstc restart s called. Smlarly, f a degeneraton s detected after a small test, ths pont s also saved as a possble optmum, and a large test s dered.
4 4 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx INITIALIZATION restart = PROBABILISTIC Save ntal pont To calculate the probablty densty Nelder-Mead teraton wth projecton (cf. Fg. 5) Save ntal pont RESTART To calculate the probablty densty PROBABILISTIC LARGE TEST SMALL TEST T. of evaluatons > Cmax T Converge The smplex s: small flat degenerated Save the smplex best pont END restart = PROBABILISTIC T3 Pont already known as a local optmum restart = PROBABILISTIC T4 Smplex flat T5 restart = PROBABILISTIC Save local optmum (SMALL LARGE TEST and return to the same pont) (not SMALL TEST and pont not on the bounds and small smplex) T6 restart = SMALL TEST Possble local optmum LARGE TEST PROBABILISTIC and not return to the same pont and pont on the bounds restart = LARGE TEST Save the smplex best pont (possble local optmum) T7 SMALL TEST and not return to the same pont and not on the bounds T8 restart = LARGE TEST t SMALL TEST and ponts not on the bounds and smplex not small and smplex degenerated Degeneracy case restart = SMALL TEST Fg.. Restarts and convergence tests lnkng n GBNM. Once the GBNM algthm termnates, the lst of possble local (eventually global) optma makes the results of the search. In practce, the calculaton of many local global optma s a beneft of the method n comparson wth global optmzers that provde a sngle soluton (e.g., evolutonary algthms). Fnally, t
5 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 5 should be noted that, n der to fnd all the local optma, the number of restarts should be larger equal to the number of local optma. Typcally, me restars are requred because searches started at dfferent ponts may converge to the same local soluton. 3. Numercal results In Secton 3., the choce of GBNM parameters s dscussed. Results on an analytcal functon are gven n Secton 3.and composte lamnate desgn problems are addressed n Sectons 3.3 and 3.4. The GBNM method s compared to an evolutonary algthm (EA). The evolutonary algthm  has a steady-state structure  wth real encodng, contnuous crossover, and Gaussan mutaton of varance r mut ¼ðx max x mn Þ =6. F far comparsons, the parameters of the EA chosen f each test are the ones that perfm best n ndependent trals among all combnatons of populaton szes ( 5), mutaton probabltes (.5.) and crossover probabltes (.4.5). 3.. GBNM parameters choce 3... The Gaussans length parameter, a In ths wk, a s set to., whch means that one standard devaton away from the Gaussan mean pont covers about % of the doman Ponts kept f the probablty calculaton Three strateges have been compared n terms of probablty of not fndng at least one local mnmum, P nfm : the x Õs used n Eqs. () and () are () the startng ponts, () the startng and local convergence ponts, () all the ponts sampled durng the search. One should remember that local convergence ponts are never duplcated (test T3 on Fg. ). From prelmnary tests, t has been observed that strategy () s memy and tme consumng, wth degraded perfmance wth respect to strateges () and (). F these reasons, t has not been consdered f further testng. Strateges () and () are tested wth N r varyng from to, on three functons, 8 f ðx ; x Þ¼ þ :ðx x Þ þð x Þ þ ð x Þ þ snð:5x Þ snð:7x x Þ x ; x ½; 5Š; >< f ðx ; x Þ¼ 4 :x þ 3 x4 x þ x x þð 4 þ 4x Þx x ; x ½ 3; 3Š; f 3 ðx ; x Þ¼ x 5: x 4p þ 5 x p 6 þ cosðx Þþ 8p >: x ½ 5; Š; x ½; 5Š; ðþ f has 4 local mnma, f 6, and f 3 3 (see Fg. 3). f s known as the Sx Humps Camel Back functon and f 3 as the BrannÕs rcos functon. F the frst and the second strateges, results after 5 analyss and based on ndependents runs are presented n Tables and, respectvely. The second strategy perfms best, ndependently of N r. It shows that the startng and local convergence ponts effcently summarze the topology of the basns of attracton. Ths scheme s chosen to update p Number of random ponts, N r If N r s equal to, the rentalzaton s random. If N r s large, the ntal ponts are dstrbuted based on the ntal and convergence ponts of past searches, whch nduces a regular grd-lke pattern. tng N r to a small number, larger than, gves a based-random rentalzaton. It should be seen as a compromse between the grd and the random strateges. Optmum value of N r depends on the test functon: f the basns of attracton are regularly dstrbuted, restarts followng a regular pattern (.e., N r large) are optmal, and vce versa. From the tests results on the three multmodal functons presented n Table, the optmal strategy s N r large f f and f 3, whle t s N r =f f. N r = s chosen as a compromse f general functon optmzaton. 3.. GrewankÕs functon mnmzaton Consder the mnmzaton of the GrewankÕs test functon, F ðx ;...; x n Þ¼ X n x Yn x cos pff ; 4n ¼ ¼ x ½ ; Š; ðþ where the dmenson s n=and the global mnmum s. at x =., =,n. Ths functon has many local mnma. Fg. 4 shows t n the one-dmensonal case (n=), x[,]. Table 3 compares GBNM and the best pont n the populaton of an evolutonary algthm (EA) at,, 5 and, functon calls. Table 3 averages ndependent runs where the startng pont of the GBNM s randomly selected. The followng crteron s used to consder the EA has convergedd to the global mnmum: n k^x x k < ; ðþ where ^x s the best pont found, and x * the global mnmum. The man observaton s that the GBNM method fnds, on average, better objectve functon values, wth a hgher probablty of fndng the global mnmum than the EA does. The advantage of the GBNM method s substantal at a low number of analyses, and slowly decreases as numercal cost grows.
6 6 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 5 Functon f 5 Functon f3 4 3 x x (a) x -5 5 (c) x.5 Functon f.5 x (b) x Fg. 3. Test functons: contour plots Bucklng load maxmzaton Composte lamnates are made of stacked layers where each layer has ented fbers melted n an sotropc matrx (see sketch n Fg. 5). The desgn problems addressed here am at fndng the optmal entaton of the fbers wthn each layer, h, where h s a contnuous varable bounded by and 9. The plates are analyzed Table Probablty P nfm of not fndng at least one of the local mnma (C max =5 analyses, runs, only the startng ponts are kept f the probablty densty calculaton) N r f f f 3 (random restart) Table Probablty P nfm of not fndng at least one of the local mnma (C max =5 analyses, runs, startng and convergence ponts are kept f the probablty densty calculaton) N r f f f 3 (random restart)
7 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 7 F(x) x Fg. 4. GrewankÕs test functon (n=). usng the classcal lamnaton they and an elastc lnear bucklng model (see ). Consder a smply suppted carbon-epoxy square plate, subjected to n-plane compressve loads N x =N y, as shown n Fg. 5. The plate s balanced and symmetrc and has 3layers, each of whch are.5 mm thck. The elastc materal propertes of the layers are E =5 GPa, E =5 GPa, G =5 GPa, m =.35. The lamnate s desgned to maxmze ts bucklng load (assocated to the most crtcal bucklng mode). Snce the plate s balanced and symmetrc, there are eght contnuous desgn varables, the ply entatons, whch are bounded between and 9. Ths problem has a unque mnmum, all ples ented at 45. te also that the outermost ples have me nfluence on the bucklng behav than the nnermost ples. Table 4 compares GBNM and the evolutonary algthm (EA), showng the stackng sequences found, after 3, 5 and analyses based on ndependents runs. At 3 evaluatons, the bucklng load of the desgns proposed by GBNM and EA are equvalent. One notces that the frst local search of GBNM has not always converged at that pont. A steady dfference between GBNM and EA desgns at 3 analyses can be seen: GBNM, whch s by nature a me ented search, converges faster on the me senstve outerples than EA does, and vce versa on the nnerples. From 5 evaluatons on, GBNM converges me accurately to the optmum than EA does Composte plate longtudnal stffness maxmzaton A 6-ply balanced and symmetrc plate, made of glass-epoxy, s to be desgned by maxmzng the longtudnal stffness E x (see [7, p. 47]). The elastc propretes f the glass-epoxy layers are E =45 GPa, E = GPa, Table 3 Comparson of GBNM (N r =) and the best of an EA populaton on GrewankÕs functon, runs (average ± standard devaton) analyses analyses 5 analyses, analyses Mnmum functon value Probablty of fndng the global mnmum Mnmum functon value Probablty of fndng the global mnmum Mnmum functon value Probablty of fndng the global mnmum Mnmum functon value Probablty of fndng the global mnmum GBNM 9.3±6.79 /.56±.499 /.947±.74 5/.98±.4 3/ EA ±5.537 / 4.86±.9 /.9±.96 /.57±. 9/ θ = fber entaton N y N x a a Fg. 5. Smply suppted rectangular plate subjected to n plane loads.
8 8 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx Table 4 Bucklng load maxmzaton, runs, N r = (stackng sequence: average ± standard devaton) 3 analyses GBNM [±45.4 ±44.97 ±45.±45.38 ±45.38 ±44.97 ±43.8 ±49.65] s Std. Dev. ±.47 ±.54 ±.83 ±4.6 ±4.37 ±.53 ±7.46 ±3.47 EA [±45.9 ±44.9 ±45.3 ±44.55 ±44.78 ±45. ±45.6 ±44.85] s Std. Dev. ±.75 ±.96 ±.67 ±3. ±3.66 ±5.9 ±8.8 ± analyses GBNM [±45. ±45.3 ±45. ±45.8 ±45.4 ±44.97 ±45.5 ±45.8] s Std. Dev. ±.7 ±. ±.39 ±.4 ±.9 ±.46 ±.9 ±4. EA [±45.3 ±44.95 ±44.99 ±44.95 ±44.8 ±45. ±44.7 ±46.45] s Std. Dev. ±.9 ±.8 ±.6 ±.9 ±. ±3. ±4.49 ±.3 analyses GBNM [±45. ±45. ±45. ±45. ±44.99 ±45. ±44.98 ±45.] s Std. Dev. ±.±.±.3 ±.5 ±.4 ±.6 ±.5 ±.44 EA [±44.96 ±44.98 ±44.96 ±45.7 ±44.99 ±44.9±45.3 ±45.7] s Std. Dev. ±.6 ±.6 ±.7 ±.95 ±.±.7 ±.7 ±4.95 G =4.5 GPa and m =.3. The plate s balanced and symmetrc, so there are four fber entatons to be found. Ths problem presents 6 local mnma whch are all the combnatons of the and 9 entatons. The global maxmum has all ples ented at. Ths example shows that local solutons exst f smple composte lamnate desgn problems. Table 5 gves the average number of local optma found after analyses based on GBNM runs as a functon of N r. The frst startng pont s chosen randomly. One observes that the average number of optma found grows wth N r. Ths s expected snce the optma are regularly dstrbuted n the doman. Meover, wthn a budget of functon evaluatons, the global optmum s always found because ts basn of attracton s the largest one. Table 5 GBNM E x maxmzaton, analyses, runs N r Average number of optma found Standard devaton Acknowledgment The frst auth would lke to express hs thanks to the Brazlan fundng agency CNPq f the fnancal suppt durng ths research. 4. Concludng remarks A local/global optmzaton method based on probablstc restart has been presented. Local searches are perfmed by an mproved Nelder Mead algthm where desgn varables can be bounded and some search falure cases prevented. The method, called Globalzed and Bounded Nelder Mead search, does not need senstvtes and constructvely uses computer resources up to a gven lmt. It yelds a lst of canddate local optma, whch contan, wth an ncreasng probablty n terms of computer tme, global solutons. The GBNM method s smple n ts prncples, and the afementonned features make t partcularly useful n an engneerng desgn context. It has been found to compare favably to an evolutonary optmzaton algthm n terms of convergence speed, accuracy of results and ablty to fnd local optma. Appendx A. A Nelder Mead algthm wth bounded varables The Nelder Mead method  s the most popular drect search method f mnmzng unconstraned real functons. It s based on the comparson of functon values at the n+ vertces x of a smplex. A smplex of sze a s ntalzed at x based on the rule (see ), x ¼ x þ pe þ Xn qe k ; ¼ ; n; ða:þ k¼ k6¼ where e are the unt base vects and p ¼ a p n ffffff pffffffffffffffffffffff n þ þ n ; q ¼ a p n ffff pffffffffffffffffffffff n þ : ða:þ
9 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx 9 Smplex ntalzaton P : smplex pont f : objectve functon value at P Ph : smplex pont where the objectve functon assumes ts hghest value Ps : smplex pont where the objectve functon assumes ts second hghest value Determne Ph, Ps, Pl, Pm fh, fs, fl Pl : smplex pont where the objectve functon assumes ts lowest value Pm : centrod smplex pont (not consderng Ph) r : reflecton coeffcent =. Reflecton: Pr = Pm + r (Pm-Ph) If Pr s out of the doman: Projecton on bounds β: contracton coeffcent = / γ : expanson coeffcent = fr < fl fr <= fs fr < fh new Nelder Mead teraton Expanson: Pe = Pm + γ (Pr-Pm) If Pe s out of the doman: Projecton on bounds fe < fr Replace Ph by Pr Contracton : Pc = Pm + β(ph-pm) fc > fh Replace Ph by Pe Replace Ph by Pr Replace all P s by (P + Pl)/ Replace Ph by Pc Converged end Fg. 6. Nelder Mead algthm wth bounded varables. The smplex vertces are changed through reflecton, expanson and contracton operatons n der to fnd an mprovng pont (see Fg. 6). The algthm termnates when the vertces functon values become smlar, whch s measured wth the nequalty, vffffffffffffffffffffffffffffffffffffffffffffffffffff ux t nþ ðf f Þ < e; X nþ f ¼ f ; ða:3þ n n þ ¼ where e s a small postve scalar. The cumulatve effect of the operatons on the smplex s, roughly speakng, to stretch the shape along the descent drectons, and to zoom around local optma. Two comments on the propertes of the algthm are added. Frstly, the Nelder Mead algthm may fal to converge to a local optmum, whch happens n partcular when the smplex collapses nto a subspace. Secondly, the method may escape a regon that would be a basn of attracton f a pontwse descent search f the smplex s large enough. ¼ Ultmately, as the sze of the smplex decreases, the algthm becomes local. The gnal Nelder Mead algthm was conceved f unbounded doman problems. Wth bounded varables, the ponts can leave the doman after ether the reflecton the expanson operaton. It s straghtfward to account f varables bounds by projecton, ( f ðx < x mn Þ; x ¼ x mn ; ða:4þ f ðx > x max Þ; x ¼ x max : The flowchart of the Nelder Mead method shown n Fg. 6 dffers from the gnal method only n the ntalzaton (Eq. (A.)) and n the bounded varables. An mptant sde effect of accountng f the bounded varables through projecton s that t tends to make the smplex collapse nto the subspace of the saturated varables. A specfc convergence test, based on a small smplex rentalzaton at the pont of convergence s then requred (see small test n Secton.).
10 M.A. Luersen, R. Le Rche / Computers and Structures xxx (4) xxx xxx References  Törn AA, Zlnskas A. Global optmzaton. Berln: Sprnger-Verlag; 989.  Bäck T. Evolutonary algthms n they and practce. Oxfd Unversty Press; 996.  Shang Y, Wan Y, Fromherz MPJ, Crawfd L. Toward adaptve cooperaton between global and local solvers f contnuous constrant problems. n: CPÕ Wkshop on cooperatve Solvers n Constrants Programmng, Pahos, Cyprus, December.  Durand N, Allot J-M. A Combned Nelder Mead smplex and genetc algthm. Avalable from:  Okamoto M, naka T, Ocha S, Tomnaga D. nlnear numercal optmzaton wth use of a hybrd genetc algthm ncpatng the modfed Powell method. Appl Math Comput 998;9:63 7.  Törn AA. A search-clusterng approach to global optmzaton. Towards Global Optm 978;:49 6.  Goldberg DE, Voessner S. Optmzng global local search hybrds. In: GECCO 99 Genetc and Evolutonary Computaton Conference, Orlando, FL, USA, 999. p. 8.  Moscato P. On Evoluton, search, optmzaton, genetc algthms and martal arts: towards memetc algthms. Caltech Concurrent Computaton Program, C3P Rept 86, 989.  Barhen J, Protopopescu V, Rester D. TRUST: A determnstc algthm f global constraned optmzaton. Scence 997;76(May):94 7.  Hu X, Shonkwller R, Sprull MC. Random restarts n global optmzaton. Techncal Rept, School of Mathematcs, Gega Insttute of Technology, Atlanta, January 994.  Hckernell FJ, Yuan Y-X. A smple multstart algthm f global optmzaton. OR Trans 997;:.  Nelder JA, Mead R. A smplex f functon mnmzaton. Comput J 965;7:38 3.  Duda OR, Hart PE, Stk DG. Pattern classfcaton. nd ed.. New Yk: John Wley & Sons;.  Wrght MH. Drect search methods: one scned, now respectable. In: Dundee Bennal Conference n Numercal Analyss, Harlow, UK, 996. p  Syswerda G. A study of reproducton n generatonal and steady state genetc algthms. In: Rawlns GJE, edt. Foundatons of genetc algthms. San Mateo, CA: Mgan Kaufmann; 99.  Berthelot J-M. Composte materals mechancal behav and structural analyss. Mechancal engneerng seres. - Sprnger; 999.  Haftka RT, Gürdal Z. Elements of structural optmzaton. 3rd rev. and expanded ed.. Kluwer Academc Publshers; 99.
Preamble INTRODUCTION TO GENETIC ALGORITHM (GA) A non-tradtonal optmzaton method. Stochastc n nature wth probablstc transton rules Tradtonal optmzaton method s determnstc n nature wth specfc rule for movng
Convergence heurstcs Heurstc algorthms Govann Rghn Unversty of Mlan Department of Computer Scence (Crema) Convergence heurstcs Convergence heurstcs are based on analoges wth physcal systems: each soluton
Mean Varance Mappng Optmzaton MVMO operates on a sngle soluton rather than a set of solutons lke n many Evolutonary Algorthms. The nternal search range of all varables s restrcted to [, ]. However, the
46 CHAPTER 4 SINGLE OBJECTIVE CENTRALIZED CONGESTION MANAGEMENT 4.1 INTRODUCTION The centralzed congeston management s based on the COPF approach. The COPF requres the submsson of detaled prvate nformaton
Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789-794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
Integer Programmng All Integer Lnear Programmng Objectves To dscuss the need for Integer Programmng (IP) To dscuss about the types of IP To explan Integer Lnear Programmng (ILP) To dscuss the Gomory Cuttng
Genetc Algorthms A Self Adaptve Hybrd Genetc Algorthm Felpe P. Espnoza Department of Cvl & Envronmental Engneerng Unversty of Illnos 49 Newmark Lab, MC-5 5 N. Mathews Ave. Urbana, IL 68 email@example.com
Unversty of Vctora Department of Economcs ECON 546: Themes n Econometrcs Lab. Exercse Introducton The purpose of ths lab. exercse s to show you how to use EVews to estmate the parameters of a regresson
ournal of mathematcs and computer Scence 9 (2014) 133-138 Smulatng Nonhomogeneous Posson Pont Process Based on Mult Crtera Intensty Functon and Comparson wth Its Smple Form Artcle hstory: Receved August
MATH 118 Class Notes For Chapter 5 By: Maan Omran Secton 5.1 Central Tendency Mode: the number or numbers that occur most often. Medan: the number at the mdpont of a ranked data. Ex1: The test scores for
Algorsmes IRWLS May 2015 1 Introducton Ths report studes the Iteratvely Re-Weghted Least Squares (IRWLS) algorthm for fndng the Maxmum Lkelhood (ML) estmators of a Generalsed Lnear Model (GLM) coeffcents.
Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent
Computer Applcatons n Electrcal Engneerng Realzaton of unform approxmaton by applyng mean-square approxmaton Jan Purczyńsk West Pomeranan Unversty of Technology 71-126 Szczecn, ul 26 Kwetna 1, e-mal: janpurczynsk@pspl
On the Optmal Control of a Cascade of Hydro-Electrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;
6.896 opcs n Algorthmc Game heory February 6, 200 Lecture 4 Lecturer: Constantnos Daskalaks Scrbe: Jason Bddle, Alan Deckelbaum NOE: he content of these notes has not been formally revewed bhe lecturer.
UC Berkeley, CS 174: Combnatorcs and Dscrete Probablty (Fall 2010) Solutons to Problem Set 3 1. (MU 3.3) Suppose that we roll a standard far de 100 tmes. Let X be the sum of the numbers that appear over
Goal Programmng Tutoral Dr. Dylan Jones Professor of Operatonal Research Logstcs and Operatonal Research Group Unversty of Portsmouth, UK Hstory Charnes and Cooper (1955, 1961) As a varant of or extenson
CHAPTER Soluton of Algerac and Transcendental Equatons. INTRODUCTION One of the most common prolem encountered n engneerng analyss s that gven a functon f (, fnd the values of for whch f ( = 0. The soluton
NOTE Communcated by Thomas Detterch Combned 5 2 cv F Test for Comparng Supervsed Classfcaton Learnng Algorthms Ethem Alpaydın IDIAP, CP 592 CH-1920 Martgny, Swtzerland and Department of Computer Engneerng,
1223 DOE Based Robust Optmzaton Consderng Tolerance Bands of Desgn Parameters Jongsoo LEE and Byongchul AHN The paper descrbes a robust optmzaton method to account for the tolerance of desgn varable and
SCHEDULING OF CONSTRUCTION PROJECTS BY MEANS OF EVOLUTIONARY ALGORITHMS Magdalena Rogalska 1, Wocech Bożeko 2,Zdzsław Heduck 3, 1 Lubln Unversty of Technology, 2- Lubln, Nadbystrzycka 4., Poland. E-mal:firstname.lastname@example.org
The Development of Web Log Mnng Based on Improve-K-Means Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna email@example.com Abstract.
Data Envelopment Analyss Data Envelopment Analyss (DEA) s an ncreasngly popular management tool. DEA s commonly used to evaluate the effcency of a number of producers. A typcal statstcal approach s characterzed
Global Optmzaton Algorthms wth Applcaton to Non-Lfe Insurance Problems Ralf Kellner Workng Paper Char for Insurance Economcs Fredrch-Alexander-Unversty of Erlangen-Nürnberg Verson: June 202 GLOBAL OPTIMIZATION
Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal
Learnng Objectves 9.1 The Cumulatve Sum Control Chart 9.1.1 Basc Prncples: Cusum Control Chart for Montorng the Process Mean If s the target for the process mean, then the cumulatve sum control chart s
Volume, No, Copyrght All rghts reserved Integrated Publshng Assocaton REVIEW ARTICLE ISSN 976 59 Analyss of Axally Vbratng Varable Cross Secton Isotropc Rod Usng DTM Mohammad Rafee, Amr Morad Mechancal
Regresson LINEAR REGRESSION Defn: a Scatterplot shows the relatonshp between two quanttatve varables measured on the same populaton unts. The values of the response varable are plotted on the Y-axs and
Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa
0 th World Congress on Structural and Multdscplnary Optmzaton May 9-24, 203, Orlando, Florda, USA On the optmal topologes consderng uncertan load postons János Lógó, Erka Pntér 2, Anna Vásárhely 3 Department
Secton 135 Test of Hypothess for Populaton Proporton As dscussed n Secton 114-115, nference about the probablty of success n ndependent Bernoull trals s same as the mean of ndependent observatons on bnary
Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a two-stage stratfed cluster desgn. 1 The frst stage conssted of a sample
Lecture 9 Load Flow, Economc Dspatch Lecture 9 Load Flow, Economc Dspatch The load flow equatons are of the form, Where ( δ, ) ( δ, ) = V Q= Q V δ δ = R δ nb V V = R V npq Where nb s the number of buses,
3 Classcal Iteratve Methods for the Soluton of Lnear Systems 3. Why Iteratve Methods? Vrtually all methods for solvng Ax = b or Ax = λx requre Om 3 ) operatons. In practcal applcatons A often has a certan
College of Management, NCTU Operaton Research II Sprng, 009 Chap Nonlnear Programmng General Form of Nonlnear Programmng Problems Ma f( S.T. g ( b for =,, m 0 No algorthm that wll solve every specfc problem
D Executve Board Annex 9 Page A/R ethodologcal Tool alculaton of the number of sample plots for measurements wthn A/R D project actvtes (Verson 0) I. SOPE, PIABIITY AD PARAETERS Scope. Ths tool s applcable
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye firstname.lastname@example.org email@example.com firstname.lastname@example.org Abstract - Stock market s one of the most complcated systems
Nested case-control studes: Should one break the matchng? Ørnulf Borgan Department of Mathematcs Unversty of Oslo Based on ongong work wth Ruth Keogh 1 Outlne: Cohort samplng desgns and "classcal" nference
UNIVERSITY OF TRENTO DIPARTIMENTO DI INGEGNERIA E SCIENZA DELL INFORMAZIONE 38123 Povo Trento (Italy), Va Sommarve 14 http://www.ds.untn.t GENETICALLY-DESIGNED ARBITRARY LENGTH ALMOST DIFFERENCE SETS G.
Random Walks on Rooted rees Felx Lazebnk, Wenbo L and Ryan Martn Department of Mathematcal Scences, Unversty of Delaware, Newark, DE 19716 Abstract For arbtrary postve ntegers h and m, we consder the famly
Graph Theory and Cayley s Formula Chad Casarotto August 10, 2006 Contents 1 Introducton 1 2 Bascs and Defntons 1 Cayley s Formula 4 4 Prüfer Encodng A Forest of Trees 7 1 Introducton In ths paper, I wll
MODULE 11a Topcs: The vector norm of a matrx Let denote a norm on R m and R n. Typcally, we thnk of x = x = max x, but t can be any norm. We defne the vector norm of a matrx A by A = max x =1 Ax. We say
14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed
Ch-Square Analyses NCCTM Oct. 10, 003 Ch-Square Analyses Ch-square analyss s one of the more complcated analyses n the AP Statstcs currculum. The complcaton s a result of several features: 1) the pluralty
Soft Sensor for The Adaptve Catalyst Montorng of a Mult-tube Reactor, Fnal Report for NSIS Competton 2006 Martn Macas Czech Techncal Unversty n Prague Faculty of Electrcal Engneerng Department of Cybernetcs
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
Chapter 7 Random-Number Generaton Banks, Carson, Nelson & Ncol Dscrete-Event System Smulaton Purpose & Overvew Dscuss the generaton of random numbers. Introduce the subsequent testng for randomness: Frequency
1. The Categores of Neural Network Learnng Rules There are many types of Neural Network Learnng Rules, they fall nto two broad categores: supervsed learnng, and unsupervsed learnng. Block dagrams of the
CS838-1 Advanced NLP: Condtonal Random Felds Xaojn Zhu 2007 Send comments to email@example.com 1 Informaton Extracton Current NLP technques cannot fully understand general natural language artcles. However,
Amercan J. of Engneerng and Appled Scences (): 8-6, 009 ISSN 94-700 009 Scence Publcatons Optmal Bddng Strateges for Generaton Companes n a Day-Ahead Electrcty Market wth Rsk Management Taken nto Account
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada firstname.lastname@example.org Abstract Ths s a note to explan support vector machnes.
Optmal portfolos usng Lnear Programmng models Chrstos Papahrstodoulou Mälardalen Unversty, Västerås, Sweden Abstract The classcal Quadratc Programmng formulaton of the well known portfolo selecton problem,
21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech-2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS
HYPOTHESIS TESTING OF PARAMETERS FOR ORDINARY LINEAR CIRCULAR REGRESSION Abdul Ghapor Hussn Centre for Foundaton Studes n Scence Unversty of Malaya 563 KUALA LUMPUR E-mal: ghapor@umedumy Abstract Ths paper
MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to
Seventh Internatonal Water Technology Conference Egypt 28-30 March 2003 EFFECT OF SEEPAGE FORCES ON NILE RIVER BANK STABILITY Medhat Azz Assoc. Prof., Secretary General, Nle Research Insttute, NWRC, Caro,
Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent
Ch Squared Ft Measured Data (Charge Voxels) Pulse Reconstructon (Charge on one Pad) Ht Reconstructon (Charge n one Pad Row) Track Fndng (Combnng Hts) Track Fndng (Combnng Pulses) Track Fttng: Ch Squared
Uncertanty n measurements of power and energy on power networks E. Manov, N. Kolev Department of Measurement and Instrumentaton, Techncal Unversty Sofa, bul. Klment Ohrdsk No8, bl., 000 Sofa, Bulgara Tel./fax:
ANOVA (III) 1 Randomzed Complete Block Desgns (RCBD) Defn: A Randomzed Complete Block Desgn s a varant of the completely randomzed desgn that we recently learned. In ths desgn, blocks of expermental unts
fferentable comparatve statcs wth payoff functons not dfferentable everywhere Lus C. Cchon* Unversdad Carlos III de Madrd, epartamento de Economa, Calle Madrd, 26, Getafe 28903, Madrd, pan Receved 4 ecember
3. STATICS O RIGID BODIES In the precedng chapter t was assumed that each of the bodes consdered could be treated as a sngle partcle. Such a vew, however, s not always possble, and a body, n general, should
Recent Progresses on Lnear Programmng and the Smplex Method Ynyu Ye www.stanford.edu/ ~ yyye K.T. L Professor of Engneerng Management Scence and Engneerng and Insttute of Computatonal and Mathematcal Engneerng
CHAPTER 8 COMPLEX VECTOR SPACES 8. UNITARY AND HERMITIAN MATRICES Problems nvolvng dagonalzaton of complex matrces and the assocated egenvalue problems requre the concept of untary and Hermtan matrces.
THE TITANIC SHIPWRECK: WHO WAS MOST LIKELY TO SURVIVE? A STATISTICAL ANALYSIS Ths paper examnes the probablty of survvng the Ttanc shpwreck usng lmted dependent varable regresson analyss. Ths appled analyss