An Analyss of Dynamc Severty and Populaton Sze Karsten Wecker Unversty of Stuttgart, Insttute of Computer Scence, Bretwesenstr. 2 22, 7565 Stuttgart, Germany, emal: Karsten.Wecker@nformatk.un-stuttgart.de Abstract. Ths work ntroduces a general mathematcal framework for non-statonary ftness functons whch enables the exact defnton of certan problem propertes. The propertes nfluence on the severty of the dynamcs s analyzed and dscussed. Varous dfferent classes of dynamc problems are dentfed based on the propertes. Eventually, for an exemplary model search space and a (1, λ)-strategy, the nterrelaton of the offsprng populaton sze and the success rate s analyzed. Several algorthmc technques for dynamc problems are compared for the dfferent problem classes. 1 Introducton Dynamc optmzaton problems are an area of ncreasng mportance and ncreasng nterest. Ths s reflected n the number of publcatons at recent conferences as well as n emergng applcatons n ndustry, e.g. control tasks and dynamc schedulng. In research, dynamc problems are prmarly used to demonstrate adaptaton of evolutonary algorthms and to motvate manfold dfferent technques to cope wth dynamcs. Nevertheless, we observe a sgnfcant lack of establshed foundatons to examne and compare problems as well as technques. Ths work presents one possble approach to reach a common framework for expermental as well as theoretcal examnatons of dfferent dynamc problems. The framework s used n ths work for a theoretcal analyss of the nfluence of populaton sze and certan technques to cope wth dynamc problems on the success rate of algorthms based on local mutaton operators. 2 Related Work Optmzaton of dynamc problems s prmarly drven by the expermental analyss of specal optmzaton technques on dfferent problems. One classcal benchmark functon s the movng peaks problem whch was recently mplemented dfferently n varous problem generators (Grefenstette, 1999; Branke, 1999b; Morrson & De Jong, 1999). Those problems are usually determned by the poston of the peaks and ther heght and wdth. Nevertheless the parameterzaton of the dynamcs dffers from problem generator to problem generator. Also there
s no common bass for the comparson of dfferent dynamc functon optmzaton problems. An overvew on dynamc problems as well as technques s gven n (Branke, 1999a). Two standard technques used to mprove evolutonary optmzaton of dynamc problems are memory for prevous solutons and an ncrease of the populaton dversty. Memory can be ntroduced ether for each ndvdual by usng a dplod representaton (e.g. Goldberg & Smth, 1987; Lews, Hart, & Rtche, 1998) or by an explct global memory (e.g. Mor, Imansh, Kta, & Nshkawa, 1997; Branke, 1999b; Trojanowsk & Mchalewcz, 1999). One possblty to ncrease the dversty are random mmgrants wthn a hypermutaton operator (Cobb & Grefenstette, 1993; Cobb, 199; Grefenstette, 1999). One focus of ths work s the analyss of the populaton sze for dynamc problems and local mutaton operators. The populaton sze was already often target of varous examnatons. De Jong and Spears (1991) and Deb and Agrawal (1999) analyzed the nteracton of populaton sze and genetc operators. Mahfoud (1994) derved a lower bound for the populaton sze n order to guarantee nchng n sharng technques. Mller (1997) deduced optmal populaton szes for nosy ftness functons. An overvew on topcs n populaton szng can be found n (Smth, 1997). Other theoretcal nvestgatons of evolutonary algorthms and dynamc problems may be found n Rowe (1999) and Ronnewnkel, Wlke, and Martnetz (2). 3 Dynamc Problem Framework Ths secton ntroduces a general framework to descrbe and characterze dynamc ftness functons. The goal of ths framework s to establsh a bass for comparson and classfcaton of non-statonary functons as well as for theoretcal results on problem hardness and algorthms power. Such results are possble snce the framework enables the exact defnton of problem propertes. The basc dea of the followng defnton s that each dynamc functon conssts of several statc functons where for each statc functon a dynamc rule s gven. The dynamcs rule s defned by a sequence of sometrc, e.g. dstance preservng, coordnate transformatons and ftness rescalngs. The possble coordnate transformatons are translatons and rotatons around a center. Defnton 1 (Dynamc ftness functon (maxmzaton)). Let Ω be the search space wth dstance metrc d : Ω Ω IR. A dynamc ftness functon F ( F (t)) t IN wth F (t) : Ω IR for t IN s defned by n IN components consstng of a statc ftness functon f : Ω IR (1 n) wth optmum at Ω, f () = 1, and a dynamcs rule wth ( coordnate transformatons where d(c (t) ftness rescalngs s (t) t IN c (t) ) t IN wth c(t) : Ω Ω (ω 1 ), c ( (t) (ω) 2 )) = d(ω 1, ω 2 ) for all ω 1, ω 2 Ω and wth s(t) IR +.
The resultng dynamc ftness functon s defned as { F (t) (ω) = max S (t) 1 f 1(C (t) (t) 1 (ω)),..., S n where C (t) = C (,t) and C (t1,t2) are the accumulated co- are the ordnate transformatons and S (t) accumulated ftness rescalngs. = c (t2) = S (,t)... c (t1+1) f n (C (t) n } (ω)) and S (t1,t2) = t 2 t=t1+1 s(t) The placement of each optmum at poston and the ntal optmal ftness 1 has been chosen for smplfcaton of the analyss only. It does not put any constrant on the functon snce the rescalng of the ftness value and the postonng may be acheved by the frst dynamcs rule. Furthermore t s assumed that at each tme step only one of the component ftness functons attans the optmal ftness value,.e. there s at each tme step only one global optmum. Before problem propertes are consdered n detal a few examples are gven how problems from recent publcatons may be studed n the gven framework. One example are the movng peaks problems (e.g. Branke, 1999b; Morrson & De Jong, 1999) whch can be realzed by one component functon for each peak and an addtonal functon f. Then, the moton of the peaks can be gven by the coordnate transformatons and the peak heghts may be changed by ftness rescalng. Note, that ths framework allows no random nfluences,.e. each ftness functon produced by above paper s problem generators defnes a new dynamc ftness functon wthn the framework. The ftness functon by Trojanowsk and Mchalewcz (1999) dvdes the search space n dfferent segments whch hold each a peak where the heght of the peaks s changng accordng to a schedule. In ths framework the statc functons may be defned on the accordng segment only. Where most other dynamc problems exhbt only coordnate translatons, Wecker and Wecker (1999) presented a ftness functon wth rotaton as coordnate transformatons whch may be reproduced easly wthn the framework. Usng Defnton 1 several problem propertes may be dentfed whch nfluence the hardness of a dynamc problem. The followng defnton formalzes a few basc problem propertes nherent n the dynamcs of the problem. ( ) Defnton 2 (Basc problem propertes). Let F = f, (c (t) ) t IN, (s (t) ) t IN ( ) and F j = f j, (c (t) j ) t IN, (s (t) j ) t IN be two components of a dynamc ftness functon F. Then the followng propertes are defned wth respect to the coordnate transtons and a set of tme steps T IN. wth constant dynamcs: const c (F ) ff t T c (t) statonary: stat c (F ) ff t T c (t) = d weak perodc: wperod c (F ) ff t1,t 2 T,t 1<t 2 C (t1,t2) homogeneous: homo c (F, F j ) ff t T c (t) = c (t) j = c (t+1) = d The propertes const s (F ), stat s (F ), wperod s (F ), and homo s (F, F j ) are analogously defned wth respect to the ftness rescalngs where c s replaced by s and d by 1. Addtonally the followng property s defned usng the ftness rescalngs.
alternatng: alter s (F ) ff t T 1 < S(t) j S (t) < s(t+1) s (t+1) j In general, a dynamc ftness functon s denoted to have constant dynamcs f t holds that 1 n (const c (F ) const s (F )). Ths s also true for the other defned propertes. The next secton defnes certan severty measures and analyzes how the severty s affected by the problem propertes. 4 Problem Propertes and Severty Ths secton presents n the followng defnton four dfferent severty measures and dscusses the nfluence of the basc problem propertes on the severty n the remander of the secton. Defnton 3 (Severty of dynamcs). Let Ω be a search space, da IR the maxmal dstance ( wthn Ω, and F) a dynamc ftness functon consstng of components f, (c (t) ) t IN, (s (t) ) t IN where 1 n. Then the followng severty measures are defned mnmal general severty: mnsev(f (t) ) = mn ω Ω maxcomp(ω) maxmal general severty: maxsev(f (t) ) = max ω Ω maxcomp(ω) average general severty: avgsev(f (t) ) = 1 Ω ω Ω maxcomp(ω) d(c where maxcomp(ω) = max (t 1) 1 n (ω),c (t) (ω)) da optmum s defned as optsev(f (t) ) = d(opt(t 1),opt (t) ) da exsts a (1 n) such that opt (t) t holds that S (t). Moreover, the severty of the where for each opt (t) there = C (t) () and for all j {1,..., n} \ {} f (C (t) ()) > S (t) j f j (C (t) j ()). Frst, the nfluence of the problem propertes on the general severty measures s analyzed. The general severty s determned by the transformatons n the coordnate space only. As long as there are only lnear translatons n a homogeneous problem from one generaton to the next, the average severty equals both the mnmal and the maxmal severty for each component functon. In an nhomogeneous problem the dfferences are determned by the dfferences n component functons. As soon as a rotatng transformaton s ntroduced the mnmal and the maxmal severty dffer wthn one component functon snce the mnmal severty can be zero (n case of rotaton only) and the maxmal severty can be rather large dependng on the rotaton angle and the sze of the search space. Nevertheless the severty of the optmum s not only determned by the general severty but also by the property alternatng of the ftness rescalngs. If a problem s alternatng at a tme step t the optmum s changng from one component functon to another component functon,.e. the severty s not predctable from the coordnate transformatons only. Note that the
alternatng property s strongly correlated to the statonarty and homogenety (stat s homo s alternate ) The followng problem classes are dentfed for complete problems as well as perods of problems. The orderng does not mply a strct ncrease n problem dffculty. Class (both parts statonary): statc optmzaton task (severty ) Class 1 (constant and homogeneous coordnate translatons, not alternatng): pure trackng task wth constant severty,.e. the statc ftness landscape s movng as a whole Class 2 (constant and nhomogeneous coordnate translatons, not alternatng): pure trackng task wth constant severty but n a changng envronment Class 3 (nconstant coordnate translatons, no jumpng): ether not so easly predctable trackng task or rather chaotc or random problem Class 4 (rotatng coordnate translatons, not alternatng): trackng task wth dfferent nherent degrees of severty Class 5 (alternatng, statonary coordnate translatons): oscllaton between several statc optma, the severty depends on the poston of the component functons optma Class 6 (alternatng, non-statonary coordnate translaton): trackng and determnaton of movng and oscllatng optma, the severty depends on the component functons dynamcs For each problem class dfferent optmzaton technques are useful. In the next secton the nterdependences are analyzed between the populaton sze and some memorzng rsp. dversty preservng technques. The mpact of these results on the optmzaton of the problem classes s dscussed at the end of the next secton. 5 Analyss of Technques and Populaton Sze Ths secton analyzes one generaton of an optmzer n a dynamc envronment. In partcular, the nfluence of the populaton sze on the probablty to get close to the next poston of the optmum s of nterest. The analyss uses an abstract problem as well as an abstract optmzer. The problem s only consdered as far as the problem propertes above are concerned. For the algorthm only some underlyng workng prncples are assumed nstead of analyzng a concrete nstance of evolutonary computaton. The search space s a two-dmensonal raster 1 1. We assume the current optmum to be placed close to the center such that no newly created ndvdual by a local mutaton les outsde the search space. Furthermore we consder a dstance metrc whch s defned by vertcal and horzontal crossngs of raster boundares (cf. Fgure 1). Smplfyng t s assumed that the optmum s moved horzontally. In the analyss the followng numbers are of nterest. 4 ponts have exactly dstance to a gven pont and the number of ponts whch are at most steps off s 2( + 1) + 1.
1 dstance 1 severty = 6 4 6 5 6 8 7 6 7 8 8 7 8 8 trackng probablty.8.6.4 severty 2.2 severty 4 severty 6 severty 1 1 2 3 4 5 6 7 8 9 1 populaton sze Fg. 1. Left: The hatched pont marks the poston of the optmum n the last generaton whch was now moved horzontally sx steps. The regon on the rght denotes those ponts whch are wthn a dstance 2 of the new optmum s poston. The numbers n the squares denote the dstance to the prevous optmum. Rght: Probablty to get close to the optmum (wthn dstance 1) for an algorthm wth local mutaton only. Frst, a pure mutaton based local search algorthm s analyzed. It s assumed that the algorthm performs a mutaton wth step sze 1 2 wth a probablty accordng to the bnomal dstrbuton. Pr[] = 1 2 4 1 ( 41 2 + The maxmal step sze s 2. Moreover, each mutaton reaches a new pont n the search space (Pr[] = ). Gven that the parent ndvdual s stuated at last generatons optmum whch moves by severty s, the number of ponts wthn a dstance d from the new optmum and an exact dstance s d d s + d equals s + d + 1 d N d,s (d ) = 2d + 1 2 2 (cf. left part of Fgure 1). Then, the probablty to ht a pont wthn dstance d from the new optmum by creatng one offsprng equals Pr[< d(s)] = N d,s (d )Pr[d ] s d d s+d Note, that no recombnaton s consdered n the scope of ths work. Assumed that the algorthm uses a (1, λ) strategy, the probablty that at least one offsprng s wthn a dstance d of the optmum results as Pr[< d(s) of λ] = 1 (1 Pr[< d(s)]) λ The results n Fgure 1 (rght) show the dependence of the trackng probablty of the populaton sze. Wth ncreasng severty of the optmum a large ) (1)
1 severty 4, dstance 1 1 severty 5, dstance 1 trackng probablty.8.6.4 mutaton only memory, 5 nd, +.2 memory, 5 nd, - memory, 1%, + memory, 1%, - 1 2 3 4 5 6 7 8 9 1 populaton sze trackng probablty.8.6.4 mutaton only memory, 5 nd, +.2 memory, 5 nd, - memory, 1%, + memory, 1%, - 1 2 3 4 5 6 7 8 9 1 populaton sze trackng probablty 1.8.6 severty 6, dstance 1.4 mutaton only memory, 5 nd, +.2 memory, 5 nd, - memory, 1%, + memory, 1%, - 1 2 3 4 5 6 7 8 9 1 populaton sze trackng probablty 1.8.6.4.2 severty 1, dstance 1 mutaton only memory, 5 nd, + memory, 5 nd, - memory, 1%, + memory, 1%, - 1 2 3 4 5 6 7 8 9 1 populaton sze Fg. 2. Results for usng an external memory for prevous good solutons. Indvduals from the memory are nserted nto the offsprng populaton. + denotes that the optmum s contaned n the memory and the contrary. populaton sze becomes more mportant. Ths result s especally of nterest for pure trackng tasks lke n class 1 and class 2 problems. It gves a gudelne for choosng an approprate populaton sze. In some algorthms wth local mutaton an adaptaton of the step sze s used (e.g. n evoluton strateges). However, ths s not consdered n ths analyss. Note, that the results do not scale wth ncreasng step sze snce the basc probablty for httng a pont n the search space decreases. For problems of class 3 or 4 the determnaton of the populaton sze s dffcult snce the severty s varyng. For the same reason (self-)adaptve technques must be questoned crtcally for class 3 and 4. Second, an external memory s added to the algorthm n whch certan good solutons from prevous generatons are stored. A memory sze of 4 ndvduals s assumed. In order to analyze the behavor wth memory two cases are dstngushed: the successful case where the optmum (or a pont close to the optmum) s contaned n the memory and the unsuccessful case. Furthermore, two replacement strateges are consdered. On the one hand, each generaton 5 ndvduals are chosen randomly from the memory and nserted nto the off-
.35 severty 8, dstance 1.3 severty 2, dstance 1.3.25 trackng probablty.25.2.15.1 mutaton only.5 mmgrants, 5 nd mmgrants, 2% 1 2 3 4 5 6 7 8 9 1 populaton sze trackng probablty.2.15.1.5 mmgrants, 5 nd mmgrants, 2% memory, 1%, + 1 2 3 4 5 6 7 8 9 1 populaton sze Fg. 3. Results for ntroducng random mmgrants nto the populaton: small severty and requred mnmal dstance 1 to the optmum (left) and hgh severty and requred mnmal dstance 1 to the optmum (rght) sprng populaton and, on the other hand, 1% of the offsprng populaton are chosen from the memory. Now the probablty from above can be modfed for the successful case wth the replacement number k { 5, λ 1 } ( ) k 39 Pr[< d(s) of λ(memory)] = 1 (1 Pr[< d(s)]) λ k 4 and for the unsuccessful case Pr[< d(s) of λ(memory)] = 1 (1 Pr[< d(s)]) λ k. Fgure 2 shows the results for the algorthm wth memory. Obvously wth low severty the trackng probablty worsens even n the successful cases wth small populaton szes. But already wth severty 5 the algorthm replacng 1% of the populaton shows small advantages. Nevertheless, the worsenng n case of memory wthout the optmum are stll bgger as the gans n the successful case. If only small severty values occur, memory seems only to be useful for problems of class 5 snce there s a hgh chance that the optmum s contaned n the memory. However, as soon as bgger severty values occur the dagrams show almost neglgble worsenngs n the unsuccessful case and sgnfcant mprovements n the successful case. As a consequence memory s always useful n problems of classes 5 and 6 wth hgh severty even f the memory of problems n class 6 mght have a rather small chance of contanng the optmum. Eventually, the method of random mmgrants s analyzed n ths theoretcal setup on ts drect nfluence n fndng the optmum or gettng close to the optmum. We consder agan two scenaros: on the one hand 5 random ndvduals are njected and on the other hand 2% of the populaton are replaced randomly. The trackng probablty can be derved smlarly to the prevous case where the basc probablty depends on the sze of the search space and the mnmal allowed dstance to the optmum. The results are shown n Fgure 3. As expected
ths method shows a sgnfcant drawback wth low severty and small expected dstance to the optmum. Nevertheless, wth hgh severty and a large allowed dstance to the optmum ths method can help to ncrease the probablty consderably. Ths shows that the method s ndeed able to fnd the correct regon wth a certan probablty. If we stll assume n the memory model that only one ndvdual out of 4 s a successful ndvdual, the random mmgrants are able to beat the memory model n the successful case. The assumpton on the memory s not unlkely snce the trackng regon covers only approx. 2% of the search space. Ths result can explan the good performance of the mechansm n certan problems usually wth recombnaton whch helps to combne several ndvduals gettng anywhere close to the optmum. Wth respect to the problem classes, ths method should be consdered for class 6 wth a rather hgh severty snce t has a guaranteed mprovement n contrary to the memory approach. 6 Concluson Ths work presents a framework for the classfcaton and comparson of dynamc problems. Ths framework s used for an analyss how the offsprng populaton sze and two specal technques for dynamc problems affect the trackng probablty of a search algorthm based on a local mutaton operator. Wthn the context of a (1, λ)-strategy, the analyss gves concrete nformaton for whch problem class whch algorthmc technque and whch populaton sze should be at least consdered. Snce especally n dynamc problems the populatons sze can be a crtcal parameter more evaluatons can mpose hgher dynamcs more nvestgatons are necessary n that drecton. Besdes the concrete results n ths work, the framework for dynamc problems enables more theoretcal nvestgatons of correlatons between problem propertes, parameter settngs, algorthmc technques, and adaptaton measures. In partcular, future work wll consder more complex models of algorthms ncorporatng recombnaton operators and other performance measures coverng dfferent aspects of adaptaton. References Branke, J. (1999a). Evolutonary approaches to dynamc optmzaton problems: A survey. In J. Branke & T. Bäck (Eds.), Evolutonary algorthms for dynamc optmzaton problems (pp. 134 137). (part of GECCO Workshops, A. Wu (ed.)) Branke, J. (1999b). Memory enhanced evolutonary algorthms for changng optmzaton problems. In 1999 Congress on Evolutonary Computaton (pp. 1875 1882). Pscataway, NJ: IEEE Servce Center. Cobb, H. G. (199). An nvestgaton nto the use of hypermutaton as an adaptve operator n genetc algorthms havng contnuous, tme-dependent nonstatonary envronments (Tech. Rep. No. 676 (NLR Memorandum)). Washngton, D.C.: Navy Center for Appled Research n Artfcal Intellgence.
Cobb, H. G., & Grefenstette, J. J. (1993). Genetc algorthms for trackng changng envronments. In S. Forrest (Ed.), Proc. of the Ffth Int. Conf. on Genetc Algorthms (pp. 523 53). San Mateo, CA: Morgan Kaufmann. De Jong, K. A., & Spears, W. M. (1991). An analyss of the nteractng roles of populaton sze and crossover n genetc algorthms. In H.-P. Schwefel & R. Männer (Eds.), Parallel Problem Solvng from Nature: 1st Workshop, PPSN I (pp. 38 47). Berln: Sprnger. Deb, K., & Agrawal, S. (1999). Understandng nteractons among genetc algorthm parameters. In W. Banzhaf & C. Reeves (Eds.), Foundatons of Genetc Algorthms 5 (pp. 265 286). San Francsco, CA: Morgan Kaufmann. Goldberg, D. E., & Smth, R. E. (1987). Nonstatonary functon optmzaton usng genetc algorthms wth domnance and dplody. In J. J. Grefenstette (Ed.), Proc. of the Second Int. Conf. on Genetc Algorthms (pp. 59 68). Hllsdale, NJ: Lawrence Erlbaum Assocates. Grefenstette, J. J. (1999). Evolvablty n dynamc ftness landscapes: a genetc algorthm approach. In 1999 Congress on Evolutonary Computaton (pp. 231 238). Pscataway, NJ: IEEE Servce Center. Lews, J., Hart, E., & Rtche, G. (1998). A comparson of domnance mechansms and smple mutaton on non-statonary problems. In A. E. Eben, T. Bäck, M. Schoenauer, & H.-P. Schwefel (Eds.), Parallel Problem Solvng from Nature PPSN V (pp. 139 148). Berln: Sprnger. (Lecture Notes n Computer Scence 1498) Mahfoud, S. W. (1994). Populaton szng for sharng methods (Tech. Rep. No. IllIGAL 945). Urbana, IL: Illnos Genetc Algorthms Laboratory, Unversty of Illnos at Urbana-Champagn. Mller, B. L. (1997). Nose, samplng, and effcent genetc algorthms. Unpublshed doctoral dssertaton, Unversty of Illnos at Urbana-Champagn, Urbana, IL. (IllIGAL Report No. 971) Mor, N., Imansh, S., Kta, H., & Nshkawa, Y. (1997). Adaptaton to changng envronments by means of the memory based thermodynamcal genetc algorthm. In T. Bäck (Ed.), Proc. of the Seventh Int. Conf. on Genetc Algorthms (pp. 299 36). San Francsco, CA: Morgan Kaufmann. Morrson, R. W., & De Jong, K. A. (1999). A test problem generator for non-statonary envronments. In 1999 Congress on Evolutonary Computaton (pp. 247 253). Pscataway, NJ: IEEE Servce Center. Ronnewnkel, C., Wlke, C. O., & Martnetz, T. (2). Genetc algorthms n tmedependent envronments. In L. Kallel, B. Naudts, & A. Rogers (Eds.), Theoretcal aspects of evolutonary computng (pp. 263 288). Berln: Sprnger. Rowe, J. E. (1999). Fndng attractors for perodc ftness functons. In W. Banzhaf, J. Dada, A. E. Eben, M. H. Garzon, V. Honavar, M. Jakela, & R. E. Smth (Eds.), Proc. of the Genetc and Evolutonary Computaton Conf. GECCO-99 (pp. 557 563). San Francsco, CA: Morgan Kaufmann. Smth, R. E. (1997). Populaton sze. In T. Bäck, D. B. Fogel, & Z. Mchalewcz (Eds.), Handbook of Evolutonary Computaton (pp. E1.1:1 5). Brstol, New York: Insttute of Physcs Publshng and Oxford Unversty Press. Trojanowsk, K., & Mchalewcz, Z. (1999). Searchng for optma n non-statonary envronments. In 1999 Congress on Evolutonary Computaton (pp. 1843 185). Pscataway, NJ: IEEE Servce Center. Wecker, K., & Wecker, N. (1999). On evoluton strategy optmzaton n dynamc envronments. In 1999 Congress on Evolutonary Computaton (pp. 239 246). Pscataway, NJ: IEEE Servce Center.