Peak Analysis for Characterizing Evolutionary Behavior.

Transcription

1 Peak Analysis for Characterizing Evolutionary Behavior. Jeffrey O. Pfaffmann Abstract Evolutionary Algorithms (EAs) use a simplified abstraction of biological evolution to interact with a fitness landscape over multiple generations. The traditional approach to exploring evolutionary motivated search relies on performance comparisons of competing designs for a common problem domain. This approach has been useful in developing an intuition of when one mechanism is superior to another. However, this intuition has developed in the absence of a clear understanding of how fitness landscape topology impacts the search process in high-dimensional space, due to the lack of high-dimensional visualization tools. Proposed is a behavior measure derived from a known set of all problem domain optima, which is used as system of landmarks. Using these landmarks, evolutionary system progress can be tracked in the problem domain and characterized with an information theoretic metric. Thus, this technique provides an intuition that takes into account the problem domain topology, allowing the behavior of different algorithm configurations to be compared as they interact with a given topology. I. INTRODUCTION Evolutionary Algorithms (EAs) are a broad family of stochastic search techniques for determining the global optimum in complex problem domains. EAs borrow the darwinian framework of variation and selection to traverse a specific problem domain. The exact implementation of this framework can vary widely, but typically follows a philosophy espoused by one of three main branches: Genetic Algorithms [], Evolutionary Programming [2], and Evolutionary Strategies [3]. These systems also exhibit a behavior described by the No-Free Lunch (NFL) theorem [4]. This theorem can be summarized as no single algorithm will perform in a superior way in all cases, and all algorithms will perform similarly in general. Thus, a given algorithm will perform well in certain situations, while a different algorithm will be better in other situations. This feature can also be observed when comparing different parameter tunings of the same algorithm. Following from the NFL theorem, researchers have developed an intuition for when one algorithm is superior to another. This understanding has developed from learning performance observations as the primary metric. While this developed understanding is useful, it is based on indirect information of how the EA interacts with the problem domain by focusing on convergence speed of competing search algorithms. Thus, performance metrics measure the problem domain topology indirectly, and do not provide a fine grain understanding how these search techniques move through the search space. In low-dimension problems, EA manipulation of the topology can be directly visualized. For Jeffrey Pfaffmann is in the Department of Computer Science, Lafayette College, Easton, Pennsylvania, USA. ( pfaffmaj@lafayette.edu). higher-dimension problems, this same level of visualization is not available. This work proposes an information theoretic based metric, called peak analysis, which utilizes the set of all known optima as landmarks to determine an evolutionary system s progress through a given problem domain. In this proposed metric, the optima set provides a map for observing how the search mechanism, or relocation heuristic, is behaving. To characterize relocation heuristic behavior the entropy is calculated for peak visitation and then aggregated over multiple runs to determine a mean visitation entropy. A peak visitation is defined as the nearest optima peak for each sampled parameter set produced by the EA, with the total of visitations equaling the total number of generated parameter sets. Tracking the relocation heuristic allows an intuition to be developed that indicates when one algorithm is more effective than another at interacting with important aspects of the problem domain topology. Here effectiveness is not tied directly to performance, instead it is tied to convergence and fitness landscape visitation. Understanding how a specific EA interacts with a problem domain topology is very important to ultimately understanding the capability of these stochastic techniques. If we observe these EAs in terms of relocation heuristic behavior, we can begin to rigorously examine problem domains as attractor topologies defined by the optima set, with each peak producing an attractor basin. Thus, the optima set becomes a map for tracking how the EA moves through the space, which can be used to observe this behavior over several iterations to determine a generalized behavior. To obtain these statistics, each generated individual can be used to compute the nearest peak and the distance to that peak. Using the occurrence of each nearest peak, the entropy can be computed to determine the level of encoded information. A baseline entropy level can be computed for both the equiprobable case and random search. As a specific EA implementation focuses on a particular peak the entropy will reduce. As will be shown in the results section, particular types of EAs have a certain entropy profile, while parameter tuning of a particular EA produce small changes in that value. Information encoding is used to identify the feature that EAs are exploring their environmental topology to capture information about that topology. Information theoretic metrics have previously been used to explore stocastic learning algorithms [5], and are a important component of algorithm analysis techniques like Kolmogorov Complexity [6]. Peak analysis is an inherently empirical approach for characterizing EA behavior after the algorithm has been executed. Similar entropy based metrics have been used to influence the behavior of genetic systems, to search a topology more effi /9/$ IEEE Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.

2 ciently or make large topologies tractable. A good example of this approach is the Option Set Entropy (OSE) metric [7], which attempts to estimate phase changes in the behavior space of different Agent-Based Model configurations. Using OSE, a multi-agent exploratory system can decide whether to further sample a specific model configuration to better estimate the underlying stochastic effects or to explore new configurations. What is interesting in the OSE approach is the information encoding represents an approximation of the exact model being explored, which is used in a decision process whether to explore deeply or broadly. There are also approaches alternate to peak analysis that are theoretical in nature [8][9] that divide the problem domain into regions and a conceptualized EA is proven to have certain behaviors in relation to the idealized topology. These theoretical approaches have great merit, providing a formal approach to defining EA behavior. How peak analysis can make a contribution is by examining existing systems to determine how they progress through a given problem domain when the optima set is known. Note, this requirement does restrict this technique to well understood problem domains or problem generators. Closest Peak (Ordered by fitness) Closest Peak (Ordered by fitness) A: Randomly Generated Individuals B: Evolutionary Algorithm Generated Individuals II. PEAK ANALYSIS As discussed above, peak analysis relies on a known optima set, which is the set of all peaks in the defined problem domain. The availability of these peaks must either be divined from an existing problem domain, or defined in a constructed domain. This work uses a problem generator (TCG-2) developed by Schmidt and Michalewicz [], which was designed for the generation of constraint based problems. The use of a problem generator provides flexibility in the creation of problem domains, allowing different topologies to be created for experimentation. This work uses two topologies, which are varied for problem dimension and the number of peaks used to construct a problem domain. Both the problem dimension and peak count is varied with one of three values, producing nine variations on a common topology and a total of 8 different problem domains. Additional details on each topology are discussed in the methods section below (see section III). To examine how different EA configurations behave on the various problem domains, three basis configurations were specified and then varied to illustrate the subtle changes in EA behavior. Two of the three EA configurations relied exclusively on point mutation, with one configuration implementing increasing levels of elitism and the other increasing the number of point mutations. The third EA configuration implemented single point crossover, and varies the level of elitism. For a baseline measurement, a random search was performed on each problem domain. Additional details for each EA configuration is provided in methods section below (see section III). As described above the proposed behavior metric uses the set of all optima to provide a mapping mechanism. This mechanism is implemented by tracking the nearest peak for each generated individual with the euclidean distance to that Fig.. Closest peak occurrence during a 5, iteration run, with individuals each iteration. peak. Fig. graphs the nearest peak for each individual during a single run, providing a graphical representation of peak frequency over time. Fig..A illustrates the random search for 5, selections, while Fig..B shows an EA running for 5, generations with a constant population of individuals. The shown experiment runs were generated with identical parameters, when applicable, with one run performed by random search and the second by an EA. The problem domain consisted of -dimensions and was constructed from 5-peaks. Immediately noticeable between the two graphs is the distribution of nearest peaks over the entire population. The random case (Fig..A) has near complete coverage in comparison to the data generated by the EA (Fig..B). Both the random and EA search results can be characterized by calculating the entropy, providing a scalar measure of encoded information. The entropy calculation for peak visitation frequency (P ) follows the traditional formulation [] n H(P )= p i log 2 p i, () i= where p i is the fraction of generated parameter sets that has peak i as the closest optima. The computed entropy is the measure of uncertainty, providing the number of bits necessary to encode the provided system. Table I gives the calculated entropy for the previously graphed experiment runs and the equiprobable case. The equiprobable case provides an ultimate baseline or maximum entropy. As the entropy decreases, the EA will have generated individuals in a more compressed space around fewer optima, which Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.

3 provides a measure of the system focusing on specific regions in the underlying topology. TABLE I COMPUTED ENTROPY FOR FIG.. equiprobable case w/5-peaks random search EA search Table I shows the random search entropy is slightly less than the equiprobable case, which is due to the influence of the underlying problem domain topology. The EA search is less than the random search and represents the system focusing on specific regions of the problem domain. Using this entropy measure, an appreciation of problem domain coverage can be determined by its relationship to the equiprobable case and other cases also exploring the problem domain. Closest Peak Distance Closest Peak Distance A: Randomly Generated Individuals B: Evolutionary Algorithm Generated Individuals Fig. 2. Closest peak distance graph for 5, iteration run, with individuals each iteration. As the euclidean distance is calculated between each individual and the nearest peak, this information is retained and tracked. Fig. 2 graphs the euclidean distance to the closet peak, which is generated from the same data as represented in Fig.. The difference between the random experiment run (Fig. 2.A) and EA run (Fig. 2.B) illustrates the EA structuring how it maneuvers through the problem domain. This measure allows convergence to be examined by observing the statistical trends of the curve. Note, this distance only indicates convergence to a peak, not convergence to a specific peak. If convergence to the global optimum were the focus, then the optima set could be aggregated into fitness groups allowing for a performance metric to be developed. For this work, simplicity was focused on, thus distance was computed for all peaks with the assumption they were equally relevant. Similarly, the euclidean distance is used for simplicity when a scale-invariant measure would be more appropriate. If a scale-invariant measure was used, comparisons between problem domains of different dimensionality could be performed. Note, this work can be easily expanded to do so, but for the data presented here, comparisons are only made between problem domains of the same dimensionality. III. METHODOLOGY The approach taken with this work is to provide a set of problem domain topologies that a second set of EA configurations can manipulate. Comparisons are then made, using similar problem domain topologies combined with the different EA configurations. As described previously, peak analysis requires that the entire optima set be known, when each optima location is a peak. This requirement can be met by either determining all peaks in a known problem domain, or constructing a problem domain from a set of peaks. Determining the existing peak set is possible for simple problem domains, but tracking how an evolutionary system traverses a simple domain is also tractable. Doing the same for higher dimensional problem domains becomes less tractable and opens the question of utility when search is required for this task. An effective approach is to derive a problem domain from a set of known peaks, leaving this work to rely on problem generators. Problem domain generators also allow for a wide array of closely related problem domain topologies to be generated, which allows the question of algorithm stability to be examined. For this work the Schmidt and Michalewicz TCG-2 problem generator is used, and is detailed in section III-A below. The EA was developed in C++ specifically to support this and other related work. For the numerically intensive components of the software, the GNU Scientific Library [2] was employed. Any number of other software packages could also be used, as the technique relies on how the EA interacts with the problem domain and not on an intrinsic feature set of the software itself. The configurations for each algorithm are provided in section III-B below. A. Problem Domain Generation TCG-2 allows a variety of parameters to be tuned, including problem dimension and number of optima peaks. A second feature of TCG-2 is that a random generator is used in the problem domain construction allowing a wide variety of problem domains to be constructed. This work takes a conservative approach by selecting two specific TCG- 2 parameter sets, and then varying problem dimension and peak count. This approach produces two sets of closely related topologies each providing nine different problem domains. The basis for each topology set was chosen by performing parameter sweeps on the TCG-2 generator for two dimensional problem domains constructed with 5 peaks. Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.

4 Graphs of the 3-dimensional landscapes were compared and two topologies chosen that differed for three parameters. The criteria for selecting the basis topologies focused on surfaces that contained well defined structure but not highlyrugged, this allowed for problem domains that were both solvable and non-trivial. Fig. 3 shows the two surfaces that were chosen, displaying the two-dimensional problem case constructed with 5-peaks. For the actual experiments the problem dimensions were increased to a greater number of dimension, while the 5-peak construction was used for one of the varied TCG-2 parameters. Topology (min = , max =.9999) scaled by a penalty constant (W ) from the objective function (G(x)). The constraint violation is determined by computing the distance of the individual to the nearest feasible region, while the penalty constant is set to. The fourth static parameter (σ peak ) is the size of the actual optima peak in the problem domain. TABLE II TCG-2 STATIC PARAMETER SET. feasibility (ρ). active constraints (a) component min. distance (d). peak width (σ peak ).4 topology random generator seed Y axis X axis Table III gives the parameters that were varied between the two topologies. The first two parameters contribute to the configuration of the feasible regions, with feasible components (m) specifying the number of components created in the problem domain and the complexity indicating the chance that regions will be smaller. The third parameter, peak decay (α), controls the shape, or slope, of the problem domain surrounding the peak. TABLE III TCG-2 DYNAMIC PARAMETER SET. Fig Topology 2 (min = , max = ).8.6 Y axis X axis Example 2-dimension problem plots, each generated with 5 peaks. Table II gives the parameters that were common to all topologies. The problem domain generator uses the concept of feasible regions in the n-dimension space. The feasibility parameter (ρ) specifies the total size of these combined regions, with the component minimum distance (d) regulating the nearness of these regions. The active constraints (a) were not used, which act at the global optimum, instead a static penalty function is exclusively used. The penalty function has the form fitness(x) =G(x) W CV (x), (2) where fitness is computed for an individual represented by vector x by subtracting the constraint violation (CV (x)) Topology 2 feasible components(m) 6 22 search space complexity (c).8. peak decay (α).5.9 ranges problem dimension (n) {5,, 5} peak count {, 5, 5} Table III also provides how the problem dimension (n) and peak count are varied. The problem dimension was kept to a computationally tractable size for a large number of experiments, while the peak count was increased non-linearly to better illustrate the EA behaviors in response to a problem domain with few- verses many-peaks. B. Evolutionary Algorithm Design For this work a very simple Evolutionary Algorithm implementation was used with slight variations that solved for the highest optima solution. Each evolutionary algorithm was run for 5, generations, with a constant population size of individuals. A standard roulette wheel [3] selection mechanism is used to determine the individuals to vary for the next generation. Point mutation is used for all EA configurations, and implemented using a gaussian probability density with a small variance (σ rand =.). Three different EA configurations were implemented and then applied to the 8 different problem domains described previously. These EA configurations include: point mutation with elitism; point mutation without elitism; combined point mutation and crossover with elitism. All three EA configurations were run with a small parameter change, Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.

5 TABLE IV EVOLUTIONARY ALGORITHM CONFIGURATIONS. elitism parameter values only mutation yes retained {, 2, 4, 6} individuals (r) only mutation no mutation {2, 3, 4} count (l) mutation & crossover yes retained individuals (r) {, 2, 4, 6} provided in Table IV. The point mutation with elitism varies the number of unmodified individuals propagated from one generation to the next. The point mutation without elitism varies the number of point mutations generated to each varied individual. Point mutation with crossover and elitism, maintains a single crossover point for all configurations, but varies the number of unmodified individuals propagated from one generation to the next. The three different EA configuration sets produce a total of different configurations, which is combined with the 8 problem domains producing 98 different experiment combinations. To resolve for variation between different random generator seed values, each experiment configuration is run for different seeds and the results aggregated. A static pool of random seeds was initially generated from a random generator using the sample date as the global seed value. A static pool is used to ensure that all 98 experiment configurations have a common starting point in the random generator sequence space. This produced a set of 9,8 individual experiment runs that the following results are derived from. IV. RESULTS Table V give the baseline entropy measures, with the three equiprobable cases for the -, 5-, and 5-peak problem domains. As discussed previously, the equiprobable cases makes the assumption that each peak will be visited with equal probability, and give the maximum entropy that can be generated for each respective system. Table V also provides the computed mean entropy (H) from random runs for each of the 8 different problem domain topologies, and the number of peaks and problem dimension (n) is varied. The standard deviation (σ) is provided, indicating that individual deviance from the mean entropy is very low. Another point of interest is that the computed mean entropies are slightly lower that the equiprobable cases and break into natural categories based on the number of peaks used to construct the system. The fact that entropy categories do not develop from the problem dimension for the random search is expected, because the parameters are generated using a flat distribution. Table VI gives the computed mean entropies for problem domain topology. The information is organized primarily by EA configuration to illustrate the performance as the problem dimension increases. Again, the mean entropy values fall TABLE V CALCULATED ENTROPY FOR RANDOM SEARCH AND THE EQUIPROBABLE CASES. EQUIPROBABLE CASES -peaks 5-peaks 5-peaks H TOPOLOGY -peaks 5-peaks 5-peaks n H σ H σ H σ TOPOLOGY 2 -peaks 5-peaks 5-peaks n H σ H σ H σ into categories that are driven by the number of peaks. If table VI is compared to table V, it can be observed that the EA 5-peak entropy category falls below the same category for the 5-peak random search, but some were near the 5-peak random search entropy category. The same can be seen for the EA 5-peak category, with values falling below the 5-peak random category and slightly above the -peak category. Similarly, the EA -peak category has the same relationship to the random search mean entropies. What does not follow from this data presentation, is a set of well defined categories for problem dimension. Typically, increase in problem dimensionality is associated with a failure to converge, or convergence to a suboptimal region. Based on this information, we can see an entropy increase with the growth of problem dimension. But there are also cases when the entropy both decreases and switches from increasing to decreasing. One possible explanation is the size of the standard deviations (σ), which are all quite large in comparison to the random search. These standard deviations give insight into the diversity of the individual populations that make up the mean entropy values. Additionally, the largest standard deviations are greater than one, but they also only occur for EAs that are using elitism to a very high degree (retained individuals, r =6) and the dimensionality is very low. To better understand how the mean entropy changes for the various EA configurations used, the data is reorganized by EA type, problem dimension, and varied EA parameter. Then the data is presented by peak count for both topologies, as shown in table VII. Again, the entropy categories derived from peak counts is seen in this table. This table also includes the mean entropy velocity (Δ), or the difference between the mean entropy for the current varied parameter value and the table item just above the current value. What this information shows is that as each varied EA parameter is increased the velocity decreases for all problems specified by topology category, peak count, and problem dimension. Thus, even though a mean entropy may initially increase as Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.

6 TABLE VI COMPUTED MEAN ENTROPY (H) AND STANDARD DEVIATION (σ) FOR THE DIFFERENT EA CONFIGURATIONS FROM MANIPULATION PROBLEM DOMAIN TOPOLOGY SET. MUTATION WITH ELITISM -peaks 5-peaks 5-peaks r n H σ H σ H σ MUTATION WITHOUT ELITISM -peaks 5-peaks 5-peaks l n H σ H σ H σ MUTATION AND CROSSOVER -peaks 5-peaks 5-peaks r n H σ H σ H σ more individuals are preserved from one generation to the next, as the parameter increases the velocity will slow and move to the negative. These results illustrate that the mean visitation entropy provides for a comparison of different techniques in terms of how an EA manipulates the problem domain topology. It also provides a method for characterizing when a particular EA technique responds to parameter tuning. What the mean entropy does not provide is an immediate measure of convergence. Fig. 4 provides an example of a convergent system, which uses topology set, problem dimension 5, and is constructed from 5 peaks. The EA that is converging used crossover with elitism, with 6 individuals retained for the next generation. The peak visitation graph (Fig. 4.B) is also given, which illustrates the system focusing on several peaks until Closest Peak (Ordered by fitness) Closest Peak Distance A: Peak Visitation Frequency B: Distance to Nearest Peak Fig. 4. A convergent EA on topology with a problem dimension 5 and constructed from 5 peaks. it converges to the global optimum. The individual entropy for the EA shown in fig 4 is Fig 5 provides an example of a semi-convergent system, with the system is falling in and out of a common set of peak basins. The problem domain configuration was from topology set 2, problem dimension 5, and is constructed from 5 peaks. The semi-convergent EA used crossover with elitism, with 6 individuals retained for the next generation. This case is more interesting than the EA shown in Fig. 4, because it illustrates how a system can begin to converge, but then fallout of a convergent state as slightly better cases are generated outside of the local optimum basin. This case also illustrates a modality that EAs can exhibit as they oscillate between different behavior profiles. The individual entropy for the EA shown in Fig. 5 is Based on Fig. 4 and 5 there are two possible approaches for capturing the fine grain behavior of convergence. One approach for determining fine grain convergence is examining the distance to the nearest peak, which is computed while determining the nearest peak. By examining the trend of the curve for nearest peak distance the curves will approach zero. A second approach, is to divide the peak visitation frequency into non-overlapping windows and measure the visitation entropy for each window. This visitation entropy set can then be analyzed to determine the different modes of entropy the system transitions through. This approach would be similar to looking for phase transitions using the Option Set Entropy discussed previously. Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.

7 TABLE VII COMPUTED MEAN ENTROPY (H), STANDARD DEVIATION (σ), AND VELOCITY (Δ) FOR EA CONFIGURATIONS MANIPULATING BOTH PROBLEM DOMAIN TOPOLOGIES. MUTATION WITH ELITISM TOPOLOGY SET TOPOLOGY SET 2 -peaks 5-peaks 5-peaks -peaks 5-peaks 5-peaks n r H Δ σ H Δ σ H Δ σ H Δ σ H Δ σ H Δ σ MUTATION WITHOUT ELITISM TOPOLOGY SET TOPOLOGY SET 2 -peaks 5-peaks 5-peaks -peaks 5-peaks 5-peaks n l H Δ σ H Δ σ H Δ σ H Δ σ H Δ σ H Δ σ MUTATION AND CROSSOVER TOPOLOGY SET TOPOLOGY SET 2 -peaks 5-peaks 5-peaks -peaks 5-peaks 5-peaks n r H Δ σ H Δ σ H Δ σ H Δ σ H Δ σ H Δ σ Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.

8 Closest Peak (Ordered by fitness) Closest Peak Distance A: Peak Visitation Frequency B: Distance to Nearest Peak Fig. 5. A semi-convergent EA on topology 2 with a problem dimension 5 and constructed from 5 peaks. V. CONCLUSION Presented is a technique, called peak analysis, for characterizing Evolutionary Algorithm (EA) behavior in highdimensional space, using an information theoretic metric that is also empirical in nature. This metric tracks how the system progresses through the problem domain, using the optima set as a map. By tracking which peak each generated individual is near over several evolutionary generations, the entropy can be computed and used to measure how the system traverses the fitness landscape. In other words, by using a computed entropy measure, whether an EA remains tightly grouped around a small set of peaks or more completely searches the problem domain space can be calculated. This work has shown that as parameters are varied there is an associated change in the entropy value, such that the velocity (or difference) between entropy values produces greater changes towards a lower entropy. Thus, this metric provides a linkage between EA parameter tuning and search behavior in a high-dimensional search space. This metric provides an alternative to performance based metrics, such that this metric ties the problem domain topology to the EA system behavior. Performance based metrics also respond to the underlying topology, but only secondarily through the fitness value. Peak analysis requires that the optima set be known, which must either be determined or computed. While it can be argued that this requirement is a drawback, a counter argument can be made that having a solid understanding of the explored topology gives a stronger intuition of how the evolutionary technique is interacting with the fitness landscape. Future enhancements of the proposed technique have the potential to provide a higher-resolution understanding of how a fitness landscape is manipulated. This can be accomplished by examining how the entropy measure changes over time and determining when phase changes occur. The most immediate direction for this work is exploring additional EA configurations. One potentially interesting configuration is Evolutionary Strategies (ES), which are specifically designed to climb optima gradients, making this technique a good test for the peak analysis metric. A second direction is to determine if correlations exist between the nearness of peaks in the optima set and the behavior of an EA to become trapped in certain regions. Finally, there are open questions that can be explored, such as what effect does point mutation produce verses crossover and what is contributed to the general EA behavior. The presented technique provides a useful metric that is not explicitly bound to the performance of an EA. Instead this technique relies on the underlying problem domain topology to reveal an understanding of the EA behavior within that topology and provides another avenue for exploring the stochastic nature of Evolutionary Algorithms. More generally, an initial exploration of evolutionary behavior that examines how a conceptualized species traverses an ecological niche set, represented by topology optima, without relying directly on a fitness measure for analysis. REFERENCES [] D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Boston, MA: Kluwer Academic Publishers, 989. [2] D. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence., 3rd ed. Piscataway, NJ: IEEE Press, 26. [3] T. Bäck, Evolutionary Algorithms in Theory and Practice. NY: Oxford University Press, 996. [4] D. Wolpert and W. Macready, No free lunch theorems for optimization, IEEE Transactions on Evolutionary Computation, vol., no., pp , April 997. [5] D. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge, UK: Cambridge University Press, 23. [6] M. Li and P. Vitányi, An Introduction to Kolmogorov Complexity and its Applications, 2nd ed., ser. Graduate Texts in Computer Science, D. Gries and F. B. Schneider, Eds. New York, NY: Springer, 997. [7] S. A. Brueckner and H. V. D. Parunak, Resource-aware exploration of the emergent dynamics of simulated systems, in AAMAS 3: Proceedings of the second international joint conference on Autonomous agents and multiagent systems, ACM, Ed., New York, NY, USA, 23, pp [8] L. Kallel, B. Naudts, and A. Rogers, Eds., Theoretical Aspects of Evolutionary Computing, ser. Natural Computing Series. Berlin: Springer, 2. [9] C. R. Reeves and J. E. Rowe, Genetic Algorithms: Principles and Perspectives, ser. Operation Research/Computer Science Interfaces Series. Springer, 22, vol. 2. [] M. Schmidt and Z. Michalewicz, Test-case generator TCG-2 for nonlinear parameter optimization, in Parallel Problem Solving from Nature PPSN VI, Proceedings of the 6th International Conference, September 8-2 Paris, France, ser. Lecture Notes in Computer Science, M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. M. Guervós, and H.-P. Schwefel, Eds., vol. 97. Heidelberg, Germany: Springer-Verlag, 2, pp [] F. M. Reza, An Introduction to Information Theory. New York, NY: Dover Publications, Inc, 994. [2] M. Galassi, GNU Scientific Library Reference Manual, 2nd ed. [3] A. Eiben and J. Smith, Introduction to Evolutionary Computing, ser. Natural Computing Series. Berlin: Springer, 27. Authorized licensed use limited to: Lafayette College. Downloaded on February 23,2 at :2:2 EST from IEEE Xplore. Restrictions apply.