Evolutionary Computation

Transcription

1 Evolutionary Computation Hsien-Chung Wu 1 Department of Mathematics, National Kaohsiung Normal University, Kaohsiung 802, Taiwan 1 hcwu@nknucc.nknu.edu.tw

2 Contents 1 Evolutionary Computation Genetic Algorithms A Simple Optimization Problem The Schema for GAs Real-Coded Genetic Algorithms Messy Genetic Algorithms Selection in Genetic Algorithms Evolution Strategies Comparison of Evolution Strategies and Genetic Algorithms Evolutionary Programming Genetic Programming The Real-Coded Genetic Algorithms for Numerical Optimization Genetic Operators Mutation Group Crossover Group Adaptive Probabilities of Genetic Operators The Synergy of Multiple Crossover Operators Constraint-Handling Methods Searching the Boundary of Feasible Region Methods Based on Penalty Functions Different Types of Evolutionary Algorithms An Evolution Program: The GENOCOP System Nonlinear Optimization: GENOCOP II GENOCOP III Genetic Algorithms and Simplex Method The Simplex Crossover GA A Simplex Genetic Algorithm Hybrid Optimization with the Interval Genetic Algorithms Simulated Annealing The Interval Genetic Algorithm Implementation Virus-Evolutionary Genetic Algorithms Genetic Algorithms with the Concept of Age Individual Aging in Genetic Algorithms Genetic Algorithms with Age Structure Guided Evolutionary Simulated Annealing Integrating Adaptive Mutations and Family Competition into GAs Adaptive Mutations Genetic Algorithm

3 2 CONTENTS Individual Representation and Initialization Crossover Operators Mutation Operators Adaptive Rules AMGA Parameter Settings Neural Networks with Genetic Algorithms Applications of GA-Based Optimization of Neural Network Connection Topology Evolving Neurocontrollers Using Evolutionary Programming Tuning of a Neuro-Fuzzy Controller by GA Coding Strategy of the NFLC Parameters Optimization by GA Genetic Fuzzy Systems Introduction to Fuzzy Systems Linguistic Variables and Fuzzy IF-THEN Rules Fuzzy Logic and Approximate Reasoning The Compositional Rule of Inference Fuzzy Rule Base and Fuzzy Inference Engine Fuzzifiers and Defuzzifiers A GA-Based Method for Tuning Fuzzy Logic Controllers A Genetic Algorithms for Optimizing Takagi-Sugeno Fuzzy Rule Bases Fuzzy System Parameters Discovery by Bacterial Evolutionary Algorithm Integrating Fuzzy Knowledge by Genetic Algorithms Genetic-Based New Fuzzy Reasoning Models with Application to Fuzzy Control Implementation of Evolutionary Fuzzy Systems

4 Chapter 1 Evolutionary Computation The common term evolutionary computation comprises techniques such as genetic algorithms, evolution strategies, evolutionary programming and genetic programming. Their principal mode of operation is based on the same genetic concepts, a population of competing candidate solutions, random combinations and alteration of potentially useful structures to generate new solutions and a selection mechanism to increase the proportion of better solutions. The different approaches are distinguished by the genetic structures that under adaption and the genetic operators that generate new candidate solutions (Cordón el al.[9, p.47]). Each cell of a living organism embodies a strand of DNA distributed over a set of chromosomes. The cellular machinery transcribes this blueprint into proteins that are used to build the organism. A gene is a functional entity that encodes a specific feature of the individual such as hair color. The term allele describes the value of gene, which determines the manifestation for an attribute, e.g., blonde, black, brunette hair color. Genotype refers to the specific combination of genes carried by a particular individual, whereas the term phenotype is related to the physical makeup of an organism. Each gene is located at a certain position within the chromosome, which is called locus (Cordón el al.[9, p.48]). Progress in natural evolution is based on three fundamental processes: mutation, recombination and selection of genetic variants. The role of mutation is the random variations of the existing genetic material in order to generate new phenotypical traits. Recombination hybridises two different chromosomes in order to integrate the advantageous features of both parents into their offspring. Selection increases the proportion of better adapted individuals in the population. Darwin coined the term survival of the fittest to illustrate the selection principle that explains the adaption of species to their environment. The term fitness describes the quality of an organism, which is synonymous with the ability of the phenotype to reproduce offspring in order to promote its genes to future generations (Cordón el al.[9, p.48]). 1.1 Genetic Algorithms Genetic algorithms (GAs) are general-purpose search algorithms that use principles inspired by natural population genetics to evolve solutions to problems. They were first proposed by Holland [22]. A GA operates on a population of randomly generated solutions, chromosomes often represented by binary strings. The population advances toward better solutions by applying genetic operators, such as crossover and mutation. In each generation, favorable solutions generate offspring that replace the inferior individuals. Crossover hybridises the genes of two parent chromosomes in order to exploit the search space and constitutes the main genetic operator in GAs. The purpose of mutation is to maintain the diversity of the gene pool. An evaluation or fitness function plays the role of the environment to distinguisch between good and bad solutions (Cordón el al.[9, p.49]). 3

5 4 CHAPTER 1. EVOLUTIONARY COMPUTATION A Simple Optimization Problem We will discuss the actions of a genetic algorithm for a simple parameter optimization problem. Let us note first that, without loss of generality, we can assume maximization problems only. Moreover, we may assume that the objective function f takes positive values on its domain; otherwise we can add some positive constant C. Now suppose we wish to maximize a function of k variables, f(x 1,, x k ) : R k R +. Suppose further that each variable x i can take values from a domain D i = [a i, b i ] R. We wish to optimize the function f with some required precision: suppose six decimal places for the variables values is desirable. Let us denote by m i the smallest integer such that (b i a i ) mi 1. Then a representation having each variable x i coded as a binary string of length m i clearly satisfies the precision requirement. Additionally, the following formula interprets each such string x i = a i + decimal(string 2 ) b i a i 2 mi 1, where decimal(string 2 ) represents the decimal value of that binary string (Michalewicz [36, p.33]). Now each chromosome (as a potential solution) is represented by a binary string of length m = k i=1 m i; the first m 1 bits map into a value from the range [a 1, b 1 ], the next group of m 2 bits map into a value from the range [a 2, b 2 ], and so on; the last group of m k bits map into a value from the range [a k, b k ]. To initialize a population, we can simply set some popsize number of chromosomes randomly in a bitwise fashion. In each generation we evaluate each chromosome (using the function f on the decoded sequences of variables), select new population with respect to the probability distribution based on fitness values, and alter the chromosomes in the new population by mutation and crossover operators. After some number of generations, when no further improvement is observed, the best chromosome represents an (possibly the global) optimal solution. Often we stop the algorithm after a fixed number of iterations depending on speed and resource criteria. For the selection process (selection of a new population with respect to the probability distribution based on fitness values), a roulette wheel with slots size according to fitness is used. In order to implement the selection process, crossover and mutation, we introduce the inverse transform method to generate a discrete random variable. Inverse Transform Method Suppose we want to generate the value of a discrete random variable X having probability mass function P {X = x j } = p j, j = 0, 1, 2,, where p j = 1. j To accomplish this, we generate a random number u, and set x 0 if u < p 0 x 1 if p 0 u < p 0 + p 1 x =. x j.. if j 1 i=1 p i u j i=1 p i Since, for 0 < a < b < 1, P {a U < b} = b a, we have that P {X = x j } = P and so X has the desired distribution. { j 1 p i U < i=1 } j p i = p j i=1

6 1.1. GENETIC ALGORITHMS 5 The preceding discussions can be written algorithmically as Generate a random number u If u < p 0 set x = x 0 and stop If u < p 0 + p 1 set x = x 1 and stop If u < p 0 + p 1 + p 2 set x = x 2 and stop. (1.1) If the x i are ordered so that x 0 < x 1 < x 2 < and if we let F denote the distribution function of X, then k k F (x k ) = P {X x k } = P {X = x i } = and so x will equal x j if j 1 i=1 p i = F (x j 1 ) u < F (x j ) = j i=1 p i. In other words, after generating a random number u we determine the value of x by finding the interval [F (x j 1 ), F (x j )] in which u lies or, equivalently, by finding the inverse of F (u). It is for this reason that the above is called the discrete inverse transform method for generating X. Furthermore, we have the following algorithm i=0 i=1 p i Generate a random number u If u < F (x 0 ) set x = x 0 and stop If u < F (x 1 ) set x = x 1 and stop If u < F (x 2 ) set x = x 2 and stop (1.2). (Ross [46, p.45-46] Selection Process We now construct a roulette wheel as follows: (i) Calculate the fitness value eval(v i ) for each chromosome v i for i = 1,, popsize. (ii) Find the total fitness of the population F = popsize i=1 eval(v i ). (iii) Calculate the probability of a selection p i for each chromosome v i for i = 1,, popsize: p i = eval(v i). F (iv) Calculate a cumulative probability q i for each chromosome v i for i = 1,, popsize: q i = i p i. j=1 Therefore, we have constructed a probability mass function P {X = v i } = p i, where popsize i=1 p i = 1. The selection process is based on spinning the roulette wheel popsize times; each time we select a single chromosome for a new population as in the algorithms (1.1) and (1.2). (i) Generate a random number r from the range [0, 1].

7 6 CHAPTER 1. EVOLUTIONARY COMPUTATION (ii) If r < q 1 then select the first chromosome v 1 ; otherwise select the ith chromosome v i such that q i 1 < r q i. Obviously, some chromosomes would be selected more than once. This is in accordance with the Schema Theorem: the best chromosomes get more copies, the average stay even, and the worst die off (Michalewicz [36, p.34]). Crossover Now we ready to apply the recombination operator, crossover, to the individuals in the new population. One of the parameters of a genetic system is probability of crossover p c. This probability gives us the expected number p c popsize of chromosomes which undergo the crossover operation. We proceed in the following way. For each chromosome in the new population, we follow the following procedure. (i) Generate a random number r from the range [0, 1]. (ii) If r < p c, select given chromosomes for crossover. The above procedure can be regarded as having a mass function P {X = 1} = P {crossover occurs} = p c and P {X = 0} = P {crossover does not occur} = 1 p c, i.e., X is a Bernoulli random variable. Now we mate selected chromosomes randomly. For each pair of coupled chromosomes we generate a random integer number pos from the range [1, m 1] (m is the total length number of bits in a chromosome). The number pos indicates the position of the crossing point. Two chromosomes (b 1, b 2,, b pos, b pos+1,, b m ) and (c 1, c 2,, c pos, c pos+1,, c m ) are replaced by a pair of their offspring, (b 1, b 2,, b pos, c pos+1,, c m ) and (c 1, c 2,, c pos, b pos+1,, b m ) (Michalewicz [36, p.35]). Mutation The next operator, mutation, is performed on a bit-by-bit basis. Another parameter of the genetic system, probability of mutation p m, gives us the expected number of mutated bits p m m popsize. Every bit (in all chromosomes in the whople population) has a equal chance to undergo mutation, i.e., change from 0 to 1 or vice versa. So we proceed in the following way. (i) Generate a random number r from the range [0, 1]. (ii) If r < p m, mutate the bit. Following selection, crossover, and mutation, the new population is ready for its next evaluation. This evaluation is used to build the probability distribution for the next selection population, i.e., for a construction of a roulette wheel with slots sized according to current fitness values. The rest of evolution is just cyclic repetition of the above steps (Michalewicz [36, p.35]). Example (Michalewicz [36, p.36]) We now run a simulation of a genetic algorithm for function optimization. We assume that the population size popsize = 20, and the probabilities of genetic operators are p c = 0.25 and p m = Let us assume also that we maximize the following function f(x 1, x 2 ) = x 1 sin(4πx 1 ) + x 2 sin(20πx 2 ), where 3.0 x and 4.1 x Let us assume further that the required precision is four decimal places for each variable. The domain of variable x 1 has length 15.1; the precision requirement implies that 2 17 < This means that 18 bits are required as the first part of the chromosome. The domain of variable x 2 has length 1.7; the precision requirement implies 2 14 < This means that 15 bits are required as the second part of the chromosome. The total length of a chromosome (solution vetcor) is then m = = 33 bits; the first 18 bits code x 1 and remaining 15 bits code x 2.

8 1.1. GENETIC ALGORITHMS 7 Let us consider an example chromosome ( ). The first 18 bits represent x 1 = decimal( ) 12.1 ( 3.0) = The next 15 bits represent x 2 = decimal( ) = So the chromosome ( ) corresponds to (x 1, x 2 ) = ( , ). The fitness value for the chromosome is f( , ) = To optimize the function f using a genetic algorithm, we create a population of popsize = 20 chromosomes. All 33 bits in all chromosomes are initialized randomly. Assume that after the initialization process we get the following population v 1 = ( ) v 2 = ( ) v 3 = ( ) v 4 = ( ) v 5 = ( ) v 6 = ( ) v 7 = ( ) v 8 = ( ) v 9 = ( ) v 10 = ( ) v 11 = ( ) v 12 = ( ) v 13 = ( ) v 14 = ( ) v 15 = ( ) v 16 = ( ) v 17 = ( ) v 18 = ( ) v 19 = ( ) v 20 = ( ) During the evaluation phase we decode each chromosome and calculate the fitness function values

9 8 CHAPTER 1. EVOLUTIONARY COMPUTATION from (x 1, x 2 ) values just decoded. We get eval(v 1 ) = f( , ) = eval(v 2 ) = f( , ) = eval(v 3 ) = f( , ) = eval(v 4 ) = f( , ) = eval(v 5 ) = f( , ) = eval(v 6 ) = f( , ) = eval(v 7 ) = f( , ) = eval(v 8 ) = f( , ) = eval(v 9 ) = f( , ) = eval(v 10 ) = f( , ) = eval(v 11 ) = f( , ) = eval(v 12 ) = f( , ) = eval(v 13 ) = f( , ) = eval(v 14 ) = f( , ) = eval(v 15 ) = f( , ) = eval(v 16 ) = f( , ) = eval(v 17 ) = f( , ) = eval(v 18 ) = f( , ) = eval(v 19 ) = f( , ) = eval(v 20 ) = f( , ) = It is clear that the chromosome v 15 is the strogest one, and the chromosome v 2 is the weakest one. Now the system constructs a roulette wheel for the selection process. The total fitness of the population is F = 20 i=1 v i = The probability of a selection p i for each chromosome v i for i = 1,, 20 is p 1 = eval(v 1 )/F = p 2 = eval(v 2 )/F = p 3 = eval(v 3 )/F = p 4 = eval(v 4 )/F = p 5 = eval(v 5 )/F = p 6 = eval(v 6 )/F = p 7 = eval(v 7 )/F = p 8 = eval(v 8 )/F = p 9 = eval(v 9 )/F = p 10 = eval(v 10 )/F = p 11 = eval(v 11 )/F = p 12 = eval(v 12 )/F = p 13 = eval(v 13 )/F = p 14 = eval(v 14 )/F = p 15 = eval(v 15 )/F = p 16 = eval(v 16 )/F = p 17 = eval(v 17 )/F = p 18 = eval(v 18 )/F = p 19 = eval(v 19 )/F = p 20 = eval(v 20 )/F = The cumulative probabilities q i for each chromosome v i for i = 1,, 20 are q 1 = q 2 = q 3 = q 4 = q 5 = q 6 = q 7 = q 8 = q 9 = q 10 = q 11 = q 12 = q 13 = q 14 = q 15 = q 16 = q 17 = q 18 = q 19 = q 20 = Now we are ready to spin the roulette wheel 20 times; each time we select a single chromosome for

10 1.1. GENETIC ALGORITHMS 9 a new population. Let us assume that a random sequence of 20 numbers from the range [0, 1] is The first number r = is greater than q 10 and smaller than q 11, menaing the chromosome v 11 is selected for the new population; the second number r = is greater than q 3 and smaller than q 4, meaning the chromosome v 4 is selected for the new population, etc. Finally, the new population consists of the following chromosomes v 1 = ( )(v 11 ) v 2 = ( )(v 4 ) v 3 = ( )(v 7 ) v 4 = ( )(v 11 ) v 5 = ( )(v 19 ) v 6 = ( )(v 4 ) v 7 = ( )(v 15 ) v 8 = ( )(v 5 ) v 9 = ( )(v 11 ) v 10 = ( )(v 3 ) v 11 = ( )(v 15 ) v 12 = ( )(v 9 ) v 13 = ( )(v 6 ) v 14 = ( )(v 8 ) v 15 = ( )(v 20 ) v 16 = ( )(v 1 ) v 17 = ( )(v 10 ) v 18 = ( )(v 13 ) v 19 = ( )(v 15 ) v 20 = ( )(v 16 ) Now we are ready to apply the recombination operator, crossover, to the individuals in the new population (vectors v i ). The probability of crossover p c = 0.25, so we expect that 5 = chromosomes undergo crossover. We proceed in the following way. For each chromosome in the new population, we generate a random number r from the range [0, 1]. If r < 0.25, we select a given chromosome for crossover. Let us assume that the sequence of random numbers is This means that the chromosomes v 2, v 11, v 13, and v 18 were selected for crossover (We are lucky: the number of selected chromosomes is even, so we can pair them easily. If the number of selected chromosomes were odd, we would either add one extra chromosome or remove one selected chromosome this choice is made randomly as well). Now we mate selected chromosomes randomly: say, the first two, v 2 and v 11, and the next two, v 13 and v 18 are coupled together. For each of these two pairs, we generate a random integer number pos from the range [1, 32]. The number pos indicates the position of the crossing point. The first pair of chromosomes is v 2 = ( ) and v 11 = ( )

11 10 CHAPTER 1. EVOLUTIONARY COMPUTATION and the generated number pos = 9. These chromosomes are cut after the 9th bit and replaced by a pair of their offspring v 2 = ( ) and v 11 = ( ) The second pair of chromosomes is v 13 = ( ) and v 18 = ( ) and the generated number pos = 20. These chromosomes are replaced by a pair of their offspring v 13 = ( ) and v 18 = ( ) The current version of the population is v 1 = ( ) v 2 = ( ) v 3 = ( ) v 4 = ( ) v 5 = ( ) v 6 = ( ) v 7 = ( ) v 8 = ( ) v 9 = ( ) v 10 = ( ) v 11 = ( ) v 12 = ( ) v 13 = ( ) v 14 = ( ) v 15 = ( ) v 16 = ( ) v 17 = ( ) v 18 = ( ) v 19 = ( ) v 20 = ( ) The next operator, mutation, is performed on a bit-by-bit basis. The probability of mutation p m = 0.01, so we expect that (on average) 1% of bits would undergo mutation. There are m popsize = = 660 bits in the whole population; we expect (on average) 6.6 mutations per generation. Every bit has equal chance to be mutated, so, for every bit in the population, we generate a random number r from the range [0, 1]; if r < 0.01, we mutate the bit. This means that we have to generate 660 random numbers. In a simple run, 5 of these numbers were smaller than 0.01; the bit number and the random number are listed below Bit position Random number The following table translate the bit position into chromosome number and the bit number within

12 1.1. GENETIC ALGORITHMS 11 the chromosome Bit position Chromosome number Bit number within chromosome This means that four chromosomes are affected by the mutation operator; one of the chromosomes (the 13th) has two bits changed. The final population is listed below; the mutated bits are typed in boldface. We drop primes for modified chromosomes: the population is listed as new vectors v i v 1 = ( ) v 2 = ( ) v 3 = ( ) v 4 = ( ) v 5 = ( ) v 6 = ( ) v 7 = ( ) v 8 = ( ) v 9 = ( ) v 10 = ( ) v 11 = ( ) v 12 = ( ) v 13 = ( ) v 14 = ( ) v 15 = ( ) v 16 = ( ) v 17 = ( ) v 18 = ( ) v 19 = ( ) v 20 = ( ) We have just completed one iteration (i.e. one generation). It is interesting to examine the results of the evaluation process of the new population. During the evaluation phase, we decode each

13 12 CHAPTER 1. EVOLUTIONARY COMPUTATION chromosome and calculate the fitness function values from (x 1, x 2 ) values just decoded. We get eval(v 1 ) = f( , ) = eval(v 2 ) = f( , ) = eval(v 3 ) = f( , ) = eval(v 4 ) = f( , ) = eval(v 5 ) = f( , ) = eval(v 6 ) = f( , ) = eval(v 7 ) = f( , ) = eval(v 8 ) = f( , ) = eval(v 9 ) = f( , ) = eval(v 10 ) = f( , ) = eval(v 11 ) = f( , ) = eval(v 12 ) = f( , ) = eval(v 13 ) = f( , ) = eval(v 14 ) = f( , ) = eval(v 15 ) = f( , ) = eval(v 16 ) = f( , ) = eval(v 17 ) = f( , ) = eval(v 18 ) = f( , ) = eval(v 19 ) = f( , ) = eval(v 20 ) = f( , ) = Note that the total fitness of the new population F is , much higher than total fitness of the previous population Also, the best chromosome now v 11 has a better evaluation than the best chromosome v 15 from the previous population Now we are ready to run the selection process again and apply the gentic operators, evaluate the next generation, etc. After 1000 generations the population is v 1 = ( ) v 2 = ( ) v 3 = ( ) v 4 = ( ) v 5 = ( ) v 6 = ( ) v 7 = ( ) v 8 = ( ) v 9 = ( ) v 10 = ( ) v 11 = ( ) v 12 = ( ) v 13 = ( ) v 14 = ( ) v 15 = ( ) v 16 = ( ) v 17 = ( ) v 18 = ( ) v 19 = ( ) v 20 = ( )

14 1.1. GENETIC ALGORITHMS 13 The fitness values are eval(v 1 ) = f( , ) = eval(v 2 ) = f( , ) = eval(v 3 ) = f( , ) = eval(v 4 ) = f( , ) = eval(v 5 ) = f( , ) = eval(v 6 ) = f( , ) = eval(v 7 ) = f( , ) = eval(v 8 ) = f( , ) = eval(v 9 ) = f( , ) = eval(v 10 ) = f( , ) = eval(v 11 ) = f( , ) = eval(v 12 ) = f( , ) = eval(v 13 ) = f( , ) = eval(v 14 ) = f( , ) = eval(v 15 ) = f( , ) = eval(v 16 ) = f( , ) = eval(v 17 ) = f( , ) = eval(v 18 ) = f( , ) = eval(v 19 ) = f( , ) = eval(v 20 ) = f( , ) = However if we look at carefully at the progress during the run, we may discover that in earlier generations the fitness values of some chromosomes were better than the value of the best chromosome after 1000 generations. For example, the best chromosome in generation 396 had value of This is due to the stochastic errors of sampling. It is relatively easy to keep track of the best individual in the evolution process. It is customary (in genetic algorithm implementations) to store the best ever (elitist) individual at a seperate location; in that way, the algorithm would report the best value found during the whole process (as opposed to the best value in the final population)(elitism) (Michalewicz [36, p.44]) The Schema for GAs A schema is built by introducing a don t care symbol into the alphabet genes. A schema represents all strings, which match it on all positiosn other than. For example, let us consider the strings and schemata of the length 10. The schema ( ) matches two strings {( ), ( )}, and the schema ( ) matches four strings {( ), ( ), ( ), ( )}. Of course, the schema ( ) represents one string only: ( ), and the schema ( ) represents all strings of length 10. It is clear that every schema matches exactly 2 r strings, where r is the number of don t care symbols in a schema template. On the other hand, each string of the length m is matched by 2 m schemata (Michalewicz [36, p.45]). There are two important schema properties, order and defining length. The Schema Theorem will be formulated on the basis of these properties. The order of the schema S, denoted by o(s), is the number of 0 and 1 positions, i.e., fixed positions (non-don t care positions), present in the schema. The order defines the speciality of a schema. For example, the following three schemata, each of length 10, S 1 = ( ), S 2 = ( 00 0 ), S 3 = ( ),

15 14 CHAPTER 1. EVOLUTIONARY COMPUTATION have the following orders o(s 1 ) = 6, o(s 2 ) = 3, and o(s 3 ) = 8, and the schema S 3 is the most specific one. The notion of the order of a schema is useful in calculating survival probabilities of the schema for mutations (Michalewicz [36, p.46]). Assume the chromosome is composed of three bits only. The solution space can be represented by a cube, whose axes correspond to the bits. The corners are labelled by bit strings with 000 located at the origin and 111 at the upper, right, rear corner. The bottom plane contains all bit strings that end with a 0 or in other words match the schema 0, the scehma 00 covers two instances 000 and 001 located at the left, front vertical edge. In the general case of an n-dimensional hypercube, a schema of order m corresponds to a n m-dimensional hyperplane (Cordón el al.[9, p.59]). The defining length of the schema S, denoted by δ(s), is the distance between the first and the last fixed string positions. It defines the compactness of information contained in a schema. For example, δ(s 1 ) = 10 4 = 6, δ(s 2 ) = 9 5 = 4, and δ(s 3 ) = 10 1 = 9. Note that the schema with a single fixed position has a defining length of zero. The notion of defining length of a schema is useful in calculating survival probabilities of the schema for crossover (Michalewicz [36, p.46]). Let us assume, the population size popsize = 20, the length of a string is m = 33 as in the previous example. Assume further that (at time t) the population consists of the following strings v 1 = ( ) v 2 = ( ) v 3 = ( ) v 4 = ( ) v 5 = ( ) v 6 = ( ) v 7 = ( ) v 8 = ( ) v 9 = ( ) v 10 = ( ) v 12 = ( ) v 13 = ( ) v 14 = ( ) v 15 = ( ) v 16 = ( ) v 17 = ( ) v 18 = ( ) v 19 = ( ) v 20 = ( ) Let us denote by ξ(s, t) the number of strings in a population at time t, matched by scehma S. For example, for a given schema S 0 = ( 111 ), ξ(s 0, t) = 3, since there are 3 strings, namely v 13, v 15, and v 16, matched by the schema S 0. Note that the order of the schema S 0, o(s 0 ) = 3, and its defining length δ(s 0 ) = 7 5 = 2 (Michalewicz [36, p.47]). Another property of a schema is its fitness at time t, eval(s, t). It is defined as the average fitness of all strings in the population matched by the schema S. Assume there are p strings {v i1,, v ip }

16 1.1. GENETIC ALGORITHMS 15 in the population matched by a schema S at time t. Then eval(s, t) = p j=1 eval(v ij ). p During the selection step, an intermediate population is created: popsize = 20 single string selections are made. Each string is copied zero, one, or more times, according to its fitness. As we have seen before, in a single string selection, the string v i has probability p i = eval(v i )/F (t) to be selected, where F (t) = 20 i=1 eval(v i) is the total fitness of the whole population at time t. After the selection step, we expect to have ξ(s, t + 1) strings matched by schema S. Since (i) for an average string matched by a schema S, the probability of its selection (in a single string selection) is equal to eval(s, t)/f (t), (ii) the number of strings matched by a schema S is ξ(s, t), and (iii) the number of single string selections is popsize, it is clear that ξ(s, t + 1) = ξ(s, t) popsize eval(s, t). F (t) We can rewrite the above formula: taking into account that the average fitness of the population F (t) = F (t)/popsize, we can write ξ(s, t + 1) = ξ(s, t) eval(s, t). (1.3) F (t) In other words, the number of strings in the population grows as the ratio of the fitness of the schema to average fitness of the population. This means that an above average schema receives an increasing number of strings in the next generation, a below average scheme receives decreasing number of strings, and an average schema stays on the same level. We also see that t 1 ξ(s, t) = ξ(s, 0) i=0 eval(s, i). F (i) The long-term effect of the above rule is also clear. If we assume that a schema S remains above average by ɛ%, i.e., eval(s, t) = F (t) + ɛ F (t) (we assume eval(s, i)/ F (i) = eval(s, t)/ F (t) for all i = 0, 1,, t 1), then ξ(s, t) = ξ(s, 0)(1 + ɛ) t and ɛ = eval(s, t) F (t) F (t) (ɛ > 0 for the above average schemata and ɛ < 0 for the below average schemata). We call the equation (1.3) the reproductive schema growth equation (Michalewicz [36, p.48]). Let us return to the example schema S 0. Since there are 3 strings, namely v 13, v 15, and v 16 (at time t) matched by the schema S 0, the fitness eval(s 0 ) of the schema is eval(s 0, t) = ( )/3 = At the same time, the average fitness of the whole population is F (t) = 20 i=1 eval(v i) = = , popsize 20 and the ratio of the fitness of the schema S 0 to the average fitness of the population is eval(s 0, t)/ F (t) =

17 16 CHAPTER 1. EVOLUTIONARY COMPUTATION This means that if the schema S 0 stays above average, then it receives an exponentially increasing number of strings in the next generations. In particular, if the schema S 0 stays above average by the constant factor of , then, at time t + 1, we expect to have = 4.19 strings matched by schema S 0 (i.e. most likely 4 or 5), at time t + 2, = 5.85 such strings (i.e. very likely 6 strings), etc. The intuition is that such a schema S 0 defines a promising part of the search space and is being sampled in an exponentially increased manner (Michalewicz [36, p.49]). Let us check these predictions on our running example for the schema S 0. In the population at the time t, the schema S 0 matched 3 strings v 13, v 15 and v 16. In the previous example, we simulated the selection process using the same population. The new population consists of the folowing chromosomes v 1 = ( )(v 11 ) v 2 = ( )(v 4 ) v 3 = ( )(v 7 ) v 4 = ( )(v 11 ) v 5 = ( )(v 19 ) v 6 = ( )(v 4 ) v 7 = ( )(v 15 ) v 8 = ( )(v 5 ) v 9 = ( )(v 11 ) v 10 = ( )(v 3 ) v 11 = ( )(v 15 ) v 12 = ( )(v 9 ) v 13 = ( )(v 6 ) v 14 = ( )(v 8 ) v 15 = ( )(v 20 ) v 16 = ( )(v 1 ) v 17 = ( )(v 10 ) v 18 = ( )(v 13 ) v 19 = ( )(v 15 ) v 20 = ( )(v 16 ) Indeed, the schema S 0 now (time t + 1) matches 5 strings v 7, v 11, v 18, v 19, and v 20. However, selection alone does not introduce any new points (potential solutions) for consideration from the search space; selection just copies some strings to form an intermediate population. So the second step of the evolution cycle, recombination, takes the responsibility of introducing new individuals in the population. This is done by two genetic operators: crossover and mutation (Michalewicz [36, p.49]). Let us start with crossover and consider the following example. As discussed earlier, a single string from the population, say v 18 = ( ), is matched by 230 schemata; in particular, the string is matched by these two schemata S 0 = ( 111 ) and S 1 = (111 10). Let us assume further that the above string was selected for crossover. Assume that (according to experiments from the previous example, where v 18 was crossed with v 13) the generated crossing site pos = 20. It is clear that the schema S 0 survives such a crossover, i.e., one of the offspring still matches S 0. The reason is that the crossing site preserves the sequence 111 on the fifth, sixth, and seventh positons in the string in one of the offsprings: a pair v 18 = ( ), v 13 = ( ) would produce v 18 = ( ), v 13 = ( ).

18 1.1. GENETIC ALGORITHMS 17 On the other hand, the schema S 1 would be destroyed: none of the offspring would match it. It would be clear that the defining length of a schema plays a significant role in the probability of its destruction and survival. Note that the defining length of the schmema S 0 was δ(s 0 ) = 2, and the defining length of the scehma S 1 was δ(s 1 ) = 32. In general, a crossover site is selected uniformly among m 1 possible sites. This implies that the probability of destruction of a schema S is p d (S) = δ(s) m 1, and consequently, the probability of schema survival is p s (S) = 1 δ(s) m 1. Indeed, the probabilities of survival and destruction of our example schemata S 0 and S 1 are p d (S 0 ) = 2 32, p s(s 0 ) = 30 32, p d(s 1 ) = = 1, p s(s 1 ) = 0. It is important to note that only some chromosomes undergo crossover, and the selective probability of crossover is p c. This means that the probability of a schema survival is in fact p s (S) = 1 p c δ(s) m 1. Again, referring to our example schema S 0 and the running example (p c = 0.25) p s (S 0 ) = = Note also that even if a crossover site is selected between fixed position in a schema, there is still a chance for the schema to survive. For example, if both strings v 18 and v 13 started with 111 and ended with 10, the schema S 1 would survive crossover (however, the probability of such event is quite small). Because of this, we would modify the formula for the probability of schema survival p s (S) 1 p c δ(s) m 1. So the combined effect of selection and crossover gives us a new form of the reproductive schema growth equation [ ] eval(s, t) δ(s) ξ(s, t + 1) ξ(s, t) 1 p c. (1.4) F (t) m 1 The equation (1.4) tells us about the expected number of strings matching a schema S in the next generation as a function of the actual number of strings matching the schema, relative fitness of the schema, and its defining length. It is clear that the above-average schemata with short defining length would still be sampled at exponentially increased rates. For the schema S 0 eval(s 0, t) F (t) [ 1 p c δ(s ] 0) m 1 = = This means that the short, above-average schema S 0 would still receive an exponentially increasing number of strings in the next generations: at time t+1, we expect to have = 4.12 strings matched by S 0 (only slightly less than 4.19 a value we got considering selection only), at time t + 2, = 5.67 such strings (again, slightly less than 5.85) (Michalewicz [36, p.51]). The next operator to be considered is mutation. The mutation operator randomly changes a single position within a chromosome with probability p m. The change is from 0 to 1 and vice versa.

19 18 CHAPTER 1. EVOLUTIONARY COMPUTATION It is clear that all of the fixed positions of a schema must remain unchanged if the schema survives mutation. For example, consider again a single string from the population, say, and the schema v 19 = ( ) S 0 = ( 111 ). Assume further that the string v 19 undergoes mutation, i.e., at least one bit is flipped, as happened in the previous example. Since v 19 was mutated at the 8th position, its offspring v 19 = ( ) is still matched by the schema S 0. If the selected mutation positions were from 1 to 4, or from 8 to 33, the resulting offspring would still be matched by S 0. Only 3 bits (fifth, sixth, and seventh the fixed bit positions in the schema S 0 ) are important: mutation of at least one of these bits would destroy the schema S 0. Clearly, the number of such important bits is equal to the order of the schema, i.e., the number of fixed positions. Since the probability of the alteration of a single bit is p m, the probability of a single bit survival is 1 p m. A single mutation is independent from other mutations, so the probability of a schema S surviving a mutation (i.e. sequence of one-bit mutation) is p s (S) = (1 p m ) o(s). Since p m 1, the probability can be approximated by p s (S) 1 o(s) p m. Also the probability of destruction is approximated by p d (S) o(s) p m. Again, referring to our example schama S 0 and the running example with p m = p s (S 0 ) = = The combined effect of selection, crossover, and mutation gives us a new form of the reproductive schema growth equation [ ] eval(s, t) δ(s) ξ(s, t + 1) ξ(s, t) 1 p c F (t) m 1 o(s) p m. (1.5) Equation (1.5) tells us about the expected number of strings matching a schema S in the next genneration as a function of the actual number of strings matching the schema, the relative fitness of the schema, and its defining length and order. Again, it is clear that above-average schemata with short defining length and low-order would still be sampled at exponentially increased rates. For the schema S 0 [ eval(s 0, t) 1 p c δ(s ] 0) F (t) m 1 o(s 0) p m = = This means that the short, low-order, above-average schema S 0 would still receive an exponentially increasing number of strings in the next generations: at time t + 1, we expect to have = 4.00 string matched by S 0 (not much less than 4.12 a value we got considering selection only, or than 4.12 a value we got considering selections and crossovers), at time t + 2, = 5.33 such strings (again, not much less than 5.85 or 5.67) (Michalewicz [36, p.52]). Note that eqiuation (1.5) is based on the assumption that the fitness function f returns only positive values; when applying GAs to optimization problems where the optimization function may return negative values, some additional mapping between optimization and fitness function is required.

20 1.1. GENETIC ALGORITHMS 19 Theorem (Michalewicz [36, p.53])(schema Theorem) Short defining length, low-order, aboveaverage schemata receive exponentially increasing trials in subsequent generations of a genetic algorithm Real-Coded Genetic Algorithms Fixed-length and binary-coded strings for the representation of candidate solutions historically tend to dominate in research and applications of GAs. This trend results mainly from a number of theoretical results such as Holland s schema theorem obtained for binary representation. More recently, researchers proposed the use of non-binary representations in GAs more adequate for a certain class of optimization problems (Cordón el al.[9, p.64]). Using a real number representation seems particularly natural for optimization problems in the continuous domain. The chromosome in real-coded GAs is a vector of floating-point numbers whose size equals the dimension of the continuous search space, or in other words each gene corresponds to a particular parameters of the problem. The real-coded GAs blur the distinction between genotype and phenotype, since in many problems the real number vector already embodies a solution in a natural way (Cordón el al.[9, p.65]) Messy Genetic Algorithms The GAs described so far encode candidate solutions by chromosomes of fixed length in which the locus of a gene determines its functionality. Messy GAs relax the fixed-locus assumption and operate with chromosomes of flexible length in which genes can be arranged in any order (Goldberg [20]). In a messy GA, each gene, in addition to the bit representing its value, contains an additional integer defining its functionality. For example, the standard binary string 1001 is transformed into a string pairs (1, 1)(2, 0)(3, 0)(4, 1), where the first element serves as a label that uniquely defines the gene functionality. The second element represents the usual allelic value of the gene. Notice that the messy coding scheme no longer requires a particular order of genes. For example, the chromosome (4, 1)(3, 0)(1, 1)(2, 0) with the shuffled genes also lends itself to represent the same bit string 1001 (Cordón el al.[9, p.67]). The schema theorem states that the frequency of shcemata with above average fitness, low order and short defining length increases in the next generation. Assume that the genes labelled 1 and 4 in the above example are tightly coupled in a way that instances of the building block 1 1 have a high fitness compared to other chromosomes in th population. According to the schema theorem, the large defining length δ(1 1) = 3 limits the frequency increase of the schema since every crossover operation disrupts the schema. A chromosome in which the genes 1 and 4 are adjacent, is less prone to disruption by crossover. Messy GAs are able to form better building blocks since the position independent coding allows them to permute the order of genes. Assume the genes in the example are arranged in the order (3, 0)(2, 0)(4, 1)(1, 1). The defining length of the permuted schema 11 decrease because genes 1 and 4 become adjacent (Cordón el al.[9, p.67]). The more flexible coding scheme of messy GAs comes with the prize of incomplete or ambiguous chromosomes. Under-specification occurs whenever an incomplete chromosome lacks a particular gene. In the 4-bit example from above, the string (2, 0)(4, 1))(1, 1) is incomplete because it does not contain a gene with label 3. Over-specification constitutes a problem whenever the string contains multiple, ambiguous alleles for the same gene. The string (2, 0)(4, 1)(3, 0)(2, 1)(1, 1) embodies two genes with label 2 having ambiguous alleles 1 and 0. Messy GAs require some additional processing when mapping genotypes to phenotypes in order to handle under- and over-specification (Cordón el al.[9, p.67]). Over-specification is resolved by a first come, first serve rule, which only considers the leftmost gene of multiple occurence. In the above example, the coding scheme only express the leftmost gene (2, 0) whereas it discards the second gene (2, 1). Under-specification turns out to be a more difficult problem. Messy GAs can deal with under-specification under the assumption that a partial string