Méta-heuristiques pour l optimisation

Transcription

1 Méta-heuristiques pour l optimisation Differential Evolution (DE) Particle Swarm Optimization (PSO) Alain Dutech Equipe MAIA - LORIA - INRIA Nancy, France Web : Mail : Alain.Dutech@loria.fr 3 mai 2010

2 Context : Neuro-Sciences Question : influence/role of pre-frontal-cortex in reinforcement-like learning in rats? (Context of Operant Conditionning, work with Alain Marchand, Bordeaux). Real Data PFC-lesion/normal SARSA State : w/o memory Policy Fitting Optimize Softmax-Policy Likelyhood Likelyhood Page 2

3 Global Optimization Search space S x = (α, γ, β) T Objective function to minimize f is unknown f : S R f : policy likelyhood (vector? traj?) sampled. Global Optimization non-convex objective function. unconstrained state space. Meta-heuristics : Simulated Annealing, Tabu Search, Ant Colony Optimization, Evolutionary Algorithm (Differential Evolution), Particle Swarm Optimization, Variable Neighborhood Search, etc. Ceci n est pas exhaustif! Page 3

4 Differential Evolution (DE) [Storn and Price, 1995] D-dimensionnal parameter vector x = (x 0,, x D ) T (1) Population of N parameter vectors at generation g Evolution of one parameter vector 1. mutation : 2. cross-over : X [g] = { x i [g]}, i = 0,, N (2) x i [g + 1] = m i = f (X [g]) (3) { m j i for some j x j i [g] otherwise (4) Page 4

5 DE canonical : uninformed evolution Mutation : r 1, r 2 and r 3 randomly different in [0,, N] Cross-Over : n and L randomly chosen x i [g + 1] = v i = x r1 [g] + F.( x r2 [g] x r3 [g]) (5) { v j i for j = n D,, n + L 1 D x j i [g] otherwise (6) Page 5

6 DE canonical : uninformed mutation x 2 min x i [g] x r3 [g] x r2 [g] x r1 [g] ( x r2 [g] x r3 [g]) x r1 [g] + F.( x r2 [g] x r3 [g]) v i x 1 v i = x r1 [g] + F.( x r2 [g] x r3 [g]) Page 6

7 DE canonical : informed mutation x 2 ( x best [g] x i [g]) x i [g] x r3 [g] x r2 [g] x best [g] v i x r1 [g] min x i [g] + λ.( x best [g] x i [g]) +F.( x r2 [g] x r3 [g]) ( x r2 [g] x r3 [g]) x 1 v i = x i [g] + λ.( x best [g] x i [g]) + F.( x r2 [g] x r3 [g]) Page 7

8 DE canonical : cross-over x i [g] v i = f (X [g]) x i [g + 1] j = j = j = (n = 2, L = 3) Page 8

9 Particle Swarm Optimisation (PSO) [Kennedy and Eberhart, 1995] D-dimensionnal parameter vector and velocity x = (x 0,, x D ) T (7) v = (v 0,, v D ) T (8) Population of N parameter vectors and velocity at generation g Initialisation Evolution of one parameter vector 1. velocity : 2. position : (9) X [g] = {( x i [g], v i [g])}, i = 0,, N (10) v i [g + 1] = f (X [g]) (11) x i [g + 1] = x i [g] + v i [g + 1] (12) Page 9

10 PSO canonical Velocity : ϕ 1 and ϕ 2 are accelaration constants R 1 and R 2 are uniformly chosen in [0, 1] x ibest best past position of i x i,nbest best position in neighbors of i (past??) v i [g + 1] = v i [g] + ϕ 1.R 1.( x ibest [g] x i [g]) + ϕ 2.R 2.( x i,nbest [g] x i [g]) (13) Position : x i [g + 1] = x i [g] + v i [g + 1] (14) Page 10

11 Constraints Enforce constraints on particles Projection into constrained space Periodic Search Space [Zhang et al., 2004] Page 11

12 Neighbors What is x i,nbest [g]? x k [g] ɛ x i [g] x j [g] Sphere, radius ɛ Page 12

13 Neighbors What is x i,nbest [g]? x i [g] Small world Page 13

14 Neighbors What is x i,nbest [g]? x i [g] Small world with random added links [Saxena and Vora, 2008] Page 14

15 DE+PSO Hybrid algorithm [Zhang et al., 2009] Initialisation Population of N particles. Only one particle in ɛ-radius sphere. Update best & neighbors nb iter multiple of I? Y DE N PSO Special x i == x i,nbest N nb iter nb max? Y End Page 15

16 DE in hybrid algorithm Mutation : j randomly different in [0,, N] x ibest best past position of i x i,nbest best position in neighbors of i (past??) v i = x i,nbest [g] + F.( x ibest [g] x jbest [g]) (15) Cross-Over : D i randomly chosen[0,, N], C R in [0, 1] x i [g + 1] = { v j i if j = D i or with proba C r x j i [g] otherwise (16) Page 16

17 PSO in hybrid algorithm Canonical version of PSO (small world + 2 rnd links). Prevent beeing stuck with null-velocity when ( x i == x i,nbest ) x i [g + 1] = x i,nbest [g] + δ. rand (17) v i [g + 1] = x i [g + 1] x i [g] (18) Page 17

18 B2 problem One global minimum of 0 at (0, 0) B2(x 1, x 2) = x x cos(3πx1) 0.4 cos(4πx2) Page < x j < 100,j = 1, 2

19 ES problem One global minimum of 1 at (π, π) (π, π) Es(x 1, x 2) = cos(x 1) cos(x 2). exp( ((x 1 x 2) 2 + (x 2 π) 2 )) Page 19

20 GP problem One global minimum of 3 at (0, 1) GP(x 1, x 2) = [1 + (x 1 + x 2 + 1) 2.(19 14x 1 + 3x x2 + 6x1x2 + 3x 2 2 )] 2 < x j < 2, j = 1, 2 [30 + (2x 1 3x 2) 2.(18 32x x x2 36x1x2 + 27x 2 2 )] Page 20

21 SH problem 760 local minima, 18 global minima of SH(x 1, x 2) = j cos[(j + 1)x 1 + j] j cos[(j + 1)x 2 + j] j=1 j=1 10 < x j < 10, j = 1, 2 Page 21

22 Références I Kennedy, J. and Eberhart, R. (1995). Particle swarm optimization. In Proc. of the IEEE Int. Conf. on Neural Networks (ICNN 95), volume 4, pages Saxena, A. and Vora, M. (2008). Novel approach for the use of small world theory in particle swarm optimization. In 16th Int. Conf. on Advanced Computing and Communications (ADCOM 2008), pages Storn, R. and Price, K. (1995). Differential evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces. Technical Report TR , International Computer Science Institute. Page 22

23 Références II Zhang, C., Ning, J., Lu, S., Ouyang, D., and Ding, T. (2009). A novel hybrid differential evolution and particle swarm optimization algorithm for unconstrained optimization. Operations Research Letters, 37 : Zhang, W.-J., Xie, X.-F., and Bi, D.-C. (2004). Handling boundary constraints for numerical optimization by particle swarm flying in periodic search space. In Congress on Evolutionary Computation (CEC), pages , Prtland, OR. Page 23