Ant Colony Optimization (ACO)

Size: px

Start display at page:

Download "Ant Colony Optimization (ACO)"

Sydney Taylor
10 years ago
Views:

1 Ant Colony Optimization (ACO) Exploits foraging behavior of ants Path optimization Problems mapping onto foraging are ACO-like TSP, ATSP QAP

2 Travelling Salesman Problem (TSP) Why? Hard, shortest path problem NP hard, in fact (no polynomial time algorithm) Well studied (can compare results to those known) GA [Freisleben and Merz] Iterated Lin-Kernighan etc. Easy to explain

3 TSP Connect n points, visiting points only once such that total path length is minimized. Path is closed; i.e. start point and end point are same. Symmetric: d = d ji It isn t easy Asymmetric: d d ji

4 TSP It s easy to create nonoptimal tours

5 Ant System (AS) for TSP Ants build tours by moving on graph Add cities one at a time until tour is complete Transition from node i to j depends on: Whether node visited (ants have a tabu list) Inverse of distance, Pheromone deposits, Transition rule: p k ( t) η =1/ d τ [ ] α τ [ η ] = [ ] α τ [ ] il ηil l k J i β β

node visited (ants have a tabu list) Inverse of distance, Pheromone deposits,

6 Transition Equation k J i is the tabu list Constants: α, β α = 0, no cooperation, greedy search β= 0 no preference for shortest edge Note: p k (t), even though pheromone concentrations same, tabu list may be different p k ( t) = [ ] α τ [ η ] α [ τ il ] [ ηil ] l k J i β β

Note: p k (t), even though pheromone concentrations same, tabu list

7 Pheromone Laying When tour complete, pheromone updated: T k (t) tour by k th ant for iteration t Q a system parameter, value is relatively unimportant = ) ( ), ( 0 ) ( ), ( ) ( / ) ( t T j i if t T j i if t L Q t k k k k τ

parameter, value is relatively unimportant = ) ( ), ( 0

8 Pheromone Decay System performs poorly without pheromone decay: Saturates, local optimum found Pheromone decay: τ (0) small τ ( t) (1 ρ). τ ( t) + τ ( t) τ m k = 1 ( t) = τ k ( t)

9 Ant Density and Initial Positioning Number of ants, m, is constant for run: Too many, reinforcement too rapid Too few, pheromone decay prevents trail formation m = n found to be reasonable Initial positioning: One per node Random

few, pheromone decay prevents trail formation m = n found

10 Elitest Ants Stolen from Genetic Algorithms (GAs) Elitist ant is one which reinforces edges belonging to best tour, T + with length L + Each iteration, e elitist ants are added to other ants so that edges in T + get extra e.q/l + Improves algorithm slightly

length L + Each iteration, e elitist ants are added to other

11 Complexity and Results Time complexity: O(t.n 2.m) = O(t.n 3 ), n=m T = # iterations, n = # cities, m = # ants Initial results: Disappointing, did not match Lin-Kernighan Better for small TSP (Oliver 30 city) Rapid convergence, but only fair for 70 cities

results: Disappointing, did not match Lin-Kernighan Better for

12 Swarm Intelligence: Bonabeau et al Alg 2-1 Edge (top) initialization Ant initialization Next Edge Choice Tour Length Calculation Best tour update

13 Swarm Intelligence: Bonabeau et al Alg 2-1 (bottom) Ant pheromone updates k th Ant updates Elitest Ant updates Save new edge Pheromone values

14 Performance Results best Oliver30 Best Tour Average Std Dev. AS-TSP TS SA

15 Rapid Convergence for best tour: 1. Tends Figure to avoid 2.12 getting trapped in local optima 2. Maintains diversity in solutions as branching factor > 2 Swarm Intelligence: Bonabeau et al

16 Improvements Another transition rule: J J j arg max = u J k i, where p k ij ( t ) = k i {[ ( t) ]. [ ] } β τ η J iu l if q > iu q [ τ ( t )] [ η ] ij k J i 0 if q [ τ ( t )] [ η ] il ij β il β q 0 q uniformly distributed [0,1], q 0 is tunable

17 Observations If q > q 0, same transition as before If q q 0, make greedy choice Uses heuristic knowledge about problem: distances and pheromone trail q 0, ~1 choose locally optimal solutions q 0, ~0 all locally optimal solutions evaluated Apply Simulated Annealing to q 0

pheromone trail q 0, ~1 choose locally optimal solutions q 0, ~0

18 Swarm Intelligence: Bonabeau et al Figure 2-13 Maintains diversity Standard deviation remains high

19 Little initial pheromone, choose all edges with same probability Figure 2-14 Considerable Diversity Remains during run Swarm Intelligence: Bonabeau et al

20 Improved Pheromone Updating All ants deposit pheromone on tour completion ACS, best ant since beginning of trial updates network Encourages ants to search within vicinity of best tour found Exploration is more directed, more focussed Pheromone updates applied ONLY to edges on best path, (i,j) edges belonging to T +

within vicinity of best tour found Exploration is more directed, more focussed

21 Pheromone Updates Two forms: During creation of tour Upon tour completion by all ants

22 Pheromone Trail Updates τ ( 1 ρ ). τ ( ) + ρ. τ ( ) ( t) t t τ ( t) = 1/ L +

23 For Each Ant Transition: Local Pheromone Updates ( 1 ). ( t). 0 τ ( t) ρ τ + ρ τ τ 0 = initial value on pheromone trail Set up to be 1/(n.L nn ), L nn = length of tour found by nearest neighbor (NN) heuristic Run NN demo When ant visits an edge, the pheromone concentration decreases, making them less attractive forcing exploration of not yet visited edges. Effect of local updating is to shuffle tours. Edge desirability changes dynamically as ants visit them.

24 Candidate Lists Maintain list of preferred cities to be visited from a given city (size cl), ordered by distance Unvisited cities examined first, next city chosen according to: j J J arg max = u J k i, where p k ij ( t ) = k i {[ ( t) ]. [ ] } β τ η J iu l if q > iu q [ τ ( t )] [ η ] ij k J i 0 if q [ τ ( t )] [ η ] il ij β il β q 0

25 ACS-TSP Swarm Intelligence: Bonabeau et al Edge initialization Ant initialization Compute tour length Next Edge Choice Local Update Tour Length Calculation

26 Swarm Intelligence: Bonabeau et al Alg 2-2a Best tour update Ant pheromone Updates for best tour Save new edge Pheromone values

27 Results (25 runs) best ACS- TSP SA EN SOM 50 City (I) City (II) City (III) City (IV) City (V)

28 Results N/A N/A (N/A) N/A N/A (N/A) (N/A) ( ) KroA100 (100 cities) (N/A) (549.18) (N/A) (542.37) Eil75 (75 cities) (N/A) (427.86) (N/A) (427.96) Eil50 (50 cities) SA #iter SA best EP #iter EP best GA #iter GA best ACS- TSP #iter ACS-TSP best best

29 Adding Local Heuristics To scale, add local optimization: Works in combination with ACS For TSP, use 2-opt, 3-opt and Lin-Kernighan Eliminate 2 edges and reconnect two resulting paths to generate a different tour Run 2-opt demo

30 ACS-3-opt Same as ACS, except: Each ant tour is minimized upon tour creation For k = 1 to m do T k (t) 3-opt(T k (t)) {apply local opt to each tour} End For

31 ACS-3-opt Results (stsp) best ACS-3-opt ACS-3-opt STSP (GA) STSP (GA) Best Average Best Average d (198 cities) lin (318 cities) att (532 cities) rat (783 cities)

32 ACS-3-opt Results (atsp) best ACS-3-opt ACS-3-opt ATSP ATSP Best Average Best Average p (43 cities) ry48p (48 cities) ft (70 cities) kro124p (124 cities) ftv (170 cities)

33 Other Potential Improvements All r best ants to update trail, instead of single ant Reduce probability of being trapped in local opt. Allow β pheromone sensitivity -- to vary Remove pheromone from edges belonging to worst tours; i.e. those below average Increase convergence speed Use improved local search Several potential projects

34 Observations Pheromone concentrations vary on edges Some strongly marked, some weakly Strong on tour (likely) Weak point to alternative solutions Diversity of solutions useful in a dynamic environment We don t start from scratch Swarm may be Superior in a Dynamic environment

35 Other methods Min-Max AS [Stützle and Hoos] Iteration s best ant updates pheromone trail Pheromone concentrations limited to [τ min,τ max ] Trails initialized to τ max Bounding concentrations prevents stagnation Also, ( τ τ ( )) τ ( t) max t Strong trails reinforced less than weak Results better than AS, comparable with ACS

36 Elitest Mechanisms Bullnheimer et al Sort m ants according to tour length, L i (t) Ants update edges based upon their rank Used σ elitest ants: best σ 1 deposit trail Weighted according to max {0, σ µ} + τ ( t ) ( 1 ρ ). τ ( t ) + σ. τ ( t ) + τ ( t ) where τ τ τ + r µ ( t ) ( t ) ( t ) = = = Q / σ 1 µ = 1 L + τ ( t ) µ ( t ), µ ( σ µ ). Q / L ( t ), if ant µ uses (i, j), 0 otherwise

Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem

Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem TR/IRIDIA/1996-5 Université Libre de Bruxelles Belgium Marco Dorigo IRIDIA, Université Libre de Bruxelles, CP 194/6,