Ant Colony Optimization (ACO) Exploits foraging behavior of ants Path optimization Problems mapping onto foraging are ACO-like TSP, ATSP QAP
Travelling Salesman Problem (TSP) Why? Hard, shortest path problem NP hard, in fact (no polynomial time algorithm) Well studied (can compare results to those known) GA [Freisleben and Merz] Iterated Lin-Kernighan etc. Easy to explain
TSP Connect n points, visiting points only once such that total path length is minimized. Path is closed; i.e. start point and end point are same. Symmetric: d = d ji It isn t easy Asymmetric: d d ji
TSP It s easy to create nonoptimal tours
Ant System (AS) for TSP Ants build tours by moving on graph Add cities one at a time until tour is complete Transition from node i to j depends on: Whether node visited (ants have a tabu list) Inverse of distance, Pheromone deposits, Transition rule: p k ( t) η =1/ d τ [ ] α τ [ η ] = [ ] α τ [ ] il ηil l k J i β β
Transition Equation k J i is the tabu list Constants: α, β α = 0, no cooperation, greedy search β= 0 no preference for shortest edge Note: p k (t), even though pheromone concentrations same, tabu list may be different p k ( t) = [ ] α τ [ η ] α [ τ il ] [ ηil ] l k J i β β
Pheromone Laying When tour complete, pheromone updated: T k (t) tour by k th ant for iteration t Q a system parameter, value is relatively unimportant = ) ( ), ( 0 ) ( ), ( ) ( / ) ( t T j i if t T j i if t L Q t k k k k τ
Pheromone Decay System performs poorly without pheromone decay: Saturates, local optimum found Pheromone decay: τ (0) small τ ( t) (1 ρ). τ ( t) + τ ( t) τ m k = 1 ( t) = τ k ( t)
Ant Density and Initial Positioning Number of ants, m, is constant for run: Too many, reinforcement too rapid Too few, pheromone decay prevents trail formation m = n found to be reasonable Initial positioning: One per node Random
Elitest Ants Stolen from Genetic Algorithms (GAs) Elitist ant is one which reinforces edges belonging to best tour, T + with length L + Each iteration, e elitist ants are added to other ants so that edges in T + get extra e.q/l + Improves algorithm slightly
Complexity and Results Time complexity: O(t.n 2.m) = O(t.n 3 ), n=m T = # iterations, n = # cities, m = # ants Initial results: Disappointing, did not match Lin-Kernighan Better for small TSP (Oliver 30 city) Rapid convergence, but only fair for 70 cities
Swarm Intelligence: Bonabeau et al Alg 2-1 Edge (top) initialization Ant initialization Next Edge Choice Tour Length Calculation Best tour update
Swarm Intelligence: Bonabeau et al Alg 2-1 (bottom) Ant pheromone updates k th Ant updates Elitest Ant updates Save new edge Pheromone values
Performance Results best Oliver30 Best Tour Average Std Dev. AS-TSP 420 420.4 1.3 TS 420 420.6 1.5 SA 422 459.8 25.1
Rapid Convergence for best tour: 1. Tends Figure to avoid 2.12 getting trapped in local optima 2. Maintains diversity in solutions as branching factor > 2 Swarm Intelligence: Bonabeau et al
Improvements Another transition rule: J J j arg max = u J k i, where p k ij ( t ) = k i {[ ( t) ]. [ ] } β τ η J iu l if q > iu q [ τ ( t )] [ η ] ij k J i 0 if q [ τ ( t )] [ η ] il ij β il β q 0 q uniformly distributed [0,1], q 0 is tunable
Observations If q > q 0, same transition as before If q q 0, make greedy choice Uses heuristic knowledge about problem: distances and pheromone trail q 0, ~1 choose locally optimal solutions q 0, ~0 all locally optimal solutions evaluated Apply Simulated Annealing to q 0
Swarm Intelligence: Bonabeau et al Figure 2-13 Maintains diversity Standard deviation remains high
Little initial pheromone, choose all edges with same probability Figure 2-14 Considerable Diversity Remains during run Swarm Intelligence: Bonabeau et al
Improved Pheromone Updating All ants deposit pheromone on tour completion ACS, best ant since beginning of trial updates network Encourages ants to search within vicinity of best tour found Exploration is more directed, more focussed Pheromone updates applied ONLY to edges on best path, (i,j) edges belonging to T +
Pheromone Updates Two forms: During creation of tour Upon tour completion by all ants
Pheromone Trail Updates τ ( 1 ρ ). τ ( ) + ρ. τ ( ) ( t) t t τ ( t) = 1/ L +
For Each Ant Transition: Local Pheromone Updates ( 1 ). ( t). 0 τ ( t) ρ τ + ρ τ τ 0 = initial value on pheromone trail Set up to be 1/(n.L nn ), L nn = length of tour found by nearest neighbor (NN) heuristic Run NN demo When ant visits an edge, the pheromone concentration decreases, making them less attractive forcing exploration of not yet visited edges. Effect of local updating is to shuffle tours. Edge desirability changes dynamically as ants visit them.
Candidate Lists Maintain list of preferred cities to be visited from a given city (size cl), ordered by distance Unvisited cities examined first, next city chosen according to: j J J arg max = u J k i, where p k ij ( t ) = k i {[ ( t) ]. [ ] } β τ η J iu l if q > iu q [ τ ( t )] [ η ] ij k J i 0 if q [ τ ( t )] [ η ] il ij β il β q 0
ACS-TSP Swarm Intelligence: Bonabeau et al Edge initialization Ant initialization Compute tour length Next Edge Choice Local Update Tour Length Calculation
Swarm Intelligence: Bonabeau et al Alg 2-2a Best tour update Ant pheromone Updates for best tour Save new edge Pheromone values
Results (25 runs) best ACS- TSP SA EN SOM 50 City (I) 5.88 5.88 5.89 6.06 50 City (II) 6.05 6.01 6.03 6.25 50 City (III) 5.58 5.65 5.70 5.83 50 City (IV) 5.74 5.81 5.86 5.87 50 City (V) 6.18 6.33 6.49 6.70
Results N/A N/A (N/A) N/A N/A (N/A) 103000 21761 (N/A) 4820 21282 (21285.44) KroA100 (100 cities) 173250 580 (N/A) 325000 542 (549.18) 80000 545 (N/A) 3480 535 (542.37) Eil75 (75 cities) 68512 443 (N/A) 100000 426 (427.86) 25000 428 (N/A) 1830 425 (427.96) Eil50 (50 cities) SA #iter SA best EP #iter EP best GA #iter GA best ACS- TSP #iter ACS-TSP best best
Adding Local Heuristics To scale, add local optimization: Works in combination with ACS For TSP, use 2-opt, 3-opt and Lin-Kernighan Eliminate 2 edges and reconnect two resulting paths to generate a different tour Run 2-opt demo
ACS-3-opt Same as ACS, except: Each ant tour is minimized upon tour creation For k = 1 to m do T k (t) 3-opt(T k (t)) {apply local opt to each tour} End For
ACS-3-opt Results (stsp) best ACS-3-opt ACS-3-opt STSP (GA) STSP (GA) Best Average Best Average d198 15780 15781.7 15780 15780 (198 cities) lin318 42029 42029 42029 42029 (318 cities) att532 27693 27718.2 27686 27693.7 (532 cities) rat783 8818 8837.9 8806 8807.3 (783 cities)
ACS-3-opt Results (atsp) best ACS-3-opt ACS-3-opt ATSP ATSP Best Average Best Average p43 2810 2810 2810 2810 (43 cities) ry48p 14422 14422 14422 14440 (48 cities) ft70 38673 38679.8 38673 38683.8 (70 cities) kro124p 36230 36230 36230 36235.3 (124 cities) ftv170 2755 2755 2755 2766.1 (170 cities)
Other Potential Improvements All r best ants to update trail, instead of single ant Reduce probability of being trapped in local opt. Allow β pheromone sensitivity -- to vary Remove pheromone from edges belonging to worst tours; i.e. those below average Increase convergence speed Use improved local search Several potential projects
Observations Pheromone concentrations vary on edges Some strongly marked, some weakly Strong on tour (likely) Weak point to alternative solutions Diversity of solutions useful in a dynamic environment We don t start from scratch Swarm may be Superior in a Dynamic environment
Other methods Min-Max AS [Stützle and Hoos] Iteration s best ant updates pheromone trail Pheromone concentrations limited to [τ min,τ max ] Trails initialized to τ max Bounding concentrations prevents stagnation Also, ( τ τ ( )) τ ( t) max t Strong trails reinforced less than weak Results better than AS, comparable with ACS
Elitest Mechanisms Bullnheimer et al Sort m ants according to tour length, L i (t) Ants update edges based upon their rank Used σ elitest ants: best σ 1 deposit trail Weighted according to max {0, σ µ} + τ ( t ) ( 1 ρ ). τ ( t ) + σ. τ ( t ) + τ ( t ) where τ τ τ + r µ ( t ) ( t ) ( t ) = = = Q / σ 1 µ = 1 L + τ ( t ) µ ( t ), µ ( σ µ ). Q / L ( t ), if ant µ uses (i, j), 0 otherwise