MuACOsm A New Mutation-Based Ant Colony Optimization Algorithm for Learning Finite-State Machines

Size: px

Start display at page:

Download "MuACOsm A New Mutation-Based Ant Colony Optimization Algorithm for Learning Finite-State Machines"

Coleen Freeman
10 years ago
Views:

National Research University of IT, Mechanics and Optics St.

1 MuACOsm A New Mutation-Based Ant Colony Optimization Algorithm for Learning Finite-State Machines Daniil Chivilikhin and Vladimir Ulyantsev National Research University of IT, Mechanics and Optics St. Petersburg, Russia Evolutionary and Combinatorial Optimization GECCO 2013 July 8, 2013

2 Motivation: Reliable software Systems with high cost of failure Energy industry Aircraft industry Space industry We want to have reliable software Testing is not enough Verification is needed ACO for Learning FSMs 2

industry We want to have reliable software Testing is

3 Introduction (1) Automated software engineering Model-driven development Automata-based programming Software specification Model Code ACO for Learning FSMs 3

4 Introduction (2) Finite-state machine Software specification Model Code ACO for Learning FSMs 4

5 Finite-State Machine S set of states s 0 S initial state T/z 3 Σ set of input events 1 Δ set of output actions δ: S Σ S transition function λ: S Σ Δ actions function T/z 1 A/z 3 Example: two states 2 events = {A, T} actions = {z 1, z 2, z 3, z 4 } A/z 2 ACO for Learning FSMs 5

function λ: S Σ Δ actions function T/z 1 A/z 3 Example: two states 2

6 Automata-based programming Design programs with complex behavior as automated-controlled objects Events e 1 e 2 Output actions z 1 z 2 z 2 z 3 z 4 Automated-controlled object Finite-state machine Events Actions Controlled object ACO for Learning FSMs 6

actions z 1 z 2 z 2 z 3 z 4 Automated-controlled object

7 Automata-based programming: advantages Model before programming code, not vice versa Finite-state machine Model Code Possibility of program verification using Model Checking You can check temporal properties (LTL) ACO for Learning FSMs 7

Code Possibility of program verification using Model

8 Issues Hard to build an FSM with desired structure and behavior Several problems of learning FSMs were proven to be NP-hard One of the solutions metaheuristics ACO for Learning FSMs 8

9 Learning finite-state machines with metaheuristics N states number of states Σ input events Δ output actions X = (N states, Σ, Δ) search space Test examples Software specification Modeling environment Fitness function f: X R ACO for Learning FSMs 9

states, Σ, Δ) search space Test examples Software

10 Approaches to learning FSMs Greedy heuristics problem-specific Reduction to SAT and CSP problems fast problem-specific Evolutionary algorithms (general) slow ACO for Learning FSMs 10

11 Proposed approach Based on Ant Colony Optimization (ACO) Non-standard problem reduction Modified ACO algorithm ACO for Learning FSMs 11

12 Solution representation Transition table Output table δ Event λ Event State A T State A T z 1 z z 2 z 3 ACO for Learning FSMs 12

13 Canonical way to apply ACO Reduce problem to finding a minimum cost path in some complete graph Vertices FSM transitions: <i S, j S, e Σ, a Δ> Each ant adds transitions to its FSM 1 1 [T/z 1 ] 1 2 [A/z 2 ] ACO for Learning FSMs 13

transitions: <i S, j S, e Σ, a Δ> Each ant adds

14 Canonical ACO: example 2 states 2 events 1 action ACO for Learning FSMs 14

15 Canonical ACO: issues Number of vertices in the construction graph grows as (N states ) 2 Σ Δ No meaningful way to define heuristic information Later we show that canonical ACO is ineffective for FSM learning ACO for Learning FSMs 15

meaningful way to define heuristic information Later we

16 Proposed algorithm: MuACOsm Mutation-Based ACO for learning FSMs Uses a non-standard problem reduction Modified ACO ACO for Learning FSMs 16

17 Problem reduction: MuACOsm vs. canonical Canonical ACO Nodes are solution components Full solutions are built by ants Proposed MuACOsm algorithm Nodes are full solutions (FSMs) Ants travel between full solutions ACO for Learning FSMs 17

solutions are built by ants Proposed MuACOsm algorithm

18 FSM Mutations T/z 3 Change transition action T/z 1 1 A/z 3 Change transition end state 2 A/z 2 T/z 3 T/z T/z 1 A/z 1 T/z 1 A/z A/z 2 A/z 2 ACO for Learning FSMs 18

state 2 A/z 2 T/z 3 T/z 3 1 1 T/z 1 A/z 1

19 MuACOsm problem reduction Construction graph nodes are FSMs edges are mutations of FSMs Example ACO for Learning FSMs 19

20 Real search space graph ACO for Learning FSMs 20

21 Part of real search space (1) ACO for Learning FSMs 21

22 Part of real search space (2) ACO for Learning FSMs 22

23 Heuristic information u η uv = max(η min, f(v) f(u)) v Finite-state machines ACO for Learning FSMs 23

24 ACO algorithm A 0 = random FSM Improve A 0 with (1+1)-ES Graph = {A 0 } while not stop() do ConstructAntSolutions UpdatePheromoneValues DaemonActions ACO for Learning FSMs 24

25 Constructing ant solutions Use a colony of ants An ant is placed on a graph node Each ant has a limited number of steps On each step the ant moves to the next node A A 1 A 2 A 3 A 4 ACO for Learning FSMs 25

26 Ant step: selecting the next node Mutation A 1 f(a 1 )=8 P = 1 P new A 1 P = P new A 2 f(a 2 )=12 A 2 A f(a)=10 A A 3 f(a 3 )=0 A 3 Go to best mutated FSM А 4 f(a 4 )=9 Probabilistic selection A 4 uv uv p Av uw uw ACO for Learning FSMs 26 w { A1, A2, A3, A4}

27 Pheromone update Ant path quality = max fitness value on a path τ uv best Update largest pheromone value deployed on edge (u, v) Update pheromone values: τ = ( 1 ρ) τ uv uv + τ best uv pheromone evaporation rate ρ 0,1 ACO for Learning FSMs 27

28 Differences from previous work Added heuristic information Changed start node selection for ants Coupling with (1+1)-ES More experiments (later) More comparisons with other authors Harder problem ACO for Learning FSMs 28

29 Simple problem: Artificial Ant Toroidal field N N M pieces of food s max time steps Fixed position of food and the ant Goal build an FSM, such that the ant will eat all food in K steps Field example: John Muir Trail ACO for Learning FSMs 29

30 Artificial Ant: Fitness function f n food s max s s max last 1 n food number of eaten food pieces s max max number of allotted steps s last number of used steps eaten food f used time steps ACO for Learning FSMs 30

31 Simple problem: Artificial Ant Two fields: Santa Fe Trail John Muir Trail Comparison: Canonical ACO Christensen et al. (2007) Tsarev et al. (2007) Chellapilla et al. (1999) Santa Fe Trail ACO for Learning FSMs 31

32 Canonical ACO State count Canonical ACO Success rate, % MuACOsm Success rate, % ACO for Learning FSMs 32

33 Fitness evaluation count Santa Fe Trail (Christensen et al., 600 steps) MuACOsm Christensen et al Number of FSM states ACO for Learning FSMs 33

34 Fitneess evaluation count John Muir Trail (Tsarev et al., 2007): 200 steps 10 6 MuACOsm Genetic algorithm Number of FSM states MuACOsm is 30 times faster for FSMs with 7 states ACO for Learning FSMs 34

35 Harder problem: learning Extended Finite-State Machines (1) ACO for Learning FSMs 35

36 Harder problem: learning Extended Finite-State Machines (2) Input data: Number of states C and sets Σ and Δ Set of test examples T T i =<input sequence I j, output sequence O j > NP-hard problem: build an EFSM with C states compliant with tests T ACO for Learning FSMs 36

37 Learning EFSMs: Fitness function Pass inputs to EFSM, record outputs Compare generated outputs with references Fitness = string similarity measure (edit distance) ACO for Learning FSMs 37

38 Experimental setup 1. Generate random EFSM with C states 2. Generate set of tests of total length C Learn EFSM 4. Experiment for each C repeated 100 times 5. Run until perfect fitness 6. Record mean number of fitness evaluations ACO for Learning FSMs 38

39 Learning random EFSMs ACO for Learning FSMs 39

40 Conclusion Developed new ACO-based algorithm for learning FSMs and EFSMs MuACOsm greatly outperforms GA on considered problems Generated programs can be verified with Model Checking ACO for Learning FSMs 40

41 Future work Better FSM representation to deal with isomorphism Use novelty search Employ verification in learning process Fitness Function MuACOsm Model Checking ACO for Learning FSMs 41

42 Acknowledgements We thank the ACM for the student travel grants ACO for Learning FSMs 42

43 Thank you for your attention! MuACOsm A New Mutation-Based Ant Colony Optimization Algorithm for Learning Finite-State Machines Daniil Chivililikhin Vladimir Ulyantsev [email protected] This presentation online: gecco-2013-presentation.pdf

Extended Finite-State Machine Inference with Parallel Ant Colony Based Algorithms

Extended Finite-State Machine Inference with Parallel Ant Colony Based Algorithms Daniil Chivilikhin PhD student ITMO University Vladimir Ulyantsev PhD student ITMO University Anatoly Shalyto Dr.Sci.,