GPU-BASED TUNING OF QUANTUM-INSPIRED GENETIC ALGORITHM FOR A COMBINATORIAL OPTIMIZATION PROBLEM

Transcription

1 GPU-BASED TUNING OF QUANTUM-INSPIRED GENETIC ALGORITHM FOR A COMBINATORIAL OPTIMIZATION PROBLEM Robert Nowotniak, Jacek Kucharski Computer Engineering Department The Faculty of Electrical, Electronic, Computer and Control Engineering Technical University of Lodz XIV INTERNATIONAL CONFERENCE SYSTEM MODELLING and CONTROL June 27-29, 2011 Łódź Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

2 PRESENTATION OUTLINE 1 QUANTUM-INSPIRED GENETIC ALGORITHMS 2 NVIDIA CUDA TM TECHNOLOGY 3 TUNING EXPERIMENTAL RESULTS 4 SUMMARY Robert Nowotniak, Jacek Kucharski System Modelling and Control, 2011

4 QUANTUM-INSPIRED GENETIC ALGORITHMS Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

5 QUANTUM-INSPIRED GENETIC ALGORITHMS Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

6 QUANTUM ELEMENTS IN EVOLUTIONARY ALGORITHMS 1 Representation of solutions Instead of exact points in a search space, probability distributions of sampling the space 2 Initialization 3 Genetic operators 4 Evaluation Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

7 QUANTUM ELEMENTS IN EVOLUTIONARY ALGORITHMS 1 Representation of solutions (bits qubits) Instead of exact points in a search space, probability distributions of sampling the space 2 Initialization 3 Genetic operators 4 Evaluation Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

8 QUANTUM ELEMENTS IN EVOLUTIONARY ALGORITHMS 1 Representation of solutions (bits qubits) Instead of exact points in a search space, probability distributions of sampling the space 2 Initialization 3 Genetic operators 4 Evaluation Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

9 CLASSICAL BITS VS QUBITS Geometrical representation of Qubit on the Bloch sphere Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

12 QUBITS AND BINARY QUANTUM GENES qubit (quantum bit): ψ = α 0 + β 1 where: α, β C, α 2 + β 2 = 1 Pr ψ : F {0,1} [0, 1] Pr ψ ({0}) = α 2 Pr ψ ({1}) = β 2 1 ψ = }{{} 2 }{{} 2 α β β α ψ 0 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

13 QUBITS AND BINARY QUANTUM GENES qubit (quantum bit): ψ = α 0 + β 1 where: α, β C, α 2 + β 2 = 1 Pr ψ : F {0,1} [0, 1] Pr ψ ({0}) = α 2 Pr ψ ({1}) = β 2 1 ψ = }{{} 2 }{{} 2 α β β α ψ 0 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

14 QUBITS AND BINARY QUANTUM GENES qubit (quantum bit): ψ = α 0 + β 1 where: α, β C, α 2 + β 2 = 1 Pr ψ : F {0,1} [0, 1] Pr ψ ({0}) = α 2 Pr ψ ({1}) = β 2 ψ = }{{} 3 } {{ 3 } α β β 1 α ψ 0 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

15 QUBITS AND BINARY QUANTUM GENES qubit (quantum bit): ψ = α 0 + β 1 where: α, β C, α 2 + β 2 = 1 Pr ψ : F {0,1} [0, 1] Pr ψ ({0}) = α 2 Pr ψ ({1}) = β 2 ψ = }{{} }{{} 1 1 α β β α 1 ψ 0 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

16 SIMPLE GENETIC ALGORITHM In Simple Genetic Algorithm, solutions to technical problems are encoded as binary strings, for example: population of solutions binary gene chromosome Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

23 QUANTUM-INSPIRED GENETIC ALGORITHMS In Quantum-Inspired Genetic Algorithms, each individual encodes probability distribution of sampling the search space quantum population quantum gene quantum chromosome Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

28 NVIDIA R TESLA TM S Tesla TM s consists of: 4 CUDA GPU cards, each: 30 streaming multiprocessors (SMs) 8 cores each (separate ALUs) 16 KB of shared memory highly effective (zero-overhead) tasks scheduler 4 GB global memory Total: = 960 processor cores Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

29 NVIDIA R TESLA TM S Tesla TM s consists of: 4 CUDA GPU cards, each: 30 streaming multiprocessors (SMs) 8 cores each (separate ALUs) 16 KB of shared memory highly effective (zero-overhead) tasks scheduler 4 GB global memory Total: = 960 processor cores Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

30 THREAD HIERARCHY ON CUDA TM GPU In CUDA, threads are grouped in blocks and blocks constitute a grid. The unit of thread scheduling is warp (32 threads). Grid of Thread Blocks Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

31 PROPOSED APPROACH TO PARALLELIZATION Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

32 GPU-BASED IMPLEMENTATION OF QIGA Two levels: 1 Coarse-grained parallelization In a grid, there can be several hundred blocks evolving independent populations with same or different parameters simultaneously. 2 Fine-grained parallelization On the population level, each individual can be evaluated and transformed in a separate GPU thread. Thus, the whole population can be represented as a block of threads. Hundreds of populations with same or different parameters can be evolved in parallel, simultaneously Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

33 GPU-BASED IMPLEMENTATION OF QIGA Two levels: 1 Coarse-grained parallelization In a grid, there can be several hundred blocks evolving independent populations with same or different parameters simultaneously. 2 Fine-grained parallelization On the population level, each individual can be evaluated and transformed in a separate GPU thread. Thus, the whole population can be represented as a block of threads. Hundreds of populations with same or different parameters can be evolved in parallel, simultaneously Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

36 NUMERICAL EXPERIMENT TEST: COMBINATORIAL OPTIMIZATION Knapsack problem (hard version, strongly correlated set of items) Number of items: 250 Comparison: 1 Simple Genetic Algorithm (SGA) popsize = 100, P c = 0.65, P m = Quantum-Inspired Genetic Algorithm (QIGA) popsize = 10, other parameters (rotation angles) as in [ 1 ] 3 Tuned Quantum-Inpsired Genetic Algorithm 1 Han, K. H., Kim, J. H.: Genetic quantum algorithm and its application to combinatorial optimization problem. Proceedings of the 2000 Congress on Evolutionary computation, 2000 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

37 NUMERICAL EXPERIMENT TEST: COMBINATORIAL OPTIMIZATION Knapsack problem (hard version, strongly correlated set of items) Number of items: 250 Comparison: 1 Simple Genetic Algorithm (SGA) popsize = 100, P c = 0.65, P m = Quantum-Inspired Genetic Algorithm (QIGA) popsize = 10, other parameters (rotation angles) as in [ 1 ] 3 Tuned Quantum-Inpsired Genetic Algorithm 1 Han, K. H., Kim, J. H.: Genetic quantum algorithm and its application to combinatorial optimization problem. Proceedings of the 2000 Congress on Evolutionary computation, 2000 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

38 QIGA EXECUTION TIME COMPARISON Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

39 QIGA EXECUTION TIME COMPARISON Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

40 SPEEDUP ON CUDA TM 1 Pentium-III 500MHz (Visual C++ 6.0) experiments / second (according to [ 1 ]) 2 Intel Core i7 2.93GHz (1 core, ANSI C) 7.33 experiments / second 3 NVidia GTX 295 (CUDA C) 890 experiments / second (about 120x speedup) 4 8 GPUs (GTX295+GTX285+Tesla s1070+tesla C2070) 3089 experiments / second (over 400x speedup) The speedup gained allows efficient meta-optimization (parameters tuning) of the algorithms 1 Han, K. H., Kim, J. H.: Genetic quantum algorithm and its application to combinatorial optimization problem. Proceedings of the 2000 Congress on Evolutionary computation, 2000 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

41 SPEEDUP ON CUDA TM 1 Pentium-III 500MHz (Visual C++ 6.0) experiments / second (according to [ 1 ]) 2 Intel Core i7 2.93GHz (1 core, ANSI C) 7.33 experiments / second 3 NVidia GTX 295 (CUDA C) 890 experiments / second (about 120x speedup) 4 8 GPUs (GTX295+GTX285+Tesla s1070+tesla C2070) 3089 experiments / second (over 400x speedup) The speedup gained allows efficient meta-optimization (parameters tuning) of the algorithms 1 Han, K. H., Kim, J. H.: Genetic quantum algorithm and its application to combinatorial optimization problem. Proceedings of the 2000 Congress on Evolutionary computation, 2000 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

42 META-OPTIMIZATION (PARAMETERS TUNING) Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

43 RESULTS OF META-OPTIMIZATION Meta-fitness of the algorithm: knapsack profit at the end of evolution Subject to meta-optimization: rotation angles in quantum genes state space rotation angles θ meta-fitness [ 1 ] 1 Han, K. H., Kim, J. H.: Genetic quantum algorithm and its application to combinatorial optimization problem. Proceedings of the 2000 Congress on Evolutionary computation, 2000 Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

44 PERFORMANCE COMPARISON Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

47 SUMMARY In our research: 1 Quantum-Inspired Genetic Algorithm has been implemented in NVidia CUDA TM technology 2 Over 400x speedup has been gained on 8 GPU devices 3 The speedup allows efficient meta-optimization of selected parameters (rotation angles in quantum genes state space) 4 Real-Coded Evolutionary Algorithm has been used as an overlaid meta-optimizer 5 Tuned QIGA algorithm performs much better in the considered combinatorial optimization problem Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

48 MY SELECTED RECENT PAPERS 1 R.Nowotniak, J. Kucharski, Meta-optimization of Quantum-Inspired Evolutionary Algorithm, 2010, Proceedings of the XVII International Conference on Information Technology Systems, ISBN R.Nowotniak, J. Kucharski, Building Blocks Propagation in Quantum-Inspired Genetic Algorithm, 2010, Scientific Bulletin of Academy of Science and Technology, Automatics, 2010, ISSN R. Nowotniak, Survey of Quantum-Inspired Evolutionary Algorithms, 2010, Proceedings of the FIMB PhD students conference, ISSN S.Jeżewski, M. Łaski, R. Nowotniak, Comparison of Algorithms for Simultaneous Localization and Mapping Problem for Mobile Robot, 2010, Scientific Bulletin of Academy of Science and Technology, Automatics, ISSN Ł. Jopek, R. Nowotniak, M. Postolski, L. Babout, M. Janaszewski, Application of Quantum Genetic Algorithms in Feature Selection Problem, 2009, Scientific Bulletin of Academy of Science and Technology, Automatics, ISSN Robert Nowotniak, Jacek Kucharski System Modelling and Control, / 19

49 Thank you for your attention Robert Nowotniak, Jacek Kucharski System Modelling and Control, 2011

50 SELECTED APPLICATIONS 1 Simultaneous Localization and Mapping (SLAM) problem for mobile robots[ 2 ] 2 Segmentation of titanium alloys images obtained with X-Ray microtomography[ 3 ] 2 Jeżewski, S., Łaski, M., Nowotniak, R.: Comparison of Algorithms for Simultaneous Localization and Mapping Problem for Mobile Robot, Scientific Bulletin of Academy of Science and Technology,. Automatics, Jopek, Ł., Nowotniak, R., Postolski, M., Babout, L.., Janaszewski, M.: Application of Quantum Genetic Algorithms to Feature Selection Problem, Scientific Bulletin of Academy of Science and Technology, Automatics, 2010 Robert Nowotniak, Jacek Kucharski System Modelling and Control, 2011