Parallel Discrepancy-based Search

Transcription

1 Parallel Discrepancy-based Search T. Moisan, J. Gaudreault, C.-G. Quimper Université Laval, FORAC research consortium February 21 th 2014 T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

2 Upsides of Parallelization Parallel computing is a growing domain. Governments and industries are intensively investing in it. It is a huge opportunity for constraint programming. It can lead to unpreceded performances. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

3 Downsides of Parallelization Parallelization is hard. Speedup can stall very fast due to communication. It is difficult to reproduce in a parallel environment the search strategies that work well sequentially. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

4 Outline 1 Constraint programming 2 Depth-first Search (DFS) 3 Limited Discrepancy Search (LDS) 4 Parallel computing in constraint programming 5 Parallel Discrepancy-based Search (PDS) 6 Theoretical and statistical analysis 7 Experimentations with an industrial case T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

5 Constraint programming Uses variables with a finite domain; Uses specific constraints to modelize the problem: All-different: variables must have different values; Regular: variables must follow a defined regular expression. Each constraint includes a specialized algorithm to filter the search space; Can include an objective function. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

6 Exemple 4 0ZQZ 3 L0Z0 2 0Z0L 1 ZQZ0 x 1 x 2 x 3 x 4 4 0L0Z 3 Z0ZQ 2 QZ0Z 1 Z0L0 x 1 x 2 x 3 x 4 dom(x 1 ) = {1,2,3,4} dom(x 2 ) = {1,2,3,4} dom(x 3 ) = {1,2,3,4} dom(x 4 ) = {1,2,3,4} x i x j i < j x j x i j i i < j x j x i i j i < j T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

7 Espace de recherche Ensemble des assignations partielles des variables Cet espace est typiquement représenté par un arbre n-aire. 4 0Z0Z 3 Z0Z0 2 0Z0Z 1 L0Z0 x1 x2 x3 x4 4 0Z0Z 3 Z0Z0 2 0Z0Z 1 Z0Z0 x1 x2 x3 x4 4 0Z0Z 3 Z0Z0 2 QZ0Z 1 Z0Z0 x1 x2 x3 x4 4 0L0Z 3 Z0Z0 2 QZ0Z 1 Z0Z0 x1 x2 x3 x4 4 0Z0Z 3 ZQZ0 2 0Z0Z 1 L0Z0 x1 x2 x3 x4 4 0L0Z 3 Z0Z0 2 0Z0Z 1 L0Z0 x1 x2 x3 x4 x x x 4 0L0Z 3 Z0Z0 2 QZ0Z 1 Z0L0 x1 x2 x3 x4 x x x x 4 0L0Z 3 Z0Z0 2 0ZQZ 1 L0Z0 x1 x2 x3 x4 x x 4 0L0Z 3 Z0ZQ 2 QZ0Z 1 Z0L0 x x x x x1 x2 x3 x4 T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

9 Depth-first Search (DFS) The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

10 Depth-first Search (DFS) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

18 Limited Discrepancy Search (LDS) Harvey and Ginsberg (1995) The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one. A discrepancy is a deviation from the first choice of the value ordering heuristic LDS proposes to visit the leaves in increasing order of discrepancy. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

19 Limited Discrepancy Search (LDS) Harvey and Ginsberg (1995) The value ordering heuristic function orders the children of a node from the most likely one to lead to a solution to the least likely one. A discrepancy is a deviation from the first choice of the value ordering heuristic LDS proposes to visit the leaves in increasing order of discrepancy. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

20 LDS Iterations 0 (0 discrepancy) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

21 LDS Iterations 1 (1 discrepancy) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

22 LDS Iterations 2 (2 discrepancies) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

23 LDS Iterations 3 (3 discrepancies) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

24 LDS Overhead versus DFS Nodes in a binary tree with n variables: 2 2 n 1 Node visits with a DFS: 2 2 n 1 Node visits with a LDS: 4 2 n n 3 By the time DFS visits the entire tree, LDS will visit half of the leaves. The leaves that LDS visits are those that have fewer than n/2 discrepancies. If the value ordering heuristic is as good as a random heuristic, LDS finds a solution by the time DFS completes. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

26 Portfolio Il s agît d une course entre les processeurs pour obtenir une solution. Chaque processeur recherche une solution avec un algorithme qui lui est propre Les processeurs peuvent s aider en échangeant des informations. Il est difficile d étendre à un nombre arbitraire de processeurs. Il faut définir des paramètres de recherche pour chaque processeur. Ces paramètres sont souvent établis manuellement. La paramétrisation est fortement lié au problème à résoudre. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

27 Work-stealing Xie et Davenport Processeur 1 Processeur 2 A B X Y C D Z V E W T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

28 Work-stealing Processeur 1 Processeur 2 A B X Y C D Z V E W T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

29 Work-stealing Processeur 1 Processeur 2 Je n'ai plus de travail. A B X Y C D Z V E W T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

30 Work-stealing Processeur 1 Processeur 2 C D A B X Y E Z V W T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

31 Work-stealing Processeur 1 Processeur 2 A B X Y E Z V C W D T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

32 Work-stealing Processeur 1 Processeur 2 A B X Y E Z V C W D T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

33 Division de l espace de recherche Approche très courante Divise théoriquement la charge de travail également Peu probable que les sous-arbres soient de même taille Ne peut pas être utilisé sur un grand nombre de processeurs Problème NP-Difficile Si les sous-arbres ne sont pas de même taille Les processeurs n auront pas la même charge de travail La durée totale de la tâche sera plus longue T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

35 Our Objectives Having a search strategy that performs well sequentially, we want to parallelize it with four main goals in mind: Search strategy preservation Workload balancing Robustness No communication T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

36 Intuition behind PDS Each leaf is assigned to a processor. The leaves are implicitly assigned in a round-robin fashion. To split the task, let s say that p 1 takes odd leaves and p 2 the even leaves T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

37 Intuition behind PDS Each leaf is assigned to a processor. The leaves are implicitly assigned in a round-robin fashion. To split the task, let s say that p 1 takes odd leaves and p 2 the even leaves. p 1 p T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

38 PDS overview p 1 p 2 p 1 & p T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

41 Implementing PDS p : its processor id ρ : the number of processors n : the number of variables in the subtree k: the number of required discrepancies l : the processor id implicitly assigned to the leftmost leaf The processor will branch in the subtree if (p l) mod ρ < ( ) n k n k l (l + 1) mod (l + 2) mod... (l + n k )mod T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

42 Branching in PDS n 1 k n 1 k 1 l l + n 1 k mod T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

44 PDS overhead versus LDS Processors Node visits Overhead Speedup n n n 2n 4 25% n 3n 5 43% ρ 2 n + 2 n n i=1 i k=0 1 min(ρ, ( i 2 k) ) i T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

45 Speedup for some number of processors n =11 variables 1200 Speedup Number of processors (ρ) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

46 Speedup for some number of processors Speedup n =11 variables n =15 variables n =17 variables n =20 variables n =30 variables Number of processors (ρ) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

47 Average computation time to find a solution according to the number of variables Average computation time (number of visited nodes) PDS ρ = Nb variables T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

48 Average computation time to find a solution according to the number of variables Average computation time (number of visited nodes) PDS ρ =1 PDS ρ =2 PDS ρ =4 PDS ρ =16 PDS ρ =32 PDS ρ = Nb variables T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

49 Balancing Theorem Let n be the number of variables in the problem. If a branch is cut from the search tree, the number of leaves removed from the workload of each processor differs by at most n p 1 p 2 p 1 p 1 p 2 p 2 p 1 p 2 T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

51 Planning and scheduling wood finishing operations We minimize order lateness. The production is done with one-to-many operations. There is multiple ways to produce the same product. Output A Input X Operation 1 Output B Output C Input Y Operation 2 Output D Output E There are setup constraints that restrict sequencing of the operations. This is a problem for which LDS was efficient thanks to a specialized value ordering heuristic (Gaudreault et.al (2010)). T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

52 Instances from a Canadian Forest-Product Company We worked with industrial instances: 65,142 variables 50,238 constraints 42 discrete decision variables whose domains have cardinality 6 Used LDS/PDS to fix those variables. Once the discrete decision variables are known, the remaining continuous variables define a linear program that can be easily solved to optimality continuous decision variables Linear program solved with Cplex. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

53 Results 1e7 2.2 PDS ρ = objective value time (s) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

54 Results 1e PDS ρ =1 PDS ρ =512 PDS ρ =1024 PDS ρ =2048 PDS ρ =4096 objective value time (s) T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

55 Results T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

56 Results 60 % T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57

57 Conclusion Theoretical analysis of DFS versus LDS Parallel Discrepancy-based Search (PDS) LDS search strategy preservation No communication Intrinsic load balancing Robust to hardware failures Theoretical analysis of this parallelization. Experimental results on large industrial instances with up to 4096 processors. T. Moisan, J. Gaudreault, C.-G. Quimper Parallel Discrepancy-based Search February 21 th / 57