Discovering Counterexamples (and Proof Ingredients) for Knuth-like k-principles

Transcription

1 Discovering Counterexamples (and Proof Ingredients) for Knuth-like k-principles Moritz Fürneisen and Janis Voigtländer University of Bonn October 9th, 2012

2 Knuth s 0-1-Principle [Knuth 1973] Informally: If a comparison-swap algorithm sorts Booleans correctly, it sorts integers correctly as well. Formally: Let (in Haskell, say): sort :: ((α, α) (α, α)) [α] [α] f :: (Int, Int) (Int, Int) f (x, y) = if x > y then (y, x) else (x, y) g :: (Bool, Bool) (Bool, Bool) g (x, y) = (x && y, x y) If for every xs :: [Bool], sort g xs gives the correct result, then for every xs :: [Int], sort f xs gives the correct result. 1

3 Parallel Prefix Computation Given: inputs x 1,..., x n and an associative operation Task: compute the values x 1, x 1 x 2,..., x 1 x 2 x n Solution: x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 2

4 Parallel Prefix Computation Alternative: x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 Or: x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 Or:... 3

5 Expressing Parallel Prefix Algorithms in Haskell Functions of type: scanl1 :: (α α α) [α] [α] For example, à la [Sklansky 1960]: sklansky :: (α α α) [α] [α] sklansky () [x] = [x] sklansky () xs = us ++ vs where t = ((length xs) + 1) div 2 (ys, zs) = splitat t xs us = sklansky () ys vs = [(last us) v v sklansky () zs] 4

6 Investigating Particular Instances Only Knuth s 0-1-Principle If a comparison-swap algorithm sorts correctly on the Booleans, it does so on arbitrary totally ordered value sets. A Knuth-like 0-1-Principle? If a parallel prefix algorithm is correct (for associative operations) on the Booleans, it is so on arbitrary value sets. Unfortunately not! 8

7 A Knuth-like Principle [V. 2008] Given: scanl1 :: (α α α) [α] [α] scanl1 () (x : xs) = go x xs where go x [ ] = [x] go x (y : ys) = x : (go (x y) ys) candidate :: (α α α) [α] [α] data Three = Zero One Two Theorem: If for every xs :: [Three] and associative () :: Three Three Three, candidate () xs = scanl1 () xs, then the same holds for every type τ, xs :: [τ], and associative () :: τ τ τ. 9

8 Why 0-1-2? And How? To get going, proof uses parametricity [Reynolds 1983], deriving a free theorem [Wadler 1989]. Proof crucially involves two specific associative functions: 1 Zero One Two Zero Zero One Two One One Two Two Two Two Two Two and 2 Zero One Two Zero Zero One Two One One One Two Two Two One Two and their behavior on lists [(Zero, ) One (, Zero) (, Two) ] and [(Zero, ) One, Two (, Zero) ], respectively. Formalisation available in Isabelle/HOL [Böhme 2007]. But: Does that really explain the why and how? What to do to get similar results for other algorithm classes? 10

9 Plan of the Talk Start telling the story of how 0-1-2, 1, 2,... were found (back in 2007, never recorded, but interesting I think). Challenge you to suggest other approaches to discover the required counterexamples and proof ingredients? Invite complaints about where the presented (deliberately naive, exploratory) approach is too ad-hoc, or unacceptably pulls a rabbit out of a hat. Provoke investigation/proposals of other algorithm classes on which one could try to play the same or similar trick(s)? 11

10 Investigating Particular Instances Only Knuth s 0-1-Principle If a comparison-swap algorithm sorts correctly on the Booleans, it does so on arbitrary totally ordered value sets. A Knuth-like 0-1-Principle? If a parallel prefix algorithm is correct (for associative operations) on the Booleans, it is so on arbitrary value sets. Unfortunately not! 12

11 Investigating Particular Instances Only Knuth s 0-1-Principle If a comparison-swap algorithm sorts correctly on the Booleans, it does so on arbitrary totally ordered value sets. A Knuth-like 0-1-Principle? If a parallel prefix algorithm is correct (for associative operations) on the Booleans, it is so on arbitrary value sets. Unfortunately not! Let s: try to find out why not, 12

12 Investigating Particular Instances Only Knuth s 0-1-Principle If a comparison-swap algorithm sorts correctly on the Booleans, it does so on arbitrary totally ordered value sets. A Knuth-like 0-1-Principle? If a parallel prefix algorithm is correct (for associative operations) on the Booleans, it is so on arbitrary value sets. Unfortunately not! Let s: try to find out why not, and why makes more sense to attempt proving, 12

13 Investigating Particular Instances Only Knuth s 0-1-Principle If a comparison-swap algorithm sorts correctly on the Booleans, it does so on arbitrary totally ordered value sets. A Knuth-like 0-1-Principle? If a parallel prefix algorithm is correct (for associative operations) on the Booleans, it is so on arbitrary value sets. Unfortunately not! Let s: try to find out why not, and why makes more sense to attempt proving, and let s try to do all that without pulling too many rabbits. 12

14 What a Knuth-like 0-1-Principle Would Be Given: scanl1 :: (α α α) [α] [α] scanl1 () (x : xs) = go x xs where go x [ ] = [x] go x (y : ys) = x : (go (x y) ys) candidate :: (α α α) [α] [α] Theorem?: If for every associative () :: Bool Bool Bool and xs :: [Bool], candidate () xs = scanl1 () xs, then the same holds for every type τ, associative () :: τ τ τ, and xs :: [τ]. 13

15 What a Knuth-like 0-1-Principle Would Be Given: scanl1 :: (α α α) [α] [α] scanl1 () (x : xs) = go x xs where go x [ ] = [x] go x (y : ys) = x : (go (x y) ys) candidate :: (α α α) [α] [α] Theorem?: If for every associative () :: Bool Bool Bool and xs :: [Bool], candidate () xs = scanl1 () xs, then the same holds for every type τ, associative () :: τ τ τ, and xs :: [τ]. Let s try to find a counterexample by property-based testing. 13

16 Getting Started Somewhat naive expression of our intent: quickcheck $ λ(candidate :: (α α α) [α] [α]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool] candidate () xs == scanl1 () xs) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ] candidate () xs == scanl1 () xs) 14

17 Getting Started Somewhat naive expression of our intent: quickcheck $ λ(candidate :: (α α α) [α] [α]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool] candidate () xs == scanl1 () xs) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ] candidate () xs == scanl1 () xs) Hmm, this is problematic in so many ways! 14

18 Getting Started Somewhat naive expression of our intent: quickcheck $ λ(candidate :: (α α α) [α] [α]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool] candidate () xs == scanl1 () xs) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ] candidate () xs == scanl1 () xs) Hmm, this is problematic in so many ways! generating (polymorphic) functions? 14

19 Getting Started Somewhat naive expression of our intent: quickcheck $ λ(candidate :: (α α α) [α] [α]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool] candidate () xs == scanl1 () xs) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ] candidate () xs == scanl1 () xs) Hmm, this is problematic in so many ways! generating (polymorphic) functions? randomness vs. exhaustiveness? 14

20 Getting Started Somewhat naive expression of our intent: quickcheck $ λ(candidate :: (α α α) [α] [α]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool] candidate () xs == scanl1 () xs) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ] candidate () xs == scanl1 () xs) Hmm, this is problematic in so many ways! generating (polymorphic) functions? randomness vs. exhaustiveness? a complex property as precondition of ==>? 14

21 Getting Started Somewhat naive expression of our intent: quickcheck $ λ(candidate :: (α α α) [α] [α]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool] candidate () xs == scanl1 () xs) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ] candidate () xs == scanl1 () xs) Hmm, this is problematic in so many ways! generating (polymorphic) functions? randomness vs. exhaustiveness? a complex property as precondition of ==>? generating/picking the type τ? 14

22 Transforming Polymorphism Away Let s try to get a grip on candidate :: (α α α) [α] [α]. 15

23 Transforming Polymorphism Away Let s try to get a grip on candidate :: (α α α) [α] [α]. A question: What can such a function do, given an operation and input list [x 0,..., x n 1 ]? 15

24 Transforming Polymorphism Away Let s try to get a grip on candidate :: (α α α) [α] [α]. A question: What can such a function do, given an operation and input list [x 0,..., x n 1 ]? The answer: Create an output list consisting of expressions built from and x 0,..., x n 1. Independently of the α-type! 15

25 Transforming Polymorphism Away Let s try to get a grip on candidate :: (α α α) [α] [α]. A question: What can such a function do, given an operation and input list [x 0,..., x n 1 ]? The answer: Create an output list consisting of expressions built from and x 0,..., x n 1. Independently of the α-type! But dependent on the input list length n! 15

26 Transforming Polymorphism Away Let s try to get a grip on candidate :: (α α α) [α] [α]. A question: What can such a function do, given an operation and input list [x 0,..., x n 1 ]? The answer: Create an output list consisting of expressions built from and x 0,..., x n 1. Independently of the α-type! But dependent on the input list length n! So, to any candidate as above corresponds a function of type Nat [Expr], where data Expr = Var Nat Op Expr Expr 15

27 Transforming Polymorphism Away Let s try to get a grip on candidate :: (α α α) [α] [α]. A question: What can such a function do, given an operation and input list [x 0,..., x n 1 ]? The answer: Create an output list consisting of expressions built from and x 0,..., x n 1. Independently of the α-type! But dependent on the input list length n! So, to any candidate as above corresponds a function of type Nat [Expr], where data Expr = Var Nat Op Expr Expr We can prove this (almost) isomorphism using parametricity/ free theorems [Reynolds 1983, Wadler 1989], and can actually program appropriate conversions. 15

28 Where Are We Now? To test: quickcheck $ λ(representation :: Nat [Expr]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) 16

29 Where Are We Now? To test: quickcheck $ λ(representation :: Nat [Expr]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) The standard (QuickCheck) approach here, for dealing with Nat [Expr], would be: generate essentially partial functions 16

30 Where Are We Now? To test: quickcheck $ λ(representation :: Nat [Expr]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) The standard (QuickCheck) approach here, for dealing with Nat [Expr], would be: generate essentially partial functions, that take on a useful / used result only for finitely many inputs 16

31 Where Are We Now? To test: quickcheck $ λ(representation :: Nat [Expr]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) The standard (QuickCheck) approach here, for dealing with Nat [Expr], would be: generate essentially partial functions, that take on a useful / used result only for finitely many inputs, while choosing some default for all others. 16

32 Where Are We Now? To test: quickcheck $ λ(representation :: Nat [Expr]) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) The standard (QuickCheck) approach here, for dealing with Nat [Expr], would be: generate essentially partial functions, that take on a useful / used result only for finitely many inputs, while choosing some default for all others. That idea is basically fine here, but we actually need some more control because of our need to check the complex precondition. 16

33 Making Partiality Explicit (and Fixing the Default ) We go from Nat [Expr] to Nat Maybe [Expr], with the interpretation that representation n = Nothing means to take on the scanl1-behavior for lists of length n. 17

34 Making Partiality Explicit (and Fixing the Default ) We go from Nat [Expr] to Nat Maybe [Expr], with the interpretation that representation n = Nothing means to take on the scanl1-behavior for lists of length n. For example, representation :: Nat Maybe [Expr] representation n n == 1 = Just [Op (Var 0) (Var 0)] n == 3 = Just [Var 0, Op (Var 1) (Var 2)] otherwise = Nothing corresponds to candidate :: (α α α) [α] [α] candidate () [x ] = [x x ] candidate () [x, y, z ] = [x, y z ] candidate () xs = scanl1 () xs 17

35 Another Problem Even representation :: Nat Maybe [Expr] is not very testable for our purposes, since we can only find out which outputs are Nothing by actually applying the function. 18

36 Another Problem Even representation :: Nat Maybe [Expr] is not very testable for our purposes, since we can only find out which outputs are Nothing by actually applying the function. What we need to shortcut the complex precondition check is explicit access to the definedness domain. 18

37 Another Problem Even representation :: Nat Maybe [Expr] is not very testable for our purposes, since we can only find out which outputs are Nothing by actually applying the function. What we need to shortcut the complex precondition check is explicit access to the definedness domain. So, we store only the non-nothing positions, as key-value-pairs: type Sparse = [(Nat, [Expr])] 18

38 Another Problem Even representation :: Nat Maybe [Expr] is not very testable for our purposes, since we can only find out which outputs are Nothing by actually applying the function. What we need to shortcut the complex precondition check is explicit access to the definedness domain. So, we store only the non-nothing positions, as key-value-pairs: type Sparse = [(Nat, [Expr])] Previous example then corresponds to (among others): sparse :: Sparse sparse = [(1, [Op (Var 0) (Var 0)]), (3, [Var 0, Op (Var 1) (Var 2)])] 18

39 Another Problem Even representation :: Nat Maybe [Expr] is not very testable for our purposes, since we can only find out which outputs are Nothing by actually applying the function. What we need to shortcut the complex precondition check is explicit access to the definedness domain. So, we store only the non-nothing positions, as key-value-pairs: type Sparse = [(Nat, [Expr])] Previous example then corresponds to (among others): sparse :: Sparse sparse = [(1, [Op (Var 0) (Var 0)]), (3, [Var 0, Op (Var 1) (Var 2)])] Actually a technical challenge now: programmatic conversion between [(Nat, [Expr])] and Nat Maybe [Expr]? 18

40 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Problem: A naive sparsification, iterating through all natural numbers and keeping those for which the result is non-nothing, does not work well. 19

41 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Problem: A naive sparsification, iterating through all natural numbers and keeping those for which the result is non-nothing, does not work well. For example, sparsify (const Nothing) would simply give :: Sparse, from which we could not learn anything. 19

42 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Problem: A naive sparsification, iterating through all natural numbers and keeping those for which the result is non-nothing, does not work well. For example, sparsify (const Nothing) would simply give :: Sparse, from which we could not learn anything. Solution: work with lazy natural numbers: data Nat = Z S Nat 19

43 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Problem: A naive sparsification, iterating through all natural numbers and keeping those for which the result is non-nothing, does not work well. For example, sparsify (const Nothing) would simply give :: Sparse, from which we could not learn anything. Solution: work with lazy natural numbers: data Nat = Z S Nat use an appropriately lazy function for lookup in [(Nat, [Expr])] 19

44 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Problem: A naive sparsification, iterating through all natural numbers and keeping those for which the result is non-nothing, does not work well. For example, sparsify (const Nothing) would simply give :: Sparse, from which we could not learn anything. Solution: work with lazy natural numbers: data Nat = Z S Nat use an appropriately lazy function for lookup in [(Nat, [Expr])] convert with care 19

45 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Problem: A naive sparsification, iterating through all natural numbers and keeping those for which the result is non-nothing, does not work well. For example, sparsify (const Nothing) would simply give :: Sparse, from which we could not learn anything. Solution: work with lazy natural numbers: data Nat = Z S Nat use an appropriately lazy function for lookup in [(Nat, [Expr])] convert with care (for convenience, actually store only the gap lengths ) 19

46 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Solution: work with lazy natural numbers: data Nat = Z S Nat use an appropriately lazy function for lookup in [(Nat, [Expr])] convert with care (for convenience, actually store only the gap lengths ) Example: representation :: Nat Maybe [Expr] representation n n == 1 = Just [Op (Var 0) (Var 0)] n == 3 = Just [Var 0,...] otherwise = Nothing 19

47 Excurse: from Nat Maybe [Expr] to [(Nat, [Expr])] Solution: work with lazy natural numbers: data Nat = Z S Nat use an appropriately lazy function for lookup in [(Nat, [Expr])] convert with care (for convenience, actually store only the gap lengths ) Example: representation :: Nat Maybe [Expr] representation n n == 1 = Just [Op (Var 0) (Var 0)] n == 3 = Just [Var 0,...] otherwise = Nothing now turns into: sparse :: Sparse sparse = [(S Z, [Op (Var 0) (Var 0)]), (S Z, [Var 0,...]), (S (S (S... )), )] + 19

48 Wrapping up the Transformation We now have: quickcheck $ λ(sparse :: Sparse) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) 20

49 Wrapping up the Transformation We now have: quickcheck $ λ(sparse :: Sparse) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) Now we can: implement the complex precondition check (by handcoding or using SmallCheck machinery) 20

50 Wrapping up the Transformation We now have: quickcheck $ λ(sparse :: Sparse) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) Now we can: implement the complex precondition check (by handcoding or using SmallCheck machinery) generalize to other finite types 20

51 Wrapping up the Transformation We now have: quickcheck $ λ(sparse :: Sparse) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) Now we can: implement the complex precondition check (by handcoding or using SmallCheck machinery) generalize to other finite types experiment with different choices for τ 20

52 Wrapping up the Transformation We now have: quickcheck $ λ(sparse :: Sparse) (forall associative $ λ(() :: Bool Bool Bool) λxs :: [Bool]... ) ==> τ. (forall associative $ λ(() :: τ τ τ) λxs :: [τ ]... ) Now we can: implement the complex precondition check (by handcoding or using SmallCheck machinery) generalize to other finite types experiment with different choices for τ experiment with different generators for Sparse-candidates experiment with QuickCheck vs. SmallCheck 20

53 Some Counterexamples Thus Found candidate :: (α α α) [α] [α] candidate () [x 0 ] = [x 0 ] candidate () [x 0, x 1 ] = [x 0, (((x 0 x 1 ) x 0 ) x 0 ) (x 1 x 1 )] candidate () xs = scanl1 () xs 21

54 Some Counterexamples Thus Found candidate :: (α α α) [α] [α] candidate () [x 0 ] = [x 0 ] candidate () [x 0, x 1 ] = [x 0, (((x 0 x 1 ) x 0 ) x 0 ) (x 1 x 1 )] candidate () xs = scanl1 () xs candidate :: (α α α) [α] [α] candidate () [x 0 ] = [x 0 ] candidate () [x 0, x 1 ] = [x 0, ((x 0 x 0 ) x 0 ) x 1 ] candidate () xs = scanl1 () xs 21

55 Some Counterexamples Thus Found candidate :: (α α α) [α] [α] candidate () [x 0 ] = [x 0 ] candidate () [x 0, x 1 ] = [x 0, (((x 0 x 1 ) x 0 ) x 0 ) (x 1 x 1 )] candidate () xs = scanl1 () xs candidate :: (α α α) [α] [α] candidate () [x 0 ] = [x 0 ] candidate () [x 0, x 1 ] = [x 0, ((x 0 x 0 ) x 0 ) x 1 ] candidate () xs = scanl1 () xs candidate :: (α α α) [α] [α] candidate () [x 0, x 1, x 2 ] = [x 0, (x 0 x 0 ) (x 0 x 1 ), x 0 (x 1 x 2 )] candidate () xs = scanl1 () xs 21

56 Recall: Plan of the Talk Start telling the story of how 0-1-2, 1, 2,... were found (back in 2007, never recorded, but interesting I think). Challenge you to suggest other approaches to discover the required counterexamples and proof ingredients? Invite complaints about where the presented (deliberately naive, exploratory) approach is too ad-hoc, or unacceptably pulls a rabbit out of a hat. Provoke investigation/proposals of other algorithm classes on which one could try to play the same or similar trick(s)? 23

57 References I G.E. Blelloch. Prefix sums and their applications. In J.H. Reif, editor, Synthesis of Parallel Algorithms, pages Morgan Kaufmann, S. Böhme. Much Ado about Two. Formal proof development. In The Archive of Formal Proofs N.A. Day, J. Launchbury, and J.R. Lewis. Logical abstractions in Haskell. In Haskell Workshop, Proceedings,

58 References II D.E. Knuth. The Art of Computer Programming, volume 3: Sorting and Searching. Addison-Wesley, J.C. Reynolds. Types, abstraction and parametric polymorphism. In Information Processing, Proceedings, pages Elsevier Science Publishers B.V., M. Sheeran. Searching for prefix networks to fit in a context using a lazy functional programming language. Hardware Design and Functional Languages,

59 References III J. Sklansky. Conditional-sum addition logic. IRE Transactions on Electronic Computers, EC-9(6): , J. Voigtländer. Much ado about two: A pearl on parallel prefix computation. In Principles of Programming Languages, Proceedings, pages ACM Press, P. Wadler. Theorems for free! In Functional Programming Languages and Computer Architecture, Proceedings, pages ACM Press,