Differential Privacy Tutorial Simons Institute Workshop on Privacy and Big Data. Katrina Ligett Caltech

Transcription

1 Differential Privacy Tutorial Simons Institute Workshop on Privacy and Big Data Katrina Ligett Caltech 1

2 individuals have lots of interesting data π 2

3 individuals have lots of interesting data π...we want to compute on it 2

4 3

5 3

6 3

7 3

8 3

9 3

10 3

11 3

12 3

13 3

14 finding statistical correlations genotype/phenotype associations correlating medical outcomes with risk factors or events publishing aggregate statistics noticing events/outliers intrusion detection disease outbreaks datamining/learning tasks use customer data to update strategies 4

15 5

16 5

17 John Doe tax forms No personally John Doe identifiable irs 1040information was released John Doe error in form 1099 Katrina Ligett data privacy Katrina Ligett aol search debacle Katrina Ligett Ligett DBLP Katrina Ligett computer science news Katrina Ligett Caltech rankings Katrina Ligett weather Pasadena Jane Smith youtube Jane Smith free tv download Jane Smith streaming tv Chris Jones childrens books Chris Jones dr seuss Chris Jones the cat and the hat Chris Jones gifts for children 6

18 John Doe tax forms No personally John Doe identifiable irs 1040information was released John Doe error in form 1099 user data privacy user aol search debacle user Ligett DBLP user computer science news user Caltech rankings user weather Pasadena Jane Smith youtube Jane Smith free tv download Jane Smith streaming tv Chris Jones childrens books Chris Jones dr seuss Chris Jones the cat and the hat Chris Jones gifts for children 7

19 John Doe tax forms No personally John Doe identifiable irs 1040information was released John Doe error in form 1099 user data privacy user aol search debacle user Ligett DBLP user computer science news user Caltech rankings user weather Pasadena Jane Smith youtube Jane Smith free tv download Jane Smith streaming tv Chris Jones childrens books Chris Jones dr seuss Chris Jones the cat and the hat Chris Jones gifts for children 7

20 John Doe tax forms No personally John Doe identifiable irs 1040information was released John Doe error in form 1099 user data privacy user aol search debacle user Ligett DBLP user computer science news user Caltech rankings user weather Pasadena Jane Smith youtube Jane Smith Ad hoc Jane Smith Chris Jones solutions are Chris Jones risky! Chris Jones Chris Jones free tv download streaming tv childrens books dr seuss the cat and the hat gifts for children 7

21 John Doe tax forms No personally John Doe identifiable irs 1040information was released John Doe error in form 1099 user data privacy user aol search debacle user Ligett DBLP user computer science news user Caltech rankings user weather Pasadena Jane Smith youtube Jane Smith Ad hoc Jane Smith Chris Jones solutions are Chris Jones risky! Chris Jones Chris Jones free tv download streaming tv childrens books dr seuss the cat and the hat gifts for children Huge opportunity for formalism. 7

22 This doesn t apply to me! I don t want to publish the whole dataset!

23 individuals hold data......what if it s sensitive? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N 9

24 individuals hold data......what if it s sensitive? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N 9 public

29 individuals hold data......what if it s sensitive? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N 18% 11 public

32 This doesn t apply to me! I don t want to publish the whole dataset!

33 This doesn t apply to me! I don t want to publish the whole dataset! not so fast...

34 This doesn t apply to me! I don t want to publish the whole dataset! not so fast... see, e.g., Korolova 2011 s Facebook microtargeting attack

35 This doesn t apply to me! I don t want to publish the whole dataset! not so fast... see, e.g., Korolova 2011 s Facebook microtargeting attack... must pay attention to all uses of sensitive data

36 what to promise about output? 14

37 what to promise about output? access to the output should not enable one to learn anything about an individual that could not be learned without access 14

38 what to promise about output? access to the output should not enable one to learn anything about an individual that could not be learned without access is this possible? 14

39 what to promise about output? access to the output should not enable one to learn anything about an individual that could not be learned without access is this possible? hint: either privacy or utility separately is easy 14

40 what if wanted to do a study about smoking and? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N 15 public

41 what if wanted to do a study about smoking and? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N 15 public

42 what if wanted to do a study about smoking and? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N there is a correlation 15 public of xxx

43 what if wanted to do a study about smoking and? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N but what if someone knew Alice is a smoker? 15 public there is a correlation of xxx

44 what to promise about output? access to the output should not enable one to learn anything about an individual that could not be learned without access

45 what to promise about output? access to the output should not enable one to learn anything about an individual that could not be learned without access not possible!

46 what to promise about output? John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N

47 what to promise about output? think of output as randomized John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N

48 what to promise about output? think of output as randomized John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N public 18%

49 what to promise about output? think of output as randomized John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N

50 what to promise about output? think of output as randomized John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N public

51 what to promise about output? think of output as randomized name DOB sex weight smoker John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N 16 public

52 what to promise about output? think of output as randomized promise: if you leave the database, no outcome will change probability by very much 16 public

53 more formally... database D a set of rows, one per person sanitizing algorithm M probabilistically maps D to event or object in outcome space

54 differential privacy [DinurNissim03, DworkNissimMcSherrySmith06] ε-differential Privacy for mechanism M: for any two neighboring data sets D1, D2, any C range(m), Pr[M(D1) C] e ε Pr[M(D2) C] 21

55 differential privacy [DinurNissim03, DworkNissimMcSherrySmith06] ε-differential Privacy for mechanism M: for any two neighboring data sets D1, D2, any C range(m), Pr[M(D1) C] e ε Pr[M(D2) C] e ε ~ (1 + ε) 21

56 differential privacy Pr[M(D1) name DOB John Doe Jane Smith Ellen Jones Jennifer Kim Rachel Waters 12/1/51 3/3/46 4/24/59 3/1/70 9/5/43 C] eε Pr[M(D2) sex weight smoker M F F F F Y N Y N N C] N N Y N N

59 differential privacy Pr[M(D1) C] e ε Pr[M(D2) C] C. Dwork

60 (ε,δ)-differential privacy Pr[M(D1) C] e ε Pr[M(D2) C]+δ C. Dwork

61 differential privacy Pr[M(D1) C] e ε Pr[M(D2) C] Is a statistical property of mechanism behavior unaffected by auxiliary information independent of adversary's computational power

62 differential privacy Pr[M(D1) C] e ε Pr[M(D2) C] promise: if you leave the database, no outcome will change probability by very much is this achievable?

63 yes! 27

64 if your output is a number... John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N 18% 28 public

65 if your output is a number... name DOB John Doe Jane Smith Ellen Jones Jennifer Kim Rachel Waters 12/1/51 3/3/46 4/24/59 3/1/70 9/5/43 sex weight smoker M F F F F Y N Y N N add noise with particular shape N N Y N N 18% 28 public

66 sensitivity of a function f f = maxd1, D2 f(d1) f(d2) for neighboring data sets D1, D2 29

67 sensitivity of a function f f = maxd1, D2 f(d1) f(d2) for neighboring data sets D1, D2 measures how much one person can affect output 29

68 sensitivity of a function f f = maxd1, D2 f(d1) f(d2) for neighboring data sets D1, D2 measures how much one person can affect output sensitivity is 1 for counting queries that count number of rows satisfying a predicate 29

69 scale noise with sensitivity f = maxd1, D2 f(d1) f(d2) [DMNS06]: on query f, can add scaled symmetric noise Lap(b) with b = f/ε, to achieve ε-differential privacy. 30

70 Laplace distribution Lap(b) p(z) = exp(- z /b)/2b variance = 2b 2 31

71 applying the Laplace mechanism 32

72 applying the Laplace mechanism single counting query: how many people in the database satisfy predicate P? sensitivity = 1 can add noise Lap(1/ε) 32

73 what if want more than one query?...composition an ε1-dp mechanism, followed by an ε2-dp mechanism, is (ε1 + ε2)-dp 33

74 what if want more than one query?...composition an ε1-dp mechanism, followed by an ε2-dp mechanism, is (ε1 + ε2)-dp can also sum both the epsilons and the deltas for (ε, δ)-dp 33

75 what if want more than one query?...composition an ε1-dp mechanism, followed by an ε2-dp mechanism, is (ε1 + ε2)-dp can also sum both the epsilons and the deltas for (ε, δ)-dp more sophisticated argument: k runs of (ε, δ)-dp mechanisms gives (ε, kδ + δ )- DP for ε = (2 k ln(1/δ )) 1/2 ε + k ε (e ε - 1) 33

76 applying the Laplace mechanism vector-valued queries of dimension d can apply composition and add noise Lap(d f/ε) in each component of output, where f is the sensitivity of each component 34

77 applying the Laplace mechanism histogram queries could again use noise Lap(d/ε) 35

78 applying the Laplace mechanism histogram queries could again use noise Lap(d/ε) but actually only need ~Lap(1/ε), since sensitivity generalizes as max L1 distance 35

79 Gaussian mechanism [DKMMN06]: Gaussian noise gives (ε, δ)-dp with σ (2 ln(2/δ)) 1/2 /ε * (max L2 distance) 36

80 Ok, but I wanted to use my database for more than a handful of statistics...

81 Ok, but I wanted to use my database for more than a handful of statistics... Data can be big in two dimensions: more rows makes privacy easier (lower sensitivity); more columns makes it harder (more queries to preserve)

82 handling an exponential number of queries John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N public

83 what handling an exponential number of queries John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Jennifer Kim 3/1/70 F 135 N N Rachel Waters 9/5/43 F 140 N N fraction over age 50? what fraction smoke and have? what fraction of males over 150 lbs?... public

84 a brief history of synthetic data BLR08: ε-dp, error log 1/3 Q n 2/3 (theory) 39

85 a brief history of synthetic data (theory) BLR08: ε-dp, error log 1/3 Q n 2/3 DNRRV09: (ε, δ)-dp, error Q o(1) n 1/2 39

86 a brief history of synthetic data (theory) BLR08: ε-dp, error log 1/3 Q n 2/3 DNRRV09: (ε, δ)-dp, error Q o(1) n 1/2 DRV10: (ε, δ)-dp, error polylog Q n 1/2 39

87 a brief history of synthetic data (theory) BLR08: ε-dp, error log 1/3 Q n 2/3 DNRRV09: (ε, δ)-dp, error Q o(1) n 1/2 DRV10: (ε, δ)-dp, error polylog Q n 1/2 HR10: (ε, δ)-dp, error log Q n 1/2 39

88 a brief history of synthetic data (theory) BLR08: ε-dp, error log 1/3 Q n 2/3 DNRRV09: (ε, δ)-dp, error Q o(1) n 1/2 DRV10: (ε, δ)-dp, error polylog Q n 1/2 HR10: (ε, δ)-dp, error log Q n 1/2 HLM12: simple & matches best bounds 39

89 a brief history of synthetic data (theory) BLR08: ε-dp, error log 1/3 Q n 2/3 DNRRV09: (ε, δ)-dp, error Q o(1) n 1/2 DRV10: (ε, δ)-dp, error polylog Q n 1/2 HR10: (ε, δ)-dp, error log Q n 1/2 HLM12: simple & matches best bounds 39

90 a brief history of synthetic data (theory) BLR08: ε-dp, error log 1/3 Q n 2/3 DNRRV09: (ε, δ)-dp, error Q o(1) n 1/2 DRV10: (ε, δ)-dp, error polylog Q n 1/2 HR10: (ε, δ)-dp, error log Q n 1/2 HLM12: simple & matches best bounds Can (sometimes) do much better than naive noise addition, with much more sophisticated techniques 39

91 exponential mechanism [MT07] select an element C range(m) with probability ~ exp(ε u(d, C)/(2 u)) where u is a scoring function 40

92 exponential mechanism [MT07] select an element C range(m) with probability ~ exp(ε u(d, C)/(2 u)) where u is a scoring function privacy obvious 40

93 exponential mechanism [MT07] select an element C range(m) with probability ~ exp(ε u(d, C)/(2 u)) where u is a scoring function privacy obvious utility... depends 40

94 [BLR08] combines exponential mechanism [MT07] for sampling complex output space sample complexity bounds from learning theory to guarantee existence of good output 41

95 [BLR08] John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Michael Ray 3/2/81 M 200 Y N Fran Michaels 9/9/54 F 155 N N Rachel Kim 1/21/77 F 130 Y Y Michelle Lo 2/2/83 F 135 N N Nira Waters 9/5/43 F 140 N N Jennifer Kim 3/1/70 F 135 N N Lisa Smith 9/5/43 F 140 N N Tony Miller 12/1/51 M 210 Y N Eve Casey 3/3/46 F 140 N N Paul Larson 4/24/59 F 160 Y Y Noelle Mason 3/1/70 F 130 N N Rachel Waters 9/5/43 F 140 Y N Shirley Wu 3/1/70 F 150 N N Rachel Waters 9/5/43 F 140 N Y Lawrence Vay 12/1/51 M 185 Y N Laura Rich 3/3/46 F 140 N N

96 [BLR08] John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Michael Ray 3/2/81 M 200 Y N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y Fran Michaels 9/9/54 F 155 N N Rachel Kim 1/21/77 F 130 Y Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N Michelle Lo 2/2/83 F 135 N N Nira Waters 9/5/43 F 140 N N Jennifer Kim 3/1/70 F 135 N N Lisa Smith 9/5/43 F 140 N N Tony Miller 12/1/51 M 210 Y N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Eve Casey 3/3/46 F 140 N N Paul Larson 4/24/59 F 160 Y Y Noelle Mason 3/1/70 F 130 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Rachel Waters 9/5/43 F 140 Y N Shirley Wu 3/1/70 F 150 N N Rachel Waters 9/5/43 F 140 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Lawrence Vay 12/1/51 M 185 Y N Laura Rich 3/3/46 F 140 N N

97 [BLR08] Size Õ(VCDIM(Q)/ε 2 ) John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Michael Ray 3/2/81 M 200 Y N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y Fran Michaels 9/9/54 F 155 N N Rachel Kim 1/21/77 F 130 Y Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N Michelle Lo 2/2/83 F 135 N N Nira Waters 9/5/43 F 140 N N Jennifer Kim 3/1/70 F 135 N N Lisa Smith 9/5/43 F 140 N N Tony Miller 12/1/51 M 210 Y N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Eve Casey 3/3/46 F 140 N N Paul Larson 4/24/59 F 160 Y Y Noelle Mason 3/1/70 F 130 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Rachel Waters 9/5/43 F 140 Y N Shirley Wu 3/1/70 F 150 N N Rachel Waters 9/5/43 F 140 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N Lawrence Vay 12/1/51 M 185 Y N Laura Rich 3/3/46 F 140 N N

98 [BLR08] Size Õ(VCDIM(Q)/ε 2 ) John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Michael Ray 3/2/81 M 200 Y N Fran Michaels 9/9/54 F 155 N N Rachel Kim 1/21/77 F 130 Y Y Michelle Lo 2/2/83 F 135 N N Nira Waters 9/5/43 F 140 N N Jennifer Kim 3/1/70 F 135 N N Lisa Smith 9/5/43 F 140 N N Tony Miller 12/1/51 M 210 Y N Eve Casey 3/3/46 F 140 N N Paul Larson 4/24/59 F 160 Y Y Noelle Mason 3/1/70 F 130 N N Rachel Waters 9/5/43 F 140 Y N Shirley Wu 3/1/70 F 150 N N Rachel Waters 9/5/43 F 140 N Y Lawrence Vay 12/1/51 M 185 Y N Laura Rich 3/3/46 F 140 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N

99 [BLR08] Size Õ(VCDIM(Q)/ε 2 ) John Doe 12/1/51 M 185 Y N Jane Smith 3/3/46 F 140 N N Ellen Jones 4/24/59 F 160 Y Y Michael Ray 3/2/81 M 200 Y N Fran Michaels name DOB 9/9/54 sex weight F smoker 155 N N Rachel Kim 1/21/77 F 130 Y Y YYY 1/11/74 F 130 N Y Michelle Lo 2/2/83 F 135 N N ZZZ 10/5/44 F 150 N N Nira Waters 9/5/43 F 140 N N Jennifer Kim 3/1/70 F 135 N N Lisa Smith 9/5/43 F 140 N N Tony Miller 12/1/51 M 210 Y N Eve Casey 3/3/46 F 140 N N Paul Larson 4/24/59 F 160 Y Y Noelle Mason 3/1/70 F 130 N N Rachel Waters 9/5/43 F 140 Y N Shirley Wu 3/1/70 F 150 N N Rachel Waters 9/5/43 F 140 N Y Lawrence Vay 12/1/51 M 185 Y N Laura Rich 3/3/46 F 140 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y YYY 1/11/74 F 130 N Y ZZZ 10/5/44 F 150 N N ZZZ 10/5/44 F 150 N N

100 [HLM12] simple to describe and to implement actually implemented and tested it state of the art in theory, performs well in practice (and quickly, despite bad worstcase news) 43

101 [HLM12] simple to describe and to implement actually implemented and tested it state of the art in theory, performs well in practice (and quickly, despite bad worstcase news)...will hear more about multiplicative weightsbased techniques in Jon s talk 43

102 I have to know all my queries in advance?!

103 interactive mechanisms so far, have discussed creating synthetic data where must know query set in advance tools exist to answer similar number of queries on the fly (correlating randomness across queries) 45

104 It seems like DP would add too much noise for my application.

105 It seems like DP would add too much noise for my application.... stop and think about what this means

106 DP connected to robustness of computation to presence or absence of individuals

107 DP connected to robustness of computation to presence or absence of individuals computation not robust? (should worry!)

108 DP connected to robustness of computation to presence or absence of individuals computation not robust? (should worry!) need more data (individuals) to get desired privacy-utility tradeoff (should think)

109 DP connected to robustness of computation to presence or absence of individuals computation not robust? (should worry!) need more data (individuals) to get desired privacy-utility tradeoff (should think) expect on real data will be robust (we can do something about this!)

110 robustness (an aside) robustness in DP sense not identical to statistical robustness---dp is worst-case rather than wrt to distribution there are connections (will mention shortly) 48

111 expect study robust on actual data idea 1: bootstrapping 49

112 bootstrap aggregation given training dataset, create many new training sets of smaller size by sampling uniformly with replacement fit your model (estimate your statistic) on each combine (e.g., voting, averaging) 50

113 bootstrap intuition a robust statistic should be stable on most reasonbly sized subsets of the data if statistic is somewhat unstable, aggregation smooths result if statistic was very stable, loss in precision should be small 51

114 bootstrap for privacy if function is not low-sensitivity but suspect it s usually stable, not clear how to guarantee DP if aggregation preserves privacy, get DP guarantee even when aggregating non-dp estimates 52

115 bootstrap for privacy = subsample and aggregate [NRS07] 53

116 subsample and aggregate: good news can use any DP aggregation function (as long as choice doesn t depend on data) private aggregation just requires adding noise scaled to sensitivity of the aggregation function privacy is immediate! 54

117 subsample and aggregate: bad news may be difficult to bound worst-case sensitivity of aggregation function default bound is max of its range (quite bad) may be difficult to get good utility guarantees 55

118 subsample and aggregate: applications underlying function might be selecting best model from among set of m options; could aggregate with a noisy max 56

119 subsample and aggregate: applications underlying function might be selecting best model from among set of m options; could aggregate with a noisy max similarly, could output a set of significant features (as for LASSO) 56

120 expect study robust on actual data idea 1: bootstrapping idea 2: check robustness before proceeding 57

121 check robustness would like to be able to test in DP manner whether computation should proceed should : e.g., whether desired function is robust (low-sensitivity) on actual data if not, halt 58

122 local sensitivity of function f on database D maxd f(d) f(d ) 1 for D neighboring data set of D 59

123 local sensitivity of function f on database D maxd f(d) f(d ) 1 for D neighboring data set of D measures how much one person can affect output on this data 59

124 propose-test-release [DL09] propose a bound on local sensitivity 60

125 propose-test-release [DL09] propose a bound on local sensitivity test in DP manner whether actual data satisfies bound 60

126 propose-test-release [DL09] propose a bound on local sensitivity test in DP manner whether actual data satisfies bound if fails, halt 60

127 propose-test-release [DL09] propose a bound on local sensitivity test in DP manner whether actual data satisfies bound if fails, halt if passes, release function with noise tailored to local sensitivity 60

128 notes on propose-test-release test could be what is L1 distance to closest database that fails local sensitivity bound? 61

129 notes on propose-test-release test could be what is L1 distance to closest database that fails local sensitivity bound? test only has sensitivity 1 61

130 notes on propose-test-release test could be what is L1 distance to closest database that fails local sensitivity bound? test only has sensitivity 1 can use conservative local sensitivity threshold 61

131 notes on propose-test-release test could be what is L1 distance to closest database that fails local sensitivity bound? test only has sensitivity 1 can use conservative local sensitivity threshold could still sometimes fail; can t get ε-dp 61

132 DP output need not be noisy! 62

133 DP output need not be noisy! PTR can be used to privately check whether distance to nearest unstable data set is far, and if so release the true f(x) 62

134 robustness, revisited 63

135 robustness, revisited robustness wrt adding/removing a few points from dataset 63

136 robustness, revisited robustness wrt adding/removing a few points from dataset robustness wrt subsampling 63

137 subsample & aggregate + propose-test-release [ST13]: can modify subsample & aggregate so that outputs true f(x) with high probability when f is subsampling stable on x 64

138 subsample & aggregate + propose-test-release [ST13]: can modify subsample & aggregate so that outputs true f(x) with high probability when f is subsampling stable on x shows that DP model selection only increases sample complexity of model selection by O(log (1/δ)/ε) 64

139 DP statistics 65

140 DP statistics connections to robustness; interquartile distance, median, linear regression [DworkLei09] 65

141 DP statistics connections to robustness; interquartile distance, median, linear regression [DworkLei09] M-estimators [Lei11, NekipelovYakovlev11] 65

142 DP statistics connections to robustness; interquartile distance, median, linear regression [DworkLei09] M-estimators [Lei11, NekipelovYakovlev11] for almost any estimator that is asymptotically normal on i.i.d. samples, DP adds asymptotically no additional perturbation [Smith11] 65

143 DP statistics connections to robustness; interquartile distance, median, linear regression [DworkLei09] M-estimators [Lei11, NekipelovYakovlev11] for almost any estimator that is asymptotically normal on i.i.d. samples, DP adds asymptotically no additional perturbation [Smith11] convergence rate of DP estimators tied to gross error sensitivity [ChaudhuriHsu12] 65

144 DP statistics connections to robustness; interquartile distance, median, linear regression [DworkLei09] M-estimators [Lei11, NekipelovYakovlev11] for almost any estimator that is asymptotically normal on i.i.d. samples, DP adds asymptotically no additional perturbation [Smith11] convergence rate of DP estimators tied to gross error sensitivity [ChaudhuriHsu12] minimizing convex loss functions [ChaudhuriMonteleoniSarwate11, Rubinstein et al. 2012, KiferSmithThakurta12] 65

145 DP statistics connections to robustness; interquartile distance, median, linear regression [DworkLei09] M-estimators [Lei11, NekipelovYakovlev11] for almost any estimator that is asymptotically normal on i.i.d. samples, DP adds asymptotically no additional perturbation [Smith11] convergence rate of DP estimators tied to gross error sensitivity [ChaudhuriHsu12] minimizing convex loss functions [ChaudhuriMonteleoniSarwate11, Rubinstein et al. 2012, KiferSmithThakurta12] model selection [SmithThakurta13] 65

146 DP statistics connections to robustness; interquartile distance, median, linear regression [DworkLei09] M-estimators [Lei11, NekipelovYakovlev11] for almost any estimator that is asymptotically normal on i.i.d. samples, DP adds asymptotically no additional perturbation [Smith11] convergence rate of DP estimators tied to gross error sensitivity [ChaudhuriHsu12] minimizing convex loss functions [ChaudhuriMonteleoniSarwate11, Rubinstein et al. 2012, KiferSmithThakurta12] model selection [SmithThakurta13] empirical investigations [VuSlavkovic09, ChaudhuriMonteleoniSarwate11, AbowdSchneiderVilhuber13] 65

147 I need to look at the data before I know what statistics to run.

148 I need to look at the data before I know what statistics to run. not DP

149 I need to look at the data before I know what statistics to run. not DP interactive (or hybrid interactive/ noninteractive ) mechanisms?

150 I need to look at the data before I know what statistics to run. not DP interactive (or hybrid interactive/ noninteractive ) mechanisms? big data force us to formalize looking at the data

151 67 summary

152 summary privacy easy to get wrong; DP provides compelling definition and useful dose of paranoia 67

153 summary privacy easy to get wrong; DP provides compelling definition and useful dose of paranoia powerful tools exist (some with no cost of privacy, and some with no noise!) 67

154 summary privacy easy to get wrong; DP provides compelling definition and useful dose of paranoia powerful tools exist (some with no cost of privacy, and some with no noise!) powerful intuition from notions of robustness 67

155 summary privacy easy to get wrong; DP provides compelling definition and useful dose of paranoia powerful tools exist (some with no cost of privacy, and some with no noise!) powerful intuition from notions of robustness many nearly ready (and quite relevant) to common big data applications 67

156 summary privacy easy to get wrong; DP provides compelling definition and useful dose of paranoia powerful tools exist (some with no cost of privacy, and some with no noise!) powerful intuition from notions of robustness many nearly ready (and quite relevant) to common big data applications no ready-to-use, commercial- grade applications: need demand! 67

157 Differential Privacy Tutorial Katrina Ligett Privacy and Data-Based Research, with Ori Heffetz. Available on SSRN. 68