A Natural Language Approach to Automated Cryptanalysis of Two-time Pads. Joshua Mason Kathryn Watkins Jason Eisner Adam Stubblefield

Transcription

1 A Natural Language Approach to Automated Cryptanalysis of Two-time Pads Joshua Mason Kathryn Watkins Jason Eisner Adam Stubblefield

2 The Two Time Pad Problem

3 Attack at Dawn doqvycswipyxac

4 Attack at Dawn doqvycswipyxac Take the Beach doqvycswipyxac

6 Attack at Dawn doqvycswipyxac Take the Beach doqvycswipyxac Attack at Dawn Take the Beach doqvycswipyxac doqvycswipyxac

7 Attack at Dawn doqvycswipyxac Take the Beach doqvycswipyxac Attack at Dawn Take the Beach doqvycswipyxac doqvycswipyxac

8 Attack at Dawn doqvycswipyxac Take the Beach doqvycswipyxac Attack at Dawn Take the Beach

9 Attack at Dawn f f = Take the Beach

10

11

12

13

14 OJNcDfoMncXzYwwQQZRXYWORT190LP

15 OJNcDfoMncXzYwwQQZRXYWORT190LP the

16 OJNcDfoMncXzYwwQQZRXYWORT190LP the QpL

17 OJNcDfoMncXzYwwQQZRXYWORT190LP the

18 OJNcDfoMncXzYwwQQZRXYWORT190LP the Man

19 Formalized by F. Rubin in 1978 Automated by E. Dawson and L. Nielson in 1996

20 Assumptions Uppercase English characters and space Space is always the most frequent character

21 P 0 P 1 = 6e f 79 61

22 P 0 P 1 = 6e f 79 61

23 P 0 P 1 = 6e 71 6f 79 61

24 P 0 P 1 = 6e 71 6f 79 61

25 P 1 P 2 =

26 P 1 P 2 =

27 P 1 P 2 =

28 Testing Methodology Trained on the first 600K characters of the Bible Attempted recovery of passages from first 600K characters of the bible

29 Percentage Correctly Recovered Dawson & Nielson P0 P1 62.7% P1 P2 61.5% P0 P1 62.6%

30 Percentage Correctly Recovered Dawson & Nielson Our Technique P0 P1 62.7% 100% P1 P2 61.5% 99.99% P0 P1 62.6% 99.96%

31 Our Assumptions Plaintext has some structure Plaintext is in a language we know

32 n-gram count 2 a 2 p 2 l 1 e 2

33

34 7 billion characters

35 7 billion characters 450 million characters

36 7 billion characters 450 million characters 4 billion characters

37 apple orange

38 a start o P 0 P 1 0e

39 a o start p(a) p(o) o P 0 P 1 0e

40 a o start p(a) p(o) p(o) p(a) o a P 0 P 1 0e

41 a o ap or start p(a) p(o) p(o) p(a) p(p a) p(r o) p(r o) p(p a) o a or ap P 0 P 1 0e

42 a o ap or app ora start p(a) p(o) p(o) p(a) p(p a) p(r o) p(r o) p(p a) p(p ap) p(a or) p(a or) p(p ap) o a or ap ora app P 0 P 1 0e

43 a o ap or start p(a) p(o) p(o) p(a) p(p a) p(r o) p(r o) p(p a) o a or ap P 0 P 1 0e 02 0e 02

44 a o ap or apa oro start p(a) p(o) p(o) p(a) p(p a) p(r o) p(r o) p(p a) p(a ap) p(o or) p(o or) p(a ap) o a or ap oro apa P 0 P 1 0e 02 0e 02

45 a o ap or apa oro start p(a) p(o) p(o) p(a) p(p a) p(r o) p(r o) p(p a) p(a ap) p(o or) p(o or) p(a ap) o a or ap oro apa P 0 P 1 0e 02 0e 02

46 Memory/Computation

47 a start b c P 2 P

48 start b c c P 2 P

49 start b c c b P 2 P

50 ba ca bb cb bc cc start b c ca ba cb bb c b cc bc P 2 P

51 start b c p(b) p(c) c b p(c) p(b) P 2 P

52 b c p(b) p(c) c b p(c) p(b) P 2 P

53 ba ca bb cb bc cc b c ca ba p(b) p(c) cb bb c b cc bc p(c) p(b) P 2 P

54 ba ca p(a b) p(a c) bb cb p(b b) p(b c) bc cc p(c b) p(c c) b c ca ba p(a c) p(a b) p(b) p(c) cb bb p(b c) p(b b) c b cc bc p(c c) p(c b) p(c) p(b) P 2 P

55 ba ca p(a b) p(a c) bb cb p(b b) p(b c) bc cc p(c b) p(c c) ca ba p(a c) p(a b) cb bb p(b c) p(b b) cc bc p(c c) p(c b) P 2 P

56 ba ca bb cb bc cc ca ba cb bb cc bc P 2 P

57 ba ca ca ba cc bc P 2 P

58 ba ca ca ba... cc bc P 2 P

59 ... P 2 P

60 ... END P 2 P

61 END P 2 P

62 b c ba ca... END P 2 P

63 Commodity Hardware System Dual Core Pentium 3 GHz Memory 8 GB Storage 1.2 TB

64 Model Build Time ~12 hours Runtime 200 ms per byte Memory Usage ~2 GB

65 Our testing methodology

66 402,590 Files 98,699 Files 520,931 Files

67 402,590 Files 98,699 Files 520,931 Files 2,590 Files 8,699 Files 20,931 Files

68 402,590 Files 98,699 Files 520,931 Files 2,590 Files 8,699 Files 20,931 Files 50 Files 50 Files 50 Files

69 Small HTML 90.64% % Documents 53.84%

70 Small Medium HTML 90.64% 92.78% % 89.04% Documents 53.84% 53.05%

71 Small Medium Large HTML 90.64% 92.78% 93.79% % 89.04% 90.85% Documents 53.84% 53.05% 52.72%

72 The Switching Problem

73 I want to remind you about our All-Employee Meeting this Tuesday, Oct. 23, at 10 a.m. Houston time at the Hyatt Regency. We obviously have a lot to talk about. Last week Well I hope you have Dad doing some of the cleaning! You know how he always has an opinion but yet no participation. Anyway I hope you're doing fine. I'm fine

74 I want to remind you about our All-Employee Meeting this Tuesday, Oct. 23, at 10 a.m. Houston time at the Hyatt Regency participation. Anyway I hope you're doing fine. I'm fine and about to Well I hope you have Dad doing some of the cleaning! You know how he always has an opinion but yet no. We obviously have a lot to talk about. Last week we reported third quarter earnings. We

75 Wu showed Word 2002 re-uses one time pad

76 T13/1510D revision 1 Working T13 Draft 1510D Revision 1.0 January 17, 2003 ATA/ATAPI Host Adapters Standard (ATA Adapter) This is an internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. The T13 Technical Committee may modify the contents. This document is made available for review and comment only. Permission is granted to members of INCITS, its technical committees, and their associated task groups to reproduce this document for the purposes of INCITS standardization activities without further permission, provided this notice is included. All other rights are reserved. Any commercial or for-profit replication or republication is prohibited. T13 Technical Editor: Tony Goodfellow Pacific Digital Corporation 2052 Alton Parkway Irvine, CA92602 USA Tel: Fax: tgoodfellow@pacificdigital.com Working T13 Draft 1532D Volume 1 Information Technology - AT Attachment with Packet Interface 7 Volume 1 (ATA/ATAPI-7 V1) Revision 2 18 February 2003 This is an internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. As such, this is not a completed standard and has not been approved. The contents may be modified by the T13 Technical Committee. This document is made available for review and comment only. Permission is granted to members of INCITS, its technical committees, and their associated task groups to reproduce this document for the purposes of INCITS standardization activities without further permission, provided this notice is included. All other rights are reserved. Any commercial or for-profit replication or republication is prohibited. T13 Technical Editor: Peter T. McLean Maxtor Corporation 2190 Miller Drive Longmont, CO USA Tel: Fax: pete_mclean@maxtor.com Reference number ANSI INCITS.*** - xxxx Printed October, 17, :56PM

77 Working T13 Draft 1532D Volume 1 Information Technology - AT Attachment with Packet Interface 7 Volume 1 (ATA/ATAPI-7 V1) Revision 2 18 February 2003 This is an internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. As such, this is not a completed standard and has not been approved. The contents may be modified by the T13 Technical Committee. This document is made available for review and comment only. Permission is granted to members of INCITS, its technical committees, and their associated task groups to reproduce this document for the purposes of INCITS standardization activities without further permission, provided this notice is included. All other rights are reserved. Any commercial or for-profit replication or republication is prohibited. T13 Technical Editor: T13/1510D revision 1 Revision 1.0 January 17, 2003 Working T13 Draft 1510D ATA/ATAPI Host Adapters Standard (ATA Adapter) This is an internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. The T13 Technical Committee may modify the contents. This document is made available for review and comment only. Permission is granted to members of INCITS, its technical committees, and their associated task groups to reproduce this document for the purposes of INCITS standardization activities without further permission, provided this notice is included. All other rights are reserved. Any commercial or for-profit replication or republication is prohibited. T13 Technical Editor: Tony Goodfellow Pacific Digital Corporation 2052 Alton Parkway Peter T. McLean Maxtor Corporation 2190 Miller Drive Longmont, CO USA Irvine, CA92602 USA Tel: Fax: tgoodfellow@pacificdigital.com Revision 1 January 17, 2003 Tel: Fax: pete_mclean@maxtor.com Reference number ANSI INCITS.*** - xxxx Printed October, 17, :56PM

78 T13 Technical Editor: T13/1510D revision 1 Revision 1.0 January 17, 2003 Working T13 Draft 1510D ATA/ATAPI Host Adapters Standard (ATA Adapter) This is an internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. The T13 Technical Committee may modify the contents. This document is made available for review and comment only. Permission is granted to members of INCITS, its technical committees, and their associated task groups to reproduce this document for the purposes of INCITS standardization activities without further permission, provided this notice is included. All other rights are reserved. Any commercial or for-profit replication or republication is prohibited. T13 Technical Editor: Tony Goodfellow Pacific Digital Corporation 2052 Alton Parkway Irvine, CA92602 USA Tel: Fax: tgoodfellow@pacificdigital.com Working T13 Draft 1532D Volume 1 Information Technology - AT Attachment with Packet Interface 7 Volume 1 (ATA/ATAPI-7 V1) Revision 2 18 February 2003 Revision 2 18 February 2003 This is an internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. As such, this is not a completed standard and has not been approved. The contents may be modified by the T13 Technical Committee. This document is made available for review and comment only. Permission is granted to members of INCITS, its technical committees, and their associated task groups to reproduce this document for the purposes of INCITS standardization activities without further permission, provided this notice is included. All other rights are reserved. Any commercial or for-profit replication or republication is prohibited. Peter T. McLean Maxtor Corporation 2190 Miller Drive Longmont, CO USA Tel: Fax: pete_mclean@maxtor.com Reference number ANSI INCITS.*** - xxxx Printed October, 17, :56PM

79 November 13, 2002 ATA/ATAPI Host Adapters Standard (ATA Adapter) This is an internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. The T13 Technical Committee may modify the contents. This document is made available for review and comment only. Permission is granted to members of INCITS, its technical committees, and their associated task groups to reproduce

80 November 13, 2002 ATA/ATAPI Host Adapters Standard (ATF; h Packet) This is no internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. The T13 Technical Committee may modify the contents. This document is made available and has not been approved. The contents may be modified by the T13 Technical technical committees, and their associated task groups to reproduce

81 November 13, 2002 ATA/ATAPI Host Adapters Standard (ATF; h Packet) This is no internal working document of T13, a Technical Committee of Accredited Standards Committee INCITS. The T13 Technical Committee may modify the contents. This document is made available and has not been approved. The contents may be modified by the T13 Technical technical committees, and their associated task groups to reproduce

82 Exact Pairwise HTML 93.79% 99.45% % 98.41% Documents 52.72% 75.91%

84 Attack at Dawn doqvycswipyxac Take the Beach doqvycswipyxac Bring me Cakes doqvycswipyxac

85 Attack at Dawn doqvycswipyxac Take the Beach doqvycswipyxac Attack at Dawn Take the Beach

86 Attack at Dawn doqvycswipyxac Bring me Cakes doqvycswipyxac Attack at Dawn Bring me Cakes

87 Take the Beach doqvycswipyxac Bring me Cakes doqvycswipyxac Take the Beach Bring me Cakes

88 Attack at Dawn Take the Beach Take the Beach Bring me Cakes Attack at Dawn Bring me Cakes

89 Attack at Dawn Take the Beach

90 Attack at Dawn Take the Beach

91 Attack at Dawn Take the Beach Take the Beach Bring me Cakes

92 Small HTML 99.96% % Documents 69.92%

93 Small Medium HTML 99.96% 99.95% % 98.33% Documents 69.92% 71.11%

94 Small Medium Large HTML 99.96% 99.95% 99.95% % 98.33% 98.34% Documents 69.92% 71.11% 69.39%