Why Can t Johnny Encrypt? A Usability Evaluation of PGP 5.0 Alma Whitten and J.D. Tygar

Transcription

1 Class Ntes: February 2, 2006 Tpic: User Testing II Lecturer: Jeremy Hyland Scribe: Rachel Shipman Why Can t Jhnny Encrypt? A Usability Evaluatin f PGP 5.0 Alma Whitten and J.D. Tygar This article has three main pints. First, effective security requires a different usability standard than ther types f cnsumer sftware. Secnd, security mechanisms are effective nly when used crrectly. Third, it attempts t utline a specific definitin f usability fr security. Security mechanisms are effective nly when used crrectly is the mst imprtant pint since risk f abandnment r misunderstanding n the part f the user is the largest threat t the successful applicatin f security mechanisms. Nte: The article name cmes frm a famus jurnalism piece n literacy called Why Can t Jhnny Read? Abut the authrs: Whitten is a CMU grad wh nw wrks are Ggle, Tygar was nce a prfessr at CMU. T illustrate, the authrs designed a usability study f PGP 5.0 fr the fllwing reasns: PGP has a well designed user interface by cnsumer sftware standards Marketed as being develped with a fcus n GUI design Marketed as making cryptgraphy accessible fr nvice cmputer users Their usability study emplyed the fllwing techniques: Cgnitive Walkthrugh Labratry User Test Whitten and Tygar s definitin f usability fr security: Security sftware is usable if the peple wh are expected t use it: Are reliably made aware f the security tasks they need t perfrm Are able t figure ut hw t successfully perfrm thse tasks Dn t make dangerus errrs Are sufficiently cmfrtable with the interface t cntinue using it The authrs identify 5 prblematic prperties f security relating t user interface design: The unmtivated user prperty Security is a secndary gal The abstractin prperty

2 Security plicies are generally systems f abstract rules fr deciding whether t grant access t resurces The lack f feedback prperty Prviding gd feedback fr security management is a difficult prblem The barn dr prperty Once a secret has been left accidentally unprtected there is n way t be sure that it has nt already been unread by an attacker. The weakest link prperty The security f a netwrked cmputer is nly as strng as it s weakest cmpnent. Using the general definitin f usability fr security, the authrs defined this fcus fr their evaluatin: If an average user f feels the need fr privacy and authenticatin, and acquires PGP with that purpse in mind, will PGP s current design allw that persn t realize what needs t be dne, figure ut hw t d it, and avid dangerus errrs, withut becming s frustrated that he decides t give up n using PGP after all? There was sme skepticism abut the fcus statement: Did the authrs predict the utcme f the user test and write the fcus statement with this result in mind? Hw d yu knw if an average user feels the need fr privacy and authenticatin? The average user was nt defined by the article, and if the scenari frces the users int PGP hw d yu knw if they wuld find the need fr this applicatin (r similar) n their wn? T evaluate this fcus, the authrs started with a cgnitive walkthrugh f the PGP 5.0 interface. They als emplyed aspects f heuristic evaluatin, anther usability technique based n adherence t high-pririty usability principles. Issues identified using cgnitive walkthrugh: Visual metaphrs are heavily used in PGP 5.0. Keys, lcks, and pens are heavily used t represent the cncepts f public and private keys as well as digital signatures. Hwever, these metaphrs break dwn since they d nt maintain cntinuity with the real wrld. Different Key Types t indicate different encryptin algrithms Invisibility f key server ptin Key management plicies names are ambiguus and autmatically assigned Sme user errrs are irreversible; deleting the public key, accidentally publicizing a key, accidentally revking a key, frgetting the pass phrase, and failing t back up the key rings Incnsistent use f security terminlgy

3 Summary f the user test: Test scenari: participant is t play a campaign crdinatr. They must send t the campaign team using PGP fr privacy and authenticatin. There were 12 participants and each was given 90 minutes t cmplete the task. Task cmpletin wuld require the fllwing tasks: Generate a key pair Get the team members public keys Make their public key available Type a secret message in an Sign the with their public key Encrypt using the team members public keys Send the Decrypt a respnse Verify the digital signature f the reply Cluttered infrmatin screens with unnecessary infrmatin cnfuse users Results 3 sent the withut encryptin 1 user frgt her pass-cde, had t create a new key pair 1 user was unable t encrypt at all failure t understand the public key mdel was widespread nly 3 users were able t decrypt the reply message 8 peple gt the public keys, but 5 needed help via 4 peple were able t send an encrypted message it was unclear if the users knew they were digitally signing r checking a signature upn encrypting r decrypting a message. Cnclusin: PGP 5.0 is nt sufficient t make cmputer security usable fr peple wh are nt knwledgeable in security. Questins raised in class abut the validity f the user test: Is the scenari t cmplex? General cnsensus was that there shuld be security sftware that wuld allw yu t cmplete the described task in 90 minutes Dcumentatin was prvided, but ne student made the bservatin that the versin f PGP he used cntained all text instructins withut step by step instructins with screen shts. The suggestin f a wizard fr first time PGP users came up. Wh are the target users fr this applicatin? Whitten and Tygar used peple wh were familiar with , but wuldn t the audience be mre specific?

4 Is the main prblem with the PGP 5.0 interface failure t hire a UI team? General cnsensus was that a UI team wuld help PGP define the answers t the fllwing questins: What is the maximum number f things a user has t learn t effectively use the sftware? Hw much f this culd be dne in the backgrund? What des the user abslutely have t be aware f (i.e. is the digital signature valid?) Lab studies generally try t decrease the number f variables, but real use cntext is full f variables that were nt cnsidered in this study like training and help frm c-wrkers One student bserved that althugh sme f these prblems were fixed in the mre recent versins f PGP 5.0, the additin f features has added t the cmplexity f the interface Ability t questins and get varied respnses depending n wh answered the questin damaged the ability t recreate the study t verify results Jhnny 2 Garfinkel and Miller, 2005 The main idea f this article is that the key prblem with PGP 5.0 was the key mdel it emplyed. They therized that by replacing this certificatin mdel with Key Cntinuity Management (KCM), nvice users wuld be able t easily encrypt and decrypt messages. They tested the effectiveness f a prttype KCM system using labratry user tests as the usability evaluatin methd. They als added anther dimensin t this user test: the presence f attackers. The attackers wuld emply ne f the three fllwing tactics t gain access t the secret cntent f messages: 1. New key attacks 2. Unencrypted message attacks 3. New identity attacks Users were allwed t questins like in Jhnny 1, but this time respnses were selected frm a standardized list. In additin, the questins and respnses were tracked as part f the study. The KCM prttype has the fllwing features; tight client integratin, visual cues in the frm f brder clrs that alter users t encryptin status, and basic autmated key distributin. The participants in the user study were brken int ne f three test circumstances: 1. N KCM 2. Clr

5 Results: 3. Clr and briefing N users had difficulty with the basic encryptin/decryptin prcess Few users understd signature integrity guarantees Authenticity f the attack message was difficult fr users t identify N user was able t avid attacks 100% f the time Grup 1 had the hardest time resisting new key attacks and unencrypted message attacks All grups had truble with the new identity attack Users has miscnceptins abut the security f sealed messages Addressing surface interface issues helped users in the scenari. Fr example, the d nt trust buttn was easy fr user t understand. Vastly simplifies encryptin/decryptin cmpared t PGP 5.0 In class discussin brught up the fllwing: Use f yellw culd be cnfusing there are cultural cnnectins between yellw and warning, althugh it is nt being used that way in this applicatin The user studies are testing very different technlgies under very different circumstances shuld it really be called Jhnny 2? Wh uses encryptin? Dissident grups, generally funded by nn-prfit rganizatins Risk t being identified because cntent f message is unacceptable t gvernment/leadership Sme security experts think everyne shuld use it all the time Activist rganizatins bad peple Used t send cnfidential infrmatin within a cmpany. Example: perfrmance evaluatins at technical research and develpment cmpany where anyne culd hack int the system. Is PGP crack-able? Chance fr cracking PGP algrithm is lwered by regular change in encryptin algrithm emplyed Class exercise: Divide int teams f 3-4 and create a user study fr ne f the fllwing bjects: Cell phne Calculatr Prtable CD Player Design the use scenari and use a member f anther grup as the participant. Think abut: Hw well des the interface supprt the participant? Hw will yu measure success/failure f different aspects f the task? Des this help yu generate ideas fr redesign f the bject?