The Need for a Usable TLS PKI Matthew Smith Usable Security and Privacy Lab, Universität Bonn
SSL / TLS / HTTPS / CA PKIs The TLS protocol family is a set of cryptographic protocols used to secure Internet traffic They are mainly used in combination with Certificate Authorities based PKIs to validate the public keys of entities Seite 2
CAs in the news Prof. Dr. Matthew Smith Seite 3
CAs in the news Trustwave to escape 'death penalty' for SSL skeleton key Moz likely to spare certificate-confession biz same fate as DigiNotar 14th February 2012 Firefox 'death sentence' threat to TeliaSonera over government spy claims Mozilla may snub telecom giant's new SSL certs 16th April 2013 Seite 4
Heartbleed XKCD Comic Seite 5
SSL CCS Injection Vulnerability Masashi Kikuchi of Lepidum CVE-2014-0224 OpenSSL s ChangeCipherSpec processing has a serious vulnerability OpenSSL 1.0.1 through 1.0.1g OpenSSL 1.0.0 through 1.0.0l all versions before OpenSSL 0.9.8y Attackers can eavesdrop and modify communication Attackers can hijack a authenticated session Usable Security and Privacy Lab Universität Bonn Seite 6
Complexity is the worst enemy of security The SSL Family SSL 1.0, 2.0 and 3.0 TLS 1.0 TLS 1.1 TLS 1.2 TLS 1.3 (draft) See the Oakland Paper by Beurdouche et al. A Messy State of the Union: Taming the Composite State Machines of TLS Seite 7
Other PKIs are also in trouble Seite 8
Problems with CAs (whom do we trust?) Approximately 100-200 trusted root CAs in Firefox, Chrome, IE Explorer, Windows, Mac OS, Linux Extended to ~400 via CA hierarchies 34% never used! (@FC 14) HTTPS only as strong as the weakest link Targeted attacks against CAs - a real world threat as Comodo and Diginotar have shown No scopes NSA/China/BND can sign certificates for any domain they wants https://www.eff.org/observatory For more details see our FC paper: You Won't Be Needing These Any More Seite 9
Seite 10
HTTPS Indicators Seite 11
Would you trust? Usable Security and Privacy Lab Universität Bonn Seite 12
Would you trust? Usable Security and Privacy Lab Universität Bonn Seite 13
Firefox 2 Warning FF2 Warning CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 3 Seite 14
FF2 Warning What users actually see Adapted from Jonathan Nightingale CyLab Usable Privacy and Security Laboratory http://cups.cs.cmu.edu/ 4 Seite 15
Seite 16
Usable Security Three seminal papers are seen as the origin of Usable Security and Privacy research* Zurko and Simon s: User-Centered Security Adams and Sasse s: Users Are Not the Enemy Whitten and Tygar s Why Johnny Can t Encrypt: A Usability Evaluation of PGP 5.0 All argued that users should not be seen as a problem to be dealt with, but that security experts need to communicate more with users, and adopt user-centered design approaches. * Garfinkel et al. Usable Security, History, Themes, and Challenges, 2014 Seite 17
USEC Evaluation Techniques Evaluating Without Users Literature Review Cognitive Walkthrough Heuristic Evaluation Model-Based Evaluation Evaluating With Users Qualitative Conceptual Model Extraction Silent Observation Think Aloud Constructive Interaction Retrospective Testing Interviews Quantitative Controlled Experiments Questionnaires Prof. Smith - Fraunhofer FKIE Seite 18
Chapter 1: End-users Seite 19
HTTPS Part 1: Security Indicators Seite 20
Microsoft IE HTTPS Indicators (old) Mozilla Firefox Safari Seite 21
The Emperor s New Security Indicators An evaluation of website authentication and the effect of role playing on usability studies (2007) Seite 22
Study Results All participants entered their passwords after HTTPS indicators were removed, including all 27 who were using their own account credentials. Seite 23
HTTPS Indicators (newer) Made more visible Security signals Green = all is well But things s<ll change on a regular basis Effec<veness s<ll isn t great Seite 24
Android HTTPS Security Indicators Mobile browsers validate TLS cer<ficates correctly display security indicators...and warn the user if something goes wrong Usable Security and Privacy Lab Universität Bonn Seite 25
Online Survey To see if users know whether they are surfing on an HTTPS protected website half of the par<cipants got the survey via HTTP the other half via HTTPS exit survey asked whether their connec<on was protected or not To find out if users understood the Browser s warning messages and warn the user if something goes wrong. presented an SSL warning message Usable Security and Privacy Lab Universität Bonn Seite 26
~50% had not seen an SSL warning message on their phone before. Online Survey - Results 745 par<cipants 47.5% of non- IT experts believed they were using a secure Internet connec<on although it was plain HTTP even 34.7% of par<cipants with prior IT educa<on thought this The risk users were warned against was rated with 2.86 (sd=.94) on a scale between 1 and 5 Many par<cipants stated they did not care about warning messages at all. Usable Security and Privacy Lab Universität Bonn Seite 27
HTTPS Part 2: Security Warnings Seite 28
Crying Wolf: An Empirical Study of SSL Warning Effectiveness (2009) Seite 29
Lab Study All recruits were given an online screening survey, and only online banking customers of the one specific bank were allowed to participate. 100 participants CMU students Recruited by fliers, emails, and participant list 5 Randomly-assigned conditions: FF2, FF3, IE7, Single page custom warning multi-page custom warning Seite 30
Library vs. Bank Library vs Bank Results Ignored Warning 100% 80% 60% 40% 20% Bank Library 0% FF2 FF3 IE7 1-page Multipage In native warning conditions, no significant difference in In native warning conditions, no significant difference in reactions reactions at at library and bank bank In new warning conditions, users more likely to heed warnings at bank than at library In new warning conditions, users more likely to heed warnings at bank than at library Seite 31
Newer HTTPS Warnings Seite 32
Alice in Warningland: A Large-Scale Field Study of Browser Security Warning Effectiveness (2013) Seite 33
Real World Analysis Studied the click-through rate for malware and HTTPS warnings Malware Firefox 7.2% Chrome 23.2% Phishing Firefox 9.1% Chrome 18.0% HTTPS Firefox 33.0% Chrome 70.2% Seite 34
Participatory Design for Security-Related User Interfaces (2015) Seite 35
Participatory Design Small groups: Analysing existing HTTPS warnings and problems to find alternative representation 15 participants (aged 22-35, 8 female) in five workshops 1. IT(male) (pilot), 2. IT (female), 3. IT (mixed), 4. Lawyers 5. Mixed (Physicist, Musician,...) Designer as neutral supporter 1. Explanation on technical background (Shared Language) and Brainstorming 2. Creating new designs (Mock-Ups) 3. Ending (Feedback) Seite 36
Brainstorming Results All groups mentioned quite similar usability aspects: Text too long and unclear Technical details unnecessary for non-experts Use of colours for recommendations and graphics for explanations helpful Capability to take action should be provided Quotes: This is censorship! A reaction to hard-fail warnings/errors I feel like a slave Security measures mostly protect the device instead of the user Task: Design a better warning! Seite 37
1 2 3 4 5 IT(male), IT (female), IT (mixed), Lawyers, Mixed Seite 38
Quote of the day Guys, look, we are actually doing what we criticized before! Seite 39
Here's My Cert, So Trust Me, Maybe? Understanding TLS Errors on the Web (2013) Seite 40
Real World Analysis Studied TLS activity of more than 300,000 users collected certificates passively at egress points of ten network sites over a nine-month period validated certificate chains using browser logic locally 98,46% of the filtered connections validate correctly, implying a false warning rate of 1,54% In a scenario with a hypothetical MITMA chance of 1 in 1.000.000 1.000.000 connections would produce 15.401 warnings out of which 15.400 would be false warnings Seite 41
Conclusion By measuring the prevalence of different types of false warnings, we provide a framework for browsers to re-evaluate their current warning mechanisms and conserve user attention. Akhawe et al. Seite 42
15.400 to 1 We found the attack! Image adapted from Bob Englehart Seite 43
PKI problems go deeper than the end-user End-users are only a small part of the TLS ecosystem Administrators are responsible for (mis)configuration webservers Developers are responsible for (mis)using TLS in their applications System architects and developers are responsible for designing error-prone systems Seite 44
Chapter 2: Administrators Seite 45
Scope of the problem We used HTTPS certificates collected by Google's web-crawler Period of 12 months ~55.7 million different hosts ~4,49 million different X.509 certificates We extracted all certificates that did not validate correctly based on the Firefox browser logic Seite 46
Common: Blaming Administrators It s all the administrators fault! Seite 47
Administrators are not the enemy! Seite 48
Find out where the problems lie ~4,49 million bad certificates We picked a random sample of 50,000 Pruned non-current certs down to 46,145 And contacted the administrators We sent 40,473 emails to webmaster@domain.com and 5,672 to addresses embedded in the certs. Of the 46,145 emails we sent 37,596 could not be delivered to the intended recipient, leaving us with 8,549 successfully delivered surveys 755 complete responses to our survey (~8%) Seite 49
Find out where the problems lie Reasons given in survey ~21% sub-domains/virtual hosts/ redirects ~16% to difficult ~16% for a small group of users ~7% NSA, PRISM & co. ~5% untrusted CA ~3% default configuration ~2% mistake Risk perception ~70% very small ~3% very high ~11% didn t know there were warnings Seite 50
Administrators wish list Lower Price for CA-signed certificates Price is perceived too high for little effort on the CA s side Free CA-signed certificates Cheaper wildcard certificates Allow CACert More trust in CACert s web of trust model Better Support for Non-Validating Certificates Support for trust-on-first-use, Pinning, TACK Better Tool Support OpenSSL command line tool too complicated Server configuration cumbersome, especially for v-hosts Auto-Update Reminder Notification of problems More details can be found in our ASIA CCS 14 paper: Why eve and mallory (also) love webmasters Seite 51
CA Financial Estimates 2013 VeriSign Class 3 Secure Server CA - G3 25603 $1,299.00 EV-SSL Pro $33,258,297.00 VeriSign Class 3 International Server CA - G3 16038 $1,299.00 EV-SSL Pro $20,833,362.00 VeriSign Class 3 Extended Validation SSL SGC CA 10151 $895.00 EV-SSL $9,085,145.00 VeriSign Class 3 Secure Server CA - G2 1933 $1,299.00 EV-SSL Pro $2,510,967.00 VeriSign International Server CA - Class 3 1661 $1,299.00 EV-SSL Pro $2,157,639.00 VeriSign Class 3 Extended Validation SSL CA 4681 $895.00 EV-SSL $4,189,495.00 VeriSign Class 3 Secure Server 1024-bit CA - G2 628 $1,299.00 EV-SSL Pro $815,772.00 VeriSign Class 3 Secure Server CA - T1 82 $1,299.00 EV-SSL Pro $106,518.00 VeriSign Class 3 Extended Validation CA - T1 33 $895.00 EV-SSL $29,535.00 $72,986,730.00 COMODO High-Assurance Secure Server CA 16888 $99.95 SSL $1,687,955.60 PositiveSSL CA 2 15516 $99.95 SSL $1,550,824.20 COMODO Extended Validation Secure Server CA 2774 $449.00 EV-SSL $1,245,526.00 COMODO SSL CA 2248 $99.95 SSL $224,687.60 COMODO High Assurance Secure Server CA 880 $99.95 SSL $87,956.00 COMODO Extended Validation Secure Server CA 2 178 $449.00 EV-SSL $79,922.00 $4,876,871.40 Trustwave Organization Validation CA, Level 2 813 $155.00 SSL $126,015.00 Trustwave Domain Validation CA, Level 1 192 $155.00 SSL $29,760.00 $155,775.00 Prof. Dr. Matthew Smith Seite 52
Chapter 3: Developers Seite 53
Trust me I m an Engineer Seite 54
HTTPS Usage on Android The default Android HTTPS API implements correct cer<ficate valida<on. What could possibly go wrong? Seite 55
HTTPS Usage on Android and ios A server needs a certificate that was signed by a trusted Certificate Authority (~130 pre-installed CAs) For non-trusted certificates a custom workaround is needed Error handling requires custom code Additional security measures such as pinning or Certificate Transparency require custom code Seite 56
It does go wrong... Q: I am getting an error of javax.net.ssl.sslexception: Not trusted server certificate. [...] I have spent 40 hours researching and trying to figure out a workaround for this issue. A: Look at this tutorial http://blog.antoine.li/index.php/2010/10/android-trusting-ssl-certificates stackoverflow.com Seite 57
SSL Sta<c Code Analysis Analysis of 13,500 popular, free apps from Google s Play Market 92.8 % of the apps use the Internet permission 91.7 % of networking API calls are HTTP(S) related 0.8 % exclusively HTTPS URLs 46.2 % mix HTTP and HTTPS 17.28 % of all apps that use HTTPS include code that fails in SSL cer<ficate valida<on 1070 include cri<cal code 790 accept all cer<ficates 284 accept all hostnames Seite 58
Manual App Tes<ng Results Cherry- picked 100 apps 21 apps trust all cer<ficates 20 apps accept all hostnames These 41 apps had an install-base of 39 185 million! Captured creden<als for: American Express, Diners Club, Paypal, bank accounts, Facebook, Twieer, Google, Yahoo, Microsog Live ID, Box, WordPress, remote control servers, arbitrary email accounts, and IBM Same<me, among others. More details can be found in our CCS paper: Why eve and mallove love Android Seite 59
Trusting all Certificates Correct HTTPS certificate validation is easy just don t do anything... What some Apps do: Seite 60
TrustManager Implementa<ons 22 different TrustManager implementa<ons NonValidatingTrustManager FakeTrustManager EasyX509TrustManager NaiveTrustManager TrustManager DummyTrustManager SimpleTrustManager AcceptAllTrustManager OpenTrustManager and all turn effec<ve cer<ficate valida<on off Seite 61
An<- Virus Example ZonerAV An<- Virus app for Android Awarded best free an<- virus app for Android by av- test.org Virus signature updates via HTTPS GET The good thing: It uses TLS Unfortunately: The wrong way static&final!hostnameverifier!do_not_verify!=!new!hostnameverifier()!!!! {!!!!!!! public&boolean!verify(string!paramstring,!sslsession!paramsslsession)!!!!!!! {!!!!!!!!!!!!!return&true;!!!!!!! }!! Zoner AV };! Seite 62
An<- Virus example Zoner fixed the bug immediately! Seite 63
How Do (Good) Apps React to MITMAs? Technically they do not endanger the user However they suffer from serious usability problems Flickr Facebook Seite 64
Common: Blaming Developers It s all the developers fault! Seite 65
Developers are not the enemy! Seite 66
Talking To Developers Finding broken HTTPS in Android and ios apps is good knowing what the root causes are is even beeer We contacted 80 developers of broken apps informed them offered further assistance asked them for an interview 15 developers agreed? Seite 67
Novice Developers This app was one of our first mobile apps and when we noticed that there were problems with the SSL certificate, we just implemented the first working solution we found on the Internet. Seite 68
Intermediate Developers We use self-signed certificates for testing purposes and the easiest way to make them working is to remove certificate validation. Somehow we must have forgotten to remove that code again when we released our app. Seite 69
Expert Developers (kind of...) [...] When I used Wireshark to look at the traffic, Wireshark said that this is a proper SSL protected data stream and I could not see any cleartext information when I manually inspected the packets. So I really cannot see what the problem is here. Seite 70
Expert Developers (time constrained) The app accepts all SSL certificates because some users wanted to connect to their blogs with self-signed certs and [ ] because Android does not provide an easy-to-use SSL certificate warning message, it was a lot easier to simply accept all self-signed certificates. vs. Seite 71
Developer Survey Summary Self-Signed Certificates Development. Developers commonly wish to use self-signed certificates for testing purposes and hence want to turn off certificate validation during testing. Self-Signed Certificates Production. A few developers wanted to use self-signed certificates in their production app for cost, effort and customer satisfaction reasons. Code Complexity. Developers described the code-level customization features of HTTPS as too complex and requiring too much effort. Certificate Pinning / Trusted Roots. Developers liked the idea of having an easy way to limit the number of trusted certificates and/or certificate authorities. Global Warning Message. Developers requested global HTTPS warning messages since they described building their own warning messages as too challenging. Seite 72
A new approach to TLS = custom code required; P = pluggable. Our modifications Existing architecture Central TLS service for Android Force TLS validation Supports self-signed certificates Certificate Pinning Standardised user interaction Alternate Cert validation strategies Force hostname verification Force certificate validation; Configurable by the users android.net.ssl TrustManagerClient (in app) uses replaced by org.apache.http.conn.ssl SSLSocketFactory uses javax.net.ssl TrustManager removed start HTTPS now secure on Android Backwards compatible, except apps that implemented pinning (19 in 13500 tested Android apps) updating them to the new pinning sytem is very easy android.net.ssl TrustManagerService (in system) Pluggable Certificate Validation: (CA-based validation, CT, AKI, TACK, etc.) Turn on/o SSLPinning, Accept all certificates on developer devices configures User options Developer options decisions warn if SSL validation fails Warn the user if connection is insecure Human Computer Interface Figure 1: This figure illustrates the process of creating an appified an SSLworld protected network connection. The Seite grey More details can be found in our CSS paper: Rethinking ssl development in 73
Chapter 4: System Design Seite 74
Problems with CAs (whom do we trust?) Approximately 100-200 trusted root CAs in Firefox, Chrome, IE Explorer, Windows, Mac OS, Linux Extended to ~400 via CA hierarchies HTTPS only as strong as the weakest link Targeted attacks against CAs - a real world threat Everybody screws up End-users Administrators Developers System Designers? https://www.eff.org/observatory For more details see our FC paper: You Won't Be Needing These Any More Seite 75
We need more usable PKIs Up-and-coming PKIs DANE (probably doomed) Certificate Transparency (Google) ARPKI (Perrig et al.) All promise better security All add components (and complexity?) How will developers cope? How will administrators cope? How will users cope? Seite 76
Frontiers of USEC Research Studies Current USEC Researcher PKI Usability Research All layers of a PKI must be usable! End Users Administrators Deploy Create Use Software Software Software Software Software Software Software Software Software Systems System Endangers Endangers Developer Frontiers of Usable Security Matthew Smith, University of Bonn & Research Center L3S Seite 77
USEC Evaluation Techniques Evaluating Without Users Literature Review Cognitive Walkthrough Heuristic Evaluation Model-Based Evaluation Evaluating With Users Qualitative Conceptual Model Extraction Silent Observation Think Aloud Constructive Interaction Retrospective Testing Interviews Quantitative Controlled Experiments Questionnaires Prof. Smith - Fraunhofer FKIE Seite 78
10 Rules for a good Crypto API? Smith & Green @ USENIX Hotsec 15 1. Easy to learn, even without crypto background 2. Easy to use, even without documentation 3. Hard to misuse. Incorrect use should lead to visible errors 4. Hard to circumvent errors except during testing/development 5. Easy to read and maintain code that uses it 6. Sufficiently powerful to satisfy (non-security) requirements 7. Easy to extend Hard to change/override core functionality 8. Appropriate to audience this means people with no crypto experience 9. Assist with/handle end-user interaction conduct developer studies 10. However, where possible integrate into standard APIs so normal developers never have to interact with crypto APIs in the first place Prof. Smith - Fraunhofer FKIE Seite 79
End-user PKI Experts are not the Enemy either! Seite 80