1 TLS all the tubes! IsWebRTC TLS Fast Yet? It can be. Making TLS fast(er)... the nuts and bolts. +Ilya
2 All communication should be secure, always, and by default! HTTPS everywhere!
3 ... HTTP TLS TCP IP Authentication am I talking to who they claim to be? Data integrity has anyone tampered with the data? Encryption can anyone see my conversation? Transport Layer Security
4 That s great, but Doesn t TLS have high computational overhead? a. Extra servers, extra ops costs, and so on? 2. Doesn t TLS incur latency overhead? a. Extra roundtrips translate to slower sites, right?
5 TLS has exactly one performance problem: it is not used widely enough. Everything else can be optimized
6 CPU + Memory let s take a peek under the hood...
7 Computational costs Asymmetric crypto (public key) is expensive (relatively speaking) O(1 ms) per handshake Symmetric crypto can easily saturate your NIC 100Mbps+ per core with sha256 and 1024 byte blocks # upgrade to latest $> openssl version # run benchmarks $> openssl speed sha $> openssl speed ecdh
8 We have deployed TLS at a large scale using both hardware and software load balancers. We have found that modern software-based TLS implementations running on commodity CPUs are fast enough to handle heavy HTTPS traffic load without needing to resort to dedicated cryptographic hardware. Doug Beaver, Facebook.
9 On our production frontend machines, SSL/TLS accounts for less than 1% of the CPU load, less than 10 KB of memory per connection and less than 2% of network overhead. Many people believe that SSL/TLS takes a lot of CPU time and we hope the preceding numbers will help to dispel that. Adam Langley, Google.
10 Rough memory usage numbers TLS compression on ~1MB / connection TLS compression off ~100KB / connection Google servers ~10KB / connection 1. Disable TLS compression (security and perf) 2. We need to improve open source libraries...
11 BoringSSL is exciting! Google s fork of OpenSSL, will be used in Chrome, Android, Internal cleanup patches, reduced resource usage, and so on. Hot off the press!
12 Elliptic Curve Ephemeral Diffie-Hellman enables Forward Secrecy. In practical deployment, we found that enabling and prioritizing ECDHE cipher suites actually caused negligible increase in CPU usage. HTTP keepalives and session resumption mean that most requests do not require a full handshake, so handshake operations do not dominate our CPU usage. Jacob Hoffman-Andrews, Twitter.
13 TLS resumption 101 Re-use negotiated parameters for the symmetric cipher Eliminates asymmetric crypto on the server via reuse of previously used parameters Eliminates full roundtrip, allowing 1-RTT connection establishment
14 TLS resumption Session identifiers Server assigns session ID Server caches parameters Client sends session ID Session is resumed Session tickets Shared state is on the server Shared state is on the client Server encrypts parameters Server sets opaque ticket Client sends opaque ticket Server decrypts ticket and resumes session
15 TLS handshake with session resumption... $> openssl s_client -connect example.com:443 -tls1 -tlsextdebug -status SSL-Session: Session Identifier Protocol : TLSv1 Cipher : RC4-SHA Session-ID: 8BE63F4825DDE238E0FE7574D D ECD BFD6FDFB861E Session-ID-ctx: Master-Key: 2FA185F11A791EFB5BA24847FA448B7A0CE73F2D095191F949A35F68CE40FD4EC389E025CCD75 Key-Arg : None Session Ticket TLS session ticket lifetime hint: 600 (seconds) TLS session ticket: e b 4c 13 9d ec-1f 1a 5a ea 89 c6 1f a7.4q.l...z b7 d5 25 4e b6 00-c2 8d ce 6c 06 8b c9 ff..%n V...l... (snip) You can enable both: older clients may not support session tickets Most servers support both, check the docs for configuration options
16 A few things to think about 1. Session identifiers a. Require a shared cache between servers for best results b. Sessions must be expired and rotated in a secure manner 2. Session tickets a. Require a shared ticket encryption key b. Shared encryption key must be rotated in a secure manner 3. Perfect Forward Secrecy (PFS) Session ticket keys have to be distributed to all the frontend machines, without being written to any kind of persistent storage, and frequently rotated. https://www.imperialviolet.org/2011/11/22/forwardsecret.html
19 Terminate TLS at the CDN edge... CDNs are not just for static content. Edge termination can significantly reduce TCP and TLS handshake costs! RTT with origin RTT with CDN edge *Before you hand over the keys to your kingdom, make your your CDN has their TLS stack optimized! You may be surprised...
20 Online Certificate Status Protocol (OCSP) Has this certificate been revoked? Stop the world and query the OCSP server DNS lookup TCP connect Wait for server response What if the OCSP check times out, gets blocked, etc? See Revocation still doesn t work.
21 Eliminating OCSP latency Chrome blocks on EV certs only Other browsers may block on all (FF) Use OCSP stapling! 1. Server retrieves the OCSP response 2. Server staples response to certificate 3. Client verifies stapled response OCSP endpoint
22 TLS handshake with stapled OCSP response... $> openssl s_client -connect example.com:443 -tls1 -tlsextdebug -status OCSP Response Data: OCSP Response Status: successful (0x0) Stapled OCSP means no blocking! Response Type: Basic OCSP Response Version: 1 (0x0) Responder Id: C = IL, O = StartCom Ltd., CN = StartCom Class 1 Server OCSP Signer Produced At: Feb 18 17:53: GMT Responses: Certificate ID: Hash Algorithm: sha1 Issuer Name Hash: F40750F016A E1F5C93E5A26D58 Issuer Key Hash: EB4234D098B0AB9FF41B6B08F7CC642EEF0E2C45 Serial Number: 0B60D5 Cert Status: good OCSP stapling increases certificate size! Is this a problem for your site? Better check.
23 How many RTTs does your certificate incur? 3+ RTT TLS handshake due to 2 RTT cert? Average certificate chain depth: 2-3 certificates Average certificate size: ~1~1.5KB Plus OCSP response Many cert chains overflow the old TCP (4 packet) CWND Upgrade your servers to use IW10!
24 Check your server, you may be surprised... nginx <1.5.6, HAProxy <1.5-dev22 incur extra RTT, even w/ IW10! Capture a tcpdump of your handshake and check the exchange Some servers will pause on large certificates until they get an ACK for the first 4KB of the certificate (doh!)
25 1-RTT non-resumed handshake with TLS False Start Client sends application data immediately after Finished. Eliminates 1RTT No protocol changes... Only timing is affected In practice Some servers break (ugh) Hence, opt-in behavior...
26 Deploying False Start... Chrome and Firefox Chrome and Firefox NPN/ALPN advertisement - e.g. http/1.1 Forward secrecy ciphersuite - e.g. ECDHE Safari Forward secrecy ciphersuite Internet Explorer Blacklist + timeout If handshake fails, retry without False Start TL;DR: enable NPN advertisement and forward secrecy to get 1RTT handshakes.
27 Ingredients for a 1-RTT TLS experience 1. False Start = 1-RTT handshake for new visitors a. New users have to perform public-key crypto handshake 2. Session resumption = 1-RTT handshake for returning visitors a. Plus, we can skip public-key crypto by reusing previous parameters 3. OCSP Stapling a. No OCSP blocking to verify certificate status 4. False Start + Session Resumption + OCSP stapling a. 1-RTT handshake for new and returning visitors b. Returning visitors can skip the public-key crypto
28 What s wrong with this picture? 300ms RTT, 1.5Mbps... It s a 2-RTT handshake we know better! At least there is no OSCP overhead! It s a 2-RTT time to first byte! Large records are buffered, which delays processing!
29 TLS record size + latency gotchas... This record is split across 8 TCP packets TLS allows up to 16KB of application per record New connection + 16KB record = CWND overflow and an extra RTT Lost or delayed packet delays processing of entire record
30 Optimizing record size 1. Google servers implement dynamic record sizing a. New connections start with 1400 byte records (aka, single MTU) b. After ~1MB is sent, switch to 16K records c. After ~1s of inactivity, reset to 1400 byte records 2. Most servers don t optimize this case at all... a. HAProxy recently landed dynamic sizing patch - yay! b. Nginx recently landed ssl_buffer_size: static override - better, but meh... TL;DR: there is no perfect record size. Adjust dynamically.
31 Quick sanity check... theory is great, but does this all work in practice?
32 Tuning Nginx TLS Time To First Byte (TTTFB) Pre 1.5.7: bug for 4KB+ certs, resulting in 3RTT+ handshakes added ssl_buffer_size: 4KB record size remove an RTT with NPN and forward secrecy 1RTT handshake https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-to-first-byte/
33 Out of the box TLS performance is poor we need to fix this. No server is perfect, plenty of work to be done to improve perf.
34 There is way too much red here Bug your CDN about fixing this!
36 Getting ready for HTTP/2 (SPDY, same thing)... better perf and lower ops costs!
37 HTTP/2 and SPDY In practice, you need TLS to deploy SPDY & HTTP/2 HTTP/2 uses a single connection to mux all requests to same origin Page load time improvement with SPDY enabled... Google News Google Sites Google Drive Google Maps Median 43% 27% 23% 24% 95th percentile 44% 33% 36% 28% Improvement over HTTP/1.1 + TLS
38 Fewer connections means that... SPDY also has advantages on the server: SPDY requests consume less resources on the server SPDY requests consume less memory but a bit more CPU SPDY requires fewer Apache worker threads Hervé Servy, Neotys. s/spdy/http2/g same results.
39 An optimized TLS deployment should... Deliver 1-RTT handshake 100% of the time TLS False Start for new visitors TLS resumption for returning visitors Ensure that server is able to send full cert chain without blocking OCSP stapling to avoid blocking Optimize data delivery 1. Optimize record size to avoid unnecessary buffering delays 2. Leverage SPDY / HTTP/2 to further reduce latency and ops costs a. Leverage HTTP/2 optimizations: unshard, un-concat, etc
40 Slides bit.ly/fasttls Learn more istlsfastyet.com Thanks! Questions? +Ilya
The case for ubiquitous transport-level encryption Andrea Bittau Stanford Michael Hamburg Stanford Mark Handley UCL David Mazières Stanford Dan Boneh Stanford Abstract Today, Internet traffic is encrypted
1 Triple Handshakes and Cookie Cutters: Breaking and Fixing Authentication over TLS Karthikeyan Bhargavan, Antoine Delignat-Lavaud, Cédric Fournet, Alfredo Pironti and Pierre-Yves Strub INRIA Paris-Rocquencourt
TOTAL VIEW ONE Technical FAQ System Overview What kind of data does TVO provide and how is it effectively delivered? TVO mirrors and records the state of every connection to deliver actionable real-time
XR: Crossroads Load Balancer and Fail Over Utility Karel Kubat 2010 This document is the introductory guide, the configuration guide and the installation guide to XR. XR is an open source load balancer
The recognized leader in proven and affordable load balancing and application delivery solutions White Paper 7 Easy Steps to Implementing Application Load Balancing For 100% Availability and Accelerated
1 USER MANUAL Legal notice Copyright 2012 TELTONIKA Ltd. All rights reserved. Reproduction, transfer, distribution or storage of part or all of the contents in this document in any form without the prior
Administration Manual Web Security Manager 4.2 www.alertlogic.com firstname.lastname@example.org February, 2014 Alert Logic, the Alert Logic logo, the Alert Logic logotype and Web Security Manager are trademarks
TLSFilter: An Application-Level Firewall for Transport Layer Security Final Report Mat Phillips email@example.com June 16, 2014 Abstract The proliferation of misconfigured and vulnerable SSL/TLS implementations
Redbooks Paper Tuning Windows Server 2003 on IBM System x Servers Phillip Dundas David Watts Windows Server 2003 1 is Microsoft s mainstream server operating system and has been now for almost four years.
Comparing Solace s Appliance- Based Guaranteed Messaging with Software Brokers Guaranteed messaging, also known as persistent messaging in the JMS world, is a quality of service whereby publisher applications
Tech Note: TechNote - Deploying CPPM with F5 BIG-IP Local Traffic Manager (LTM) Version Date Modified By Comments 0.1 July 2014 Danny Jump Early Draft Version 0.2 / 0.3 07/11/2014 Con Stathis Added sections
DEPLOYMENT GUIDE Version 1.0 Deploying F5 with Apache Web Servers Table of Contents Table of Contents Deploying the BIG-IP LTM with the Apache web server Prerequisites and configuration notes... 1 Product
Single Pass Load Balancing with Session Persistence in IPv6 Network C. J. (Charlie) Liu Network Operations Charter Communications Load Balancer Today o Load balancing is still in use today. It is now considered
FortiOS Handbook Load Balancing for FortiOS 5.0 FortiOS Handbook Load Balancing for FortiOS 5.0 November 6, 2012 01-500-99686-20121106 Copyright 2012 Fortinet, Inc. All rights reserved. Fortinet, FortiGate,
Appliance Administration Manual v6.21 This document covers all required administration information for Loadbalancer.org appliances Copyright 2014 Loadbalancer.org, Inc. Table of Contents Section A Introduction...7
Best Practices for Securing Privileged Accounts 2015 Hitachi ID Systems, Inc. All rights reserved. Contents 1 Introduction 1 2 Risk management 2 2.1 Baseline risks............................................
5 Easy Steps to Implementing Application Load Balancing for Non-Stop Availability and Higher Performance DEPLOYMENT GUIDE Prepared by: Jim Puchbauer Coyote Point Systems Inc. The idea of load balancing
Jazz Performance Monitoring Guide Author: Daniel Toczala, Jazz Jumpstart Manager The goal of this performance monitoring guide is to provide the administrators and architects responsible for the implementation
Planning for VoIP by John Q. Walker and Jeffrey T. Hicks a NetIQ Corporation whitepaper, April 2, 2002 Treating VoIP as a Major IT Project 2 Getting It Going...2 Keeping It Running Well...3 Planning, Analysis,
Using Dynamic Feedback to Optimise Load Balancing Decisions Jeremy Kerr firstname.lastname@example.org Abstract The goal of a network load balancer is to distribute a workload evenly amongst a cluster of