WXOS 5.5 SSL Optimization Implementation Guide for Configuration and Basic Troubleshooting

Transcription

1 1 WXOS 5.5 SSL Optimization Implementation Guide for Configuration and Basic Troubleshooting

2 Table of Contents 1. Introduction How Does the SSL Optimization Feature Work What Happens When an SSL Session is Resumed Application Support SSL Version and Cipher Suite Support Certificate Support Hardware Platforms License Requirements Configuration Guide Make Sure That You Have the Encryption Service License Installed Enable SSL Optimization for the HTTPS Application Definition Step 1 Update Application Definition Step 2 Update Monitor Filter List Step 3 Enable TCP Acceleration Step 4 Enable Compression Step 5 Enable SSL Optimization Step 6 Import Certificate Enable SSL Optimization for a Custom Application Definition Step 1 Create Application Definition Step 2 Update Monitor Filter List Step 3 Verify SSL Optimization Configuration Step 4 Import SSL Certificate Troubleshooting What happens if You Run an HTTPS Application with the Default Configuration What Happens if You Only Configure HTTPS for Compression What Happens if You Forget to Enable the SSL Optimization Feature What Happens if You Forget to Check the SSL Encrypted Checkbox in the Application Definition? What Happens if You Forget to Import the Correct Certificate? What Happens if You Enable SSL Optimization While a Session is Already Running How Do You Verify that Everything is Working Common Soft Quit Reasons References Internet Links Books

3 1. Introduction The goal of this Implementation Guide is to provide you with some background information about the SSL Optimization feature, to present a step by step guide to successfully deploying the feature, and finally to offer some basic troubleshooting tips. In this document, we will use the term SSL as more of a generic technology term for TCP-based encryption services (SSL or TLS), rather then meaning the actual SSL (Secure Socket Layer) or TLS (Transport Layer Security) standards. We will use the term WX as a generic family name for both the WX and WXC appliances unless otherwise specified. Please see section 4 for more background information on SSL and TLS. Also see section 1.3 for what is supported in Juniper s current SSL Optimization feature. In WXOS 5.5, Juniper is introducing an SSL Optimization feature which in essence means that the WX can apply MSR and Network Sequence Caching (NSC) compression technologies to applications encrypted using SSL encryption, significantly improving throughput. In general, the better the compression results are, the better the TCP acceleration feature will work, and this gets more data across the WAN link in every round-trip. The Juniper SSL Optimization feature is implemented in a network-centric way rather than being an application proxy-based solution. By network-centric we mean that the WX is transparently inserted into the SSL trust model rather then replacing the SSL trust model, like an application proxy-based solution would do. Using this approach, the Juniper SSL implementation does not change the trust model when it comes to the authentication part of SSL. The client and server will still perform the authentication process as usual. The client will authenticate the server by looking at the certificate that the server has sent and validate the common name, time stamp and Certificate Authority (CA) signer as it normally does in all SSL solutions. If the server requires that the client authenticate itself using a client certificate, that is again done in the normal way and has no impact on the Juniper SSL Optimization feature. In the Juniper trust model, the private key and certificate of the application/server you would like to optimize is only imported into the server-side appliance, not to each branch office appliance. This approach means that there are fewer threats towards compromising the private key, since it is not available in the branch office appliances. Another upside to this trust model approach is that replacing a branch office device is easy, since no certificates or private keys need to be installed in a branch office appliance. In the Juniper trust model, the WX appliances do not need to be configured to optimize an SSL-based application that is based on specific source/destination IP information. In the Juniper configuration, you can use a more generic application definition like TCP only for an SSL-based application. The decision about whether to optimize certain flows or not is based on whether there is a match between the certificate sent by the server and any of the imported certificates in the server-side WX appliance. This provides a quicker deployment of the feature. Based on administrator preference, an application definition can be based on source/destination IP information as well which provides flexibility in deployments. 1

4 1.1. How Does the SSL Optimization Feature Work To understand how the SSL Optimization feature works, we first need to understand how the Juniper WX fits into the SSL trust model and what happens during the establishment of an SSL session. In the first step in an SSL session establishment, the client sends a Client Hello message to the server. The key information in the Client Hello message that the WX on the server side looks for is the SSL Session ID. This information is added to the SSL session cache. Client WX/WXC Client Hello WX/WXC WX caches session ID Server The second step in the SSL session establishment is the Server Hello message from the server. The key information in the Server Hello message that the WX on the server side looks for is what SSL version, cipher suite and compression method the server selects. If the SSL version or cipher suite is not supported or compression is used, the WX will soft quit the flow. A soft quit means that the WX appliance will not perform any SSL optimization, like compression, on this flow. The WX will also match the SSL Session ID in the SSL session cache. WX temporarily caches session keys for duration of transaction Server Hello Server Certificate Server Hello Done Client Key Exchange WX Key Exchange WX checks for imported certificate WX computes session keys The third step in the SSL session establishment is the Server Certificate message from the server. The key information in the Server Certificate message that the WX on the server side looks for is the certificate sent from the server. The WX will compute a digest of the certificate from the server and then match it with one of the imported certificates. If there is a match, the WX appliance will optimize this flow. If the certificate from the server side does not match one of the imported certificates, the WX will soft quit this flow. The fourth step in the SSL session establishment is an optional step that does not have relevance for the Juniper SSL Optimization feature. The server might require the client to authenticate itself by sending the Certificate Request message to the client. The final step that has relevance for the Juniper SSL Optimization feature is the Client Key Exchange message from the client. In the Client Key Exchange message, the client has encrypted (using the server s public key that is embedded in the server certificate) the key material for the master secret (session key) that the client and server will use to exchange encrypted data, also referred to as bulk encryption. Since this message is encrypted with the public key of the server only, the corresponding private key will be able to decrypt this key material. Normally, the private key will only be available on the server which means that it will decrypt the packet, compute the master secret and send it back to the client. This still happens when using the SSL Optimization feature, but since the WX on the server side has a copy of the private key, it will also be able to compute the master secret. The server and the WX on the server side will calculate the same master secret. Once the WX has computed the master secret, it sends this information to the WX on the client side which will store this information in RAM memory for the duration of the SSL transactions within the specific flow. 2

5 From this point forward, the WX appliance will be able to apply all optimization techniques (compression, TCP acceleration, QoS) to the communication between the client and the server. The transportation between the WX appliances will use the regular compression tunnel. The compression tunnel will preferably be encrypted using the IPSec service that is available when you have the Encryption Service license installed to assure end-to-end security. This Implementation Guide will not cover the IPSec encryption setup. Note 1: In WXOS 5.5, the IPSec encryption service is not supported in a WX or WXC Stack deployment. Note 2: All of the above information is true for a new SSL session, but is slightly different when an SSL session resumes What Happens When an SSL Session is Resumed Once a client have established an SSL session with the server, it can close the TCP connection if it is idle, but still maintain the SSL session information. This means that when the client wants to send some new encrypted data to the server, it does not need to go through an entirely new SSL negotiation process, which takes a longer time and consumes a lot of CPU resource on both the client and the server side. Instead, the client can resume the already established SSL session towards the server. In this case, the client will send a Client Hello message to the server containing the existing SSL Session ID. If the server still has the SSL Session ID information in its cache, it will resume the session by sending the Server Hello message with the same SSL Session ID. If the WX on the server side has seen this SSL Session ID before the session ended, it will be able to also resume SSL optimization on this specific flow. If the WX has not seen this SSL Session ID before the session ended, it will soft quit the flow. If the server does not have the SSL Session ID information in its cache, it will send a Server Hello message to the client with a new SSL Session ID. This will trigger the client to start a new SSL session Application Support The Juniper SSL Optimization feature can be applied to any implicit SSL-based application Web (HTTPS), mail (IMAPS and POP3S), directory services (LDAPs), and so on. By an implicit SSL-based application, we mean that the application natively runs SSL as part of the session start. The example below shows an implicit SSL session establishment to an IMAPS server. As you see, the SSL Client Hello message comes directly after the regular TCP handshake. This is an implicit SSL-based application. The Juniper SSL Optimization feature does not support explicit SSL-based applications. By explicit SSL-based applications, we mean that the client has the option to start an encrypted session with the server if the client and server have that capability. The most common explicit SSL-based application is SMTPS which supports the STARTTLS option. The example below shows an explicit SSL session establishment to an SMTP server that supports the STARTTLS option. 3

6 As you see in the example above, after the normal TCP handshake the server will respond with a normal SMTP response. The client will them issue the SMTP EHLO (in packet #271) message to the server. The server will then tell the client what capabilities the server supports, one of the capabilities being the STARTTLS option. If the client is capable of also using the STARTTLS option, the client will issue the STARTTLS command (see packet #273). In packet #274, the server tells the client it is OK to start the TLS session. The client will start the actual TLS handshake in packet #275. This is an example of an explicit SSL-based application. NOTE: The above examples only show implicit and explicit SSL-based applications. Not all SMTPS implementations support the STARTTLS option; these will then be an implicit SSL-based application SSL Version and Cipher Suite Support WXOS 5.5 supports SSL v.3 and TLS 1.0. As part of the SSL session negotiation (Client Hello message), the client might tell the server it is capable of using SSL v.2 which is NOT supported in WXOS 5.5. When a server receives the Client Hello message, the server will decide if it will accept SSL v.2, deny access or tell the client to instead use SSL v.3 or TLS 1.0. In the latter case, the WX can use SSL optimization. WXOS 5.5 supports the following cipher suites originally defined in RFC 2246 and RFC 3268: NULL_WITH_NULL_NULL RSA_WITH_NULL_MD5 RSA_WITH_NULL_SHA RSA_EXPORT_WITH_RC4_40_MD5 RSA_WITH_RC4_128_MD5 RSA_WITH_RC4_128_SHA RSA_EXPORT_WITH_RC2_CBC_40_MD5 RSA_WITH_IDEA_CBC_SHA RSA_EXPORT_WITH_DES40_CBC_SHA RSA_WITH_DES_CBC_SHA RSA_WITH_3DES_EDE_CBC_SHA RSA_WITH_AES_128_CBC_SHA RSA_WITH_AES_256_CBC_SHA RSA_EXPORT1024_WITH_RC4_56_MD5 RSA_EXPORT1024_WITH_RC2_CBC_56_MD5 RSA_EXPORT1024_WITH_DES_CBC_SHA RSA_EXPORT1024_WITH_RC4_56_SHA Note: SSL or TLS sessions using compression technologies are not supported Certificate Support WXOS 5.5 supports the Privacy Enhanced Mail standard (PEM), Public-Key Cyptography Standards (PKCS12) and Distinguished Encoding Rules (DER). Up to 100 certificates/keys can be imported on the WX/WXC appliance. This means that at least 100 different applications can be optimized, but WXOS 5.5 also supports wild card certificates which in practice means even more applications are supported. 4 The certificates and private keys that have been imported are encrypted on the WX/WXC appliance flash memory. The certificate and private keys are not accessible by Juniper Networks or any other third party.

7 1.6. Hardware Platforms WXOS 5.5 and also SSL Optimization can run on all WX/WXC and WX/WXC-stack platforms, currently supported. Check release notes for details License Requirements All encryption service features, SSL and IPSec, require that an additional license be installed in each appliance. The Encryption Service license is available in the Juniper pricelist and is called ENC-xxx, where xxx equals the hardware model. For example, ENC-500 is the Encryption Service license for the WXC-500. In a WX or WXC Stack configuration, the Encryption Service license is only applied to the stack server (WX-100), not the stack clients, just like the regular bandwidth license. Note: In a WX/WXC Stack solution, the Encryption Service license will provide SSL Optimization, but not IPSec services. In a WX/WXC standalone solution, the Encryption Service license will provide both SSL and IPSec services. A customer with an existing IPSec license will, after upgrading to WXOS 5.5, have the ability to also use the SSL Optimization feature; no license update is needed. 2. Configuration Guide 2.1. Make Sure That You Have the Encryption Service License Installed There are two ways to verify if you have the Encryption Service license installed. Check that the Encryption license in visible in the Device Setup -> License Key menu. Or just check that the Encryption submenu is available in the Device Setup menu. If you need to install an Encryption Service license, you first need to create the license key on the Juniper license server. You create the license key based on a combination of the device serial number and product authorization code that you receive when you buy the Encryption Service license. The license key is then installed in the Device Setup -> License Key menu Enable SSL Optimization for the HTTPS Application Definition In WXOS 5.5, the only predefined application definition that relates to the SSL Optimization feature is the HTTPS application. By default, the HTTPS application is configured to be passed through the WX appliance without any optimization (compression, TCP and application-specific optimization). This means that there are quite a few steps to properly configure this application for the SSL Optimization feature. Below you will find the step-by-step guide. The steps below apply to both the client and server side appliance unless otherwise specified. 5

8 Step 1 Update Application Definition The first step is to update the application definition. You will find this in the menu Device Setup -> Application -> Definitions. Click on the HTTPS application link and make sure that you check the SSL Encrypted checkbox. Next click Submit to activate the change Step 2 Update Monitor Filter List The second step updating the list of monitored applications is optional, but most people will want to do this. Unless you complete this step, the monitor results for the HTTPS application will be sorted into the Other traffic category in all monitoring results, including the Flow Diagnostics tool. Go to the Device Setup -> Application -> Monitoring menu. Make sure you check the HTTPS application and then click Submit. Note: A maximum of 40 applications can be included on the Monitored Application list so you will need to de-select another application to be allowed to monitor the HTTPS application, if your application list is full Step 3 Enable TCP Acceleration The third step is to add the HTTPS application into TCP Acceleration. This is a mandatory step to allow the applicationspecific optimization to be enabled. TCP Acceleration is also mandatory if you would like to use NSC compression on this application. Go to the Acceleration -> TCP Acceleration menu. Make sure you check the HTTPS application and the click Submit Step 4 Enable Compression The fourth step is to add the HTTPS application into the compression configuration. You will need to at least add the application into the compression tunnel (minimum MSR compression) or the application will not be optimized. Go to the Compression -> Application Filter menu. Make sure to at least check the HTTPS application for Compress (MSR). Checking the Network Sequence Cache option is not mandatory but is recommended in most applications. 6

9 Step 5 Enable SSL Optimization The fifth step is to enable the SSL Optimization feature. Go to the Device Setup -> Encryption -> SSL Optimization menu. Enable the SSL Optimization feature Step 6 Import Certificate The final step is to import certificates and private keys for the applications/ server you would like to optimize. This is ONLY performed on the WX appliance on the server side (closest to the application/server) of the optimized connection. Go to the Device Setup -> Encryption -> SSL Certificates menu. Click Import in the top right hand corner. First, enter a Friendly Name for the imported certificate/private key pair that makes it easy for you to understand what this certificate/private key pair is being used for. Next select where the files that contain the certificate/private key pair are located. Depending on the way the certificate and private key pair were exported from the server, both the certificate and private key might be in the same file (PKCS12) or they could be in separate files (PEM and DER). Finally, you need to enter the Pass Phrase that was set on the private key file when it was exported from the server. Then click Submit and the certificate/key pair should be imported. There are only two options available on an imported certificate/ key pair. Either View the certificate (not the private key) or Delete the certificate. This concludes the configuration of the predefined HTTPS application definition. 7

10 2.3. Enable SSL Optimization for a Custom Application Definition If you would like to add support for a custom SSL-based application or a generic SSL-based application that is not part of the default application definition, the configuration steps are a bit different from the HTTPS application in the previous section. Again the steps below apply to both the client and server side appliance unless otherwise specified Step 1 Create Application Definition The first step in this case is to create a new application definition. Go to the Device Setup -> Application -> Definition menu. Click New Application in the top right hand corner. Start by defining an application name. Next select the application to be SSL encrypted. Finally, add the relevant information that defines this specific application. In this case, the TCP ports of IMAPS have been added. Click Submit to complete the application definition. When defining a new application, it will automatically be enabled for TCP acceleration and added into the compression configuration for both Compress (MSR) and Network Sequence Caching Step 2 Update Monitor Filter List This second step is optional, but most users will want to do this. Unless you update the list of monitored applications, the monitor results for the new application will be sorted into the Other traffic category in all monitoring results, including the Flow Diagnostics tool. Go to the Device Setup -> Application -> Monitoring menu. Make sure you check the new application and then click Submit. Note: A maximum of 40 applications can be included on the Monitored Application list so you will need to de-select another application to be allowed to monitor the new application, if your Application list is full. 8

11 Step 3 Verify SSL Optimization Configuration The third step is to verify that you have enabled SSL Optimization for the new application. Go to the Device Setup -> Encryption -> SSL Optimization menu. Make sure that the new application is checked for SSL Optimization. Note: The only way for an application to appear on the Application list is that it is checked as an SSL Encrypted application in the application definition Step 4 Import SSL Certificate The final step is to import certificates and private keys for the applications/server you would like to optimize. This is ONLY performed on the WX appliance on the server side of the optimized connection (the one closest to the application/server). This procedure is outlined in section The only thing to think about in the example above is that with three different applications in the same application definition, you might need to import three different certificate/key pairs, depending on how the certificates have been deployed in the application/server. 3. Troubleshooting The main troubleshooting tool in WXOS is the Flow Diagnostics tool. You will find this tool in the Admin -> Tools -> Flow Diagnostics menu. For the SSL Optimization feature, the main troubleshooting is done in the WX appliance on the server side of the solution. In the examples below, you will see the reports from the server side appliance unless otherwise specified. In the Flow Diagnostics tool, you can search for a specific flow. In the example above, we are using TCP source port 443 as the search criteria. After you have entered the search criteria, click Go to start the search. The result below will display the most recent flows. To view detailed information about the flow, click on the magnifying glass icon. Note: You can t use the Application drop-down menu to find HTTPS flows, since HTTPS is not a monitored application by default. Use the Device Setup -> Application -> Monitoring to update the Monitored Application list. 9

12 3.1. What happens if You Run an HTTPS Application with the Default Configuration You will see that the WX appliance will not do anything for this flow. That is because a passthrough filter is being applied. This is by design in the default configuration since HTTPS traffic is not going into the compression tunnel. The only way to do any form of optimization is to configure the application for compression in the Compression -> Application Filter menu What Happens if You Only Configure HTTPS for Compression If you look at the flow, you will see that it does get compressed, but there are several indications that you are not getting optimal performance. First, you see that TCP acceleration is not active for this flow. You also see that the application specific modules are not active. Looking at the compression details, you see very low compression results that result when the SSL Optimization feature is not active. Looking at the module error in the Network Sequence Caching module, you see that the application (HTTPS) is not configured for NSC. This is again by design in the default configuration. 10

13 3.3. What Happens if You Forget to Enable the SSL Optimization Feature Even though you will not see any error in any of the modules, you should still see. that the application optimization modules are not active. Also, the compression level is very low. Note: The compression result on the client side WX Appliance might be very different (higher) What Happens if You Forget to Check the SSL Encrypted Checkbox in the Application Definition? This applies even if you have enabled the SSL Optimization feature. Still the Application Acceleration modules are not active What Happens if You Forget to Import the Correct Certificate? You should see an error in the Application Acceleration module. If you check the details, you will see a Soft Quit error on the flow. Soft Quit means that for some reason the WX appliance is not optimizing this flow in the Application Acceleration module. If you check the specific Application Acceleration module, you will see the reason for the Soft Quit. The soft quit reason will look different on the server versus the client side WX appliance. On the server side, you will see the real issue. In this case you get the NO_CERT_MATCH error, which is self explanatory. 11

14 On the client side WX appliance, you will see the soft quit reason NOT_OPTIMISED_CHK_RMT_WX, which means that the flow for some reason is not optimized and you need to check the server side WX appliance for the real reason What Happens if You Enable SSL Optimization While a Session is Already Running In this case, you would get the SESSION_CACHE_MISS soft quit reason. This means that the WX appliance on the server side hasn t seen this SSL session before and hence can t optimize this flow. Note: An existing SSL session between a client and a server has nothing to do with an existing TCP session. While the TCP session might timeout in a matter of seconds, the SSL session timeout is much longer. For example, in modern Microsoft environments the SSL session cache timeout is 10 hours. In practice, this normally means that to restart an SSL session, you need to restart the application, for example the Web browser software How Do You Verify that Everything is Working There are two main things to look for. First, you should not have any errors on any of the modules. Note: The only exception is if you have made a conscious decision not to use NSC in a WXC appliance. In this case, you would have an error in the NSC module. Secondly, you should in general see higher compression results then a few percent. The compression results will as usual depend on the compressibility of the content, but sending the same file twice when using NSC should give you +90 percent in compression result. 12

15 3.8. Common Soft Quit Reasons Below are a collection of some common soft quit reasons. SSLV2 The server decided to accept an SSL v.2 connection from the client. SSL v.2 is not supported in WXOS 5.5. TLS11 The server decided to accept a TLS 1.1 connection from the client. TLS 1.1 is not supported in WXOS 5.5. CIPHERSUITE The server decided to use a cipher suite that is not supported by WXOS 5.5. See list of supported cipher suites in section 1.4. NO_CERT_MATCH The WX appliance on the server side could not match the certificate sent by the server with one of the imported certificates. SESSION_CACHE_MISS A previous SSL session is being resumed, but that session is not in the WX session cache. COMPRESSION The server has accepted to use compression with the SSL/TLS connection. This is not supported in WXOS 5.5. NOT_OPTIMISED_CHK_RMT_WX This message is seen on the client side WX appliance and means that for some reason, the server side WX appliance decided not to optimize this specific flow. Please check the corresponding flow in the server side WX appliance to find out the real reason for the soft quit. 4. References 4.1. Internet Links Link to the WXOS 5.5 Operators Guide Basic information about SSL/TLS and how it works Link to the original SSL v.3 specification created by Netscape Link to the IETF Web site for all relevant information in the TLS standards Setting the SSL Session cache timeout in Microsoft environments Setting the SSL session cache timeout in a Apache-SSL environment 13

16 CORPORATE HEADQUARTERS AND SALES HEADQUARTERS FOR NORTH AND SOUTH AMERICA Juniper Networks, Inc North Mathilda Avenue Sunnyvale, CA USA Phone: 888.JUNIPER ( ) or Fax: EAST COAST OFFICE Juniper Networks, Inc. 10 Technology Park Drive Westford, MA USA Phone: Fax: ASIA PACIFIC REGIONAL SALES HEADQUARTERS Juniper Networks (Hong Kong) Ltd. 26/F, Cityplaza One 1111 King s Road Taikoo Shing, Hong Kong Phone: Fax: EUROPE, MIDDLE EAST, AFRICA REGIONAL SALES HEADQUARTERS Juniper Networks (UK) Limited Building 1 Aviator Park Station Road Addlestone Surrey, KT15 2PG, U.K. Phone: 44.(0) Fax: 44.(0) Books The following book is highly recommended for anyone who would like to read more about SSL and TLS in an easy digested format. Stephen A. Thomas, SSL & TLS Essentials: Securing the Web ISBN-10: ISBN-13: If you really want to go deep into the development and deployment of SSL- and TLS-based applications, you should have look at this book. Eric Rescorla, SSL and TLS: Designing and Building Secure Systems ISBN-10: ISBN-13: About Juniper Networks Juniper Networks, Inc. is the leader in high-performance networking. Juniper offers a high-performance network infrastructure that creates a responsive and trusted environment for accelerating the deployment of services and applications over a single network. This fuels high-performance businesses. Additional information can be found at Copyright 2008, Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered trademarks, or registered service marks in this document are the property of Juniper Networks or their respective owners. All specifications are subject to change without notice. Juniper Networks assumes no responsibility for any inaccuracies in this document or for any obligation to update information in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice Apr