esafe Gateway/Mail v. 3.x Load Balancing for esafe Gateway 3.x with Cisco Web NS and CSS Switches Design and implementation guide esafe Gateway provides fast and transparent real-time inspection of Internet traffic. This document describes the firewall load balancing setup and configuration of the Cisco CSS Switches, designed to provide high availability solutions for secure gateway environments where performance requires the use of more than one Gateway, such as the Aladdin esafe Gateway. It is intended that the solution be tested after production of this document so that Aladdin can begin to recommend to its customers an approved High Availability solution for their esafe Gateways using the Cisco CSS as load balancer. This setup and configuration is a standard configuration for the Cisco CSS Switches such as the Cisco 11503 for firewall load balancing. The distinction between load balancing servers and firewalls, or in this case the Aladdin esafe Gateway, in the most general sense is that servers sit behind a single load balancing switch whereas firewalls are sandwiched between 2 load balancing switches. The reason for the different architecture configuration is that by having a CSS load balancing switch on either side of the esafe Gateway, traffic with the same source and destination address, and hence all flows to and from those addresses, can be made to go through the same esafe Gateway. This is essential for state full inspection firewalls, and appears to be how the esafe Gateway needs to work. (Last updated:.april 7, 2003 10:21 am) All attempts have been made to make the information in this document complete and accurate. Aladdin is not responsible for any direct or indirect damages or loss of business resulting from inaccuracies or omissions. The specifications in this document are subject to change without notice. COPYRIGHT No part of this Technical Document may be reproduced or transmitted in any form or by any means, except for the use of the registered user(s) without permission from Aladdin Knowledge Systems, Ltd. Copyright 2001-2, Aladdin Knowledge Systems, Ltd. All rights reserved. Partial Copyright for information on Check point products 2001-2 Check Point Software Technologies Ltd. All rights reserved. TRADEMARKS esafe is a trademark of Aladdin Knowledge Systems, Ltd. CSS and Web NS are trademarks of Cisco Systems, Inc. All other product names mentioned herein are trademarks or registered trademarks of their respective owners. Load Balancing for esafe Gateway 3.x with Cisco Web NS and CSS Switches version 3.x level 4/7/03
Table of Contents page 22 Chapter 1:Overview.......................................................... 1 Chapter 2:Configuration....................................................... 2 Chapter 3:Redundant Configuration............................................... 4 Overview of Configuration 4 The Load Balancing setup for CSS A and B looks as follows: 5 The setup for CSS C and D looks as follows: 6 Chapter 4:CSS FWLB Box-2-Box vs FWLB VIP/V-INT redundancy............................ 8 Advantages: 8 Disadvantages: 8 When to use: 8 When not to use: 8 Load Balancing for esafe Gateway 3.x with Cisco Web NS and CSS Switches version 3.x level 4/7/03
Overview page 1 Overview Firewall load balancing enables you to configure a maximum of 15 firewalls per CSS. Configuring multiple firewalls can overcome performance limitations and remove the single point of failure when all traffic is forced through a single firewall. The firewall load-balancing feature ensures that the CSS will forward all packets with the same source and destination IP addresses through the same firewall or other similar device. The CSS accomplishes this task by performing an XOR on the source and destination IP address. Because the CSS can exist on either side of a firewall, it can balance traffic over multiple firewalls simultaneously. Each firewall is active and available in the load balancing firewall algorithm. The CSS uses the source and destination IP addresses in the algorithm to calculate which firewall to use for each flow. Firewall load balancing acts as a Layer 3 device. Each connection to the firewall is a separate IP subnet. All flows between a pair of IP addresses, in either direction, traverse the same firewall. Firewall load balancing performs routing functions; it does not apply content rules to firewall load-balancing decisions. When using the Cisco CSS Content Switch with multiple Aladdin esafe Gateways, a High Availability and scaleable design can be created to allow for higher data throughput than is recommended with a single esafe Gateway solution, plus providing for the added security of redundant failover should a unit in the Gateway fail. In normal operation traffic destined for the internal network is load balanced on a connection basis across 2 or more esafe Gateways; in the event of a failure of one of the Gateways, the traffic is redistributed using the CSS Content Switch over the remaining Gateways to ensure the continued operation of applications. Additionally, a design can be created providing even more secure redundancy by using redundant CSS Content Switches for Ultra High Availability environments.
Configuration page 2 Configuration The diagram below shows two esafe Gateways called Aladdin esafe 1 and 2 sandwiched between 2 CSS Content Switches called CSS-A and B. The internal network is shown as 10.10.30.0/24 with an unprotected outside network of 192.168.10.0/24 and private addressing within the CSS Sandwich. Figure 1: Firewall Load Balancing Example First, the interfaces and circuits of the CSS switch need be configured, along with any management and access lists information specific to the device. This information has not been included here but may be specified later if required. For each path through the esafe Gateways from both the outside CSS-A and inside CSS-B Content Switches the following 2 configuration parameters define the path taken by the data traffic. Firewall index (identifies the physical firewall), local firewall IP address, remote firewall IP address, and CSS VLAN IP address
Configuration page 3 Static route that the CSS will use for each firewall or configure the OSPF protocol to dynamically learn all firewall routes Use the ip firewall command to define parameters. You must define these parameters for each path through esafe Gateway on both CSS A and B. A CSS must exist on each side of the esafe Gateway to control which esafe unit 1 or 2 is selected for each flow. Within the CSS configuration, you must configure both CSS A and B with the same firewall index number. To avoid dropping packets, the CSS directs all packets between a pair of IP addresses across the same esafe Gateway. This applies to packets flowing in either direction. If a failure occurs on one path, all traffic will use the remaining path or balance traffic on the remaining paths. Important Note:Note You must define the firewall index before you define the firewall route or the CSS will return an error message. To configure the route, refer to the ip route command. The syntax for this global configuration mode command is: a. ip firewall index local_firewall_address remote_firewall_address remote_switch_address The variables are listed below. Enter all IP addresses in dotted-decimal notation(for example, 192.168.10.1). index - The index number to identify the firewall. Enter a number from 1 to 254. local_firewall_ip address - The IP address of the firewall on a subnet connected to the CSS. remote_firewall_ip address - The IP address of the firewall on the remote subnet that connects to the remote CSS. remote_switch_ip address - The IP address of the remote CSS. b. Use the ip route firewall command to configure a static route for firewalls. You can optionally set the administrative distance for the IP route. The syntax for this command is: ip route ip_address subnet_mask firewall index distance The variables are: ip_address - The destination network address. Enter the IP address in dotted-decimal notation. subnet_mask - The IP subnet mask. Enter the mask in either: CIDR bitcount notation (for example, /24). Do not enter a space to separate the IP address from the prefix length. Dotted-decimal notation (for example, 255.255.255.0). index - An existing index number for the firewall route. For information on configuring a firewall index, refer to the ip firewall command. distance - The optional administrative distance. Enter an integer from 1 to 254. A smaller number is preferable. The default value is 1. For example the configuration for CSS-A in the example above is as follows ip firewall 1 10.10.10.33 10.10.20.33 10.10.20.41 ip firewall 2 10.10.10.37 10.10.20.37 10.10.20.42 ip route 10.10.30.0 255.255.255.0 firewall 1 1 ip route 10.10.30.0 255.255.255.0 firewall 2 1 This essentially means that the 2 esafe Gateway paths are defined at the CSS-A switch with equal cost routes, and hence will be equal cost load balanced per connection. The configuration for CSS-B in the example above is as follows, which ensures the return path goes back through the same esafe Gateway. Note the use of the default route on the return path 0.0.0.0 0.0.0.0. ip firewall 1 10.10.20.33 10.10.10.33 10.10.10.41 ip firewall 2 10.10.20.37 10.10.20.37 10.10.10.42 ip route 0.0.0.0 0.0.0.0 firewall 1 1 ip route 0.0.0.0 0.0.0.0 firewall 2 1
Redundant Configuration Overview of Configuration page 4 Redundant Configuration This configuration allows the use of a resilient/redundant pair of CSS 115xx load balancers on either side of the Aladdin esafe Gateways, for a highly redundant solution. There are in fact 2 configurations for redundancy, a more simple Box-2-Box redundancy, where one CSS 11500 is used as primary, and one as failover, and a more complex VIP/V-INT redundancy where both devices work by load balancing connections, and if one path fails, the other side can take over full responsibility for all connections. The pros and cons of both redundant configurations are discussed in this section. The setup as depicted in Figure 2, the redundant configuration is built as a CSS sandwich model with firewalls or esafe Gateways in between. The software on the CSS 11x00 switches needs to be WebNS 5.01 or higher in order to support FWLB with VIP/INT redundancy. The firewall load balancing configuration as shown below is the same regardless of the type of redundancy used. The added complexity of configuration depends on whether Box-2-Box or V Int/VIP redundancy is chosen. In the former case, one of the pair of CSS 115xx switches is chosen as the primary pair and path, and the other pairb are the secondary or failover pair. In the diagram this could mean that CSS A and B are primary, and C and D are secondary. If V Int/VIP redundancy is used, then what this gives is the ability to load balance connections across both pairs of CSS 115xx switches in either direction, with fail over to the other pair of switches in the event of failure of any interface. The failover is determined by using VRRP with virtual pairs of interfaces. There is a primary path and secondary path down each of the two pairs or CSS switches. This means that half of the traffic goes through CSS A and B as its primary path and CSS C and D as its secondary path, and the other half of the traffic goes through CSS C and D as its primary path, and CSS A and B as its secondary path. All switches are being used therefore to increase overall throughput, but with the added complexity of this type of configuration. Again the configuration of the V Int/VIP redundancy is not included here, for simplicity just the Firewall Load balancing configuration has been given. A descriptive overview of the configuration of the V Interfaces and VIPs is given here below however for the users understanding. Overview of Configuration The sandwiched configured model has the following important load balancing routing setting configured: Step 1. The routing to the VIP subnet 10.10.20.16/28 on the back-end CSS switches ( B and D) is via 2 routes pointing to: ip route 10.10.10.16 255.255.255.240 192.168.10.100 ip route 10.10.10.16 255.255.255.240 192.168.10.101 Step 2. These next hop addresses are the Virtual Interface addresses on the front-end CSS switches (A and C), where under stable conditions : 192.168.10.100 is active in CSS A and 192.168.10.101 is active in CSS C. Step 3. These two static routes are redistributed via dynamic routing towards the external, world by the Routers shown at the top of the diagram. Step 4. The routing from the Internet routers (cloud) towards the Internal LAN is dynamic over two routes, which are per session load balanced. Step 5. The front end CSS A and C switches have each a static route configured to the VIP subnet 10.10.20.16/28 via the ip firewall configured routes. (see details below) Step 6. The firewalls or esafe Gateways have the VIP subnet directly connected and don t require any routing statement to the internal CSS units, B and D. Step 7. The routing path backwards from the real servers on the internal LAN 10.10.30.0/24 to the external world (clients) is achieved by putting a default gateway in the real servers pointing to the V-INT address of the backend CSS units. Half of the real servers (or groups of servers) point to the back-end V-INT which is active on CSS B and the other half of the real servers point to the V-INT which is active on CSS D.
Redundant Configuration The Load Balancing setup for CSS A and B looks as follows: page 5 Step 8. In our case these are gateways: 10.10.20.11 and 10.10.20.12 corresponding to the active V Int active or primary on those CSS switches. Step 9. The back-end CSS units ( B and D) have a default route 0.0.0.0 0.0.0.0 with next hop the firewall interface derived from the active ip firewall index statement. (see below for details) Step 10. In the firewalls there is a static route pointing the V-INT address of the 10.10.10.0/24 vlan on the front-end switches ( A and C). Example: On esafe 1: ip route 192.168.10.0 255.255.255.0 10.10.10.35 On esafe 2: ip route 192.168.10.0 255.255.255.0 10.10.10.36 The front end CSS units ( A and C) have a static route for the external network 192.168.10.0 This setup can of course be made dynamic with OSPF. The Load Balancing setup for CSS A and B looks as follows: CSS A# ip firewall 1 10.10.10.33 10.10.10.20 10.10.10.18 ip firewall 2 10.10.10.37 10.10.10.30 10.10.10.18 ip firewall 4 10.10.10.38 10.10.10.29 10.10.10.28 ip firewall 3 10.10.10.34 10.10.10.19 10.10.10.28 ip route 10.10.10.16 255.255.255.240 firewall 1 1 ip route 10.10.10.16 255.255.255.240 firewall 2 1 ip route 10.10.10.16 255.255.255.240 firewall 3 10 ip route 10.10.10.16 255.255.255.240 firewall 4 10 CSS B# ip firewall 1 10.10.10.20 10.10.10.33 10.10.10.41 ip firewall 2 10.10.10.30 10.10.10.37 10.10.10.41 ip firewall 3 10.10.10.19 10.10.10.34 10.10.10.42 ip firewall 4 10.10.10.29 10.10.10.38 10.10.10.42 ip route 0.0.0.0 0.0.0.0 firewall 3 10 ip route 0.0.0.0 0.0.0.0 firewall 4 10 ip route 0.0.0.0 0.0.0.0 firewall 1 1 ip route 0.0.0.0 0.0.0.0 firewall 2 1 The black ip firewall statements have in both CSS A and B indices 1 and 2, and are linked to the black ip route statements having the same indices and a metric of 1. (Primary routes). The purple ip firewall statements have indices 3 and 4 and are linked to the purple ip route statements having the same indices and metric 10 (Backup routes). The ip firewall statements use three ip addresses, which are the interface addresses of: 1. Nearest primary address of firewall 2. Remote address of firewall 3. Interface address of remote CSS The ip address list in the ip firewall statements is limited to three. So, no additional hops can be installed between the two CSS units. The reason for this is that the CSS sends out icmp keepalives over this path with destination address the remote CSS, and with a TTL value of 2. The TTL value of 2 is set for security reasons to avoid any of these special icmp packets the leave the CSS sandwiched network.
Redundant Configuration The setup for CSS C and D looks as follows: page 6 The payload of the icmp keepalives has three ip addresses in it as configured in each ip firewall statement. The remote CSS who receives these icmp packets, will analyse and store the payload and verifies if it has an ip firewall statement with the same intermediate addresses. Both CSS will send out icmp messages and synchronise on the ip addresses and the indices in the payload of these packets. The setup for CSS C and D looks as follows: CSS C# ip firewall 1 10.10.10.37 10.10.10.30 10.10.10.28 ip firewall 2 10.10.10.33 10.10.10.20 10.10.10.28 ip firewall 3 10.10.10.34 10.10.10.19 10.10.10.18 ip firewall 4 10.10.10.38 10.10.10.29 10.10.10.18 ip route 10.10.10.16 255.255.255.240 firewall 1 1 ip route 10.10.10.16 255.255.255.240 firewall 2 1 ip route 10.10.10.16 255.255.255.240 firewall 3 10 ip route 10.10.10.16 255.255.255.240 firewall 4 10 CSS D# ip firewall 1 10.10.10.30 10.10.10.37 10.10.10.42 ip firewall 2 10.10.10.20 10.10.10.33 10.10.10.42 ip firewall 4 10.10.10.29 10.10.10.38 10.10.10.41 ip firewall 3 10.10.10.19 10.10.10.34 10.10.10.41 ip route 0.0.0.0 0.0.0.0 firewall 1 1 ip route 0.0.0.0 0.0.0.0 firewall 2 1 ip route 0.0.0.0 0.0.0.0 firewall 3 10 ip route 0.0.0.0 0.0.0.0 firewall 4 10 The grey ip firewall statements have in both CSS units indices 1 and 2, and are linked to the grey ip route statements having the same indices and a metric of 1. (Primary routes) The orange ip firewall statements have indices 3 and 4 and are linked to the orange ip route statements met the same indices and metric 10 (Backup routes) The layout and addressing to be used including the V Interfaces and VIPs is shown below in Figure 2 for the redundant CSS configuration.
Redundant Configuration The setup for CSS C and D looks as follows: page 7 Figure 2: esafe Load Balancing with Cisco CSS 115xx Switches in Redundant Configuration.
CSS FWLB Box-2-Box vs FWLB VIP/V-INT redundancy Advantages: page 8 CSS FWLB Box-2-Box vs FWLB VIP/V-INT redundancy Advantages: Box-2-Box Simple to configure VIP/V-INT Failover Time between 1 and 3 seconds because: Floating-static path is already up Firewall path information has been exchanged Circuits are up Active Active configuration possible More performance: All switches forward traffic No single point of failure Disadvantages: Box-2-Box VIP/V-INT Only Active Standby configuration More complex configuration possible Standby CSS units are not used for data traffic switching Currently only one physical link possible for VRRP communication When to use: Box-2-Box When Active/Standby is the expected behaviour When a dedicated 10/100 link can be configured between the CSS units. When configuration synchronization is needed VIP/V-INT When there is a common subnet between the two CSS units where the VIP/V-INT can reside on. In both Active/Active and Active/Standby configurations. When fast failover time (< 5sec) is a requirement. When not to use: Box-2-Box VIP/V-INT When Active/Active is needed When configuration synchronization is needed When a dedicated 10/100 link between Configuration complexity of this model is an issue. CSS units cannot be used Lack of 10/100 ports, or switches are. too far apart. Again, it is recommended that the setup be tested for performance and failover times with the esafe Gateways. The configuration will work on all versions of CSS code from 5.01 onwards. The newer CSS 1150x platforms use code 5.10 onwards.
page 3 Document Notes - Changes D DN20020918 Document originally written by Mark Dennis madennis@cisco.com i