Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Objectives The purpose of this lab is to demonstrate both high availability and performance using virtual IPs coupled with DNS round robin as a load sharing technique including failover. Throughout the lab all three WCGs will be configured in a managed cluster, correctly allocated virtual IPs and subsequently configured to be addressed via DNS round robin, allowing load balancing of the WCGs with active/active clustering. You will test functionality and configuration at each stage of the process. Lab Environment Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 1
Managed vs. Full Clustering Management Clustering Management-only mode enables you to administer all the nodes in a cluster at the same time. Nodes automatically share configuration information. Full Clustering Full-clustering mode uses one logical cache spread across the machines all cluster members communicate to see if any of them have a copy of a requested page. Includes management-only mode. A fully clustered Websense Content Gateway maps cached content to specific nodes in the cluster. When a node receives a request, it checks to see if the request is a hit somewhere in the cluster. If the request is a hit on a different node, the node handling the request fetches the object from the hit node and serves it to the client. Websense Content Gateway uses a proprietary communication protocol to fetch an object from other cluster nodes. If a node fails or is shut down and removed, Websense Content Gateway removes references to the missing node on all nodes in the cluster. If virtual IP failover is enabled requests destined for the missing node are handled by another node. Configuring a Managed Cluster Configure WSG-1, WSG-2, WSG-3 as a Managed Cluster using eth0 as the clustering interface. Do the following steps 2-9 on WSG-1, WSG-2 and WSG-3 1. At the command line, enter the following: route add 224.0.1.37/32 dev eth0 2. You need the multicast group address and interface name to configure through Websense Content Manager. 3. Configure > My Proxy > Basic > Clustering 4. Select Management Clustering 5. Select interface eth0 6. Use the Multicast Group Address 224.0.1.37 Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 2
7. Apply the settings and restart the gateway. 8. On each gateway, use the following command to verify the cluster communication is working correctly: tcpdump -v -ni eth0 ip multicast You should see the multicast communication displaying on each of the WSG servers, showing the source IP of the WCG interface and the destination multicast group address. Note when implementing clustering with appliance based WCGs, you do not require a dedicated interface. Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 3
Active/Active & Active/Passive - HA, Failover, Load Balancing and Redundancy High Availability High availability refers to a system or component that is continuously operational for a desirable percentage of total time. Failover Failover involves the functions of a system being assumed by a secondary system when the primary becomes unavailable through either failure or scheduled down time. Load balancing Load balancing spreads work between two or more computers, network links or other resources in order to get optimal resource utilization, maximize throughput and minimize response time. Redundancy Redundancy may take two forms: Spare components /systems may be inactive but available for use in case of the failure of the active system. Multiple components/instances may be implemented with load balancing to provide increased throughput if all are available and redundancy if any of them fail. Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 4
Virtual IP Virtual IP is used in clustering - virtual IP failover assures that if a node in the cluster fails, other nodes can assume the failed node's responsibilities, if virtual IP addressing is enabled, several virtual IP addresses could be assigned to this node. The virtual IP should be tied to the proxy interface. How it works Virtual IP addresses are not bound to any one machine and can be moved about. It works via ARP; when a machine assumes a virtual IP address it tells the router/switch that its MAC address is the address for that virtual IP address. Incoming packets are sent to the VIP address, but all packets travel through real network interfaces. Browser clients are configured to connect to the virtual IP address of the proxy. Load balancing may then be provided by DNS Round Robin It is not recommended to combine Virtual IP with hardware load balancing e.g. Layer 4 Switch Using a proprietary management protocol, Websense Content Gateway nodes communicate their status with their peers. If a node fails, its peers recognize the failure and negotiate which of the remaining nodes will mask the fault by taking over the failed node s virtual interface. Virtual IP Cluster recovery process When a cluster node fails, the processes performed during cluster creation are re-run to redistribute Virtual IP addresses and URLs to be cached. The remaining nodes may or may not retain current Virtual IP addresses or URLs - the process is not a redistribution of the failed node s associations, it is a completely new distribution Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 5
Choosing the number of VIPs dependent on number of nodes When choosing the number of VIPs to allocate to a cluster, for load balancing purposes it s important to recognize how the VIPs will be re-distributed after a node failure. In the following two node cluster, two VIPs addresses provide a balanced configuration on failover. For the following three node cluster, six VIPs provide a balanced configuration on failover. If each node had only one VIP, upon re-distribution three VIPs would be distributed to the remaining two active nodes. One node would take 2 VIPs leaving the other with a single VIP the result of which would be that load balancing would in fact be unbalanced. The calculation for determining the number of VIPs required to balance a cluster in the event of one node failing is N x (N-1) where N is the number of nodes. Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 6
Configuring HA for WCG Using Virtual IP Addresses Recall you have 3 gateways in a management cluster, assuming the event of one cluster node failing you want to distribute the remaining IPs evenly. We will assign 6 VIPs across the 3 cluster nodes. This results in 2 VIPs per gateway assuming full availability balancing to 3 VIPs per node on the failure of 1 node. On WCG-1 9. Navigate to Configure > My Proxy > Basic and Enable Virtual IPs 10. Restart the proxy 11. Navigate to Configure > Networking > Virtual IPs 12. Click on the Edit file button 13. Type the following Virtual IP details adding each one 172.29.1.150 eth0 Sub Interface 2 172.29.1.151 eth0 Sub Interface 3 172.29.1.152 eth0 Sub Interface 4 172.29.1.153 eth0 Sub Interface 5 172.29.1.154 eth0 Sub Interface 6 172.29.1.155 eth0 Sub Interface 7 14. Click the Apply button and then the Close button. 15. Verify on WCG-2 and WCG-3 the VIPs are shown under Configure > Networking > Virtual IPs Note the sub interface number must be unique for the entire cluster: -bash-3.1# ifconfig eth0... eth0:2... eth0:3... On WCG-1, WCG-2 and WCG-3 Systems 16. Go to the WCG VM command line Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 7
17. Run ifconfig on WCG-1, WCG-2 and WCG-3 18. View the sub interfaces to determine which virtual IP(s) have been assigned to each proxy. You can also get IP / Node mappings from Monitor > Networking > Virtual IP (this does not provide sub interface details) 19. Note the IP addresses mappings below. Sub Interface VIP Sub Interface VIP WCG-1 WCG-2 WCG-3 Testing VIP directly 20. Switch to the XP-1 VM 21. Set Internet Explorer to explicitly proxy to one of the VIPs assigned to WCG-3 22. Test Connectivity by using http://www.google.com 23. Test Filtering by using http://www.concordiahomes.ca Shutting down WCG-3 to redistribute VIPs 24. On WCG-3, run ifconfig and verify sub interfaces are listed 25. Using /opt/wcg/wcgadmin stop, stop the WCG services on WCG-3 hosting the explicit proxy VIP in the previous section 26. Await all services to stop Verifying re-assignment of VIP to WCG-1 and WCG-2 27. Run ifconfig on WCG-3 and verify no sub interfaces listed for eth0. You can also verify VIP re-distribution via Monitor > Networking > Virtual IP from WCG-1 or WCG-2 28. Note which VIPs are assigned to available cluster nodes. Sub Interface VIP Sub Interface VIP WCG-1 WCG-2 WCG-3 (shutdown) Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 8
Sub Interface VIP 29. You should still be set to proxy through the VIP previously allocated to WCG-3 30. From the WXP VM test Connectivity to http://www.google.com and filtering using http://www.concordiahomes.ca 31. If not yet functioning, wait 30 seconds 32. Test connectivity and test filtering again. 33. Time permitting, start the WCG services again on WCG-3 and verify the time it takes to redistribute VIPs to the node, either via a WCG GUI or verifying with ifconfig on WCG-3. Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 9
Configuring round robin Now you have successfully configured and tested the cluster with VIPs, you will now configure DNS round robin to load balance traffic across the VIPs. Round robin is a local balancing mechanism used by DNS servers to share and distribute network resource loads. You can use it to rotate all resource record (RR) types contained in a query answer if multiple RRs are found. By default, DNS uses round robin to rotate the order of resource records (RR) data returned in a query where multiple RRs of the same type exist for a queried DNS domain name. DNS resource records (RRs) describe the characteristics of a zone (or domain). Round robin provides a simple method for load balancing client use of Web servers and other frequently queried multi-homed computers. Example: Round-robin rotation A forward lookup-type query (for all A RRs that match a DNS domain name) is made for a multihomed computer (wcgproxyrr.wstrain.com) that has three IP addresses. Separate A RRs are used to map the host's name to each of these IP addresses in the zone. In the stored wstrain.com zone, the RRs appear in this fixed order: Resource Record A: An IPv4 Address record. An IPv4 address for a host wcgproxyrr IN A 10.0.0.1 wcgproxyrr IN A 10.0.0.2 wcgproxyrr IN A 10.0.0.3 IN: Query class used for Internet The first DNS client that queries the server to resolve this host's name receives the list in default order. When a second client sends a subsequent query to resolve this name, the list is rotated as follows: wcgproxyrr IN A 10.0.0.2 wcgproxyrr IN A 10.0.0.3 wcgproxyrr IN A 10.0.0.1 Enabling Round Robin In the next steps you will define new A Host records for each VIP defined in the Gateway cluster. Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 10
34. Start the Content Gateway services on WCG-3. Verify Round Robin is enabled on the Domain Controller 35. On the domain controller VM, open DNS from Start > Administrative Tools > DNS 36. Right click on the domain controller name DC-WSG and click Properties 37. Choose the Advanced tab. 38. Ensure Enable Round Robin is checked Defining A Hosts 39. In the navigation tree on the left, drill down into the domain, expand Forward Lookup Zones, and select wstrain.com 40. Right click in the right frame and choose New Host (A) Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 11
41. Add the six VIPs selected for the WCG A Host records as wcgproxyrr.wstrain.com, one for each of the VIPs assigned to the Gatwway, i.e. 172.29.1.150 172.29.1.155 Testing Proxy Addressing via hostname 42. Configure XP-1 VM to proxy to wcgproxyrr.wstrain.com Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 12
43. Verify you can still browse the internet using the new proxy settings. 44. To verifying VIP and DNS RR are both working together, from a command prompt on the DC run, and leave the command running ping wcgproxyrr.wstrain.com t 45. This should resolve to one of the IP from one of the A Host IPs entered in the Forward Lookup Zone in the DNS server. 46. Open a browser on the DC, configuring the proxy details to proxy through wcgproxyrr.wstrain.com on the required ports for HTTP/HTTPS and verify you can browse the internet. 47. Establish the WCG that owns the VIP resolved from wcgproxyrr.wstrain.com on the DC, and on the WCG owning that VIP, use WCGAdmin to stop the WCG services 48. You should see connection drop, then eventually come back up. Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 13
49. Verify you can still browse from the DC, and use ifconfig to verify which Content Gateway now owns that IP 50. You have now verified that both DNS Round Robin and VIPs are working for the managed cluster. End of Lab Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy Page 14