High Availability and Load Balancing for Basic Dialogic PowerMedia Extended Media Server (XMS) Configurations Using the Linux Virtual Server Framework
Introduction As more and more critical telecom applications move to IP based networks, providing server applications that can be reliably utilized with a minimum of down-time becomes crucial. There are a number of drawbacks to a single server solution: Downtime that occurs if there is a failure; An upgrade to a higher end server is required when the load exceeds the maximum processing capacity of the original server, with the upgrading/replacement process often complex; and The cost of the server is not linear with its performance - a server that is twice as fast would cost more than twice as much. On the other hand, clusters of servers, connected by a fast network, are emerging as a viable architecture for building high-performance and highly available applications. A cluster of inexpensive servers provide a number of benefits. For example, this type of loose-coupled architecture is more scalable, more cost-effective and more reliable than a single processor system or a tightly coupled multiprocessor system. There are challenges, however, in providing transparency, scalability and high availability of the parallel services in the cluster. This document provides a guide to configuring a pool of PowerMedia XMS servers in a high available/load balancing configuration by using the Linux Virtual Servers framework present in most Linux kernels, along with the open source Keepalived package. Basic Configuration The Keepalived package and Linux Virtual Server framework permit highly complex configurations, allowing a variety of failover and load balancing scenarios. This document, however, will be limited to a single Virtual Server (LoadBalancer) and a pool of PowerMedia XMS servers in a Network Address Translation (NAT) configuration, such as this example: 2
In this configuration, a new SIP call (INVITE) comes in on the virtual server external interface. The virtual server chooses one of the available PowerMerdia XMS servers in the pool to process the call. All subsequent SIP packets for the same call are directed to the same PowerMedia XMS server. The virtual server also periodically polls each of the PowerMedia XMS servers to verify they are operational. If one is not operational, it is removed from the pool of available PowerMedia XMS servers. When it becomes available again, it will be inserted back in the pool. Characteristics and Limitations of this Configuration There are a number of considerations to keep in mind when using Virtual Server with PowerMedia XMS: This configuration works only for SIP-based control setups (MSML, VXML, NETANN, JSR309). A PowerMedia XMS RESTful interface use case is not covered by this document. The Virtual Server recognizes only SIP using UDP for transport. SIP over TCP is not supported. 3
The PowerMedia XMS servers and the internal Virtual Server interface must be on the same IP subnet. The returning SIP packets from PowerMedia XMS servers must pass through the Virtual Server. This means that the default gateway for the PowerMedia XMS server must be the Virtual Server. If using policy based routing, only SIP traffic needs to pass through the Virtual Server. The Virtual Server is a single point of failure in this configuration. In more complex configurations, it is possible to configure a cluster of Virtual Servers sharing the same IP address. One Virtual Server can then take over from the other. SIP Session Timers must be used on all SIP calls to maintain the SIP connection table in the Virtual Server in a correct state. By using SIP Session timers, SIP user agents will periodically poll the PowerMedia XMS Server, and, in doing so, maintain the persistent SIP connection table in the Virtual Server. Failing to do so will risk the next SIP packet not being forwarded to the correct PowerMedia XMS server. Note that this is comparable to configuring UDP persistence timeouts in firewall/nat/ipsec configurations. This example configuration works only when different PowerMedia XMS servers function as autonomous systems. Typical IVR applications fall under that category. Conferencing and hairpinning between connections are not possible with this configuration, since these could span more than one PowerMedia XMS server. Software Installation Server Installation The components of the Linux Virtual Server framework are included in standard Linux distributions with kernel versions at or greater than 2.6.38. CentOS and RHEL versions 6.4 use 2.6.32.xxxx kernels. This makes them unsuitable for the Virtual Server system unless kernel patches are added and recompilation is done. Recent (June 2013) versions of the Ubuntu server are using kernels greater than 3.x. They include the kernel components to support SIP in Virtual Server configurations. As of May 2013, Ubuntu could be downloaded from http://www.ubuntu.com/server. Secondary Software The Linux Virtual Server administration utility is used to set up, maintain or inspect the Virtual Server table in the Linux kernel. Install the following ipvsadm package via the following commands: sudo apt-get install ipvsadm 4
SIPp is used to monitor the state of the different PowerMedia XMS servers. Install it via: sudo apt-get install SIP-tester Keepalived is the framework that manages the virtual server and monitors the health of the PowerMedia XMS servers via SIPp. The Keepalived package doesn t include support for SIP nor the recent features provided by the Linux Virtual Server. The Keepalived package that accompanies this document XMS-LVS_Material - was modified to include SIP support. Uncompress and execute on the command line: > make install sysv-rc-conf permits control of which Ubuntu service is started at boot time. It is the equivalent of the RHEL/Centos/Fedora ntsysv. sudo apt-get install sysv-rc-conf Finally, the popt library is required by Keepalived: sudo apt-get install libpopt sudo apt-get install libpopt-dev Configuration keepalived.conf Below is a sample Keepalived configuration file:! Configuration File for keepalived virtual_server 192.168.49.16 5060 { delay_loop 120 lb_algo rr lb_kind NAT nat_mask 0.0.0.0 persistence_timeout 60 protocol UDP pe SIP real_server 192.168.49.7 5060 { weight 1 MISC_CHECK { 5
misc_path "/home/xms/script.sh 192.168.49.7 5080" real_server 192.168.49.17 5060 { weight 1 MISC_CHECK { misc_path "/home/xms/script.sh 192.168.49.17 5100" The parameters of note are: virtual_server 192.168.49.16 5060: Definition of the Virtual Server, its IP address and the UDP/TCP port it is listening to. delay_loop 120: The number of seconds between two (2) polls to check if the PowerMedia XMS servers are alive. persistence_timeout 60 The time, in seconds, that the connection will remain in the Virtual Server SIP connection table. If no SIP message is sent on that connection during that time, the connection will be removed. It is important to choose that value carefully, and to correlate it with the value of the SIP session timers that keep the SIP connection alive. real_server 192.168.49.17 5060: The definition of the real PowerMedia XMS servers performing the work. Their IP address and the UDP port used by SIP are needed. misc_path "/home/xms/script.sh 192.168.49.17 5100" This is the path to the SIPp script that is being used to validate the status of the PowerMedia XMS sever. Two parameters are needed - the IP address of the PowerMedia XMS server and the UDP port that SIPp script instance will use. Note: it is important that each real server definition use a different port. If two SIPp instances share the same UDP port, unpredictable failures may occur. PowerMedia XMS Server Configuration The only configuration that needs to be done on the PowerMedia XMS servers is to have the Virtual Server IP address set as the default gateway. This is done on the Network screen in the PowerMedia XMS Web GUI. Launching the Keepalived Daemon and Related System Configuration To test the configuration, launch the Keepalived daemon in the foreground with all logging redirected to the console. Use the following command line: /usr/local/sbin/keepalived -n -f /etc/keepalived/keepalived.conf -d l 6
Once it is confirmed that your configuration is functioning correctly, you can launch the Keepalived daemon with the system initialization scripts Verifying the Installation The first step in verifying the installation is to make sure that incoming SIP calls are evenly distributed among the pool of available PowerMedia XMS servers. tshark or tcpdump may be used to obtain a packet capture: tshark/tcpdump i eth0 w output.pcap port 5060 This will collect the SIP messaging only. The resulting file can be viewed using the GUI-based Wireshark utility. If all PowerMedia XMS servers are up, the following scenario can be expected: 1. Make a SIP call to the Linux Virtual Server 2. Once the call is answered, disconnect the call 3. On inspection, all the SIP packets should be forwarded to the same PowerMedia XMS server 4. Repeat steps 1-3, and verify that for each call another PowerMedia XMS server is selected in a round robin fashion A scenario where a PowerMedia XMS server goes down can be simulated as follows: Disconnect or power off one of the PowerMedia XMS servers in the pool Repeat steps 1-4 and verify that the disconnected PowerMedia XMS server is not used to answer any of the calls Reconnect/power on the disconnected PowerMedia XMS server Repeat steps 1-4 and verify that the re-connected XMS server is used again to answer calls Further Exploration Performance and Non-NATed Configurations It may be desirable to achieve the high performance possible with this configuration, while avoiding having the Virtual Server act as a general-purpose router/nating machine. This can be done by configuring policy based routing for SIP. Doing so uses the Virtual Server as default gateway for SIP only, while the default gateway/router is used for the rest of the traffic. 7
Another possibility is to use the direct routing configuration of the Virtual Server. In such a configuration, incoming SIP packets pass through the Virtual Server, while outgoing SIP packets leaving the PowerMedia XMS servers do not. See the Virtual Server documentation for configuration and setup. Pool of Virtual of Servers Using Virtual Router Redundancy Protocol In the configuration presented in this document, the Virtual Server/load balancer is a single point of failure; all the signaling has to pass through it. If the Virtual Server fails for some reason, the whole configuration fails. KeepAlived lets you configure a pool of Virtual Servers/load balancers. Whenever a Virtual Server in the pool fails, another Virtual Server takes over. This is done transparently to the user, without downtime. To accomplish this, KeepAlived uses the virtual router redundancy protocol and connection table replication of the Virtual Servers across the pool of servers. References http://www.linuxvirtualserver.org http://www.keepalived.org/index.html http://www.ubuntu.com/server Open Source This article discusses one or more open source products, systems and/or releases. Dialogic is not responsible for your decision to use open source in connection with Dialogic products (including without limitation those referred to herein), nor is Dialogic responsible for any present or future effects such usage might have, including without limitation effects on your products, your business, or your intellectual property rights. 8
RFP www.dialogic.com Dialogic Inc 1504 McCarthy Boulevard Milpitas, California 95035-7405 USA Copyright 2013 Dialogic Inc. All Rights Reserved. You may not reproduce this document in whole or in part without permission in writing from Dialogic Inc. at the address provided below. All contents of this document are furnished for informational use only and are subject to change without notice and do not represent a commitment on the part of Dialogic Inc. and its affiliates or subsidiaries ( Dialogic ). Reasonable effort is made to ensure the accuracy of the information contained in the document. However, Dialogic does not warrant the accuracy of this information and cannot accept responsibility for errors, inaccuracies or omissions that may be contained in this document. INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH DIALOGIC PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN A SIGNED AGREEMENT BETWEEN YOU AND DIALOGIC, DIALOGIC ASSUMES NO LIABILITY WHATSOEVER, AND DIALOGIC DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF DIALOGIC PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY INTELLECTUAL PROPERTY RIGHT OF A THIRD PARTY. Dialogic products are not intended for use in certain safety-affecting situations. Please see http://www.dialogic.com/company/terms-of-use.aspx for more details. Due to differing national regulations and approval requirements, certain Dialogic products may be suitable for use only in specific countries, and thus may not function properly in other countries. You are responsible for ensuring that your use of such products occurs only in the countries where such use is suitable. For information on specific products, contact Dialogic Inc. at the address indicated below or on the web at www.dialogic.com. It is possible that the use or implementation of any one of the concepts, applications, or ideas described in this document, in marketing collateral produced by or on web pages maintained by Dialogic may infringe one or more patents or other intellectual property rights owned by third parties. Dialogic does not provide any intellectual property licenses with the sale of Dialogic products other than a license to use such product in accordance with intellectual property owned or validly licensed by Dialogic and no such licenses are provided except pursuant to a signed agreement with Dialogic. More detailed information about such intellectual property is available from Dialogic s legal department at 6700 de la Cote-de-Liesse Road, Suite 100, Borough of Saint- Laurent, Montreal, Quebec, Canada H4T 2B5. Dialogic encourages all users of its products to procure all necessary intellectual property licenses required to implement any concepts or applications and does not condone or encourage any intellectual property infringement and disclaims any responsibility related thereto. These intellectual property licenses may differ from country to country and it is the responsibility of those who develop the concepts or applications to be aware of and comply with different national license requirements. Dialogic, Dialogic Pro, Dialogic Blue, Veraz, Brooktrout, Diva, BorderNet, PowerMedia, ControlSwitch, I-Gate, Mobile Experience Matters, Network Fuel, Video is the New Voice, Making Innovation Thrive, Diastar, Cantata, TruFax, SwitchKit, Eiconcard, NMS Communications, SIPcontrol, Exnet, EXS, Vision, incloud9, NaturalAccess and Shiva, among others as well as related logos, are either registered trademarks or trademarks of Dialogic Inc. and its affiliates or subsidiaries. Dialogic's trademarks may be used publicly only with permission from Dialogic. Such permission may only be granted by Dialogic s legal department at 6700 de la Cote-de-Liesse Road, Suite 100, Borough of Saint-Laurent, Montreal, Quebec, Canada H4T 2B5. Any authorized use of Dialogic's trademarks will be subject to full respect of the trademark guidelines published by Dialogic from time to time and any use of Dialogic s trademarks requires proper acknowledgement. The names of actual companies and products mentioned herein are the trademarks of their respective owners. This document discusses one or more open source products, systems and/or releases. Dialogic is not responsible for your decision to use open source in connection with Dialogic products (including without limitation those referred to herein), nor is Dialogic responsible for any present or future effects such usage might have, including without limitation effects on your products, your business, or your intellectual property rights. 05/13