By PANKAJ SHARMA. Concepts of Server Load Balancing

Transcription

1 Concepts of Server Load Balancing By PANKAJ SHARMA 1

2 Introduction of Load balancing and clustering with Liferay Load balancing is one of the most popular in the world due to its impressive ease-of-use. load balancing is techniques to distributed load on multiple system. Server Load Balancing (SLB) could mean many things, for the purpose of this book it is defined as a process and technology that distributes site traffic among several servers using a network-based device. This device intercepts traffic destined for a site and redirects that traffic to various servers. The load-balancing process is completely transparent to the end user. it s configured optimally for a multiple server environment. If one server isn t sufficient to serve the high traffic needs of your site, Liferay scales to the size you need. In Figure 1-1, we see the simplest representation of SLB. Apache server based Load Balancing: Simply add load balancing to your current multiple tomcat server configuration in order to achieve high availability and build your own web cluster without burning your entire hosting budget. Load balancing can be implemented quickly and easily as an add-on to your current server solution to share the load between your web servers, using a simple script to replicate the data on the server. A load balancer performs the following functions: Intercepts network-based traffic (such as web traffic) destined for a site. 2

3 Splits the traffic into individual requests and decides which servers receive individual requests. Maintains a watch on the available servers, ensuring that they are responding to traffic. If they are not, they are taken out of rotation. Provides redundancy by employing more than one unit in a fail-over scenario. Offers content-aware distribution, by doing things such as reading URLs, inter-cepting cookies, and XMLparsing. 3

4 . Figure 1-1 Apache server based Load Balancing Concept The Concept of Architecture Load Balancing: The objective here is that the solution should distribute the load among the servers in the cluster to provide the best possible response time to the end user. In a typical clustering solution, this involves use of a load distribution algorithm, like a simple round robin algorithm or more sophisticated algorithms, that distributes requests to the servers in the cluster by keeping track of the load and available resources on the servers. In shown figure1-2 4

5 5

6 Figure 1-2 Apache server based Architecture Load Balancing Types of Load Balancing : DNS-Based Load Balancing: With DNS round robin, it is possible to give multiple IP addresses to hostname,distributing traffic more or less evenly to the listed IP addresses. For instance, let's say you had three web servers with IP addresses of , , and that we wanted to SLB was a technology or a viable product, site administrators would (andsometimes still do)employ a load balancing process known as DNS round robin.dns round robin uses a function of DNS that allows more than one IP address to associate with a hostname. Every DNS entry has what is known as an A record,which maps a hostname to an IP address (such as In the Beginning ).Usually only one IP address is given for a hostname. Under ISO's DNS server, BIND 8, this is what the DNS entry for would look like: IN , share the load for the site The configuration in the DNS server for the three IP addresses would look like this: IN A IN A IN A

7 You can check the effect using a DNS utility known as look up, which would show the following for Server: ns1.vegan.net Address: Name: Addresses: , , The end result is that the traffic destined for is distributed between the three IP addresses listed.as shown in Figure 1-3. Figure 1-3 Traffic distribution by DNS-based load balancing 7

8 we have four front-end servers, {FE1, FE2, FE3, FE4}. The user retrieves this list of servers from DNS, and then randomly connects to one in the list. In our example, let s assume that the client connects to FE3. Upon connecting, the client presents its SIP URI from which a hash is generated by the front-end server. From this hash, the server determines the location of the registrar assigned to it. Note When a user is first enabled on (or moved to) a pool, a hash is generated to determine which front-end server is the primary registration database for the user, along with the order in which the remaining frontend servers will be attempted (as the backup registrar services). For our example, our user hash results in {FE4, FE2, FE1, FE3}. This is then interpreted as the order in which the clients will attempt to register. The client attempts to register with FE3, but because it s not the primary registrar assigned to the user, FE3 redirects the client to FE4 as the correct registrar to connect to. The client successfully registers with FE4. SLB has several benefits: Flexibility: SLB allows the addition and removal of servers to a site at any time, and the effect is immediate. Among other advantages, this allows for the maintenance of any machine, even during peak hours with little or no impact to the site. A load balancer can also intelligently direct traffic using cookies, URL parsing, static and dynamic algorithms, and much more. High availability: SLB can check the status of the available servers, take any non responding servers out of the rotation, and put them in rotation when they are functioning again. This is automatic, requiring no 8

9 intervention by an administrator. Also, the load balancers themselves usually come in a redundant configuration, employing more than one unit in case any one unit fails. Scalability: Since SLB distributes load among many servers, all that is needed to increase the serving power of a site is to add more servers. This can be very economical, since many small- to medium-sized servers can be much less expensive than a few high-end servers. Also, when site load increases, servers can be brought up immediately to handle the increase in traffic.load balancers started out as PC-based devices, and many still are, but now load balancing functions have found their way into switches and routers as well. firewall Load Balancing: Firewall load balancing balances traffic flows to one or more firewall farms. A firewall farm is a group of firewalls that are connected in parallel or that have their inside (protected) and outside (unprotected interfaces connected to common network segments. Firewall load balancing requires a load-balancing device (IOS SLB) to be connected to each side of the firewall farm. A firewall farm with inside and outside interfaces would then requir two loadbalancing devices, each making sure that traffic flows are directed toward the same firewall for the duration of the connection. Firewall load balancing is performed by computing a hash value of each new traffic flow (source and destination IP addresses and ports). This is called a route lookup. The firewall load-balancing device then masquerades as the IP 9

10 address for all firewalls in the firewall farm. Firewall load balancing can detect a firewall failure by monitoring probe activity. The HSRP can be used to provide a stateless backup redundancy for multiple firewall load-balancing devices. If one device fails, a redundant device can take over its function. Multiple firewall load-balancing devices can also use statefulbackup for redundancy. Backup devices keep state information dynamically and can take over immediately if a failure occurs. Figure 1-4 Firewall Load-Balancing Concept 10

11 What About Load Balancing: First, let's be clear on what "load balancing" is. Load balancing - a technique to distribute workload across resources. It is but one component in a high-availability cluster. Liferay Portal s case, we are load balancing the volume of requests across multiple app servers, which may or may not be on physically separate hardware. Initially, this may seem sufficient, until you realize some of the components that the portal uses. What is mod_jk: mod_jk is a replacement to the elderly mod_jserv. It is a completely new Tomcat-Apache plug-in that handles the communication between Tomcat and Apache. The mod_jk connector is an Apache HTTPD module that allows HTTPD to communicate with Apache Tomcat instances over the AJP protocol. The module is used in conjunction with Tomcat's AJP Connector component. shown in figure1-5. Now the configure is much easier and more consistent. ProxyPass /servlets ajp://tc.example.com:8089 Easier when Apache needs to proxy both HTTP and AJP. Leaverage improvements proxy module. 11

12 12

13 Concept About Connectors: Figure 1-5 Apache + mod_jk Balancing Apache Tomcat uses Connector components to allow communication between a Tomcat instance and another party, such as a browser, server, or another Tomcat instance that is part of the same network. For example, the HTTP connector listens for requests over the HTTP/1.1 protocol on various TCP ports, and forwards them to the Engine associated with processing the request. Using the AJP connector, Apache Tomcat instances can exchange data with mod_jk enabled instances of Apache HTTPD, using the AJP protocol. Implementations of mod_jk are also available for integration with IIS and NES/iPlanet/Sun, but are less widely used. About AJP: AJP, an acronymn for Apache Jserv Protocol, is a binary version of HTTP that is optimized for communication between Apache HTTPD and Apache Tomcat over a TCP connection. The current version of the AJP protocol is 1.3, referred to by the standard name ajp13. ajp13 extends the earlier mod_jserv and ajp12 modules, offering significant speed improvements and SSL support. Other than the data format, differences between the standard HTTP and AJP protocols include more persistent connections (to avoid unnecessary socket creation) and a focus on connection reuse over a series of request/response cycles. 1. Apache can now talk Ajp with Tomcat directly. 2. mod_proxy_ajp is magic mojo. 3. Other proxy improvements make this even more exciting. 4. Mod_jk alternative. 13

14 Apache Module mod_proxy: Apache has a mod_proxy module available for almost all versions of apache.however, prior to apache 2.2, only reverse proxy features were available and mod_proxy_balancer was not available for load balancing.this module implements a proxy/gateway for Apache. It implements proxying capability for AJP13 (Apache JServe Protocol version 1.3), FTP, CONNECT (for SSL), HTTP/0.9, HTTP/1.0, and HTTP/1.1. The module can be configured to connect to other proxy modules for these and other protocols. Apache's proxy features are divided into several modules in addition to mod_proxy: mod_proxy_http, mod_proxy_ftp, mod_proxy_ajp, mod_proxy_balancer, and mod_proxy_connect. Thus, if you want to use one or more of the particular proxy functions, load mod_proxy and the appropriate module(s) into the server (either statically at compile-time or dynamically via the LoadModule directive).shown in figure1-6 Documentation for mod_proxy is available for: 1. Apache Apache Apache Apache 2.2 The proxy module (and other proxy extension used) must be loaded: LoadModule proxy_module modules/mod_proxy.so 14

15 15

16 Configure mod_proxy: Figure1-6 Apache + mod_proxy Concept Now enable mod_proxy in Apache, and proxy requests to the application server by adding the example below to your Apache httpd.conf (note: the files may be different on your system; the JIRA docs describe the process for Ubuntu/Debian layout): # Put this after the other LoadModule directives LoadModule proxy_module/usr/lib/apache2/modules/mod_proxy.so LoadModule proxy_http_module/usr/lib/apache2/modules/mod_proxy_http.so # Put this in the main section of your configuration (or desired virtual host, if using Apache virtual hosts) ProxyRequests Off ProxyPreserveHost On 16

17 <Proxy *> </Proxy> Order deny,allow Allow from all ProxyPass /confluence ProxyPassReverse /confluence <Location /confluence> Order allow,deny Allow from all </Location> Apache Mod]ule mod_proxy_ajp: mod_proxy_ajp is an Apache module which can be used to forward a client HTTP request to an internal Tomcat application server using the AJP protocol. This module requires the service of mod_proxy. It provides support for the Apache JServ Protocol version 1.3 (hereafter AJP13). Thus, in order to get the ability of handling AJP13 protocol, mod_proxy and mod_proxy_ajp have to be present in the server. This module is used to reverse proxy to a backend application server (e.g. Apache Tomcat) using the AJP13 protocol. The usage is similar to an HTTP reverse proxy, but uses the ajp:// prefix: 17

18 Simple Reverse Proxy: <Proxy balancer://cluster> BalancerMember ajp://app1.example.com:8009 loadfactor=1 BalancerMember ajp://app2.example.com:8009 loadfactor=2 ProxySet lbmethod=bytraffic </Proxy> ProxyPass /app balancer://cluster/app Tomcat configuration: You just have to create the AJP connector in the conf/server.xml file like that: If you are just running Tomcat standalone you would define a connector on port 8080 like this: <!-- Define an AJP 1.3 Connector on port > <Connector port="8009" enablelookups="false" redirectport="8443" protocol="ajp/1.3" URIEncoding="UTF-8" /> However we want to run Tomcat on an AJP connector, so: 1. To directory of server.xml file configuration Ubuntu path :>liferay-portal ce-ga1/tomcat /conf/server.xml 2. Comment out the normal connector above, and uncomment the AJP connector: <!-- Define an AJP 1.3 Connector on port > <Connector port="8009" 18

19 enablelookups="false" redirectport="8443" protocol="ajp/1.3" URIEncoding="UTF-8" /> 3. Adjust the port if you wish and add in the URIEncoding="UTF-8" attribute. 4. Start Tomcat normally. Configure Apache: We now need to tell Apache to talk to Tomcat over AJP. You should note that since in ubuntu OS.the Apache config and modules have been relocated.. This guide is for Ubuntu and for Apache 2.2. Also note that in Ubuntu, the necessary modules are already installed into /usr/libexec/apache2. Thanks Apple! 1. Navigate to your Apache directory: ubuntu path :</etc/apache2/ httpd.conf 2. Open httpd.conf and scroll to the large LoadModule section. Ensure you have the following uncommented: LoadModule proxy_module libexec/apache2/mod_proxy.so LoadModule proxy_ajp_module libexec/apache2/mod_proxy_ajp.so 3. In my httpd.conf, right at the bottom there is a line: In my httpd.conf, right at the bottom there is a line: 4. Create an ajp.conffile. You could put the configuration for AJP in the main httpd.conf file but I prefer to keep things separated. 19

20 touch other/ajp.conf 5. Open ajp.confand paste in the following: ProxyRequests Off <Proxy *> Order deny,allow Deny from all Allow from localhost </Proxy> ProxyPas/ ajp://localhost:8009/ ProxyPassReverse / ajp://localhost:8009/ This will forward all requests to Tomcat. You can optionally pass a list of contexts that you want forwarded, like so: ProxyRequests Off ProxyPass /somecontext ajp://localhost:8009/somecontext ProxyPassReverse/somecontextajp://localhost:8009/somecontext Adjust the port to be whatever the port is in your Tomcat AJP connector. Save and close. 6. Once again, ensure you have the line in httpd.conf that is going to load this ajp.conf file. 20

21 7. Restart Apach2e: 8. Open your terminal and type the bellow command 9. sudo /etc/init.d/apache2 start 10. sudo /etc/init.d/apache2 restart 11. sudo /etc/init.d/apache2 stop 12. sudo /etc/init.d/apache2 reload 13. #sudo/home/pankaj/pankajsharma/softwares/liferay/apache _server_cluster/liferay-portal sh.catalina.sh run Apache Module mod_proxy_balancer: This module requires the service of mod_proxy. It provides load balancing support for HTTP, FTP and AJP13 protocols Thus, in order to get the ability of load balancing, mod_proxy and mod_proxy_balancer have to be present in the server. The module mod_proxy_balancer implements stickyness on top of two alternative means: cookies and URL encoding. Providing the cookie can be either done by the back-end or by the Apache web server itself. The URL encoding is usually done on the back-end. Shown in figure1-7 With apache 2.2 mod_proxy is able to use the extension mod_proxy_balancer. Configuraion: The configuration of mod_proxy_balancer is similar to pure mod_proxy, except that balancer:// URLs may be used as a protocol instead of 21

22 when specifying destinations (workers) in ProxyPass element # map to cluster with session affinity (sticky sessions) ProxyPass /balancer! <Proxy balancer://mycluster> BalancerMember ajp://localhost:8009/ route=jvm1 BalancerMember ajp://localhost:8010/ route=jvm2 </Proxy> <Location / ProxyPass balancer://mycluster/ stickysession=jsessionid nofailover=on </Location> 22

23 Figure1-7 Apache + mod_proxy_balancer Concept How Do You Know You re Done? Liferay clustering is not just pointing a load balancer to two(or more) Liferay nodes.you are not done.why? Because there are certain application-level components that need to be either centrally managed or synchronized. Let s go through a checklist. Configuring multiple Tomcat application servers with Apache HTTP server/ Load Balancing with Apache WebServer: we can configure multiple Tomcat instances with single Apache Web Server. There are many solutions posted on the web, but it is the simplest and easiest way to do this. 23

24 Prerequisite: Apache HTTP Serve 2. Tomcat 6.1.x mod_jk.so module for Apache Web server Any text editor, i.e. Notepad, etc. We will required to two tomcat server 1 and tomcat server 2 fo version 6.1.x if you do not have Apache HTTP server goto and download the appropriate version and install on your machine. if you do not have Apache Tomcat Server goto and download the appropriate version and install on your machine. if you do not have mod_jk.so module, goto and download the appropriate connector for Apache Web Server. Given below steps follows : Step 1: Configure Apache 2.x: Httpd.conf configuration: Path in ubuntu:->/etc/apache2/httpd.conf Path in windows:->apache-home/apache2.2/conf/httdp.conf Add below code ServerName localhost LoadModule proxy_module modules/mod_proxy.so 24

25 LoadModule proxy_ajp_module modules/mod_proxy_ajp.so LoadModule proxy_balancer_module modules/mod_proxy_balancer.so LoadModule status_module modules/mod_status.so <VirtualHost *:80> ServerName localhost ServerName 1localhost ProxyRequests Off <Proxy *> Order deny,allow Allow from all </Proxy> ProxyPass / ajp://localhost:8080/ ProxyPass / ajp://localhost:8080/ ProxyPassReverse / ajp://localhost:8080/ ProxyPassReverse / ajp://localhost:8080/ </VirtualHost> <Proxy balancer://mycluster> 25

26 BalancerMember ajp://localhost:8009/ route=jvm1 BalancerMember ajp://localhost:8010/ route=jvm2 </Proxy> <Location /> ProxyPass balancer://mycluster/ stickysession=jsessionid </Location> Step 2: Configure Tomcat: you have to configure each tomcat instance for different port to run successfully. To configure Tomcat s port, goto conf directory of any tomcat instance you have and open the server.xml file in the text editor. In Tomcat-server1: Path:->home-LifeRay/Apache_server_cluster/Tomcat-server1/liferay-portal ce- ga1/tomcat/conf Every liferay-tomcat's server.xml files change the ports. Given At localhost change the port below shutdown : 8005 <Server port="8005" shutdown="shutdown"> connector : 8081 <Connector URIEncoding="UTF-8" connectiontimeout="20000" port="8081" protocol="http/1.1" redirectport="8445"/> 26

27 redirectport : 8445 ajp-connector: 8009 Set jvmroute to support load-balancing via AJP ie. <Engine name="catalina" defaulthost="localhost"> so it should look like this <Engine name="catalina" defaulthost="localhost" jvmroute="tomcat1"> Add to enablelookups="false" at AJP 1.3 Connector on port <Connector port="8009" protocol="ajp/1.3" redirectport="8445" URIEncoding="UTF-8" enablelookups="false"/> Uncomment below line for clustering tag. <Cluster classname="org.apache.catalina.ha.tcp.simpletcpcluster" /> In Tomcat-server2: Given At localhost change the port below shutdown :8006 <Server port="8006" shutdown="shutdown"> connector :8082 Connector URIEncoding="UTF-8" connectiontimeout="20000" port="8082" protocol="http/1.1" redirectport="8446"/> redirectport : 8446 ajp-connector :8010 Set jvmroute to support load-balancing via AJP ie 27

28 <Engine name="catalina" defaulthost="localhost"> so it should look like this <Engine name="catalina" defaulthost="localhost jvmroute="tomcat2"> Add to enablelookups="false" at AJP 1.3 Connector on port 8009 <Connector port="8009" protocol="ajp/1.3" redirectport="8443" URIEncoding="UTF- 8"enableLookups="false"/> Uncomment below line for clustering tag <Cluster classname="org.apache.catalina.ha.tcp.simpletcpcluster" /> Step 3:Creating configuration file of Tomcat instances for Apache Web Server now open text editor and create a file called workers.properties and enter the information of all tomcat instances you have. and save it under the conf directory of Apache. A sample worker.properties is given: # lists the workers by name worker.list=tomcat1, tomcat2, loadbalancer #Tomcat1 configuration worker.tomcat1.port=8009 //AJP1.3 Connector s port worker.tomcat1.host=localhost //IP of machine where tomcat1 is running worker.tomcat1.type=ajp13 //AJP Connector type. worker.tomcat1.lbfactor=100 //Load Balancing factor 28

29 #Tomcat2 configuration worker.tomcat2.port=8010 //AJP1.3 Connector s port worker.tomcat2.host=localhost //IP of machine where tomcat1 is running worker.tomcat2.type=ajp13 //AJP Connector type. worker.tomcat2.lbfactor=100 //Load Balancing factor #Load Balance worker configuration worker.loadbalancer.type=lb worker.loadbalancer.balanced_workers=tomcat1,tomcat2 Step 4:Configure JK module: Copy the downloaded JK module file (mod_jk.so) to the modules directory of Apache Web Server. now goto conf directory of Apache Server, and open the httpd.conf file in the text editor. and add these lines to the end of httpd.conf LoadModule jk_module modules/mod_jk.so JkWorkersFile conf/workers.properties JkLogFile logs/mod_jk.log kloglevel info now you have done all the configuration, now you need to configure the JK Module to redirect the requests to specific tomcats. It can be done by the following: To redirect specific request to specific Tomcat add this line to httpd.conf: JkMount /abc/* tomcat1 or JkMount /*.do tomcat2 Or, if you want to configure requests to be served by any of the tomcat then add 29

30 this: JkMount /*.jsp loadbalancer if any of the tomcat is stopped then all the request will be served by other tomcat and if both the tomcat are running then the requests are redirected to tomcat servers according to the lbfactor specified in the workers.properties. Now start all the Tomcat Servers and Apache Server, and its done. Step 5 To configuratin of Web.xml file: The directory of web.xml file Ubuntu path:->liferay-home/tomcat /webapps/root/web- INF/wex.xml To Insert tag <distributable/> before <filter> tag into web.xml file. To insert tag <distributable/> in both Liferay Apache tomcat server. Step 6 Catalina.sh or Catalina.bat Config : Add these below lines into Catalina.sh file. Add these below lines into Catalina.bat file for windows OS at starting. These below lines are add In both Liferay Apache tomcat server. JAVA_OPTS="-Dsli.env=ci -Djava.net.preferIPv6Addresses=true -Djava.net.preferIPv4Stack=true 30

31 -Djava.net.preferIPv6Stack=true -Xms128m -Xmx512m" Step 7 To start apache, tomcata, tomcatb and navigate to : Open your terminal and type the below commands such as sudo /etc/init.d/apache2 start sudo /etc/init.d/apache2 restart 31

32 Checklist (Logical View): Load Balancer it can be software (i.e. - Apache), or hardware (F5), orwhatever you wish, really. All it is doing is redirecting requests. Centralized Database- Hopefully, you have gotten off of HSQL and are using a real DB server. This is a JBDC connection. It is abstracted from Liferay's point of view. Any level of redundancy you have behind that JDBC connection is up to you and your DBA. Just as an example, you may choose to configure a MySQL cluster, or Oracle RAC, for DB high availability. 32

33 Ehcache This is what Liferay uses out-of-the-box for it's Hibernate level 2 cache. This needs to be configured to sync, else you will see inconsistencies depending on what node the load balancer redirects end users to. You are not forced to use Ehcache, it is simply what it ships with. You could use something like Terracotta, for example. If you do not do this, you will most definitely see inconsistencies depending on the node the end user is on, due to stale caches. Lucene This needs to be centralized. This can be done: a) via JDBC (canwork, but there may be issues with speed and table locks), b) swapped out for something like SOLR (runs as a webapp in Tomcat), or c) using the cluster link feature that can be turned on where each node maintains its own replicated cache. If you do not do this, you will see inconsistencies with search and other indexed data that is returned from the DB. Document Library This needs to be centralized. This is because each node keeps the assets locally on the file system by default. While the meta-data is in the DB, the files serve up faster this way (vs. BLOBS in a DB). So,you need to either a) point the content repository to be stored in the DB (can work but performance may suffer) via JCRHook in portal properties, or b) mount a path a shared storage (i.e. SAN or other file system storage) and configure each node to connect to this common place via AdvancedFileSystemHook in portal properties. If you do not do this, the meta-data regarding your documents will be in the DB, but when you try to retrieve them, the node may or may not find the document there, physically. 33

34 Checklist (Detailed File View): 34

35 How Do I Know It s Working?: A quick test: 1. Bring up the load balancing 2. Bring up Node 1 3. Bring up Node 2 35

36 4. Open Browser 1 and go to Node 1 directly (i.e On Browser 1 go to page 1 (i.e. in as Admin 6. Open Browser 2 (different browser session) and go to Node 2 directly (i.e- 7. On Browser 2 go to page 1 (i.e. logged in as Admin 8. In Browser 1 add a portlet to the page 9. Shut down Node Go to Browser 2 and refresh the page (i.e. CTRL F5) Quick Test Result: If Browser 2 refreshes and shows the added portlet frombrowser 1, Liferay clustering in most likely configured correctly. This is because this behavior demonstrates that not only the load balancer is redirected to the live node, but that cache replication is occurring between nodes. 36

37 37