This howto is also a bit old now. But I thought of uploading it in the howtos section, as it still works.

Transcription

1 Assalam-u-alaikum, This howto is also a bit old now. But I thought of uploading it in the howtos section, as it still works. Created : Mid 2007 Last updated: Mid 2007 The following link is very nice tutorial : but uses Debian! I have used CentOS 4.3 and all heartbeat and ipvs packages from centos43 extras. Nothing needs to be downloaded from ANYWHERE. No need to go to Ultra monkey site, etc. Make sure that example.com has proper Zone entries on your DNS server for: www1.example.com. IN A www2.example.com. IN A lb1.example.com. IN A lb2.example.com. IN A IN A lb1.example.com lb1 HOSTNAME=lb1.example.com GATEWAY= IPADDR= NETMASK= / 15

2 DEVICE=eth0 BOOTPROTO=static HWADDR=00:40:26:5F:5C:60 ONBOOT=yes TYPE=Ethernet IPADDR= NETMASK= both Webservers mkdir /media/nfs ; mkdir /media/extras mount -t nfs :/data/cdimages/centos43nfs /media/nfs echo "mount -t nfs :/data/cdimages/centos43nfs /media/nfs" >> /etc/rc.local mount -t nfs :/data/cdimages/centos43extrasnfs /media/extras/ echo "mount -t nfs :/data/cdimages/centos43extrasnfs /media/extras/" >> /etc/rc.local rpm -ivh httpd ent.centos4.i386.rpm apr c4.1.i386.rpm httpd-suexec ent.centos4.i386.rpm apr-util i386.rpm Load Balancers: /etc/modules does not exist on RedHAT and clones. On both load balancers (also placed the same in /etc/rc.local) /etc/modprobe.conf can be used but I do not know how to use it, so /etc/rc.local is the easy way out. modprobe ip_vs_dh modprobe ip_vs_ftp modprobe ip_vs modprobe ip_vs_lblc modprobe ip_vs_lblcr modprobe ip_vs_lc modprobe ip_vs_nq modprobe ip_vs_rr modprobe ip_vs_sed modprobe ip_vs_sh modprobe ip_vs_wlc modprobe ip_vs_wrr / 15

3 both LBs rpm -ivh curl rhel4.i386.rpm perl-crypt-ssleay i386.rpm perl-html-parser i386.rpm perl-ldap noarch.rpm perl-net-dns i386.rpm perl-libwww-perl noarch.rpm libidn i386.rpm perl-convert-asn noarch.rpm perl-digest-hmac noarch.rpm perl-html-tagset noarch.rpm perl-uri noarch.rpm perl-xml-sax noarch.rpm perl-digest-sha i386.rpm perl-xml-namespacesupport noarch.rpm rpm -ivh perl-mail-imapclient rf.noarch.rpm heartbeat-* ipvsadm i386.rpm Install Heartbeat packages vi /etc/sysctl.conf # Enables packet forwarding net.ipv4.ip_forward = 1 sysctl -p USE "UNAME -n" for node names vi /etc/ha.d/ha.cf logfacility local0 bcast eth0 # Linux mcast eth auto_failback off node lb1.example.com node lb2.example.com respawn hacluster /usr/lib/heartbeat/ipfail apiauth ipfail gid=haclient uid=hacluster USE "UNAME -n" for node names LB1: 3 / 15

4 vi /etc/ha.d/haresources lb1.example.com ldirectord::ldirectord.cf LVSSyncDaemonSwap::master IPaddr2:: /24/eth0/ LB2: lb2.example.com ldirectord::ldirectord.cf LVSSyncDaemonSwap::master IPaddr2:: /24/eth0/ both LBs vi /etc/ha.d/authkeys auth 3 3 md5 redhat both LBs chmod 600 /etc/ha.d/authkeys both LBs vi /etc/ha.d/ldirectord.cf checktimeout=10 checkinterval=2 autoreload=no logfile="local0" quiescent=yes virtual= :80 real= :80 gate real= :80 gate fallback= :80 gate service=http request="ldirector.html" receive="test Page" scheduler=rr 4 / 15

5 protocol=tcp checktype=negotiate Both LBs chkconfig --level 35 ldirectord off chkconfig --level 35 heartbeat on service ldirectord stop service heartbeat start service ldirectord stop Stopping ldirectord [ OK ] service heartbeat start Starting High-Availability services: ldirectord is stopped for /etc/ha.d/ldirectord.cf [FAILED] heartbeat: 2007/02/16_04:12:30 ERROR: Current node [lb1.example.com] not in configuration! heartbeat: 2007/02/16_04:12:30 info: By default, cluster nodes are named by `uname -n` and must be declared with a 'node' directive in the ha.cf file. heartbeat: 2007/02/16_04:12:30 ERROR: Configuration error, heartbeat not started. uname -n lb1.example.com OHHHHH Files corrected with uname -n output. i.e. instead of lb1, I have now changed that to lb1.example.com service heartbeat start Starting High-Availability services: ldirectord is stopped for /etc/ha.d/ldirectord.cf [ OK ] [root@lb2 /]# service ldirectord stop Stopping ldirectord [ OK ] [root@lb2 /]# service heartbeat start Starting High-Availability services: 5 / 15

6 ldirectord is stopped for /etc/ha.d/ldirectord.cf [ OK ] [root@lb2 /]# Good. Alhumdulilah. LB1: ip addr sh eth0 2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:08:c7:69:13:9d brd ff:ff:ff:ff:ff:ff inet /24 brd scope global eth0 inet /24 brd scope global secondary eth0 inet6 fe80::208:c7ff:fe69:139d/64 scope link valid_lft forever preferred_lft forever Notice LB1 has two IPs, which DO NOT SHOW in ifconfig -a output. LB2: [root@lb2 /]# ip addr sh eth0 2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:8b:60:c8:f8 brd ff:ff:ff:ff:ff:ff inet /24 brd scope global eth0 inet6 fe80::250:8bff:fe60:c8f8/64 scope link valid_lft forever preferred_lft forever [root@lb2 /]# ldirectord ldirectord.cf status ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 3978 NOTE: The above output (running) will be displayed on ACTIVE LB [root@lb2 /]# ldirectord ldirectord.cf status ldirectord is stopped for /etc/ha.d/ldirectord.cf [root@lb2 /]# / 15

7 ipvsadm -L -n IP Virtual Server version (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn /]# ipvsadm -L -n IP Virtual Server version (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn /]# NOTE: If LB1 does not show as below, try restarting heartbeat on LB1 and LB2. In the same order I have written here. ipvsadm -L -n IP Virtual Server version (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP :80 rr -> :80 Route > :80 Route > :80 Local /etc/ha.d/resource.d/lvssyncdaemonswap master status master running (ipvs_syncmaster pid: 4704) NOTE: Master running on LB1 [root@lb2 /]# /etc/ha.d/resource.d/lvssyncdaemonswap master status master stopped (ipvs_syncbackup pid: 4488) [root@lb2 /]# WWW1 and WWW2 7 / 15

8 vi /etc/sysctl.conf # When an arp request is received on eth0, only respond if that address is # configured on eth0. In particular, do not respond if the address is # configured on lo net.ipv4.conf.eth0.arp_ignore = 1 # Ditto for eth1, add for all ARPing interfaces #net.ipv4.conf.eth1.arp_ignore = 1 # Enable configuration of arp_announce option net.ipv4.conf.all.arp_announce = 2 # When making an ARP request sent through eth0 Always use an address that # is configured on eth0 as the source address of the ARP request. If this # is not set, and packets are being sent out eth0 for an address that is on # lo, and an arp request is required, then the address on lo will be used. # As the source IP address of arp requests is entered into the ARP cache on # the destination, it has the effect of announcing this address. This is # not desirable in this case as adresses on lo on the real-servers should # be announced only by the linux-director. net.ipv4.conf.eth0.arp_announce = 2 # Ditto for eth1, add for all ARPing interfaces #net.ipv4.conf.eth1.arp_announce = 2 sysctl -p on BOTH WWWs Note : /etc/network/interfaces exists in debian only it does not exist in RedHat. For that you need to use the /etc/sysconfig/network-scripts/ifcfg-* files. DEBIAN vi /etc/network/interfaces auto lo:0 iface lo:0 inet static address netmask pre-up sysctl -p > /dev/null / 15

9 WWW1 and WWW2 REDHAT / CentOS / Fedora cd /etc/sysconfig/network-scripts/ cp ifcfg-lo ifcfg-lo:0 vi ifcfg-lo:0 DEVICE=lo:0 BOOTPROTO=static IPADDR= NETMASK= ONBOOT=yes NAME=loopback both WWWs [root@www1 network-scripts]# ifconfig eth0 Link encap:ethernet HWaddr 00:0A:5E:05:97:B4 inet addr: Bcast: Mask: inet6 addr: fe80::20a:5eff:fe05:97b4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7914 errors:472 dropped:0 overruns:0 frame:472 TX packets:2875 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (2.1 MiB) TX bytes: (286.2 KiB) Interrupt:5 Base address:0x7080 lo Link encap:local Loopback inet addr: Mask: inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:26 errors:0 dropped:0 overruns:0 frame:0 TX packets:26 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1820 (1.7 KiB) TX bytes:1820 (1.7 KiB) [root@www1 network-scripts]# ifup lo:0 [root@www1 network-scripts]# ifconfig 9 / 15

10 eth0 Link encap:ethernet HWaddr 00:0A:5E:05:97:B4 inet addr: Bcast: Mask: inet6 addr: fe80::20a:5eff:fe05:97b4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:8001 errors:472 dropped:0 overruns:0 frame:472 TX packets:2923 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (2.1 MiB) TX bytes: (292.1 KiB) Interrupt:5 Base address:0x7080 lo Link encap:local Loopback inet addr: Mask: inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:26 errors:0 dropped:0 overruns:0 frame:0 TX packets:26 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1820 (1.7 KiB) TX bytes:1820 (1.7 KiB) lo:0 Link encap:local Loopback inet addr: Mask: UP LOOPBACK RUNNING MTU:16436 Metric:1 network-scripts]# network-scripts]# ifconfig eth0 Link encap:ethernet HWaddr 00:40:26:5F:5C:60 inet addr: Bcast: Mask: inet6 addr: fe80::240:26ff:fe5f:5c60/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:8349 errors:0 dropped:0 overruns:0 frame:0 TX packets:4236 errors:0 dropped:0 overruns:0 carrier:0 collisions:132 txqueuelen:1000 RX bytes: (2.0 MiB) TX bytes: (440.1 KiB) Interrupt:5 Base address:0x7080 lo Link encap:local Loopback inet addr: Mask: inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:53 errors:0 dropped:0 overruns:0 frame:0 TX packets:53 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:5347 (5.2 KiB) TX bytes:5347 (5.2 KiB) 10 / 15

11 network-scripts]# ifup lo:0 network-scripts]# ifconfig eth0 Link encap:ethernet HWaddr 00:40:26:5F:5C:60 inet addr: Bcast: Mask: inet6 addr: fe80::240:26ff:fe5f:5c60/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:8412 errors:0 dropped:0 overruns:0 frame:0 TX packets:4275 errors:0 dropped:0 overruns:0 carrier:0 collisions:132 txqueuelen:1000 RX bytes: (2.0 MiB) TX bytes: (445.2 KiB) Interrupt:5 Base address:0x7080 lo Link encap:local Loopback inet addr: Mask: inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:53 errors:0 dropped:0 overruns:0 frame:0 TX packets:53 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:5347 (5.2 KiB) TX bytes:5347 (5.2 KiB) lo:0 Link encap:local Loopback inet addr: Mask: UP LOOPBACK RUNNING MTU:16436 Metric:1 network-scripts]# Both WWWs vi /var/www/html/ldirector.html Test Page Both WWWs Configure the ServerName Directive repectively for each node: WWW1 ServerName www1.example.com WWW2 ServerName www2.example.com 11 / 15

12 Both WWWs: cd /etc/httpd/conf.d/ mv welcome.conf welcome.conf.orig service httpd start chkconfig --level 35 httpd on WWW1 (to represent some web content): conf.d]# cd /var/www/html/ html]# vi index.html WWW1 Website page WWW2 (to represent some web content): conf.d]# cd /var/www/html/ html]# vi index.html WWW2 Website page NOTE: I have used different webpage contents on both servers to actually notice the difference if one node fails. In real life, of-course the web content will be same. Now try from some webclient / browser ( ) the following: OR You will see WWW1 one time you press GO or ENTER, and will see WWW2 the second time. And it will keep switching like this proving that the load balancer is working properly and is sending connections to the web server which has lesser load. Since there are only two web servers abnd one client to check both servers one by one, the Load Balancer takes you to www1.example.com and to www2.example.com turn by turn. Great Alhumdulillah. Try stopping apache on WWW1, you would see the WWW2 page on your browser and if you 12 / 15

13 keep clicking GO/ refresh, you will keep seeing WWW2 page. As soon as you turn up WWW1 server, and press refresh on client. You will see WWW1 page. Same goes for WWW2 server. Great Lets Fail the LB now. First check which one is active? /etc/ha.d/resource.d/lvssyncdaemonswap master status master running (ipvs_syncmaster pid: 4704) So LB1 is ACTIVE LB. Let's fail that and see if heartbeat takes over or not. So we still see our web page on ( or not? service heartbeat stop Stopping High-Availability services: [ OK ] Check status again: /etc/ha.d/resource.d/lvssyncdaemonswap master status master stopped (ipvs_syncbackup pid: 5125) Check to make sure if LB2 has taken over as master: [root@lb2 /]# /etc/ha.d/resource.d/lvssyncdaemonswap master status master running (ipvs_syncmaster pid: 4743) [root@lb2 /]# NOTICE the master has stopped on LB1. Check browser on client, Working perfectly. We still get our page. If I start heartbeat on LB1 again, LB1 must remain inactive and LB2 must still remain ACTIVE. Lets see: 13 / 15

14 service heartbeat start Starting High-Availability services: ldirectord is stopped for /etc/ha.d/ldirectord.cf [ OK ] /etc/ha.d/resource.d/lvssyncdaemonswap master status master stopped (ipvs_syncbackup pid: 5125) Notice that LB2 is still ACTIVE. Alhumdulillah ================================================================ Lets connect null modem wire to serial ports of the two LBs and change appropriate settings in various config files on both LBs, so the too much broadcast for heartbeat signals on the switch is removed / minimized. Both LBs : vi /etc/ha.d/ha.cf logfacility local0 serial /dev/ttys0 baud #bcast eth0 # Linux #mcast eth auto_failback off node lb1.example.com node lb2.example.com respawn hacluster /usr/lib/heartbeat/ipfail apiauth ipfail gid=haclient uid=hacluster [root@lb2 ~]# service heartbeat stop Stopping High-Availability services: [ OK ] [root@lb1 ~]# service heartbeat stop Stopping High-Availability services: 14 / 15

15 [ OK ] Checked through browser. Found everything working. Success. Alhumdulillah. 15 / 15