Linux Virtual Server Jim Lawson VAGUE/University of Vermont jim@jimlawson.org / Jim.Lawson@uvm.edu
What is a load balancer? Front-end appliance for a web (or other service) farm Allows you to scale out rather than scale up Several vendors supply products in this space (Cisco, F5, Foundry, others)
Linux Virtual Server ( IPVS inside the kernel) Kernel-space load balancer What is LVS? Fast, efficient, reliable Somewhat featurelimited compared to commercial options
What is keepalived? Provides: Health-checking for realservers - takes malfunctioning servers out of the pool Failover for director/load-balancer
LVS NAT config Client Request VIP1 192.168.1.1 LVS director LVS gateway 10.0.0.1 Realserver 1 10.0.0.10 Realserver 2 10.0.0.11 Realservers specify director as gateway addr Realserver 3 10.0.0.12
LVS DR config Client Request VIP1 192.168.1.1 LVS director Response Request Response Request Realserver 1 192.168.1.10 VIP1 192.168.1.1 Realserver 2 192.168.1.11 VIP1 192.168.1.1 Realserver 3 192.168.1.12 VIP1 192.168.1.1
Why DR (direct-route?) Director only needs to handle request portion of traffic. In typical HTTP, request is small (typically <1K) GET /index.html HTTP/1.1 Host: www.uvm.edu Response is sent directly to gateway HTTP/1.1 200 OK...... Content-size: 22947
Why DR (direct-route?) Francois JEANMOUGIN Francois (dot) JEANMOUGIN (at) 123multimedia (dot) com 06/06/2005: I have 38 realservers behind my director, incoming traffic (to director) goes up to 20Mb/s, outgoing (from realservers LVS-DR setup) up to 60Mb/s. I have about 1200 sites hosted. 36 virtual_server entries in keepalived.conf, 30 VIPs. There's no noticable load on the poor PIII/700 director that's handling the traffic.
Why not DR? ARP problem Realservers have to be configured to not ARP for the VIP only the director should respond to ARP requests for that IP Linux: 2.2, 2.4 kernels need hidden arp patch 2.6 kernels only need arp_ignore and arp_announce set in /proc. Most other unixes (unices?): NOARP works fine Windows (since NT4SP2): ifconfig -arp
Why not DR? If you forget to set NOARP (or hidden/arp_announce) before you bring the VIP up on the realserver, the realserver will receive all traffic bound for the VIP! To avoid this, put the VIP config in a special startup script which always sets the proper flags in /proc In general, it is a good idea to have 1 VIP per service or pool
LVS scheduling algorithms rr (round-robin) lc (pick server with least # connections) wrr, wlc weighted versions of above For load balanced caching proxy servers: DH (destination hash, static, based upon destination IP) LBLC (locality-based least connection; like DH but dynamic)
lc example graph LVS with 2 realservers, serving http all day (rrd image courtesy Salvatore D. Tepedino)
lc vs rr Round-robin keeps servers more or less evenly balanced Least-connections is very good at keeping them evenly balanced BUT... thundering herd problem Newly added or recovered realservers have no active connections! Guess where everyone gets sent?
Keepalived Monitors services for availability Built-in checks: http, https, smtp, ldap, tcp Custom scripts are easy to plug in Threaded Health Checks When services go down, servers are removed from pool and users are automatically redirected to remaining available nodes
What about the director? It's a single point of failure Solution: keepalived VRRP Virtual Router Redundancy Protocol RFC 2338, election protocol, multicast Similar to Cisco's HSRP active/passive Can have VIPs staggered between directors for active/active config
What about the director? Active connection state (client IP <-> realserver) is communicated via IPVS syncd Active server informs passive server about new associations Runs over crossover cable, or LAN During a failover, gratuitous ARP is sent Failback: set PREEMPT_DELAY
CIT LVS config www.uvm.edu Client redback zorocratid coneweb Request Response carrier tangleweb orbweaver oonopid VRRP IPVS syncd Request ldap.uvm.edu peregrine fishercat porcupine smtp.uvm.edu passenger pony eagle Response