COSC1300, COSC1301. Web Servers and Web Technology

Transcription

1 School of Computer Science and Information Technology COSC1300, COSC1301 Lecture 10: Load Balancing of Web Servers

2 Lecture Overview Load Balancing What it is? Why use it? Where is it used? Benefits thereof? Load Balancing Techniques Techniques used Load Balancing Reference Architectures Sun / Java Reference Architecture 2

3 Load Balancing (LB) Load balancing (LB) is the process of distributing load across multiple devices or services in some equitable way The most common reason is To establish uniform work allocation across application servers (AS's) To establish uniform resource allocation across the resource domain Load Balancing can apply to any kind of work, Web Servers / Services Terminal Services CPU cycles, Bus Throughput We will be mostly looking at LB for network service optimisation. 3

4 Why Load Balance? Most enterprise class applications require scalable LB of some sort, hardware and networks have finite resources; it is exhaustible. when it runs out and more is needed, how do we add it? How do we tell the world about it? Do we need to tell the world? At RMIT, we use multiple load balanced servers to service 90,000 students and 5000 staff Q: How many servers do you think RMIT uses? Learning Hub: 4 servers Blackboard: Weblearn: 6 servers 1 server 4

5 Forms of Load Balancing Non-transparent (client effectively selects server) There may be occasions when a client selects the site (eg may be a different site to which might be located in Italy) But each can have links to the other(s) Unlike content-negotiation where multiple DNS entries point to the same machine, and Apache selects the document tree internally, here DNS is used to select the machine via name Distribute the load: Geographically Geometrically (as in Grid Computing) Functionally (eg secure servers) Eg. An Italian page might refer to an English page when the Italian does not exist. Client ❶ ❷ ❸ jp it en de Wikipedia.com 5

6 Forms of Load Balancing (2) Transparent (client sees only one web site) App Scaling As more resources are needed to cope with increasing traffic, LB distributes load across multiple servers in order to scale applications. Vertical scaling Horizontal scaling High availability In mission-critical apps, a level of redundancy allows a site to remain up even if multiple components fail Dynamic Reconfigurability Allows a site to change its configuration frequently if necessary without telling the world about it Can be Both The English Wikipedia could be non-transparent for language selection, but transparent within a language Client en LB 6

7 Vertical Scaling Vertical Scaling (scaling up): Adding more hardware resources to the same machine generally adding more CPU/s and Memory Example: 8 way vs 4 way machines (8 vs 4 CPUs) This is usually very expensive ($$) Easy to implement (generally no change required in your application) But you still have a single point of failure 7

8 Horizontal Scaling Horizontal Scaling (scaling out): Adding more machines to the mix, generally cheap hardware Cheap(er) - at least more linear expenditures Harder to implement (much harder than vertical) May add another protocol layer between machines Many points of failure but can usually handle failures elegantly due to redundancies 8

9 A Load Balanced Application The diagram shows a simple load balanced application Balanced at every tier Web Servers Database Servers The Web Tier uses the load balancer to talk to the DB Servers. Load Balancer INTERNET Firewall Q: How many IP Addresses do you need for the whole application? WS1 WS2 Web Servers DS1 DS2 Database Servers DB 9

10 Introducing: Load Balancers Q: How many IP Addresses do you need for the whole application? A: A minimum of four One for each Web App One for each DB Server One more..? For which component? 10

11 Introducing: Load Balancers Given what you know about DNS Servers How many IP addresses can you bind to one name? So if you have a web application on multiple application servers how does a client decide which component in the application to send to? Does the client need to make that decision? if not so who makes it? 11

12 Introducing: Load Balancers The DNS name/entry is usually bound to the Load Balancer IP Address (each load balancer needs an IP address). Q: Why would this device require an IP address? The load balancer usually just acts as a forwarding device, and forwards packets (that come in on port 80; HTTP Requests) to/from a client to/from an Application Server So it is the load balancer, not the client, that decides which part of the application a packet goes to. so the client needs to talk to the load balancer, hence the LB needs an IP address and a DNS entry. 12

13 Load Balancer Example Ringo Barlow Baker Load Balancer AS1 AS2 Our load balancer has a single entry point from the internet, and two application servers Request Sequence: Ringo, Baker, Barlow So Ringo's request is forwarded to Application Server AS1, and Baker's request is forwarded to AS2 Then Barlow's request arrives. Where does it go? 13

14 Load Balancer Example In the previous schematic we see that each client is forwarded to a different application server. The requests are thus balanced by the Load balancer (follow the colored lines) So how do we decide which AS to use? Algorithms of Load Balancing The simplest algorithm is called Round-Robin (everyone gets a turn). In this case Barlow would have its request forwarded to the next available Application Server (AS)? 14

15 Load Balancer Algorithms Load Balancers decide how to balance a load based on different application-specific algorithms. These are essentially rules that govern the overall behavior of the Load Balancer. These rules can be applied to External events Network traffic, traffic distribution, etc Internal events AS response times, AS status (down,up, ) For more detail, see: "Understanding CSM Load Balancing Algorithms" adbde.shtml 15

16 LB Parameters Some of the most important parameters affecting LB are: Node availability A list of nodes that the LB can forward requests to, The node status (busy, free, offline, resetting, ). Can be as simple as the LB pinging the AS. Advance knowledge of traffic behaviour (app-specific) The LB knows a traffic pattern in advance (eg central ATM services, some web services) Can make resources available in advance or schedule subsequent transactions (eg balance update audit trail) Statistical estimate of traffic behaviour (app-generic) Using network traffic modeling algorithms Trying to predict future needs based on present trends 16

17 LB Parameters (2) Some other parameters affecting LB are: Propagation Delay (PD) The act of generating statistics (ie network stats or load stats) on a server takes time, thus this adds latency to requests. Session Persistence ('Stickiness') Most web pages generate multiple TCP sessions, which should all normally go to the same server in order to enable memory caching. Q: Why is this required? Is this ALWAYS required? May affect efficiency of LB algorithm if stickiness not properly distributed 'failover' to a different server if a server goes down is not trivial Transaction complexity Operations may range from a simple static HTML read to a multi-record DB update. Multiple AS updates to DS's may affect LB algorithm non-trivially 17

18 LB Algo: Random (R) The simplest load balancing algorithm is Random Directs the network connection to any available server, Treats all servers as equal, regardless of the number of connections or response time. Treats all network transactions as equally complex Has no memory, so it treats all transactions as idempotent (same effect no matter which server, or sequence) True for HTTP, and HTTP GETs. Q: Is it true for HTTP forms? True for session cookies? Makes the grandiose assumption that all traffic is equal which is clearly false, so not appropriate for web use. 18

19 LB Algo: Round Robin (RR) The next simplest load balancing algorithm is Round-Robin Directs the network connection to the next available server, allocated in some sequence Treats all servers as equal, regardless of the number of connections or response time. Treats all network transactions as equally complex Has no memory, so it treats all transactions as idempotent (same effect no matter which server, or sequence) True for HTTP, and HHTP GETs. Q: Is it true for HTTP forms? True for session cookies? It is simple and sometimes very effective because no propagation delay or caching hinders the algorithm. 19

20 LB Algo: Least Connections (LC) The Least Connections method directs network connections to the server with the fewest connections. Although it may not be intuitively obvious that the mathematical predictor would provide effective load balancing, in fact, it is quite successful, according to Cisco. On Web sites where there is a collection of servers with similar performance, the predictor is effective in smoothing distribution when a server becomes bogged down. On Web sites where there are large differences in the capacity of various servers, the predictor is also very effective. In maintaining the same number of connections to all servers, those servers that are capable of processing (and thus terminating) connections the fastest receive more connections over time. A server deemed to be twice as powerful as another server receives about twice as many connections per second. 20

21 LB Algo: Weighted RR / LC The weighted method allows you to assign a performance weight to each server. Administrators can assign a weight to each real server, and the Load Balancer uses this weight to determine the percentage of the current number of connections to give each server. Servers with a higher weight value receive a larger percentage of connections at any one time. Weight usually allocated on server properties such as Capacity, CPUs, etc 21

22 LB Algos: Weighted Methods EG: In a configuration with 5 servers, the percentage of connections is calculated as follows: Server Number of Connections Weight of server1 = 7 Weight of server2 = 8 Weight of server3 = 2 Weight of server4 = 2 Weight of server5 = 5 Total weight of all servers This distribution results in server 1 getting 7/24 of the current number of connections, server 2 getting 8/24, server 3 getting 2/24, and so on. If a new server, server 6, is added with a weight of 10, it receives 10/34, and so on. 22

23 LB Algos: Pros and Cons Random Pros: Extremely simple to implement Cons: Too simple to actually work well Round Robin (eg Pros: Better than random allocation because the requests are equally divided among the available servers in an orderly fashion. Cons: Round robin algorithm is not enough for load balancing based on processing overhead required and if the server specifications are not identical to each other in the server group. 23

24 LB Algos: Pros and Cons (2) Weighted Methods: Pros: Adapts according to the capacity of the servers in the group. Useful in app-specific cases involving function-specific servers in a cluster Cons: Does not consider advanced load balancing requirements such as processing times for each individual request. 24

25 LB Algorithm Adaptation An essential component of load balancing is LB adaptation (also called intelligent forwarding) For example, the simple Round-Robin method on its own as described earlier is not adaptive. It treats all network transactions as independent It treats all network transactions as identical in complexity It treats all application servers as independent and identical It does not distinguish between transactions requiring database server access, from those that do not. which are strong points of weakness of this algorithm. 25

26 The need for intelligent forwarding What happens to the client requests if some of the application servers are down? What happens if the network destination is unreachable, or the network is saturated, or worse yet the network on half the application servers is congested and the other half of the application servers is relatively load free? Will the client experience be uniform for all clients (Ringo, Barlow, Baker)? Ringo Barlow Baker Load Balancer? The above may seem obvious, but what if there were 10 AS's or which 3 were down? AS1 AS2 26

27 The Health-Check Every load balancer has a list of nodes (eg. available web servers) it can forward requests to. This list is usually availability based Availability may be as simple as the load balancer pinging each Web Server or requesting an Static HTML document from the Web Server So the load balancer can keep this list up to date and adapt the load dynamically. 27

28 Break 28

29 Load Balancer Issues: Sessions An important class of web transactions are when these involve sessions. When a transaction is session-based, the session information must be kept somewhere. If it is kept on the AS, what happens when the next transaction goes to a different AS? If it is kept on the LB and transferred to each AS as needed, what happens when the LB goes down what failover is available? If it is kept on the client, then it persists as long as the client persists, which is the desired behaviour anyway. Q: How can we store session information on the client in a HTTP transaction? 29

30 Load Balancer Issues: Sessions Q: How can we store session information on the client in a HTTP transaction? Q: What if we don't, then what do we need to ensure? A load balancer required to maintain persistence must support sticky sessions. A sticky session specifies that once a session is created by a given server, subsequent requests from the user will continue to be routed to that same server in order to preserve session information. The question comes back to idempotence: In a session, all transactions are not equal since they depend on session information (eg trans time, seq, result, etc ). We can make them all equal by storing the session information with the transaction. This may not always be possible. Q: How did we do that before, in this course? 30

31 Load Balancer Issues: Sessions Q: How can we store session information on the client in a HTTP transaction? A: Cookies!!! Two scenarios are possible: A load balancer may not need to support sticky sessions if all the session information is in the transaction itself and this is carried through to all resources that need it (eg database server) Idempotent NO sticky sessions LB algorithm free to allocate server without regard to session A load balancer can be deployed with cookie based persistence in which case, the cookie is used to relay session information, that the load balancer uses to redirect to the server that created the session. Not Idempotent Sticky Sessions LB algorithm constrained in its resource allocation 31

32 LB: Putting it all together For session-enabled LB (LB assigns cookie) 1. REQ: Client LB 2. REQ: LB AS (next available) say, AS3 3. RESP: AS3 + (session indicating AS3) LB 3 4. RESP: LB 3 Client 3 In a subsequent transaction 1. REQ: Client 3 LB (sees session indicating AS3) 2. REQ: LB AS3 3. RESP: AS3 + (session indicating AS3) LB 3 4. RESP: LB 3 Client 3 32

33 LB: Sticky Sessions Here is the entire concept of how Load Balancers work and how servers are clustered with load balancers in order to provide high availability. Client Choose Node ❶ ❷❸ ❹ } Monitors Fallover Manager Sends List Updates Node List Node Manager Node is Up Node is Down Sends Node List Provides List Dispatch Manager Sends Request Session Stickiness URI Node Controller LB Node Controller Sends One Node Asks "Which Node?" Node Chooser Sends One Node Load Balancer 33

34 LB: Putting it all together (2) Q: What happens if the Web Server the client is stickied to becomes unavailable half way through a client request A: In other words, what happens if session info is lost? Either the client is forwarded to a new AS if session can be transferred, the client is logged out and a new session is started. This is up to the implementation of the Load Balancer and/or application. Some applications keep session information in a database thus allowing another web server to restart the session if needed. This may add to the load non-trivially 34

35 High Availability (HA) A system is highly available if: It recovers quickly from failure Mean time between failure (MTBF) is typically very low (<1% of uptime, say) Automatic and transparent failover to a new component whenever possible 35

36 Continuous Availability (CA) A system is continuously available if: It continues to appear to work even when components fail within the system. This is due to: Automatic failover to a new component By repeating the transaction there if idempotent By restarting the session if not Redundant operation Multiple component performing the same action and a vote taken on the result Used particularly in mission-critical operations (eg. Spacecraft, some medical equipment) 36

37 What does HA have to do with Load Balancers? Load Balancing is a technique which provides HA as well as Scalability. Note the distinction between HA and CA CA would be nice, but very expensive to implement. HA is often sufficient. The transaction idempotence of HTTP often makes CA unnecessary There is no point in doing LB if not in order to ensure HA/CA; it is the raison d etre of LB. 37

38 Continuous Availability (2) CA is different to HA in that it must be able to handle the integrity of partially completed transactions or operations. So in an instance where a web server drops off mid-request, another can continue doing its work and the process is transparent to the end user. Although CA is desirable and ideal, the small downtime required to realise this is rarely needed. Only the most critical applications typically found in the military, defense, telecommunications or financial industries, have such a high standard for availability It is very important to note, that high availability does not mean continuous availability. 38

39 Measuring Availability Compare reliability and availability Reliability is typically used when referring to the number of faults a system experiences within a given time interval, Availability is a measure of the system s overall ability to offer resources and services despite any faults. 39

40 Measuring Availability (2) There are two standard metrics used to measure the availability of a system: Mean-time-between failure (MTBF) Assume that a system is repaired, replaced or patched, after each failure. Assumed that the component is then returned to service immediately after the failure. Mean-time-to-recovery (MMTR) May include individual components or entire systems. 40

41 Measuring Availability: An example Availability = MTBF / (MTBF + MTTR) Web Server MTBF = 4380 hrs since last failure Web Server MTTR = 3 hrs to discover and restart A = % = 4380 Hrs / (4380 Hrs + 4 Hrs) Three nines: 99.9 Significance? 41

42 The Myth of the Nines A popular marketing message that has been very visible across the computing industry the last few years is the concept of a piece of software or hardware having the ability to deliver Five Nines of uptime This means 5.26 minutes of downtime a year. Reasonable, do you think? It is also important to be aware of what is meant by failure, and whether partial failure is included in this definition. 42

43 Five Nines: Statistics Uptime (%) 90 % 95 % 99 % 99.9 % % % % Downtime per Year 876 hours (36.5 days) 438 hours (18.25 days) 87.6 hours (3.65 days) 8.76 hours minutes minutes seconds 43

44 Five Nines: The reality 44

45 Five Nines the reality Five Nines equals percent uptime for the component or system. Obtaining 100% uptime is often an expensive, futile, or impossible task. Three Nines of availability is a much more practical percentage to aim for. If this level of availability can be achieved, then the component or system could easily be considered a highly available component or system for just about any production environment. The more nines the more $$ 45

46 Single Point of Failure and Fail Over Single Point of Failure A single point of failure is typically a hardware or software component whose loss or failure results in a denial of service. Typically these components are lacking a paired, redundant or complimentary substitute. A fully-realized high availability system removes any single point of failure within the system. Fail Over Typically when a service is no longer available, it is started on another component within the system. This transfer of service following a failure is known as a fail over. For example, many high availability servers will be configured with at least two network interface cards. This guards against total component failure if one card fails 46

47 Case Study: Sun High Availability This shows a typical Sun Java ES reference architecture. Notice the component redundancy at each level. Think for a moment about geographical location of each component. Will each component be in the same server room? Why or why not? 47

48 Other factors that affect Availability People: When people are involved, human errors that delay recovery or cause outages are likely scenarios that should be considered Maintaining Availability is generally as much about people as it is about technology. Organisation: The service support team needs adequate organisation structures to ensure that they have the correct skills available or on call when required. Procedures: Service support staff require correctly documented, tested, and maintained procedures. Skills: Prompt fault detection and recovery requires business and technology skills. Environment: Storms, fires, earthquakes, floods and man-made disasters can affect availability. Infrastructure: IT Systems are reliant upon power continuity, power quality, cooling, air flow, and air filtering for proper operation. 48

49 Lecture Summary Load Balancing What it is? Why use it? Where is it used? Benefits thereof? Load Balancing Techniques Techniques used Load Balancing Reference Architectures 49

50 School of Computer Science and Information Technology COSC1300, COSC1301 Next Lecture 11: Advanced Research Topics

51 Next Lecture Overview Here we cover some further advanced research topics relevant to WSWT to give you a flavour of what research in this area is all about. We will cover: The Mobile Context Content Distribution Networks Distributed Caching and Cache Invalidation Fragment Caching (AJAX) Data Hoarding Beyond WSWT? An overview of where to go further for those interested in this topic. 51