5 Performance Management for Web Services. Rolf Stadler School of Electrical Engineering KTH Royal Institute of Technology.

Transcription

1 5 Performance Management for Web Services Rolf Stadler School of Electrical Engineering KTH Royal Institute of Technology April 2008 Overview Service Management Performance Mgt QoS Mgt Resource Control Mechanisms Service Quality Cluster Resources Resource abstractions 1

2 Outline 5.1 An architecture for a best-effort service. 5.2 An architecture with service differentiation. 5.3 A decentralized architecture with service differentiation. The lecture focuses on cluster-based web services. Architectures in 5.1 and 5.2 have been developed by groups at the IBM T.J. Watson Research Laboratory in NY, USA. Architecture in 5.3 is based on work in our lab. Clients Scenario for Web Services IP 2 IP 1 Internet IP S Web Server S IP n 2

3 Cluster-based Web Services DMZ Business Logic Database Internet clients entry point layer-4/7 switch application servers database servers 5.1 An Architecture for a Best-effort Service 3

4 Problem Setting Scenario (Web) clients connect via the Internet to a (virtual) web server S with IP address IP s. S is realised as a cluster of servers, network components, etc. Servers have identical functionality and content. Management Problem Allow for Scalability: System design must support dynamic increase (or decrease) of performance of web server S with respect to requests/sec processed and response time per request. Support Manageability: Allow for server monitoring and control operations from a management station, specifically monitoring response times to requests, adding and removing of server resources, setting and modifying performance policies on S. Load Balancing using the Address Dispatcher Server 1 from Internet IP Router Address Dispatcher (Layer-4 Switch) Server 2 to Internet IP Router Server n 4

5 The Address Dispatcher Approach [GH99] The system includes a TCP-level packet forwarder, also referred to as a layer 4 switch. In this architecture, the component is called Address Dispatcher (AD). The AD routes TCP packets from the same sender (IP address, port number) to a particular server S i. The system includes a set of servers (S i ) i with identical content. The AD and the servers S i share the same virtual IP address [GH99]. Incoming client to server traffic follows a different route than outgoing server to client traffic. L-E Model for Address Dispatcher Mgt Control Parameters (l j ) j, O L, G L, (w i ) i Mgt State Abstractions TCP conn arrival and departure rates Legislator Performance metrics from S i Response time sample query S i. Intensity Estimator Weights for WRR (w i ) i with w i >=0, SUM i (w i ) =1 Request Request Executor TCP conn table: (IP 1, P 1 )->S i1 (IP 2, P 2 )->S i2... 5

6 L-E Model for Address Dispatcher Executor Maintains resource state in form of a TCP connection table. Allocates servers to new TCP connections, following a weighted round robin (WRR) algorithm. Legislator Computes weights w i for WRR as the control policy for executor Takes as input the following metrics [GH99]): AD: Request rate; Servers S i : number of active processes, memory utilization on S i; Response time for sample query from AD to S i. Design of the legislator is based on heuristics that have been proved in praxis. (Management) control parameters for legislator: (l j ) j : Relative weights for above metrics O L : Frequency of executing legislator G L : Sensitivity threshold for changing weights w i (w i ) i : Predefined values for w i The Management Architecture for Address Dispatcher (1) Management Station P Server 1 A from Internet IP Router Address Dispatcher L E CE Server 2 A to Internet IP Router Server n P: Management program L: Legislator E: Executor CE: Estimator and collector of perf. metrics A: Script-enabled agent, estimating perf. metrics on S i A 6

7 The Management Architecture for Address Dispatcher (2) Real-time re-configuration: The management station can take S i out of service by setting w i :=0; It can add a new S j by increasing the number of servers by 1, and letting the executor know the LAN address of the new server. In the discussed example, the legislator runs a simple, heuristic functions to compute the control policy. Other control policies are possible, for instance policies that block requests. 5.2 An Architecture enabling Service Differentiation 7

8 Service Differentiation for Web Services Supporting different grades for each web service allows to define performance objectives (QoS) for different service operations, customers, etc. These objectives are typically expressed in terms of response time. An Approach to realize service differentiation [GP03]: Service provider defines mapping (customer, service, operation, grade) class Resources within the web cluster are allocated per class. Requests within a class are treated equally. This allows to differentiate between several operations (e.g., getquote, buyshare) of the same service or several grades (e.g., gold, silver) of the same service (e.g., StockUtility). Comparing Multiclass Networks with Web Services with Service Differentiation Network Service Web Service Key Resources capacity of output ports: schedulable region; buffer size, link bandwidth processing capacity in requests/sec Service type flow request (operation) Performance Characteristics per service class peak rate, average rate, etc. required CPU time Performance (QoS) Objectives per service class max end-to-end delay, loss for packets; max block rate for flows max response times for requests, max drop rates for requests 8

9 A Management Architecture for Service Differentiation (1) Client Client L4 Switch Gateways Gateways Gateways Server Server Nodes Server Nodes Nodes Client Publish-Subscribe Control Network Monitoring and Control Path Request Path Management Console Global Resource Manager [GP03] A Management Architecture for Service Differentiation (2) The architecture can be understood as an extension of the architecture in 5.1. The functions of the network dispatcher in 5.1. is split into a layer-4 switch and several gateways. The gateway component enables service differentiation and response time objectives per class. The global resource manager is the management application that controls the operation of the gateways in a feedback control loop. It attempts to optimize a cluster utility function under the constraint of achieving QoS objectives. 9

10 Elements of the Architecture: Gateway (1) Server S 1 Classifier Buffer Manager Scheduler Router Server S 2 Server S 3 Gateway g Server S 4 Control policy for gateway g: (N g,s ) s, (w g,c ) c Elements of the Architecture: Gateway (2) The gateways controls the amount of server resources allocated to each class. This allows to control the response time experienced by requests of each class. Requests received by a gateway are classified and stored in FIFO buffers. There is one buffer per class. A scheduler serves the buffers, dispatches requests to servers using a WFQ discipline, guarantees a min flow for each class limits the number of concurrent requests outstanding Control parameters for the scheduler, set by the global resource manager : N g,s : max. number of concurrent requests server s can handle from gateway g. w g,c : min. number of concurrent requests from class c all servers can handle from gateway g. 10

11 Elements of the Architecture: Global Resource Manager Offered Load Service Time Utility Functions Server capacities (N s ) s Performance Objectives Global Resource Manager Resource Configuration (N g,s ) gs (w g,c ) gc The global resource manager periodically computes N g,s and w g,c. It uses a queuing model to predict the performance of the system for given values of N g,s and w g,c and applies a dynamic programming algorithm to find the values that maximize a cluster utility function. This function captures the business value of meeting the performance objectives of the various classes. See [GP03] for details. L-E Model for Global Resource Manager and Gateways response time objectives parameters for utility function Mgt State Abstractions offered load service times Estimators Legislator Global Resource Manager control policy (N g,s ) s, (w g,c ) c server capacities (N s ) s Capacity Estimator Requests Gateway g Executor gateway state (N g,s ) s, (w g,c ) c 11

12 Determining the Capacity of a Server N s Throughput (request/sec) capacity N s Number of Concurrent Requests Recommended Reading [GH99] Goldszmidt, G.; Hunt, G. : Scaling Internet services by dynamic allocation of connections, Proceedings of the Sixth IFIP/IEEE International Symposium on Integrated Network Management, 1999, pp There are differences in terminology between lecture and paper, most important: Address Dispatcher (lecture) stands for Network Dispatcher (paper); Legislator (lecture) stands for Manager (paper). [GP03] R. Levy et al: Performance Management for cluster-based web services, Eighth IFIP/IEEE International Symposium on Integrated Network Management (IM 2003), Colorado Springs, Colorado, USA, March 24-28, [AS06] C. Adam and R. Stadler: A Middleware Design for Large-scale Clusters Offering Multiple Services, IEEE electronic Transactions on Network and Service Management (etnsm), vol. 3, no. 1,