Computing Service Architecture: central monitor-absence load balancing

Transcription

1 Computing Service Architecture: central monitor-absence load balancing Satoshi Azuchi, Sojeong Hong, Jinoh Kim May 5, 26 Abstract We propose decentralized load balancing by priority and migration based load control mechanism, which is done autonomously. Each computing server s load balancing is achieved by dynamically changing its priority and control task acceptance rate based on a computing server s priority. In addition by providing task set level migration, we can enhance the effectiveness of our decentralized load balancing mechanism. For security, we provide mutual authentication and capability based access control. We implemented a mock of computing service architecture and support our claims by presenting several sets of experiment results in this report. 1 Introduction With tremendous growth of personal computers, distributed computing has proliferated. The distributed computers enable to solve complex tasks such as scientific computation without having an expensive super computer. The typical examples of the distributed computing are Grid and volunteer computing based on donation of computer cycles such as SETI@home [2]. The volunteer computing assumes heterogeneous computing environment due to the characteristic of voluntary participation of the computing resources, while the Grid is built upon homogeneous controlled environment. While deploying distributed computing, even though it exploits a lot of CPU cycles, some problems have been announced as well. One of the problems is that distributed computing systems are vulnerable to single point of failure, especially monitor server which is responsible to manage and control the distributed computing resources. Employing additional computing hardwares for duplication can solve this problem, but it requires extra costs. Moreover, it does not achieve only by replicating the monitor server; rather it requires complex and accurate fault tolerant protocols and mechanisms. Hence, we propose a new distributed priority-based computing architecture to utilize the computing resource more efficiently and be tolerant against single point of failure. The traditional distributed computing systems do not much consider the priority-based computing services. There may be some tasks simultaneously, and each of them requires different set of resources (in this paper, we use job and task interchangeably). Therefore, the priority-based computing services can be applied for resource management and allocation in distributed computing environment. This functionality is used to apply policies for resource utilization in some sense as well. Security concerns are also an important part of the distributed computing. In particular, donationbased computing has more potential problems than the Grid computing. As mentioned above, an institution organize the Grid, so it is simple to protect the resources. Contrarily, it is difficult to protect resources in the voluntary computing environment because there is lack of well-organized management functions. This arises some security issues in distributed computing in terms of network attacks as well as authentication and authorization. 1

2 In order to solve these problems, we consider the central monitor-absence architecture; that is, fully distributed. By removing the monitor server, we can save cost and reinforce fault-tolerant capability because we do not worry about the master server down. In addition, by providing priority-based computing services, we can offer users to have different degrees of computer cycles. However, removing the master server requires another load balancing method because the master server is in charge of load balancing. For this reason, we propose a central monitor-absence load balancing method for our architecture. 2 Computing Server Farm Design In this section we elaborate our computing server farm design. First, we introduce overall architecture. Then a scenario is described and each component is explained precisely. 2.1 Architecture The proposed architecture provides virtual machine-based load balancing without any central load balancing monitor, capability-based access control, and mutual authentication. The overall architecture consists of computing server farm, service management server, and clients as shown in Figure 1. Each component s role is explained in the following sub-sections. Service Management Server 1. Auth. 2. Subscription 3. Capability Computing Server Farm 3. Auth. 4. Job submission 5. Result Client Figure 1: Design overview 2.2 Scenario Before using the computing service, the user should buy/receive a degree of computer cycle, so called subscription or service level agreement. Based on the client s payment or client s role (if the priority is decided based on the user s role in a company), the priority is decided by the service management server. The service management server makes capability and signs the capability. Then it sends the capability token to the client. 2

3 When a client wants to run his job on the computing server farm, it requests for the availability of the server usage with capability token; the server selection can be done randomly (we assume that a client knows all server s address). When the server receives the request, it checks the capability s validity such as expiry date and signature of service management server. Then it compares the received client s priority and the machine s priority. If the client s priority is higher or equal to the machine s priority, it accepts the request and asks the client to submit the job. Otherwise, it forwards the request to another machine. In the computing server farm side, each machine maintains its priority and changes the priority dynamically based on its workload. Specifically, if a server has more jobs than a certain number it increases its priority so that it can receive jobs from a fewer clients who have higher priority. The rejected client can contact other computing servers iteratively until it can submit its job. Our design supports an additional load balancing via task set migration across computing servers (due to the implementation decision, we provide task set level migration using Xen s migration function instead of single task migration, process migration). The task set migration takes places as follows. A computing server has a threshold, which is the maximum number of tasks a computing server can hold. An administrator can decide the threshold based on their resource capability (e.g. CPU speed, size of memory, etc). When the number of jobs reaches to the threshold, the computing server stops accepting clients task submissions. Then it communicates with other computing servers and migrates a task set to another computing machine, whose workload is the least. If all computing server is too overloaded to accept task set migration, a computing server wait until some of its tasks end and start to accept clients task submissions again; the client with the highest priority will get this chance. 2.3 Major Components Computing Server Load Balancer : A server farm consists of a set of autonomous load balancing enabled computing servers. Each computing server maintains priority variable P, which is dynamically adjusted based on each server s workload. This work is done by the load balancer. The load balancer s responsibilities are listed as follows : Work distribution inside a machine : when a load balancer receives a task from a client, it sends to computing daemon on top of virtual machine. Priority Adjustment : A load balancer has pre-defined work unit t. If the workload is greater than t, it increases P by one. If the workload gradually increases and reaches to 2t, P is increased by one again and so forth. Job Migration Decision : A load balancer monitors the total number of tasks it holds. When it reaches to the threshold, which pre-defined by the administrator based on the server s capability, the load balancer negotiate with other computing servers and trigger the task set migration; the load balancer selects the computing server whose workload is the least. Computing Daemon : The accepted task is actually executed by the computing daemon on a virtual machine. Access Guard : Each computing server also has an access guard. The access guard verifies the expiry date and the digital signature [9] of the service management server in the submitted capability to validate the capability s integrity and authenticity. Then the access guard checks if the priority in the capability is greater or equal to the computing machine s current priority P. If the client s priority is less than P, it forwards the request to another computing server. Otherwise, it accepts the client s request and let the load balancer accept the job. 3

4 2.3.2 Service Management Server The service management server is the entity that issues capability to the client. The capability includes 1) identity of a client, 2) priority, 3) expiration time, 4) the service management server s digital signature. The priority of a client can be decided based on how much the client pays for the computing service subscription. Note that the service management server does not interact with any computing server in the farm. Hence, as long as a client has a valid capability token, it can use computing server farm even if the service management server is off-line Client The client is the entity that submits jobs to the computing server farm. Before any job submission, it should receive a capability from the service management server. When a client wants to submit a job to the computing server farm, it requests job submission to a server randomly. Note that at the first request, the client does not submit the job actually. If the computing server has lower priority and it accepts the client s request, the client actually submits the job. 3 Implementation All components are implemented using C and for experiment and result parsing, we used Perl. Computing servers are installed on three machines in the itlabs instructional cluster. The service management server and a set of clients (processes called by Perl script) are installed on tera.cs and oxygen.cs respectively. 3.1 Xen installation and virtual machine migration We installed Xen 3. onto three Ubuntu 5.1 machines in the itlab instructional cluster. Xen installation process automatically compiled Xen and Linux kernel for Xen (called XenoLinux) based on kernel This XenoLinux always runs on Xen in order to provide control interface of Xen. Then, we confirmed that we were able to run virtual machines on Xen and to migrate them from the one machine to another. We used Debian (sarge) for this testing. To support OS migration, a root disk image must be shared among machines. First, we tried to use the global network block device (GNBD). However, it didn t work on our environment. Thus, we now use NFS to share a root disk image. 3.2 Computing server We implemented and installed load balancer, which assigns jobs to virtual machines and manage all jobs on a computing server, on three machines (on Domain ) in instructional cluster. Each machine has four virtual machines (Ubuntu 5.1) and computing daemon is running on top of the virtual machines to compile and run submitted tasks. When the task ends, it reports to the hosting load balancer. When a migration takes place, the load balancer transfers the information about the migrated task set and update its database. Note that the computing daemon, which has each task s owner information (e.g. IP address), is also migrated. Hence, when a task is finished, the daemon can notify to the new load balancer on the new machine. 3.3 Multi-threaded TCP/IP communication We implemented a communication module to provide middleware functions based on TCP/IP sockets among clients, the service management server, computing servers, and virtual machines. Each component has a server daemon to receive any service requests from other components. To request services of any other components, transient connections are established because permanent connections can be 4

5 limited by system configurations. After communicating, the communication channels are closed. In particular, since there can be a lot of clients, transient connection management is better in terms of scalability. As the architecture, the communication module is designed using multi-threaded event-driven model. Thus each component waits for an event, and a thread is spawned whenever an event comes. Then corresponding callback function is called to provide appropriate service. By providing well-defined APIs, the module works as a middleware in the system. Table shows the APIs provided by the communication module to applications. API Name comm. nonce comm. subscr client comm. submit client comm. forward task comm. send result comm. nego migrate Description Exchange mutual authentication data Request/reply subscription of clients Request/reply submission of tasks Forward tasks to a VM Return the result of a task Exchange VM migration information Table 1: Communication APIs 3.4 Public key infrastructure based authentication We implemented standard authentication protocol based on public key infrastructure. First, for public key authentication, we used X.59 [12] package provided by OpenSSL; X.59 specifies the standard for standard formats for public key certificates and a certification hierarchies. Service management server is considered as root certificate authority (which has a certificate signed by its own private key and issues certificates to other entity) and a key generator (it generates public key pair for computing servers). In our system, the service management server issues certificate for computing servers and clients. In our system, we do not consider connection hijacking so that when the client could verify the server s IP address, we consider it as the servers are authenticated. To authenticate a server, a client sends a nonce (random number) to the server, and then the server is supposed to sign the nonce. The server sends the signed nonce and its public key certificate. We assume that all clients have the root certificate authority s certificate already. When the client verifies the signature, it sends its data to the server, otherwise it conclude the server is not valid one. To protect the capability content, especially to prevent for the client to modify its priority, service management server sign the capability content and the computing servers checks the validity of capability using server s public certificate. This signature generation and verification is implemented using PKCS#7 [13] package, provided by OpenSSL. 4 Experiment Results In this section, we show comparison of computing servers workload and clients waiting/turnaround time based on task submission traffic with diverse properties. Also, we show the effect of task set migration to load balancing. The traffic is generated using Poisson random number generator. At each time interval (4 second), client processes are generated based on the random number. We control the rate of task submission using λ value. Also, we manipulate priority distribution among clients to show its effect on server; details are explained in the following sub-sections. 5

6 3 # of tasks legolas # of tasks gimli # of tasks aragorn 25 2 # of tasks sec Figure 2: Well-balanced computing servers workload 3 # of tasks legolas # of tasks gimli # of tasks aragorn 25 2 # of tasks Figure 3: Computing servers workload based on weighted random selection. aragon, gimli, and legolas are selected with.6,.3, and.1 probability as a first choice sec 6

7 4.1 Server workload with respect to diverse traffic For the first test case, the traffic with λ = 3 (45 job submissions per minute) is generated for 1 minutes. In this case, the service management server randomly selects the priorities of clients and a client also selects computing servers randomly to submit its task. Figure 2 shows that computing servers workload based on the given traffic. The three machines workloads are well balanced as we expected. After about 6 seconds, no traffic is not generated. However, the computing servers still have running tasks. That is because the remaining clients, whose priority was less than computing servers, are accepted by computing servers since they become less busy. Since with the first case traffic, it is not sufficient to show our priority based load balancing works properly. We generated different kinds of client set that select one computing server with.6 probability, i.e. this server (aragon) is considered a more popular computing server compared to the other two computing servers. Hence when a client starts to contact computing servers there are more chances that aragon will be selected. The other two computing servers are chosen with.3 and.1 probabilities (the corresponding servers are gimli and legolas respectively). If a client gets rejected from the computing server it choose with weighted random selection, the client contacts another computing server. Figure 3 shows the fairly well-balanced servers work load even if the clients chooses aragorn as a first attempt with higher probability. So far, we tested computing servers with the set of clients whose priorities are selected randomly between 1 and 5 (the lowest and the highest priority in our system) by service management server. What if the priories of clients are not well distributed? We generated two skewed distributions, clients with only high priorities (4 or 5) and clients with only low priorities (1 or 2). Figure 4 shows how workload changes on a computing server with respect to different priority set clients. The first workload peak is generated by the clients with evenly distributed priorities. The second workload peak is generated by the clients with 1 or 2 priorities and the last one is derived by the high priority clients. Except the different priority combination, other conditions (λ = 3, computing servers are selected randomly by clients) are set to the same values for the equality. Since the computing server control its workload based on priority, it could block the lower level clients task submission dynamically. However, in case of high priority client traffic, server could not reject clients requests a lot since clients priority were higher than computing servers. 4.2 Task set migration effect How does the task set migration affect on load balancing? If computing server can migrate tasks when it is overloaded, it is obvious that task migration is beneficial for load balancing. Here we will show the effect of task set migration by comparing workload on computing servers with virtual machine migration and without it. In order to trigger the migration, we generate a special client task submission pattern by giving them all priority 5. Clients select three computing servers with weighted probability of.6,.3, and.1 as we presented before. Note that we do not consider load balancing based on priority here to show only the effect of task set migration. Figure 5 shows the case of no migration. The workloads of three systems are the same as the load injected by 6:3:1 proportion. This is because the overloaded system denies submitted jobs when it reaches to the threshold and the clients choose another machine. In the case of migration in Figure 6, workloads of three systems are a little different from the injected loads. However, the overloaded system shows significantly shorter peak duration. This is very important. The overloaded system is aware of over-utilized situation, and it attempts migration immediately. Therefore it can be easily free from overloaded situation. Around 55 seconds, however, the system experiences relatively long overloaded period. This is because the system cannot migrate more virtual machines due to system configuration (the maximum number of virtual machine which can be migrated is set to 2 in the configuration in each system). For the results, it is clear that virtual machine migration is a great idea for load balancing. 7

8 25 pri 1/2 pri 4/5 all pri 2 15 # of tasks 1 5 sec Figure 4: Work load of a computing server with different clients priority combinations; 1-5 priority distribution, 1 or 2 priority distribution, 4 or 5 priority distribution from the left to the right 4.3 Comparison of clients with respect to priority We have seen that our priority and migration based load balancing works well. In this section we will show the efficiency of our system in the point of client. We measured clients waiting time, turnaround time and the number of rejections by computing servers. In order to manipulate the workload in computing servers, we generate two kinds of tasks (1 minute and 3 minute durations tasks) with same rate of task submissions. In order to submit task, a client communicates with computing servers iteratively, i.e. if a computing serveer rejects a client, the client communicates with other computing servers. If all servers reject a client s request, the client waits for 1 seconds and starts to pull the computing servers again. Figure 7 shows the average waiting time and turn around time with respect to clients priorities. The lower priority clients have waited longer time than the higher priority clients. In addition, when the computing servers are more overloaded (i.e. with 3 minute tasks), the waiting time become much longer than light workload on computing servers. Figure 8 shows the average number of rejections from computing servers with respect to clients priorities. The clients with priority 1 waited for about 85 times when the task duration was 1 minute and about 33 timers when the task duration was 3 minute, on average. The clients with priority 2 have rejected for very a few times when the task duration was 1 minute and about 13 times when the task duration was 3 minutes. The clients with priority 3 have rejected about 25 time only when the task duration was 3 minutes. The higher priority clients were not rejected during our experiment settings. The result is what we exactly expected. Hence, our design can provide differential service based on priorities. 8

9 3 # of tasks legolas # of tasks gimli # of tasks aragorn 25 2 # of tasks Figure 5: Workload on computing servers with weighted random selection without task set migration. The client has only 5 priorities. sec 3 (2) (3) # of tasks legolas # of tasks gimli # of tasks aragorn 25 (1) 2 (1) aragon -> gimli (2) aragon -> legolas (3) gimli -> legolas # of tasks Figure 6: Workload on computing servers with weighted random selection with task set migration. The client has only 5 priorities. sec 9

10 9 8 waiting time with 6s job waiting time with 18 s job turnaround time with 6s job turnaround time with 18s job 7 6 time (sec) 5 4 time (sec) Figure 7: The average clients waiting and turnaround time based on their priorities. 18-second tasks derive more workload on computing servers than 6-second tasks priority 35 # of rejects with 6s job # of rejects with 18 s job 3 25 # of rejects Figure 8: The average number of rejections by computing servers with respect to clients priorities. 18- second tasks derive more workload on computing servers than 6-second tasks priority 1

11 5 Related Work Network load balancing is a critical area to offer online services because users do not want the services are procrastinated or unresponsive. In particular, server load balancing is important since unavailability is one of the crucial measures the sites [4] [14]. There are several ways to provide load-balancing function for sites servers. Server clustering is a promising technique to build scalable server architecture [18]. Geographical load balancing is another way to achieve the distribute the load [5]. It distributes the load using network proximity information, so the closest site is selected for the service, while the server clustering is usually take the CPU load of each system into account to distributed the load. Grid computing enables users to obtain higher throughput computing by taking advantage of many networked computers. Especially, it uses the CPU resources of individual computers which are connected networks to solve large-scale computation problems [7] [15]. Volunteer computing is a new paradigm: it uses donated computing resources, while grid computing uses well-organized recourses. The typical example is SETI@home [2] run on BOINC distributed computing infrastructure [1]. Experimental work to build grid computing environment using Xen [3] for scientific computing was provided by [11]. In this work several virtual machines are installed on a machine to provide grid computing environment. In addition, PVM [8], which virtualizes a number of heterogeneous machines as a single distributed parallel processor, is a famous tool for scientific and engineering computing. Virtual Machine migration mechanisms have been developed because encapsulating the state of a running process which is necessary for the process migration is difficult [6] [17]. [6] integrated live OS migration into Xen which enables rapid movement of interactive workloads within clusters and data centers. [6] implemented VM migration system called VMotion which is a part of the VMware Virtual Center Product. 6 Lessons learned Priority based load balancing enables us to eliminate a central monitor since all servers achieve self-load balancing autonomously. However, one problem takes place; a client who has low priority cannot submit his job to any of computing servers if all servers are busy, i.e. the priorities of all computing servers are higher than that of the client. This problem can be alleviated 1) by allocating a certain number of clients that a computing server would accept even if the server has higher priority, or 2) by having a dedicated server for low priority clients. In the case of the first approach, we can also employ a tunable policy that the number of acceptable lower clients can be changed by the server administrator s decision. More precisely, if the number of acceptable lower clients is, the system does not consider the unlimited waiting of the low priority client. In the opposite way, if the value is the number of possible jobs a server can hold, the server does not follow our priority based load balancing. Hence, the value should be determined heuristically. Task set migration might not be the right choice to show how task migration among computing servers would be beneficial for the load balancing. We could use mobile processes (such as Aglet [16] or D Agents [1]) for single task level migration. To balance load, we only took the iterative contact to servers into account. The iterative way works pretty good in our experiments, but we are not sure that there might be same results if we use more complicated or refined variables (e.g., different processing time for each job). With respect to this concern, the recursive contact can be a good approach, in which the servers forward the job by negotiations. 7 Conclusion The traditional way of load balancing based on centralized monitor is vulnerable to single point of failure. If we multiplicate the monitor, then the cost increase is increased not only due to the monitor itself cost 11

12 but also due to the management cost. Hence, we proposed priority and migration based decentralized load balancing. Priority based autonomous load balancing provide load balancing and differential service based on how much a client pays for the service. Experiments show that even if there is skewed preference among computing servers, our system could achieve load balancing. Furthermore by providing task set migration, we also enhance the effectiveness of our load balancing mechanism by moving task from the overloaded server to the unloaded server. Last, capability-based on access control make computing servers independent from the service management server. Mutual authentication can protect clients data from attacker who might impersonate computing/service management server and computing server from malicious users. References [1] D. P. Anderson. BOINC: A system for public-resource computing and storage. In GRID, pages 4 1, 24. [2] D. P. Anderson, J. Cobb, E. Korpela, M. Lebofsky, and D. Werthimer. Seti@home: an experiment in public-resource computing. Commun. ACM, 45(11):56 61, 22. [3] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proceedings of the nineteenth ACM symposium on Operating systems principles, volume 37, 5 of Operating Systems Review, pages , New York, Oct ACM Press. [4] T. Bourke. Server Load Balancing. O Reilly, 5th edition, 21. [5] V. Cardellini, M. Colajanni, and P. S. Yu. Geographic load balancing for scalable distributed web systems. In MASCOTS, pages 2 27, 2. [6] C. Clark, K. Fraser, S. Hand, and J. Hansen. Live migration of virtual machines. In In proceedings of the 2nd Symposium on Networked Sytems Design and Implementation, 25. [7] I. Foster and C. Kesselman, editors. The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco, CA, [8] A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V. Sunderam. PVM: Parallel virtual machine: a users guide and tutorial for networked parallel computing. MIT Press, Cambridge, MA, USA, [9] Goldwasser, Micali, and Rivest. A digital signature scheme secure against adaptive chosenmessage attacks. SICOMP: SIAM Journal on Computing, 17, [1] R. Gray, G. Cybenko, D. Kotz, and R. Peterson. and d. rus. d agents: Applications and performance of a mobile-agent system, 21. [11] M. Hardt and R. Berlich. Xen: Scientific use cases and performance comparisons, 25. [12] R. Housley, W. Ford, W. Polk, and D. Solo. RFC 2459: Internet X.59 public key infrastructure certificate and CRL profile, Jan Status: PROPOSED STANDARD. [13] B. Kaliski. Pkcs #7: Cryptographic message syntax, version 1.5. RFC 2315, March [14] C. Kopparapu. Load Balancing Servers, Firewalls, and Caches. Wiley, 22. [15] K. Krauter, R. Buyya, and M. Maheswaran. A taxonomy and survey of grid resource management systems for distributed computing. Softw, Pract. Exper, 32(2): , 22. [16] D. B. Lange and M. Oshima. Mobile agents with Java: The aglet API. World Wide Web Journal, [17] M. Nelson, B.-H. Lim, and G. Hutchins. Fast transparent migration for virtual machines. In In proceedings of Usenix 25 Annual Technical Conference, pages ,

13 [18] T. Schroeder, S. Goddard, and B. Ramamurthy. Scalable web server clustering technologies. Network, IEEE, 14(3):38 45, May 2. 13