Transcription

1 REBELS: REmote Execution BasEd Load-balancing System A. Puliato, O. Tomarchio, G. Haring, G. Kotsis Ist. di Informatica e Telecomunicazioni Dept. of Applied Computer Science Universita' di Catania UniversityofVienna Viale A. Doria Catania, Italy Lenaugasse 2/ Vienna, Austria [ap, tomarchio]@iit.unict.it [haring, gabi]@ani.univie.ac.at Abstract We discuss the general objectives and the design of base mechanisms for managing resource sharing in a heterogeneous distributed environment. Given the need for simple, safe use of the computing and communication facilities present onthenetwork, the proposed environment should assist the user in submitting requests to the distributed system in a transparentway. The user should not need specic knowledge of hardware and software architectures available or of the mechanisms for their activation and execution. As a specic example, we will focus our attention on the remote execution of processes, with the aim to balance the load on the system and minimize the mean execution time. The REBELS system is described and implementation issues based on the use of Java technology are also addressed. Keywords: heterogeneous distributed systems, virtualization, access transparency, resource management, load balancing 1 Introduction A distributed system represents a collection of resources which can be accessed by dierent users in order to satisfy their own requests. Heterogeneity is one of the main features of such systems which contributes to oer a variety of services and hw/sw platforms, but that at the same time introduces extra diculties related with the usually hard task to execute the same code on dierent machines (portability). Furthermore, the access to the system is generally not transparent, i.e.the user has to be aware of system details and is required to explicitly transfer code from one machine to another, to compile and to execute it or to connect to the remote server to retrieve from it or to store on it data. However,asaside eect of the great improvements in operating systems, which nowadays provide easy access to their functionalities completely hiding any kind of detail (abstraction), users wish to exploit the full services oered by the available communication networks. The current research trend [8] is in the direction of a virtualization of the whole distributed system, which thus acquires the same capabilities as a generic processing resource providing simple and immediate access by means of high-level system calls. Proceeding along this line, we propose a reference architecture for transparent access to the resources of a distributed system which is based on a hierarchical decomposition of the system and makes wide useofagenttechnology [5]. Unlike other papers in literature, which propose algorithms for load balancing, our attention is focused on the denition of an architecture for load balancing, which could result completely independent from both the specic balancing algorithm used and from the topology of the system. We dene and implement adequate mechanisms to allow a user to browse and execute an application regardless of the physical location of the application code. Emerging systems like the one presented in [3], address CPU time allocation, but they often do not include generalized resource access and allocation, as we provide. The proposed architecture can be easily extended to implement basic functions of a distributed operating systems. In this paper we present REBELS (REmote Execution BasEd Load-balancing System), a system for the remote execution of applications in a distributed environment, that gives powerful mechanisms for both computational and communication load balancing. REBELS has a graphic environment and, since it is developed in Java [7], it is multiplatform and can be easily used in heterogeneous distributed systems. The remaining of the paper is organized as follows. Section 2 introduces two system primitives for transparent access to system resources and describes a reference architecture for resource management. Further details on the interaction among the dierent layers of this architecture are provided in section 2.1. In Section 3, we present REBELS, a Java-based implementation of the proposed architecture. Performance indices related to dierent evaluation scenario are discussed in section 4. Finally, in Section 5 we present the conclusions and provide some pointers for further developments. 1

2 2 System architecture The main goals of our proposal can be summed up as a total virtualization of network resources, (1) to provide simple and transparent access to network resources, (2) to increase the exibility and eciency in the usage of distributed resources and (3) to ensure security in the access of resources. To implement these primitives, an adequate system reference architecture has to be set up. Many of the solutions proposed in literature consider a restricted set of workstations (systems with no more than computers are generally recommended) [3]. This is due to the fact that, when the number of nodes presentinthe system increases, the overhead that the load balancing mechanism involves (in terms of trac produced on the communication network), makes the introduced benets very limited. Another common hypothesis is assuming that the applications are resident inallthemachines of the system [1]. Finally, the presence of only one node devoted to do all the activities associated with Load Balancing, which is also the only point to access the system [6], is often assumed. In this paper we present a multi-layer resource management architecture, consisting of the functional blocks in Fig. 1. The User Layer is the point from which the user accesses the services provided by the system friendly and transparently. The functions carried on by the User layer can be summarized in the following points: (1) to control the access to the system through appropriate authentication strategies (2) to nd and show the accessible applications in a user friendly way (3) to accept the requests of execution and send them to the broker layer (4) to execute the application by using: the services put at our disposal by the System Layer, the couple of best nodes, and the data supplied by the user (5) to manage any error condition. The Broker Layer implements the load balancing mechanisms for optimizing the use of the resources put at our disposal by the system. It intercepts the user's requests and, once the computational load has been evaluated, allocates the resources in order to minimize the processing times. These are the functions carried on by this level: (1) to accept the execution requests of an application coming from the user layer (2) to select the couple of nodes from which totake and execute the application, according to the load level of the system and the algorithm of load balancing used (3) to send the results obtained at point (2) to the user layer. The Resource Layer includes all the system resources, that is, the nodes where the applications are saved and the nodes enabled for their execution. The presence of nodes presenting both the above-mentioned features is also possible. The functions carried on can be summarized as follows: (1) monitoring the level of use of the resources (2) transferring such information to the broker layer, for executing the strategies of load balancing. The System Layer consists of two units: the rst one gives the support for remote execution, while the second one enables the transfer of the results of processing from the node that executed the application to the user. The functions carried on by this level are : (1) allowing the migration of the code of the application from the server tothenodethatwillhave to execute it (2) instantiating the application correctly (3) transferring the results to the user. 2.1 Interaction among layers The resources in the Resource Layer are divided into clusters, according to criteria such as locality (e.g. grouping together resources in the same networking domain) or functionality (e.g. resources providing similar functions). Inside each cluster we have two kinds of nodes with respect to the storage and the execution of the applications: Computational Node (CN) Figure 1: Layer Structure of our Architecture Application Server (AS) Computational Nodes are those on which applications can be executed (execution nodes). Application Servers are nodes which act as servers to store applications (storage nodes). The system therefore comprises centres to collect applications (AS) which play a role equivalent to that of the mass storage subsystem in a traditional processing system. CNs, on the other hand, have the dual role of system access points and processing resources on which to execute applications. In general, however, nothing prevents a generic node from simultaneously having the role of CN and AS: it can, in fact, both store and execute applications. Each cluster typically consists of a group of close computers (belonging to the same LAN). This way we can overcome both the issue of the limited number of 2

3 workstation (many clusters can be present), and the bottleneck caused by the manager, because each cluster will have its own manager. The proposed architecture, as well as having a good scalability and an intrinsically parallel nature, has a good level of fault tolerance. In fact, the system would be able to perform its functions even if the manager of a cluster were broken. The computation skill would probably be reduced (some CNs would be missing), and some applications would no longer be available (the only AS containing such application might all be resident in this cluster). However, the system would continue to operate, and the resources contained in the active clustersmightbeusedagain. As weshowed in Fig. 1, the Broker Layer has the structure of a n-ary tree. The leaves of the tree, called Terminal Nodes (TN), manage the resource clusters and interface with the resource layer. All the nodes of the tree are a point of access for the user layer. The highest the point of access in the hierarchy is, the best the visibility of the resources by the user is. Let us dene Internal Nodes (IN) the non-terminal nodes of the tree. The root node (RN), that is, the root of the tree, is an IN without parents. The query generated by a user logged in at an IN of level i is propagated to the IN of level i +1, until it reaches the terminal nodes. They therefore communicate the availability of the resources of their cluster to meet the query, according to the current state of the system. Through the propagation of the information obtained from the TNs, the INs select the best couple of nodes and communicate it to the father node, until the IN to which the application was requested is reached. The latter noties the results to the User Layer that, exploiting the System Layer, executes the application. The parameters to be taken into consideration for the selection of the optimal couple (AS CN) are: State of the resource. For a CN we need to monitor the workload of the processor, the free memory, the swap memory, the number of page-in and page-out. Other static parameters such as the type of cpu used and the quantity of total memory are supplied while doing the conguration of the system. Conversely, for an AS we need to know the disk throughput and the average access time. Load level of the network, that is, the average delay of transmission among the nodes of the system. Since the transfer of the code of an application from an AS to a CN is considered, while selecting the best couple we also have to consider the time required for such transfer. In fact, the execution of the application on a less powerful CN (but closer to the AS) might be convenient, rather than using a faster, but very distant CN. A software module is therefore active on all the system resources (CN and AS). It operates as an agent, and is in charge of monitoring the performance parameters of the resource and communicating them to the TN manager of the cluster, for enabling it to execute the optimal selection. Furthermore, each TN gives the IN of the higher level an indication of the delay of communication with the resources of its cluster and of the distance (delay) with all the other TNs. Each TN therefore has to monitor the state of the network continuously, in order to know the delay of communication with the nodes of its cluster and with the other TNs. The mechanism which enables us to implement such function is described below. 3 REBELS implementation In this section we present the system REBELS (REmote Execution BasEd Load-balancing System) that is an implementation of the approach to the load balancing for remote execution that we described before. REBELS is available at the following Web address: REBELS enables us to perform the remote execution of applications present in the network, by balancing the computational and communication load according to adequate strategies of Load Balancing. Since a distributed system typically consists of heterogeneous machines, in the implementation of REBELS we decided to use a multiplatform language that in this specic case is Java by Sun MicroSystems [7]. Furthermore, due to its high features of portability, we assumed that the applications executable by the user are written in the same language. This assures the possibility to execute the same application on dierent hardware/software platforms, with no need for recompiling their code. As we specied below, we will take into consideration both the execution of batch-like Java applications, and of interactive type 1. The architecture at the basis of REBELS is the one shown in Fig. 1. The classes implementing broker, resource, user and system layer are grouped in packages with the same name. In the following sections we analyze the implementation selections in detail, regarding the single functional units. 3.1 Resource Layer An agent is instanced on each resource (both AS and CN). Its task is to monitor their state and to communicate any change to the terminal node associated with the cluster. An ExecutionManager is also instanced on each CN. Its task is to nd the application from the AS (through the services provided by the System Layer), to create the execution environment of the application (working directory 1 From now on,by application we will mean Java applications 3

4 indicated by the user) and to launch the application with the appropriate parameters dened by the user. Once the execution is completed, the execution Manager sends the results (working directory) to the module User Proxy, which noties the user about the completion of the execution. We need to outline that, once the selection of the couple of best nodes has been made, the broker layer is not involved in the actual execution of the application. In fact, the involved entities are the User Proxy associated with the user and the execution Manager, that communicate through the services put at our disposal by the System Layer. One of the tasks carried out by the Resource Layer is the monitoring of the use of the resources. For accessing some of such parameters, we need to use functions that depend on the operating system and the hardware architecture of the machine to be examined. Some of the software modules needed for reading such information cannot therefore be eectively developed by using Java, since the overhead introduced is often excessive. Furthermore, Java (due to its portability features) does not have adequate mechanisms for the interaction with the operating system. For recalling some low-level primitives, the use of languages such ascismore appropriate. It allows a more complex and direct monitoring of the harware and software features of the machine. The agents of monitoring of the resources require such potentialities, and therefore the issue of the integration of modules written in C within REBELS aroused. The system CORBA [9] gives us a good solution to this issue. In fact, CORBA enables us to integrate software modules developed in dierent languages and prepared for the execution on dierent hardware architectures. The agents in C were therefore integrated in REBELS by dening some interfaces in IDL (Interface Description Language). 3.2 Broker Layer The Broker Layer is implemented through a tree structure whose nodes (IN) are the decisional elements of the load balancing system. The nal elements of such tree are called Terminal Node (TN), and their role is to manage the clusters of resources. Each TN gives the IN information regarding the volume of communication (delay) in its cluster, and between it and the other clusters. Furthermore, each TN keeps a table of state, whose entries consist of the state of the generic resource of its cluster. By default, the TN has a passive role with regards the updating of this table. In fact, it assumes that the resource charges of communicating any variation of its state. Finally, each TN keeps a second table containing the applications present in the Application Servers of its cluster. This table is propagated to the INs of the Broker Layer, in order to give an image of the applications present in the subtree that derived from them. The strategy of load balancing takes place through a distributed process involving the TNs and the INs. The Terminal Nodes collect the information about the state of the resources and the computational requirements of the single applications. The Internal Nodes implement a cost function ; for the selection of the best couple of resources, and also take into consideration the delays of communication on the network. The cost function ; used by an IN might be dierent than the one used by the other INs. In fact, the proposed architecture gives the greatest exibility. Furthermore, as we have already pointed out, our purpose is not to dene an algorithm of load balancing (there is a wide literature regarding this topic [4, 6, 3, 1]), but to supply a valid support for experimenting existing strategies and implementing any new ones. However, the used cost functions gave valid results, as it is shown by the tests carried out. REBELS can be used with dierent algorithms of balancing active simultaneously, and is a platform for developing, implementing and testing load balancing strategies. 3.3 User Layer The task of the User layer is to supply the user with a vision of the resources and of the applications present in the system, leaving out of consideration their location. Each userisassociated with an agent called User Agent that gives the point of access to the system. Its task is to send the execution queries of an application to the system. The User Agent creates an instance of the class UserProxy for each query. Such agent will establish a connection with the CN in charge of executing the application by interacting (as we said before) with the module ExecutionManager present in the CN. The connection between these two entities is released at the completion of the execution of the application required by the user. Figure 2: The REBELS Browser GUI After typing login and password at the prompt of REBELS, and indicating the IN to connect to, the user is presented the window of Fig. 2, where the applications 4

5 present in the system, catalogued according to the areas of interest, can be browsed. Each application is indicated only once, even if it is present in several AS. For obtaining information about an application, we just need to select it and press on the button VIEW. A window containing a description of the main features of the application will appear. Once the execution of the application is completed, REBELS shows a window indicating the return state of the process. 3.4 System Layer The System Layer supplies two support services to the execution of the applications used by Resource, Broker and User layers. The rst service consists of transferring the results of the application from the CN to the user that had required their execution. This is made through by transferring a portion of le system the implemented classes adapt the dierent representations of the le system among dierent operating systems, as well as actually transferring the les contained in the directory indicated. The second service provided is the migration of the code of an application from a node to another where it will be adequately instanced and executed. Such service is based on the use of RMIClassLoader, that allows the dynamic loading at runtime of Java classes from dierent sources. 4 Performance Evaluation In this section we present some experimental measures carried out during the execution of REBELS, considering dierent topologies of system and various load distributions. As we have already said before, two versions of REBELS have been implemented. Their main dierence is that the rst one uses only Java, while the second one uses CORBA for enabling the integration of modules written in C++. In fact, from an analysis of the performances of the version Java, we noticed that the agents of monitoring of the state of resources occupied a very high cpu time. For assuring a suitable updating of the state of the system, we noticed that the sampling period T c to be used is about 3sec. Once we xed T c =3sec, we evaluated that on a processor Pentium 133 MHz with 32 MB of RAM the overhead introduced by the agents of monitoring implemented in Java was on an average of 20 ; 25%. This value is too high, and takes cpu time to the processing of the applications. We thus developed native methods for monitoring the system resources in language C++, and we made their use possible by the TNs, through Visibroker by Visigenic. In the same operational conditions, we therefore noticed an overhead contained between 1 ; 2%. The version Java-CORBA is therefore more eective, and the following tests were therefore carried out on it. The purpose of the presented experiment is to verify the skills of load balancing provided by REBELS. The assumed topology is the one shown in Fig. 3. Overall, 16 computers have been used with processor Pentium 133 MHz with 16 Mbyte RAM and operating system Windows 95. The resources are divided in two clusters each consisting of 2 AS and 4 CNs. The Broker layer consists of a root node directly connected to the TN managing the two clusters. Figure 3: Test topology used The test lasted 12 hours. During this time the computational load on the CNs of the system was monitored. The test application is the same used in the previous experiment, and consists of sorting numbers selected at random. The execution time required on a single CN is 70sec. The system was loaded dierently, according to hours, and the distribution shown in Fig. 4. In an hour range the execution queries of the test application take place according to an exponential distribution. Figure 4: Load distribution The trend of the distribution of the load on the system shown in Fig. 4 was obtained by discretizing the load curve of an academic environment obtained by Calzarossa and Serazzi in [2]. The rate of generation of 5

6 queries in the dierent hourswas obtained by making the load on the system to result equal to 100% in the hour of largest use (between 11:00 and 12:00) and by proportionally reducing such value in the other hours, according to the scale factor obtained from Fig. 4. In particular, if n is the number of available CNs and the service rate of the single CN, the highest service rate S of the whole system is: S = n The highest arrival rate S will be therefore: S = S = n In the case considered, n =8 and = 1. Thus 70 S =0:11428req=sec. Such value is the exponential generation rate of the queries in the range of largest use. For example, from 15 to 18, the rate used will be exactly the half, since the ratio between the use of the system in such hours and the largest use is 0:5. 5 Conclusions We presented REBELS an environment to assist the user in submitting queries to a distributed system in a transparent way. The user does not need specic knowledge of hardware and software architectures available or of the mechanisms for their activation and execution. We focused our attention on remote execution of processes, with the aim to balance the load on the system and minimize the mean execution time. The REBELS architecture was described and implementation issues based on the use of Java technology were addressed. A performance evaluation of REBELS was carried out and a comparison between a pure Java implementation and the use of CORBA to access C modules devoted to monitor the status of the resources was also provided. References [1] D. Arredondo, M. Errecalde, S. Flores, F. Piccoli, M. Printista, and R. Gallard. Load distribution and balancing support in a workstation-based distributed system. ACM Operating Systems Review, 31(2), April [2] M. Calzarossa and G. Serazzi. A Characterization of the Variation in Time of Workload Arrival Patterns. IEEE Transactions on Computers, C-34(2):156{162, February Figure 5: Load balancing through REBELS Thus, by loading the system according to the described scheme, we could obtain the graphic of Fig. 5, which shows the average use of CPU of the CNs present in the system. As we can see, the 8 curves have a similar progress, which conrms an actual load balancing on the resources. In particular, we notice how during the period of highest load, between 11 and 12, all CNs are actually subject to the largest use (100%), while in the areas where the load is limited we notice considerable dierences. For example, between 13 and 14, when the interarrival rate is 10 times lower than the highest one, a CN has a use of 40% while other have only 20%. This because in low load conditions the applications are often completed before the following query is done, and the same CN can therefore be selected again. Furthermore, this test enabled us to evaluate the mean time spent by REBELS for selecting the best couple (AS, CN). The value obtained is of about 500msec, thus negligeable in comparison with the total execution time of the application (70sec). [3] A. Chavez, A. Moukas, and P. Maes. Challenger: A Multi-agent System for Distributed Resource Allocation. The First International Conference on Autonomous Agents, AA'97, February [4] D. L. Eager, E. D. Lazowska, and J. Zahorjan. Adaptive Load Sharing in Homogeneous Distributed Systems. IEEE Transactions on Software Engineering, SE-12(5), May [5] M. R. Genesereth and S. P. Ketchpel. Software agents. Communications of the ACM, 37(7):48{53, [6] T.V. Gopal, N.S. Karthic Nataraj, C. Ramamurthy, and V. Sankaranarayanan. Load balancing in heterogeneous distributed systems. Microelectronics Reliability, 36(9), September [7] J. Gosling. The Java Language Environment: a White Paper. Technical report, Sun Microsystems, May [8] T. G. Lewis. Where is client/server software headed? IEEE Computer, pages 49{55, April [9] R. Zahavi T.J. Mowbray. The essential CORBA: Systems Integration Using Distributed Objects. JohnWiley &Sons, Inc.,