Agent-Based Grid Load Balancing Using Performance-Driven Task Scheduling

Transcription

1 Agent-Based Grid Load Balaning Using Performane-Driven Task Sheduling Junwei Cao *1, Daniel P. Spooner, Stephen A. Jarvis, Subhash Saini and Graham R. Nudd * C&C Researh Laboratories, NEC Europe Ltd., Sankt Augustin, Germany Department of Computer Siene, University of Warwik, Coventry, UK NASA Ames Researh Center, Moffett Field, California, USA address for orrespondene: ao@rl-nee.de Abstrat Load balaning is a key onern when developing parallel and distributed omputing appliations. The emergene of omputational grids extends this problem, where issues of ross-domain and large-sale sheduling must also be onsidered. In this work an agent-based grid management infrastruture is oupled with a performane-driven task sheduler that has been developed for loal grid load balaning. Eah grid sheduler utilises preditive appliation performane data and an iterative heuristi algorithm to engineer loal load balaning aross multiple proessing nodes. At a higher level, a hierarhy of homogeneous agents are used to represent multiple grid resoures. Agents ooperate with eah other to balane workload in the global grid environment using servie advertisement and disovery mehanisms. A ase study is inluded with orresponding experimental results to demonstrate that both loal shedulers and agents ontribute to overall grid load balaning, whih signifiantly improves grid appliation exeution performane and resoure utilisation. 1. Introdution A omputational grid is an emerging omputing infrastruture that enables effetive aess to high performane omputing resoures [11]. Resoure management and sheduling are key grid servies, where issues of load balaning represent a ommon onern for most grid infrastruture developers. While these are established researh areas in parallel and distributed omputing, grid omputing environments present a number of new hallenges, inluding: Cross-domain: The proess of grid resoure sheduling enompasses multiple administrative domains, where one domain may be unaware of the resoures offered by another. The lak of entral ownership and ontrol means that it is diffiult to map an entire set of tasks to permutations of the available resoures. Large-sale: As a grid an enompass a large number of high performane omputing resoures that are not entirely dediated to the environment, omputational apabilities an vary signifiantly over time. These dynami properties must be addressed when implementing grid information and resoure management infrastrutures. In our previous work an agent-based methodology was developed for large-sale distributed software systems with highly dynami behaviours [6, 7]. This has been used in the implementation of an agent-based resoure management infrastruture for grid omputing [8, 9], where eah agent represents a loal grid resoure and ats as a servie provider of high performane omputing power. Agents ooperate with eah other to disover available resoures for tasks using a tehnique of servie advertisement and disovery. The researh presented in this paper adopts the agent-based methodology to grid load balaning, ahieved by oupling the agent system with a performane-driven task sheduler that has been developed for loal grid load balaning. Eah loal sheduler uses an iterative heuristi algorithm based on the preditive performane data for eah appliation. The algorithm aims to minimise makespan and proessor idle time, whilst meeting the deadlines set for eah task. The algorithm is based on an evolutionary proess and is therefore able to absorb system hanges suh as the addition or deletion of tasks, or hanges in the number of hosts or proessors available in the loal domain. At a higher level, the hierarhy of homogeneous agents are responsible for dispathing tasks from grid users to eah of the available loal grid shedulers; this is orhestrated 1 This work was arried out when the author was with the University of Warwik.

2 by some mathmaking and deision-making poliies that are also driven by performane predition data. Eah agent is only aware of neighbouring agents and servie advertisement and disovery requests are only proessed among neighbouring agents, whih provides the possibility for saling over large wide-area grid arhitetures. Appliation performane predition provides the essential funtionality that enables the grid load balaning apabilities desribed in this work. The PACE toolkit [15] is used to provide this apability for both the loal shedulers and the grid agents. The main omponents of the PACE toolkit inlude appliation tools, resoure tools, and an evaluation engine. The PACE evaluation engine an ombine appliation and resoure models at run time to produe performane data (suh as total exeution time). In the work desribed in [5] an ASCI (Aelerated Strategi Computing Initiative) kernel appliation, Sweep3D, is used to illustrate the performane predition apabilities of PACE. The validation results show that a high level of auray an be obtained, ross-platform omparisons an be easily undertaken, and the proess benefits from a rapid evaluation time. These features allow PACE preditive data to be used on-the-fly for grid resoure load balaning. There are a number of approahes to sheduling and load balaning parallel and distributed systems. Unlike bath queuing systems, suh as Condor [13] and LSF [19], that address resoure management within a loal grid, the loal grid sheduler desribed in this work is based on appliation performane predition. While AppLeS [4] and Ninf [14] are also based on performane evaluation tehniques, they utilise the NWS [18] resoure monitoring servie, as opposed to the performane predition apabilities provide by the PACE toolkit. Nimrod [3] has a number of similarities to this work, inluding a parametri engine and heuristi algorithms [1] for sheduling obs. The kernel of our loal grid sheduler is a geneti algorithm (GA) [20]. Some existing systems use the Globus toolkit [10] to integrate with the grid omputing environment, inluding Condor- G [12], Nimrod/G [2] and Ninf-G [17]. The sheduler introdued in this work is distributed for grid omputing using an agent-based methodology; where agents are used to ontrol the query proess and to make resoure disovery deisions based on internal logi rather than relying on a fixed-funtion query engine. Agent-based resoure disovery is also used in [16], where eah agent either represents a user appliation, a resoure, or a mathmaking servie. Rather than using a olletion of predefined speialised agents, this work uses a hierarhy of homogeneous agents that an be reonfigured with different roles at run time. The paper is organised as follows: setion 2 introdues the theory and implementation of the shedulers for loal load balaning; in setion 3, the agent-based implementation of a grid load balaning system is desribed; a ase study with experimental results is inluded in setion 4 and the paper onludes in setion Performane-Driven Task Sheduling In this work, a loal grid is onsidered to be a network of proessing nodes (suh as a multiproessor or a luster of workstations). Load balaning issues are addressed at this level using task sheduling algorithms driven by PACE performane preditions. 2.1 Geneti Algorithm Consider a grid resoure P with n proessing nodes. A PACE resoure model i an be used to desribe the performane information of eah proessor P i. { P i = 1,2 n} { i = 1,2 n} P = i,..., (1) ρ = ρi,..., (2) A set of parallel tasks T is onsidered to be run on P, where a PACE appliation model inludes the performane related information of eah task T. Additionally, T is speified with a requirement of the appliation exeution deadline from the user. T = { T = 1,2,..., m} (3) σ = { σ = 1,2,..., m} (4) δ = { δ = 1,2 m} (5),..., A shedule is defined by a set of nodes P P (orresponding ρ ρ ) alloated to eah task T and a start time at whih the alloated nodes all begin to exeute the task in unison. The exeution time for eah task T is a funtion, t x ( ρ, σ ), provided by the PACE evaluation engine. The ompletion time of eah task T is defined as: η = τ + t x ( ρ, σ ). (6) The makespan,, for a partiular shedule, whih represents the latest ompletion time of any task, is subsequently defined as: 1 m { η } ω = max, (7)

3 The goal is to minimise funtion (7) with respet to the shedule, at the same time, η δ should also be satisfied as far as possible. In order to obtain near optimal solutions to this ombinatorial optimisation problem, the approah taken in this work is to find shedules that meet the above riteria through the use of an iterative heuristi method in this ase a geneti algorithm. The proess involves building a set of shedules and identifying solutions that have desirable harateristis. These are then arried into the next generation. The tehnique requires a oding sheme that an represent all legitimate solutions to the optimisation problem. Any possible solution is uniquely represented by a partiular string, S k, and strings are manipulated in various ways until the algorithm onverges on a near optimal solution. In order for this manipulation to proeed in the orret diretion, a method of presribing a quality value (or fitness) to eah solution string is required. The algorithm for providing this value is alled the fitness funtion f v. host task ordering map of map of map of map of map of map of task #3 task #5 task #2 task #1 task #6 task #4 time task #1 task #2 task #3 task #4 task #5 task #6 Figure 1. An Example Coding Sheme and Corresponding Gantt Chart The oding sheme we have developed for this problem onsists of two parts: an ordering part, whih speifies the order in whih the tasks are to be exeuted and a mapping part, whih speifies the alloation of proessing nodes to eah task. The ordering of the taskalloation setions in the mapping part of the string is ommensurate with the task order. An example of a solution string and its assoiated shedule are shown in Figure 1. The exeution times of the various tasks are provided by the performane predition system and are assoiated with the task obet for evaluation by the fitness funtion f v. A ombined ost funtion is used whih onsiders makespan, idle time and deadline. It is straightforward to alulate the makespan, k, of the shedule represented by any solution string S k. The nature of the idle time should also be taken into aount. Idle time at the front of the shedule is partiularly undesirable as this is the proessing time whih will be wasted first, and is least likely to be reovered by further iterations of the GA or if more tasks are added. Solutions that have large idle times are penalised by weighting pokets of idle time to give k, whih penalises early idle time more than later idle time. The ontrat penalty k is derived from the expeted deadline times and task ompletion time. The ost value for a shedule, represented by a solution string S k, is derived from these metris and their impat predetermined by: f k m i W ωk + W ϕk + W θ k = m i W + W + W The ost value is then normalised to a fitness value using a dynami saling tehnique: f k v max max k min (8) f f =, (9) f f where f max and f min represent the best and worst ost value in the sheduling set. The geneti algorithm utilises a fixed population size and stohasti remainder seletion. Speialised rossover and mutation funtions are developed for use with the two-part oding sheme. The rossover funtion first splies the two ordering strings at a random loation, and then reorders the pairs to produe legitimate solutions. The mapping parts are rossed over by first reordering them to be onsistent with the new task order, and then performing a single-point (binary) rossover. The reordering is neessary to preserve the node mapping assoiated with a partiular task from one generation to the next. The mutation stage is also two-part, with a swithing operator randomly applied to the ordering parts, and a random bit-flip applied to the mapping parts. 2.2 System Implementation A loal grid sheduling system is developed in Java to implement the algorithm desribed above. It uses the PACE evaluation engine for appliation performane predition data with several other modules inluded for

4 task management, exeution, and resoure monitoring. The system implementation is illustrated in Figure 2. Request Info. Task Management Task Info. Appliation Models Results Task Exeution Shedule Info. GA Sheduling Evaluation Results PACE Evaluation Engine Resoure Info. Resoure Monitoring Resoure Info. Resoure Models Figure 2. The System Implementation of a Performane-Driven Loal Grid Sheduler Task management. Requests are passed to the task management module where they queue for sheduling and exeution. Eah task is given a unique identifiation number and awaits the attention of the GA sheduler. Task management also interfaes with the operations on the task queue, inluding adding, deleting or inserting tasks. The task queue is regarded by the GA sheduling as the optimisation set of tasks T. Resoure monitoring. The resoure monitoring is responsible for gathering statistis onerning the proess nodes on whih tasks may exeute. These statistis inlude availability, load average and idle time. Currently, only host availability is supported, where the resoure monitor queries eah known node every five minutes. This is provided to the GA sheduler as the urrently available resoures P on whih tasks an be sheduled. Resoure monitoring is also responsible for organising the GA sheduling results and resoure availabilities into servie information that an be advertised. This is only used when a grid environment is onsidered and will be introdued in the following setions. Task exeution. This is responsible for exeuting the program assoiated with a task on a sheduled list of proessors. Currently, MPI and PVM based programs are supported and the exeutable programs must be pre-ompiled and available in all loal file systems. It is envisaged that this will be extended as the system matures. The PACE evaluation engine is used to produe preditive data on the task exeution time given appropriate appliation and resoure models. While PACE evaluations omplete relatively quikly (usually in the order of a few tenths of a seond), as the searh spae of the GA is large, it is not usually possible to prealulate all required performane estimates ahead of time. A demand-driven evaluation sheme is used, oupled with a ahe of past evaluations. When a partiular evaluation result is requested, the ahe is searhed. If the result already exists, it is returned to the sheduler. Otherwise the PACE evaluation engine exeutes and the result is added to the ahe before being returned. The geneti algorithm and orresponding system implementation provide a fine-grained solution to dynami load balaning on loal grid resoures and aim to maximise resoure utilisation and meet task deadlines. However, this annot provide load balaning servies to a global grid sine the GA does not sale to thousands of proessing nodes and tasks. An additional mehanism for distributing over wide-area grid resoures is required. This is ahieved through the introdution of an agentbased resoure advertisement and disovery framework. 3. Agent-based Grid Load Balaning The problem whih is onsidered in this setion is the disovery of grid resoures that provide the optimum exeution performane for globally grid-submitted tasks. This researh is not aimed at o-alloating a task onto more than one grid resoure. Most MPI or PVM based programs use tightly oupled parallelism and it is diffiult to ahieve high performane when relatively slow ommuniation mediums are involved. 3.1 Agent-based Servie Disovery While agent-based servie disovery and its appliation to grid resoure management has been desribed in detail in previous work [6, 7, 8, 9], a brief introdution is provided whih demonstrates how the onepts apply to grid load balaning. This is extended to inlude a detailed system implementation in setion 3.2. A performane-driven sheduler (desribed in setion 2) is responsible for managing the proessing nodes at a loal level and sheduling the inoming tasks to ahieve loal load balaning. Eah agent provides a high-level representation of eah loal sheduler and therefore haraterises these loal resoures as highperformane omputing servie providers in the wider grid environment. This higher-level representation is enhaned by organising the agents into a hierarhy, where the servie information provided at eah loal grid resoure an be advertised throughout the hierarhy and agents an ooperate with eah other to disover available resoures. Servie advertisement. An agent an advertise servie information to both upper and lower agents.

5 Different strategies an be used to ontrol these proesses, whih has an impat on the system effiieny. Servie information an be pushed to or pulled from other agents, a proess that is triggered by system events or through periodi updates. Servie disovery. Disovering available servies is also a ooperative ativity. Within eah agent, its own servie is evaluated first. If the requirement an be met loally, the disovery ends suessfully. Otherwise servie information from both upper and lower agents is evaluated and the request dispathed to the agent whih is able to provide the best requirement/resoure math. If no servie an meet the requirement, the request is submitted to the upper agent. When the head of the hierarhy is reahed and the available servie is still not found, the disovery terminates unsuessfully (representing a request for a omputing resoure whih is not supported by the available grid). While the proesses of servie advertisement and disovery is not motivated by grid sheduling and load balaning, it an result in an indiret oarse-grained load balaning effet. A task tends to be dispathed to a grid resoure that has less workload and an meet the appliation exeution deadline. The disovery proess does not aim to find the best servie for eah request, but endeavours to find an available servie provided by a neighbouring grid resoure. While this may derease the load balaning effet, the trade-off is reasonable as grid users prefer to find a satisfatory resoure as fast and as loal as possible. The advertisement and disovery mehanisms also allow possible system salability. Most requests are proessed in a loal domain and need not to be submitted to a wider area. Both advertisement and disovery requests are proessed between neighbouring agents and the system has no entral struture whih might at as a potential bottlenek. While further work is neessary to test the salability of the system, the experiments desribed in setion 4 illustrate the ontribution of both loal shedulers and agents to the proess of grid load balaning. These experiments use the initial implementation as desribed below. 3.2 System Implementation The system arhiteture an be found in Figure 3. Agents are implemented using Java and data are represented in an XML format. Different PACE tools are integrated for different uses. The loal shedulers are performane-driven - as desribed in setion 2. The detail of how the agents utilise the PACE performane data and work with loal shedulers to ahieve grid load balaning is desribed. User Portals Agent Hierarhy Loal Shedulers Grid Resoures Agent-based grid load balaning Appliation Tools Evaluation Engine Resoure Tools PACE performane predition Figure 3. The System Arhiteture for Agent- Based Grid Load Balaning Loal shedulers are responsible for the submission of servie information to the agents (see setion 2.2). An example of this servie information an be found in Figure 4. <agentgrid type= servie > <agent> <address>gem.ds.warwik.a.uk</address> <port>1000</port> </agent> <luster> <address>gem.ds.warwik.a.uk</address> <port>10000</port> <type>sunultra10</type> <npro>16</npro> <environment>mpi</environment> <environment>pvm</environment> <environment>test</environment> <freetime>nov 15 04:43: </freetime> </luster> <agentgrid> Figure 4. An Example Servie Information The identity of a loal sheduler and its orresponding agent is provided by a tuple of the address and port used to initiate ommuniation. The hardware model and the number of proessors is also provided. The example speifies a single grid resoure, in this ase a luster of 16 SunUltra10 workstations. To simplify the problem, the proessors within eah grid node are onfigured to be homogeneous. The appliation exeution environments that are supported by the urrent implementation of the loal shedulers inlude MPI, PVM, and a test mode that is designed for the experiments desribed in this work. Under test mode, tasks are not atually exeuted and the preditive appliation exeution times are sheduled and assumed to be aurate. The latest GA sheduling makespan indiates the earliest (approximate) time that orresponding proessors beome available for more tasks. Due to the effet of GA load balaning, it is

6 reasonable to assume that all proessors within a grid have approximately the same freetime. The agents use this item to estimate the workload of eah grid resoure and make deisions on where to dispath inoming tasks. This item hanges over time and must be frequently updated. Servie advertisement is therefore important among the agents. A portal has been developed whih allows users to submit requests destined for the grid resoures. An example request is given in Figure 5. <agentgrid type="request"> <appliation> <name>sweep3d</name> <binary> <file>binary/sweep3d</file> <inputfile>binary/input.50</inputfile> </binary> <performane> <datatype>paemodel</datatype> <modelname>model/sweep3d</modelname> </performane> </appliation> <requirement> <environment>test</environment> <deadline>nov 15 04:43: </deadline> </requirement> < >unwei@ds.warwik.a.uk</ > </agentgrid> Figure 5. An Example Request Information A user is required to speify the details of the appliation, the requirements and ontat information for eah request. Appliation information inludes binary exeutable files and also the orresponding PACE appliation performane model r, whih is generated using the appliation tools embedded in the user portal. In the urrent implementation we assume that both binary and model files are pre-ompiled and available in all loal file systems. In the requirements, both the appliation exeution environment and the required deadline time r should be speified. Currently the user s address is used as the ontat information to whih the results of the submission are posted. Servie disovery proesses are triggered by the arrival of a request at an agent, where the kernel of this proess is the mathmaking between servie and request information. This math is straightforward if a loal grid sheduler an provide the required appliation exeution environment. The expeted exeution ompletion time for a given task on a given resoure an be estimated using: { ( ρ, )} η = ω + min t σ. (10) r x r P P, P Φ, ρ ρ, ρ Φ For a homogeneous loal grid resoure, the PACE evaluation funtion is alled n times. If ηr δ, the r resoure is onsidered to be able to meet the required deadline. Otherwise, an appropriate loal resoure is not onsidered available for the inoming task. This performane estimation of loal grid resoures at the agent level is simple but effiient. However, when the task is dispathed to an available resoure, the real situation may differ from the senario onsidered in (10). The loal GA sheduler may hange the task order and advane or postpone a speifi task exeution in order to balane the workload on different proessors, and in so doing maximise resoure utilisation whilst maintaining the deadline ontrats of eah user. Servie disovery for a request within an agent involves multiple mathmaking proesses. An agent always gives priority to the loal sheduler. Only when the loal resoure is unavailable is the servie information of other grid resoures evaluated and the request dispathed to another agent. In order to measure the effet of this mehanism for grid sheduling and load balaning, several performane metris are defined Performane Metris There are a number of performane riteria that an be used to desribe grid resoure management and sheduling systems. What is onsidered as high performane depends on the system requirements. In this work three ommon statistis are investigated quantitatively and used to haraterise the effet of grid load balaning. These inlude: average advane time of appliation exeution ompletion, average resoure utilisation rate and load balaning level. During a period of time t, a set of M parallel tasks T are sheduled onto grid resoures P with N proessing nodes. Note that the proessing nodes may inlude those at all loal grids. The final sheduling senario an be desribed using the alloation to eah task T (with deadline ) a set of nodes P P and a time domain [, ] during whih the alloated nodes are simultaneously utilised for task exeution. The average advane time of appliation exeution ompletion an be alulated diretly using: M ( δ η ) = ε = 1, (11) M whih is negative when most deadlines fail. The resoure utilisation rate i of eah proessing node P i is alulated as follows: ( η τ ) P i P υ i =,. (12) t

7 The average resoure utilisation rate of all proessing nodes P is: N υi i= υ = 1, (13) N where is in the range 0 1. The mean square deviation of i is defined as: d = N i= 1 ( υ υ ) N i 2, (14) and the relative deviation of d over that desribes the load balaning level of the system is: d β =1. (15) υ The most effetive load balaning is ahieved when d equals zero and equals 1. The three aspets of the system desribed above are interrelated. For example, if the grid workload is balaned aross all the proessing nodes, the resoure utilisation rate is usually high and the tasks finish quikly. These are illustrated in the ase study desribed in the next setion. 4. A Case Study Three experiments have been designed using the system implementations desribed in setions 2 and 3 to evaluate different load balaning onfigurations. Experimental results regarding the three performane metris are also inluded to illustrate the ontribution of loal shedulers and agents for grid load balaning to improve both resoure utilisation and appliation exeution Experimental Design The experimental system is onfigured with twelve agents, illustrated by the hierarhy shown in Figure 6. These agents are named S 1 S 12 (for the sake of brevity) and represent heterogeneous hardware resoures ontaining sixteen proessing nodes per resoure. As shown in Figure 6, the resoures range in their omputational apabilities. The SGI multi-proessor is the most powerful, followed by the Sun Ultra 10, 5, 1, and SPARCStation 2 in turn. In the experimental system, eah agent maintains a set of servie information for the other agents in the system. Eah agent pulls servie information from its lower and upper agents every ten seonds. All of the agents employ idential strategies with the exeption of the agent at the head of the hierarhy (S 1 ) that does not have an upper agent. S 2 (SGIOrigin2000, 16) S 6 (SunUltra5, 16) S 3 (SunUltra10, 16) S 7 (SunUltra5, 16) S 8 (SunUltra1, 16) S 1 (SGIOrigin2000, 16) S 4 (SunUltra10, 16) S 9 (SunUltra1, 16) S 10 (SunUltra1, 16) S 5 (SunUltra5, 16) S 11 (SunSPARCstati on2, 16) S 12 (SunSPARCstati on2, 16) Figure 6. Case Study: Agents and Resoures The appliations used in the experiments inlude typial sientifi omputing programs. Eah appliation has been modelled and evaluated using PACE. An example of the PACE prediations for the system S 1 (whih represents the most powerful resoure in the experiment) an be found in Table 1. Table 1. Case Study: Appliations and Requirements sweep3d [4,200] fft [10,100] impro [20,192] losure [2, 36] aobi [6,160] memsort [10,68] pi [2,128] As shown in the table, different appliations have different performane senarios whih has a signifiant impat on the task sheduling results. During eah experiment, requests for one of the seven test appliations are sent at one seond intervals to randomly seleted agents. The required exeution time deadline for the appliation is also seleted randomly from a given domain; the bounds of the appliation requirements an also be found in Table 1. The request phase of eah experiment lasts for ten minutes during whih 600 task exeution requests are sent out to the agents. While the seletion of agents, appliations and requirements are random, the seed is set to the same so that the workload for eah experiment is idential.

8 While the experiments use the same resoure onfigurations and appliation workloads, different ombinations of loal grid sheduling algorithms and global load balaning mehanisms are applied as shown in Table 2. Table 2. Case Study: Experimental Design FIFO algorithm Iterative heuristi algorithm Agent-based servie disovery Experiment Number The GA algorithm and agent-based servie disovery are introdued respetively in setions 2 and 3. The GA sheduling is ompared with the results from a simpler first-in-first-out (FIFO) algorithm. The FIFO sheduling does not hange the order of tasks. Eah task is sheduled aording to the time at whih it arrives (also driven by the PACE preditive data). All of the possible resoure alloations (a total of possibilities) are tried. As soon as the urrent best solution is found, it is fixed and will not hange as new tasks enter the system. The three performane metris are used to measure the load balaning effet at both the loal and global levels Experimental Results The experimental results are given in Table 3; this inludes the three metris applied to eah agent and to all the grid resoures in the system. Table 3. Case Study: Experimental Results Experiment Number (s) (%) (%) (s) (%) (%) (s) (%) (%) S S S S S S S S S S S S Total Experiment No. 1 In the first experiment, the FIFO algorithm is used by eah loal grid sheduler with no supporting higherlevel agent-based mehanism provided. The algorithm does not onsider makespan, idletime or deadline. Eah sheduler reeives approximately 50 task requests on average, whih results in only the powerful platforms (SGI multiproessors S 1 and S 2 ) meeting the requirements. The slower mahines inluding the Sun SPARCstations lusters S 11 and S 12 impose serious delays in task exeution with long task queues. The overall average delay for task exeution is approximately 8 minutes. It is apparent that the high performane platforms are not utilised effetively, and the lak of proper sheduling overloads resoures like S 11 that is only 44% utilised. The average utilisation of grid resoures is only 26%. The workload for eah of the proessing nodes in eah grid resoure is also unbalaned. For example the load balaning level of S 12 is as low as 46%. The overall grid workload is also unbalaned at 31%. Experiment No. 2 In experiment 2, the GA algorithm is used in plae of the FIFO algorithm although no higher-level agentbased load-balaning mehanism is applied. The algorithm aims to minimise makespan and idletime, whilst meeting deadlines. Compared to those of experiment 1, almost all metris are improved. Task exeutions are ompleted earlier and the average task exeution delay is redued to approximately 5 minutes. However, resoures suh as S 11 and S 12 remain overloaded and the GA is not able to find solutions that satisfy all the deadlines. Generally, resoures are better utilised as a result of the GA sheduling, suh as the use of S 11 whih inreases from 44% to 73%. The overall average utilisation also improves from 26% to 38%. While load balaning on eah grid resoures is signifiantly improved, the lak of any higher-level loadbalaning mehanism results in an improved overall grid load balaning to 42% (as opposed to 31% in experiment 1). Experiment No. 3 In experiment 3, the agent-based load-balaning mehanism is enabled for high-level resoure disovery. This results in a new distribution of requests to the agents, where the more powerful platform reeives more requests. As a result, the maority of task exeution requirements an be met and all grid resoures are well utilised (80% on average). The load balaning of the overall grid is signifiantly improved from 42% (in experiment 2) to 90%. The load balaning on the grid resoures suh as S 1 and S 2 are only marginally improved by the GA sheduling when the workload is higher. No other agents show an improvement in loal grid load balaning. The experimental results in Table 3 are also illustrated in Figures 7, 8 and 9, whih illustrate the effet on the performane metris given in setion 3.3. The urves indiate that different platforms exhibit

9 different trends when the system is onfigured with loal and global load balaning mehanisms. Among these, the urves for S 1, S 2, (whih are the most powerful) and S 11, S 12, (whih are the least powerful) are representative and are therefore emphasised, whilst others are indiated using grey lines. The urve for the overall grid is illustrated using a bold line. Appliation Exeution In Figure 7, it is apparent that both the geneti algorithm and agent-based servie disovery ontribute to improving the appliation exeution ompletion. The urve implies that the more a resoure is loaded the more signifiant the effet is. For example, S 1 and S 2 are not overloaded during the three experiments, and therefore the value of only hanges slightly. S 11 and S 12 are heavily overloaded during experiments 1 and 2, and therefore the improvement of in experiments 2 and 3 is more signifiant. The results for S 3 S 10 are distributed between these two extremes. The urve for the overall grid provides an average estimation for all situations, whih indiates that the agent-based mehanism ontributes more towards the improvement in appliation exeutions than the loal shedulers. (s) Experiment Number S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 Total Figure 7. Case Study: Trends I for Experiment Results on Advane Times of Appliation Exeution Completion Resoure Utilisation The urves in Figure 8 illustrate similar trends to those of Figure 7. S 1, S 2 and S 11, S 12 still represent the two extreme situations between whih the other platforms are distributed. The urve for the overall grid indiates that the agent-based mehanism ontributes more to maximising resoure utilisation. However, overloaded platforms like S 11 and S 12 benefit mainly from the GA sheduling, whih is more effetive at load balaning when the workload is high; lightly-loaded platforms like S 1 and S 2 hiefly benefit from the agent-based mehanism, whih an dispath more tasks to them. (%) Experiment Number S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 Total Figure 8. Case Study: Trends II for Experiment Results on Resoure Utilisation rate Load Balaning Curves in Figure 9 demonstrate that loal and global load balaning are ahieved in different ways. While S 1, S 2 and S 11, S 12 are two representative situations, the global situation is not simply an average of loal trends as those illustrated in Figures 7 and 8. In the seond experiment when the GA sheduling is enabled, the load balaning of loal grid resoures are signifiantly improved. In the third experiment, when the agent-based mehanism is enabled, the overall grid load balaning is improved dramatially. It is lear that the GA sheduling ontributes more to loal grid load balaning and agents ontribute more to global grid load balaning. The oupling of both is therefore a good hoie to ahieve grid load balaning at both loal and global levels. (%) Experiment Number S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 Total Figure 9. Case Study: Trends III for Experiment Results on Load Balaning Level 5. Conlusions This work addresses the problem of load balaning in a global grid environment. A GA-based sheduler has

10 been developed for fine-grained load balaning at the loal level. This is then oupled with an agent-based mehanism that is applied to load balane at a higher level. Three experiments are arried out, with defined performane metris used to measure the effet of grid load balaning. The experimental results demonstrate that the agent-based mehanism oupled with performane-driven task sheduling is suitable for grid load balaning. Load balaning an have large impat on both task exeutions and resoure utilisation. In most situations, good load balaning results in task exeution ompleting earlier with a better utilisation of grid resoures. Higher level load-balaning mehanisms oupled with loal level sheduling will provide a more effetive solution to load balaning on the overall grid than the independent use of either approah. The results presented in this paper will be ompared with other similar systems. Future enhanement to the system will inlude the integration with other grid toolkits (e.g. Globus MDS and NWS). Experiments to test the salability of the system will be arried out on a grid test-bed being built at Warwik. Aknowledgements This work is sponsored by grants from the NASA AMES Researh Center (administered by USARDSG, ontrat No. N C-9012), the EPSRC (ontrat No. GR/R47424/01), the EPSRC e-siene Core Programme (ontrat No. GR/S03058/01), and the NEC. The authors would like to express their gratitude to previous members of the group at Warwik, inluding Dr. S. C. Perry, Dr. J. S. Harper and Dr. D. J. Kerbyson, for their ontribution to this work. Referenes [1] A. Abraham, R. Buyya, and B. Nath, Nature s heuristis for sheduling obs on omputational grids, in Pro. 8 th IEEE International Conferene on Advaned Computing and Communiations, Cohin, India, [2] D. Abramson, J. Giddy, and L. Kotler, High performane parametri modelling with Nimrod/G: killer appliation for the global grid? in Pro. 14 th International Parallel and Distributed Proessing Symposium, Canun, Mexio, [3] D. Abramson, R. Sosi, J. Giddy, and B. Hall, Nimrod: a tool for performing parameterized simulations using distributed workstations, in Pro. 4 th IEEE International Symposium on High Performane Distributed Computing, Pentagon City, VA, USA, [4] F. Berman, R. Wolski, S. Figueira, J. Shopf, and G. Shao, Appliation-level sheduling on distributed heterogeneous networks, in Pro Superomputing, Pittsburgh, PA, USA, [5] J. Cao, D. J. Kerbyson, E. Papaefstathiou, and G. R. Nudd, Performane modelling of parallel and distributed omputing using PACE, in Pro. 19 th IEEE International Performane, Computing and Communiation Conferene, pp , Phoenix, AZ, USA, [6] J. Cao, D. J. Kerbyson, and G. R. Nudd, Dynami appliation integration using agent-based operational administration, in Pro. 5th International Conferene on the Pratial Appliation of Intelligent Agents and Multi-Agent Tehnology, pp , Manhester, UK, [7] J. Cao, D. J. Kerbyson, and G. R. Nudd, High performane servie disovery in large-sale multi-agent and mobile-agent systems, Int. J. Software Engineering and Knowledge Engineering Speial Issue on Multi-Agent Systems and Mobile Agents 5 (Otober 2001), [8] J. Cao, D. J. Kerbyson, and G. R. Nudd, Performane evaluation of an agent-based resoure management infrastruture for grid omputing, in Pro. 1 st IEEE International Symposium on Cluster Computing and the Grid, pp , Brisbane, Australia, [9] J. Cao, S. A. Jarvis, S. Saini, D. J. Kerbyson, and G. R. Nudd, ARMS: an agent-based resoure management system for grid omputing, Sientifi Programming, Speial Issue on Grid Computing 10 (2002), [10] I. Foster and C. Kesselman, Globus: a metaomputing infrastruture toolkit, Int. J. High Performane Computing Appliations 2 (1997), [11] I. Foster and C. Kesselman, The GRID: Blueprint for a New Computing Infrastruture, Morgan-Kaufmann, [12] J. Frey, T. Tannenbaum, I. Foster, M. Livny, and S. Tueke, Condor-G: a omputation management agent for multiinstitutional grids, in Pro. 10 th IEEE Symposium on High Performane Distributed Computing, San Franiso, CA, USA, [13] M. Litzkow, M. Livny, and M. Mutka, Condor a hunter of idle workstations, in Pro. 8 th International Conferene on Distributed Computing Systems, pp , San Jose, CA, USA, [14] H. Nakada, M. Sato, and S. Sekiguhi, Design and implementations of Ninf: towards a global omputing infrastruture, Future Generation Computing Systems 5-6 (1999), [15] G. R. Nudd, D. J. Kerbyson, E. Papaefstathiou, S. C. Perry, J. S. Harper, and D. V. Wilox, PACE a toolset for the performane predition of parallel and distributed systems, Int. J. High Performane Computing Appliations 3(2000), [16] O. F. Rana, D. Bunford-Jones, D. W. Walker, M. Addis, M. Surridge, and K. Hawik, Resoure disovery for dynami lusters in omputational grids, in Pro. 10 th IEEE Heterogeneous Computing Workshop, San Franiso, CA, USA, [17] Y. Tanaka, Ninf-G: grid RPC system based on the Globus toolkit, in The 2001 Globus Retreat, San Franiso, CA, USA, [18] R. Wolski, N. T. Spring, and J. Hayes, The network weather servie: a distributed resoure performane foreasting servie for metaomputing, Future Generation Computing Systems 5-6 (1999), [19] S. Zhou, LSF: load sharing in large-sale heterogeneous distributed systems, in Pro Workshop on Cluster Computing, [20] A. Y. Zomaya and Y. The, Observations on using geneti algorithms for dynami load-balaning, IEEE Transations on Parallel and Distributed Systems 9 (September 2001),