Supporting Interactive Application Requirements in a Grid Environment

Transcription

1 Supporting Interactive Application Requirements in a Grid Environment Antonella Di Stefano, Giuseppe Pappalardo, Corrado Santoro, Emiliano Tramontana University of Catania, Italy 2 nd International Workshop on Distributed Cooperative Laboratories Santa Margherita Ligure, April 16 18, 2007

2 Problem Statement Traditional Grid middlewares are batch-oriented For interactive applications, new issues arise: interactivity; QoS concerns; soft-real-time requirements. These aspects are related to Grid resource finding, usage and monitoring.

3 Problem Statement 1 Identify/Extract application requirements; 2 Determine the resource(s) needed for those requirements; 3 Find the node able to offer such resources; 4 Reserve the resources; 5 Enforce requirements, checking that QoS is being met, at runtime. Grid Research Lab@UniCT is working on a Grid infrastructure taking into account such aspects. Two research projects: TriGRID VLab, PI2S2/COMETA

4 To meet application requirements, it is necessary to know the amount of CPU Network Disk space Worst-case execution Time (WCET) This is needed for each thread, if the application is parallel (multi-thread).

5 The JJPRO: Java Job PROfiler Our approach: to statically analyze an application in order to extract the requirements. We developed a Java Job PROfiler that performs static bytecode analysis and determines three totals: op disk, number of opcodes related to disk operations op net, number of opcodes related to network operations op total, number of opcodes, in total wcet, worst-case execution time

6 JJPRO and Resource Usage Then, the resource usage (rates) is determined as: Disk usage, du = op disk op total Network usage, nu = op net op total Processor usage, pu = 1 du pu This is repeated for each thread. Job Profile: t, (du t, nu t, pu t, wcet t ).

7 Once the job profile has been determined (du, nu, pu, wcet), the node offering the amount of resources needed has to be found. Current approaches (e.g. EDG matchmaker/broker, Globus MDS) are based on a hierarchy of repositories. Two main disadvantages Efficiency: a latency between the time instant in which the resource is monitored and the instant in which its value is read from the repository to perform matchmaking. Scalability: increasing grid dimension implies a growth of repository information and complexity of its management.

8 Our Proposal Exploiting peer-to-peer approaches. nodes are interconnected by an overlay network. Each node is connected to, and interacts with, one or more neighbors. Search is performed by forwarding a resource request to neighbours until a node is found. Forwarding strategy is based on navigating a surface of potential field.

9 Abstraction Grid nodes + the amount of resource available on each node form a 3D surface. Each node features a mass M( ) proportional to the amount of available resource. Valleys correspond to nodes with a large amount of available resource. Hills correspond to nodes with a small amount of available resource. Algorithm: navigating the surface, using kinematic laws, in searching for the global minimum (or a minimum close to global).

10 The Algorithm The resource request is modeled as a capsule, with a certain amount of initial kinetic energy and a direction (up, down). The capsule has also a mass, which is proportional to the amount of requested resource. The capsule is attracted by nodes with lower masses the lower the mass, the higher the resource availability. When the capsule reaches a node, a difference of potential, using function PD(, ), is computed by using the masses of the current node and each neighbor....

11 The Algorithm (2) Two heuristics for node selection. If PD(Source, Dest) < 0 (descending path) the capsule gains energy and heuristic H 2 is used during descending. If PD(Source, Dest) > 0 (ascending path) the capsule loses energy and heuristic H 1 selects the next neighbor during climbing. A friction δ(source, Dest) > 0 is always considered.

12 The Algorithm (3) energy energy + PD(Source, Dest) δ(source, Dest). When the capsule reaches a minimum in the surface and it has no more energy to overcome any hill, the current node is selected for allocation. When the capsule reaches the node selected for allocation, capsule s mass is added to node s mass. This operation also corresponds to resource reservation.

13 Resource usage has to be monitored at runtime: 1 to ensure that the requested QoS is met; 2 to avoid overutilization; 3 to check soft-real-time deadlines (if present). This implies a continuous monitoring of application activities.

14 Monitoring with Aspects Our solution uses aspect-oriented techniques and computational reflection. Two components (aspects) for each thread of the application: 1 Remon monitors resource usage 2 DeadlineSensor monitors soft-real-time deadlines

15 Remon and DeadlineSensor

16 We implemented the framework described; for the resource finding algorithm, we wrote a simulator. We tested the algorithm by using three different strategies for heuristics: Deepest Slope (DS). H 1 highest mass, H 2 lowest mass. Aim: to find minimums as soon as possible and exit from local minimums as soon as possible. Minimum Energy (ME). H 1 lowest mass, H 2 lowest mass. Aim: to mimic a physical process by guiding the capsule always towards lower energy zones. Random (R). H 1 random mass, H 2 lowest mass. Aim: to evaluate the advantages of a choice based on mass values rather than a random approach.

17 Simluation Results TESTBED: square grid, resource allocation requests with a mass randomly chosen in [0, 1]. FUNCTIONS: PD(N 1, N 2 ) = M(N 1 ) M(N 2 ), δ(n 1, N 2 ) = M(N 1 ) M(N 2 ) H PD(, ) = 0 PARAMETERS (w.r.t. capsule s initial energy): Optimality. How far each solution is from the best one. The difference of the masses of the found node and the global minimum. Fairness. The flatness of the surface, it is an evaluation of how much the resource usage is equally balanced. Cost. The number of steps performed before finding the node.

18 Optimality ME DS R FF Initial Energy FF = First Fit

19 Fairness Problem Statement ME DS R FF Initial Energy FF = First Fit

20 Cost Problem Statement ME DS R FF Initial Energy FF = First Fit

21 Remarks Problem Statement The algorithm terminates (cost is finite). The algorithm tends to find the global minimum (optimality increases). The algorithm tends to equally balance resource leasing among all grid nodes (fairness increases). First Fit is the worst strategy. DS causes the highest cost, ME features the lowest cost. If the initial energy > a threshold, optimality and fairness do not increase; only cost increases. If the initial energy > another threshold, R seems to behave like the other strategies for optimality and fairness, but not for cost. The strategy has to be chosen for each specific application, by considering the weight of each parameter.

22 and Future Work Extraction of application requirements by means of instrumentation. Resource finding and reservation by means of an effective and scalable algorithm. Resource usage monitoring and enforcing by means of a transparent solution. Integration into an existing middleware (Globus/gLite) is ongoing.

23 Supporting Interactive Application Requirements in a Grid Environment Antonella Di Stefano, Giuseppe Pappalardo, Corrado Santoro, Emiliano Tramontana University of Catania, Italy 2 nd International Workshop on Distributed Cooperative Laboratories Santa Margherita Ligure, April 16 18, 2007