Dynamic Network Resources Allocation in Grids through a Grid Network Resource Broker

INGRID 2007 Instrumenting the GRID Second International Workshop on Distributed Cooperative Laboratories Session 2: Networking for the GRID Dynamic Network Resources Allocation in Grids through a Grid Network Resource Broker Davide Adami, Stefano Giordano, Michele Pagano CNIT Research Unit Dept. of Information Engineering - University of Pisa 1

Outline Introduction Target and motivations of the research activity Grid Network Resource Broker Architecture WCBDS (Wang-Crowcroft with Bandwidth and Delay Sorting) Path Computation Algorithm Simulations results Conclusions 2

A parallel renderer/encoder Frame Sequencer GoP Assembler Output Store Parallel Rendering DivX Encoding 3

Instrumentation/Computing Grid Environment High Speed Optical Network with G-MPLS Control Plane Grid Concept A Grid is a collection of distributed computing resources over network that appear to an user or an application as one large virtual computing system 4

Grid Networking Issues 1. A network infrastructure which prevents degrading the throughput of grid applications due to network delay and network fault is required 2. It is necessary to carry out network resource scheduling as well as computing resource scheduling. 3. A network design and deployment methodology for complicated grid networking is necessary. Grid Application Grid Application Grid Application Diffserv-aware MPLS TE network 5

Scheduling Process: Enhanced Deployment Cycle Application nodes are mapped into computing resources Cumulative bandwidth requirements are given Weighted Task Interaction Graph of the application Vertex: Computational Cost Edge: Communication Cost Network Query (list of candidate solutions) Reservation 6

Grid Network-Aware Environment Grid Network Resource Broker GNRB Monitoring and Management Area Grid Application Manager Diffserv-aware MPLS TE network PC Cluster 7

GNRB Functional Blocks Grid Application Grid Application Manager Admission Control Module Path Computation Element Network Information Database Measurement Sampling Modules Visualization GUI Network Resource Scheduler Network Resources Manager Network Monitoring System Measurement Database Network Element Configuration Manager Topology Discovery Service Sampling Sampling Capturing device Capturing device 8

GNRB Architecture Network Resources Manager Policy-based provisioning Path computation Network Resources Scheduling Topology Discovery Service Network Element Configuration Manager Service provisioning GNRB and Network Monitoring System Link utilization QoS measurements (packet loss, delay, jitter) Implemented: SNMP agent for bandwidth utilization Implemented: UNIPI MPLS/DiffServ network traffic monitoring tool 9

Network Monitoring System DiffServ/MPLS network traffic monitoring Graphs for each LSP/PHB pair are available (measured and predicted values) 10

GNRB Network Services Network Topology Discovery Provides information about the topology of the network and QoS metrics associated to the links Best-effort effort connections Weighted Topology Discovery Best paths, according to a metric specified by the GAM are computed by the NRM Network resources may be allocated QoS provisioning Premium service (Peak Rate, Burst Size, Latency) Better than BE service (Mean Rate, Burst Size, Mean Latency) End-to to-end connections with QoS constraints are established 11

Path Computation Algorithm Goal Given a set of N LSP set-up requests, the basic function of the PCE is to find N network paths that satisfy their QoS constraints (Bandwidth B min, Delay D max ) QoS Metrics Bandwidth: Concave metric k l m B(p)=min[B(i,j); B(j,k);.. B(l,m)] j i Delay: Additive metric D(p)=D(i,j)+D(j,k)+ D(l,m) Path p = i,j,k,l,m 12

End-to-end Delay End-to-end Delay Propagation delay Transmission delay Queueing delay Queueing delay: Deterministic Upper Bound Delay for LSP i D i k M = + r j= 1 i S i j M r i = max burst = guaranteed size rate LSP i Node j Delay in case of WFQ scheduling discipline S i j = L R max j + Li r i L L i R max j = max packet size = max packet size LSP i = output link bandwidth 13

The WCBDS Algorithm N Requests WC Algorithm Z Requests accepted Z = N? Yes EXIT N Requests No Bandwidth Based Re-ordering WC Algorithm Wang-Crowcroft Algorithm Z Requests accepted Z = N? No Delay Based Re-ordering Yes EXIT 1. Set d ij = if B ij < B min 2. Compute the path P with the minimum delay 3. Calculate the delay D* of P 4. If D* < D max select the path P otherwise reject the request N Requests WC Algorithm Z Requests accepted Z = N? Yes EXIT No ERROR 14

Wang-Crowcroft Algorithm B 3-30 C B min = 3 Mbps D max =100 ms B min = 3 Mbps A D max =98 ms Rejected! 4-20 6-10 X 2-20 D 4 20 X 2 30 4-20 E 4-10 X 1-30 F 1) Prune the links with B av < B min 2) Find minimum delay path 1. A_B_C_F 2. A_D_E_C_F 1. D = 96.43ms 2. D = 102.49ms 3) Check if D < D max 96.43ms < 100ms 15

NS2 Software Modules Old Modules MNS - MPLS Network Simulator RSVP-TE\ns with Reservation Styles New Modules OSPF-TE\ns MPLS Recovery Strategies Path Computation Algorithm 16

New MPLS Node Architecture in NS2 OSPF-TE module RSVP-TE module 17

Network Topology MPLS Backbone (1, 50) Node17 Node0 (0.3, 100) (2, 10) LSR9 (2, 10) LSR10 Network 0 Network 1 LSR16 LSR4 Node18 (2, 20) (2, 10) Node1 LSR8 Node19 (1, 10) (2, 100) LSR5 (1, 30) (1, 10) LSR14 Node2 Node3 LSR6 (2.5, 15) LSR11 (2.5, 10) LSR7 (2.5, 10) (2, 20) LSR15 (2.5, 10) (2.5, 10) (2.5, 10) (2.5, 10) LSR13 Network 2 Node20 LSR12 (Bandwidth x Mbps, Delay y ms) 18

First Scenario Node0 Network 0 LSR4 LSR9 LSR10 LSR16 Node17 Network 1 Node18 Node1 LSR8 Node19 LSR5 LSR6 LSR14 LSR11 LSR15 Network 2 Node2 LSR7 MPLS Backbone LSR13 Node20 LSR12 Node3 Ingress LER Egress LER Bandwidth (Kbps) Delay (ms) Path Time (ms) 4 16 600 100 4-5-8-10-16 84 16 15 100 20 16-15 11 19 15 4 1800 200 15-14-11-6-4 110

Second Scenario Node0 Network 0 LSR4 LSR9 LSR10 LSR16 Node17 Network 1 Node18 Node1 LSR8 Node19 LSR5 LSR6 LSR14 LSR11 LSR15 Network 2 Node2 LSR7 MPLS Backbone LSR13 Node20 LSR12 Node3 Ingress LER Egress LER Bandwidth (Kbps) Delay (ms) Path Time (ms) 4 16 600 100 4-5-8-10-16 84 16 15 100 20 16-15 11 20 15 4 2400 200 15-13-12-7-6-4 95

Third Scenario Node0 Network 0 LSR4 LSR9 LSR10 LSR16 Node17 Network 1 Node18 Node1 LSR8 Node19 LSR5 LSR6 LSR14 LSR11 LSR15 Network 2 Node2 LSR7 MPLS Backbone LSR13 Node20 Traffic Load 75% on the path 15_14_11_6_4 Node3 LSR12 Ingress LER Egress LER Bandwidth (Kbps) Delay (ms) Path Time (ms) 4 16 600 100 4-5-8-10-16 84 16 15 100 20 16-15 11 21 15 4 1800 200 15-14-11-6-4 1287

Fourth Scenario Node0 Network 0 LSR4 LSR9 LSR10 LSR16 Node17 Network 1 Node18 Node1 LSR8 Node19 LSR5 LSR6 LSR14 LSR11 LSR15 Network 2 Traffic Load 75% on the path 15_14_11_6_4 Node2 Node3 LSR7 MPLS Backbone LSR12 LSR13 Node20 Ingress LER Egress LER Bandwidth (Kbps) Delay (ms) Path Time (ms) 4 16 600 100 4-5-8-10-16 84 16 15 100 20 16-15 11 22 15 4 2400 200 15-13-12-7-6-4 1287

Conclusion The design and deployment of grids for remote instrumentation services require the introduction of new control plane mechanisms to dynamically allocate resources in high-speed (G)-MPLS networks A centralized approach, based on a GNRB, has been designed and developed A new algorithm for the computation of path with bandwidth and delay constraints has been proposed Preliminary simulation results are promising Next step: implementation and testing in a real grid environment 23