CHAPTER 4 PROPOSED GRID NETWORK MONITORING ARCHITECTURE AND SYSTEM DESIGN

Transcription

1 39 CHAPTER 4 PROPOSED GRID NETWORK MONITORING ARCHITECTURE AND SYSTEM DESIGN This chapter discusses about the proposed Grid network monitoring architecture and details of the layered architecture. This chapter describes the design of the proposed system, the design of an automated deployment of the proposed system as a service in Grid, the network metrics used for performance evaluation and Network Cost Function. This chapter also describes about the computation of Resource Cost Value using resource metric, and Network Cost Value using network metrics. The Network Aware Resource Selection strategy is also explored in this chapter. 4.1 PROPOSED ARCHITECTURE The complex system like grid, monitoring is essential for understanding its operation, debugging, and failure detection and also for performance optimization. Due to the heterogeneity and constantly varying nature of grid, estimation of network performance is indispensable. If the status of the network path can be predicted, it is possible to use that information in grid applications. For example, the network status can be used to adapt the traffic load in order to avoid congestion on a network path. Scheduling of large data flows for data intensive applications is highly dependent on network path characteristics. For computationally intensive applications, resource broker or scheduler needs to have comprehensive and

2 40 accurate knowledge of network properties to fulfill service level agreements, ensure QoS, and to make fit choices for advance reservation. The four layered Grid network monitoring architecture is proposed and modeled with the grid scheduler in the collective layer. The proposed architecture is based on OGSA compliant layered architecture which is shown in the Figure 4.1. In the proposed approach, the CARE Resource Broker (CRB) is used for job submission (Thamarai Selvi et al 2009). Figure 4.1 Grid Network Monitoring Architecture Fabric Layer The Grid fabric layer defines protocols for the publication, discovery, negotiation, monitoring, accounting and payment of the operations

3 41 on individual resources. The resources may be computational resources, storage systems, catalogues, network resources and sensors or may be a logical entity, such as a distributed file system, computer cluster, or distributed computer pool. The Grid Resource Access and Management (GRAM) protocol is used for allocation of computational resources and for monitoring and control of computation on those resources, and Grid File Transfer. Resource and connectivity layer This layer consists of low-level middleware that provides secure and unified access to remote resources. Depending on the type of resources, different middleware can be chosen such as Globus, Unicore, Alchemi, and Storage Resource Broker. Using services of such low-level middleware layer, one can create high-level middleware services that support rapid creation and deployment of applications on global Grids. Collective Layer The proposed architecture is modeled in this layer with a grid scheduler. The Request handler which resides in CRB receives job requests from the users. The controller is in CRB which controls the scheduling, selection of the suitable resource for job submission from the matched resource list, the monitoring the execution of jobs in Grid, and also maintains the status of the submitted jobs. CRB selects the suitable resource using resource metrics. The network monitoring is fit in this layer to retrieve the network metrics which have influence on the resource selection. More sensors are deployed in grid resources to provide more network metrics so that the measurement of the network performance becomes more reliable. The sensors are the network monitoring tools and

4 42 utilities which are started through migration of mobile agents from resource broker to all resource sites when there is a need of unplanned monitoring in the Grid environment. The planned network monitoring gathers the network metrics from information repository, because the sensors are the network monitoring tools which are initiated on all the Grid resources to retrieve the network metrics periodically and update the information repository. The network performance measurement and prediction utilizes the information repository to measure and predict the performance of the Grid. Network Monitor monitors the network and collects the network metrics such as bandwidth, RTT, packet loss, jitter and stores it into information repository using agent based information aggregator. The Resource Monitor monitors the grid resources and collects the resource metrics and stores the collected information to information repository with the aid of agent based information aggregator. The agent based information aggregator aggregates the resource and network information from the Grid resources and periodically updates the information repository. It maintains the information about every physical resources and its performance of the network through end to end network monitoring across the grid infrastructure. The data manager uses the network cost function to measure the network performance which is described in the section 4.4. CRB catalogs the matched resources depends on the job requirements submitted by the user. Resource selector queries the information repository to select the suitable resource with network aware resource selection strategy and sending that information to the scheduler. The job monitor maintains its current status of the job execution and reports the progress to the user through Resource Broker. The network predictor predicts the future network performance using History Based (HB) approach and stores the predicted values into the information repository. It is also

5 43 responsible for sending the predicted value to the scheduler to take complex decisions. Application layer The application layer facilitates the use of resources in a grid environment through resource access protocols. The portal present at this layer allows the grid user to submit resource requirements to find out suitable resources for the execution of the submitted applications. It also includes software and tools to support application workflow and composition. 4.2 DESIGN OF THE GRID NETWORK MONITORING SYSTEM The proposed Grid Network Monitoring System design is based on the architecture described above. The Figure 4.2 provides the view of the Grid Network and Resource monitoring at resource level. Figure 4.2 Grid Network and Resource Monitoring at Resource Level

6 44 Grid Network Monitor The Grid network monitor initiates measurements or predictions on demand. The sensors are deployed to provide more network metrics which provides reliable network performance. The sensors are network monitoring tools and utilities like UDPmon, TCPmon, IPerf and Ping. These sensors provide network metrics like bandwidth, RTT, packet loss, and jitter which in turn facilitates the network status monitoring. Mobile Agent Generator In Grid, whenever the user submits the job through the resource broker, the mobile agent is created from mobile agent generator and it is cloned and migrated to all grid resources and starts the sensors. The sensors are the network monitoring tools which are used to retrieve the network metrics between the end-to-end node in all grid resources. Resource Monitor The mobile agent migrates from the resource broker to all resources and collects the resource metrics. The Resource Monitor monitors the resource metric and sends the monitored information to data collector. The data collector aggregates the collected information and periodically updates the local archive maintained in every CE. Data Accumulator The Grid network monitor initiates data accumulator to collect the metrics. In the Resource Broker and Grid Resources (i.e. head node), the server for all the sensors running, from that the network metrics for the link between the head to each compute node is sent to data accumulator. The accumulator extracts the necessary data from multiple compute nodes and

7 45 stores it in global archive, also called as information repository present in the head node. The resource metrics are collected from each node along with IP address of the corresponding node and a time stamp to maintain validity for the data. The agent performs this task for all computed nodes in the grid cluster and updates the global archive. Data Processor The data processor process the network data collected through deployed numerous sensors to measure the network performance using network cost function. The network cost value and resource cost value is calculated for all Grid Resources, through which the compound cost value is calculated to help the scheduler for selecting the suitable resource for job submission. Global Archive It contains aggregated information about network metrics and resource metrics of all Grid Resource. The Global archive, also called as Information Repository resides in the head node. This also stores the results of the cost function computations which are used by the data processor to measure the network performance and also for prediction. Predictor The predictor is linear and Historic-Based. This model uses standard time series forecasting techniques to predict the performance based on a history of measurements from previous behaviors on the same path. In the proposed system, Holt-Winters (HW) model is used for predicting network performance in the near future.

8 46 Visualizer This part deals with displaying the monitored network characteristics and the predicted network performance. The deliverables from this component may be a graph or a chart providing a clear vision about the network status of all grid resources. Sensors This component deals with the actual data collection in the Grid Resources. The numerous sensors are deployed using network monitoring tools and utilities like UDPMon, TCPMon, IPerf and Ping. These sensors provide metrics like bandwidth, latency, packet loss rate, jitter, round trip time (RTT) and one-way delay which in turn facilitates for network status monitoring. The communication among the model components is depicted in Figure 4.3. Figure 4.3 Work Flow of the Proposed Monitoring System

9 NETWORK METRICS FOR PERFORMANCE EVALUATION Monitoring the Grid network performance requires the analysis of various parameters like bandwidth, RTT, packet loss, jitter, etc which varies frequently depending upon the real time network conditions across the links. These parameters individually won t determine the network performance accurately. Thus combinations of the parameters are required Bandwidth The maximum amount of data per time unit that a hop or path can provide given the current utilization. The available bandwidth of a link relates to the unused, or spare, capacity of the link during a certain time period. At any specific instant in time, a link is either transmitting a packet at the full link capacity or it is idle so the instantaneous utilization of a link can only be either 0 or 1. Thus any meaningful definition of available bandwidth requires time averaging of the instantaneous utilization over the time interval of interest. The average utilization, u(t-t, T) for a time period (T-t, T) is given by u(t-t, T) = (u(x) ) over the limit T-t and T. where u(x) is the instantaneous available bandwidth of the link at time x. IPerf deals with TCP bandwidth and UDP bandwidth. Bandwidth = S / ( T Latency) (4.1) where, S is the message size, T is the message transfer time, and Latency is measured from RTT RTT Round Trip Time (RTT) is the time at which the last packet byte departs from the source t(d), and the time at which the last packet byte arrives

10 48 at the packet destination t(a). The Ping utility finds an application to measure this metric. RTT = t (A) - t (D) (4.2) Jitter Jitter is the "instantaneous packet delay variation" (IPDV) and it denotes the difference experienced by subsequent packets, I and I+1, on a one-way transit from source to destination. Iperf is used to measure it Packet Loss Packet Loss indicates the percentage of loss of data packets when the packets are transmitted between the end hosts. Packet loss may take place due to hardware fault, congestion in the channel, corruption in the data packet sent. The data packets that are discarded by the routers when the load becomes heavy also accounts for the packet loss percentage. Iperf and UDPmon Tools are used to measure packet loss rate. Packet loss percent = {1 - (Received Acks / Sent Packets)} * 100 (4.3) 4.4 NETWORK COST FUNCTION The Measurement of the network characteristics like latency, throughputs, packet loss rate, jitter, etc, is a repeated operation in any network management system and also in Grid Environment. A single network characteristic does not provide the significant information about the network performance through network resources. So an aggregation of multiple network metrics known as Network Cost Function (NCF) is needed to measure the performance of the network. The metrics considered are bandwidth, RTT, packet loss, and jitter between any nodes in the grid cluster.

11 49 The location of nodes could be inferred from the list of nodes available in Grid. The bandwidth measure is taken with respect to the average and the maximum value. BWmax value is considered because of considering the maximum possible bandwidth in the network channel. The variation of the RTT values is very wide; hence there is a need of normalization. The half normal form is used here, because the RTT values contain only the positive values and this distribution is very specific version of normal distribution. The packet loss, p and jitter are raised to the powers of the arbitrary values which are decided based on the current Grid set up. The values of the arbitrary varies [0, 1], hence the resulting values also reflect the values in [0, 1]. Packet loss rate is significant in network cost functions because it provides an estimate of both short and long-term congestion on a given data path due to packet drop which depends on the performance of the transfer protocols. IPDV is an important quality of service factor in assessment of network performance. If there is no packet loss and jitter, the NCF influenced with bandwidth, and RTT. The NCF (Network Cost Function) varies in the range [0, 1], where 0 indicates that a given node is not reachable and 1 denotes the maximum degree of usage of the link if the network is congestion free. Let BWmax denotes the maximum available bandwidth between the corresponding pair of nodes. Let BWavg denotes the mean available bandwidth between the corresponding pair of nodes, i.e., BW = BW, where n is the number of values taken for calculating the mean bandwidth for a period of time.

12 50 Let RTTmax denotes the maximum RTT between the corresponding pair of nodes, RTTavg denotes the mean RRT values measured by individual probes between the corresponding pair of nodes, i.e., RTT = RTT, where k denotes the number of values taken for calculating the mean RTT for the same period of time. And denotes how the RTT have influence on NCF. = RTT avg RTT max (4.4) Let p denotes the packet loss, varies in interval [0, 1] and tuned to balance the dependency of packet loss. Let jitter denotes the measure of the variability over time of the packet delay across a network, called as IPDV. varies in interval [0, 1] and tuned to balance the dependency of jitter. The setting of, and are depend on the Grid cluster which influences the maximum value. The NCF of end-to-end node is measured by analysing the network parameter values such as bandwidth, RTT, packet loss and jitter between that pair of nodes. The NCF of the individual links are calculated using the following expression. NCF= e (4.5) 4.5 RESOURCE COST VALUE The computation of the Resource Cost Value (RCV) of the Grid Site (GS) is based on the available Free Memory of the grid resources. The average Free Memory, FreeMem avg is calculated for each Grid Resource in a Grid environment, i.e., FreeMem = FreeMem, where, n is the

13 51 number of resources in a GS. The maximum available memory for a Grid Site, i.e., FreeMem max = max n (FreeMem k ) is also identified to evaluate the RCV, where n is the number of resources or Computing Elements (CE) available in a Grid Site. The RCV is computed by the following expression. RCV GS = FreeMem FreeMem avg max (4.6) According to the Equation (4.6), the RCV of Grid resource, RCV GS is varies in the range [0,1]. 4.6 NETWORK AWARE RESOURCE SELECTION STRATEGY The integration of network information with resource information has very much influence in the decision making process of a Grid Resource Broker. One of the major functions of the resource broker is to select the suitable resource from the list of Grid resources which are geographically distributed. The components of the network aware resource selection are shown in Figure 4.4. Figure 4.4 Network Aware Resource Selection Component

14 52 The primary selection is based on the requirements to be needed for the execution of job. In the proposed approach, the CRB is used as a primary selection to identify the matched grid resources for the specific job. The primary selection rules are defined by the job when it is submitted from CRB, which are the requirements needed to execute the job, such as software needed to execute the job, CPU Speed, and Memory, etc. The primary selection rules are used by the CRB broker to list the suitable resources to execute the job which are available in Grid environment. The execution of the job needs of transferring multiple input files and output data which lead to produce the traffic across the Grid. The amount of traffic can be reduced while selecting the node with better network connectivity. So there is a need of measuring the performance of the network using network metrics such as network cost value. The resource cost value is also computed by considering the available free memory. The Resource Discovery uses the secondary selection rule which is defined by the combination of network cost value and resource cost value called as compound cost value to select a suitable node from the CRB matched list of resources. 4.7 JOB MONITORING The Job Monitor is responsible for maintaining the status of the execution of the job to track its progress which is shown in the Figure 4.5. Grid Resource Allocation Management (GRAM) is implemented as a Web Services Resource Framework (WSRF) service in GT4 (Feller et al 2007). GRAM provides an API that allows for submitting and canceling a job request as well as checking the status of a submitted job. The job file is written using JSDL. After the job file is given as the input, it is passed to

15 53 JobMonitor. The JobMonitor uses the security provided by the Grid Security Infrastructure (GSI) of the Globus toolkit. WS-GRAM supports signature and XML encryption. It uses digital certificates to send secure XML SOAP messages between the Resource Broker and the Grid Resource. Reports job status User GRAM Job M onitor Authenticate Job M anager Submits job file Delegat ion Store Schedule Fork Scheduler Executes job User Job Store Credent ials using RFT Report Status Job State M onitor Report Status Job Event Daemon Figure 4.5 Function of Job Monitor Grid Resources have the local resource management system (LRMS) which controls jobs running on CEs. It allocates CEs to jobs, starts and stops jobs on user request and possibly restarts jobs if an error occurs. The LRMS identi es the job it manages using local job identi er (LJID). The jobmanager is the Globus GRAM which allows Grid users to start jobs on a Grid resource.

16 54 The user submits a job manifest, a document which contains the job description and the specification of the requested local resources to the jobmanager. After successful authentication and authorization, the jobmanager translates the job manifest into a form understood by the local resource management system and starts the job under a local user account i.e, user ID. A different user account is assigned to identify processes belonging to a job is to start. Then the GRAM ensures that each job is started under a user account that is distinct from accounts used by other presently running Grid jobs. First user delegation is performed, and then the job execution starts at the CEs. Then the job status and progress is updated periodically at the Resource Broker where the user submits the job. Once job execution completes, the results are reported to the user along with any possible errors like the job contact string does not match any which the job manager is handling which are reported by GRAM. The JobMonitor also identifies the current directory of the job, files in that directory, the permission set up of those files and its resource consumption. The Figure 4.6 depicts the process of job submission and monitoring. The jobmonitor represents the Grid service, which allows Grid users to start jobs on Grid resource.

17 Figure 4.6 Sequence diagram of job submission and monitoring process 55

18 NETWORK PERFORMANCE PREDICTION There are three simple linear predictors namely Moving Average, Exponential Weighted Moving Average (EWMA), and non-seasonal Holt- Winters. These predictions have some order for prediction say n based on the number of previous values taken into consideration for the prediction. A first order prediction is simpler but may not be accurate; higher the order then the predicted value is more accurate. There are also more complex linear predictors but selecting their order and linear coefficients requires a large number of past measurements. So the simple predictor Holt-Winters (HW) method is considered for predicting network performance rather than complex ones History-Based Prediction The History-Based (HB) prediction method is similar to traditional time series forecasting, where past samples of an unknown random process are used to predict the value of the process in the future Moving Average (n-ma) predictor is: Given a time series Y, the one-step n-order Moving Average (MA) i+1 = (n) -1 { Y (i-n+1) +Y (i-n+2) +..+Y (i-1) +Y (i) } (4.7) where, i is the predicted value and Y i is the actual (observed) value at time i. If n is too small, the predictor cannot smooth out the noise in the underlying measurements. On the other hand, if n is too large the predictor cannot aptly adapt to non-stationarities.

19 Exponentially Weighted Moving Average predictor is The one-step Exponentially Weighted Moving Average (EWMA) X = X + ( )X (4.8) where, is the weight of the last measurement (0< <1). Similar to the MA predictor, a higher cannot smooth out the measurement noise, while a lower is slow in adapting to changes in the time series Level Shifters and Outliers While experimenting with various predictors, it was found that the largest prediction errors are often caused by level shifts and outliers in the observed time series. Furthermore, if there is a need of manage to avoid these two characteristics in the time series forecasting, the exact choice of the predictor, or of its parameters, does not make a significant difference. A level shift is a type of non-stationarity, and it causes a significant and typically sudden change in the mean of the observed time series. An outlier is a measurement that is significantly different, beyond the typical level of statistical variations, relative to nearby measurements. One way to deal with level shifts, after they are detected, is to restart the predictor, ignoring all previous history. Outliers, on the other hand, can be just ignored. 4.9 ARCHITECTURE OF THE AUTOMATED DEPLOYMENT OF NETWORK AWARE RESOURCE MONITORING SERVICE The proposed architecture for an agent based Automated Deployment of Network aware Resource Monitoring service is shown in the Figure 4.7. The proposed mobile agent based automated deployment avoids

20 58 the maintenance costs and human errors occurring during deployment. Since mobile agents are capable of operating even without active connections between nodes, they are not affected by network failures. Further mobile agents reduces network load. Mobile agent technology is very much flexible to support the rum-time mobility through push and pull interaction models. And the characteristics persistence, cloning and migration of the mobile agents improve the reliability through replication. Figure 4.7 Automated Deployment of Network aware Resource Monitoring service A mobile agent is composed of code and data which migrates to other nodes and executes there in the node to which it migrates. The mobile agents exploit the basic communication protocols defined within IBM Aglets Workbench (Aglets 2004) for agent migration and to dispatch messages from one node to another node. Deployment Agent is a mobile agent which contains code for deployment of the services. It resides in the Resource Broker. On request from a newly arrived Grid resource, it migrates to the Grid resource and executes. The node where the Resource Broker is running is act as a Registration Node which maintains a database of the Grid resources that arrive and also it sends the IP address of the Resource Broker to the newly arrived resource. The Resource Broker contains the services to be deployed and the deployment agent. The registration node maintains a repository of IP

21 59 addresses of the resources in which the monitoring service is deployed and also the IP address of the Resource Broker. Mobile agents are used to get the IP address of the Resource Broker from the registration node and for deploying the services. For an automated deployment of service, the mobile agent migrates from the new resource to the registration node, collects the IP address of the Resource Broker and migrates back. Using the IP address of the Resource Broker, the newly arrived resource requests the Resource Broker for deployment of the services. Then the deployment agent migrates from the Resource Broker containing the services to the newly arrived resource and executes thereby deploying the services SUMMARY The monitoring of Grid resources is a momentous task because of the diversity of the computing resources and applications in Grid environments. The existing resource brokers are not considering the factors such as location of data, bandwidth availability and data transfer time while scheduling data-intensive applications on Grid Resources. This chapter presented a four layered architecture for Grid network monitoring system which is modeled with Grid scheduler. The Grid network performance is measured using Network Cost Function by analyzing the network metrics such as bandwidth, RTT, packet loss and jitter between the pair of nodes in Grid. The resource cost value is computed using resource metrics and the network cost value is computed using network metrics. The proposed system is integrated with CARE Resource Broker (CRB) which is used for job submission. The Network Aware Resource Selection Strategy is proposed for resource selection by computing the compound cost value using network cost value and resource cost value for the selection of a suitable node from the CRB matched list of resources. Once the job execution is completed, the results or errors are reported to the user which is provided by GRAM (Grid Resource Allocation and Management). The agent based automated deployment of Network aware Resource Monitoring service also explored in this chapter.