Integration of the OCM-G Monitoring System into the MonALISA Infrastructure

Transcription

1 Integration of the OCM-G Monitoring System into the MonALISA Infrastructure W lodzimierz Funika, Bartosz Jakubowski, and Jakub Jaroszewski Institute of Computer Science, AGH, al. Mickiewicza 30, , Kraków, Poland funika@agh.edu.pl, [bjakubow,jjarosz]@student.agh.edu.pl Abstract The main intention of this paper is to present a way to integrate the grid monitoring system OCM-G into the monitoring agent infrastructure of MonALISA. OCM-G is designed to monitor distributed and parallel applications and provide on-line information about their execution status as well as some parameters of nodes they are running on. To provide the user with a wide set of visualization capabilities we supply these information to the services of a scalable performance visualization infrastructure, MonALISA, through common protocols. We feed the services with data which can be then accessible by a Mon- ALISA client or repository that publishes them on a server. A sample configuration and output are provided to illustrate the idea of the operation of the integrated tools. 1 Introduction Nowadays, when Grid has become a more and more active player in HPC the importance of monitoring is not to be overestimated. It is crucial to provide information about computation nodes and jobs running on them. A product of the CrossGrid project, Grid-enabled OMIS-Compliant Monitoring system (OCM-G) [1] allows to monitor a number of the desired parameters of both execution infrastructure and interactive jobs running on it. Its compliance with On-line Monitoring Interface Specification (OMIS) [2] enables to exploit a well defined interface for interactions with data suppliers and monitoring clients, like performance analysis tools. The OCM-G features a wide palette of monitoring capabilities, including the capturing of user-defined measurement values using a special instrumentation techniques. The idea that motivated our research was how to transform the monitoring data coming from OCM-G to an easy interpretable form and make them available for a big number of clients. We concentrated our attention on Mon- ALISA [4]. Its services are capable of providing on-line chart plotting from simple datagrams containing variables values only. The services are registered in MonALISA LookUp Service to be discoverable for clients. There can also be MonALISA Repositories which are able to perform some more sophisticated actions on data and publish the information in several ways including Tomcat Server. What we worked on is an application that is intended to pass specified information from OCM-G to MonALISA. Both of these systems provide well specified

2 APIs that makes the task easier. The application is fully configurable through an XML file. Upon a short overview of related work, the paper proceeds with the presentation of the tools OCM-G and MonALISA in the context of their roles in the research. The next section covers our vision of the interconnection between them and their features we use for integration activities. Then we describe the details of our application followed by a usage example. 2 Related Work The domain of Grid computing is not short of monitoring tools, exploiting various visualisation approaches. There is a widely recognized R-GMA architecture [3] underlying a plenty of performance analysis tools. However, since R-GMA exploits the idea of relational monitoring database, the information supplied to the user can be user for longer time-related analysis. GRM [5] is a semi-online monitor of applications running in a distributed heterogeneous system. It consists of local monitors which are responsible for writing a complete trace of application as well as statistical data in a shared buffer. Buffer is then turned into a file by the main monitor. The file is then sent to the PROVE visualization tool which is capable of displaying those detailed information both off-line and semi-on-line. Autopilot [6] is another distributed monitor that uses local daemons to collect data about both system and network performance. It is using the concept of sensors and actuators that are put in the source code of application. For visualisation it uses the Virtue [7] toolkit. It defines a visualisation hierarchy for managing performance details. It shows data in form of graphs with data from different abstraction levels. SCMSWeb [8] unlike the two mentioned above have its own visual interface. It generates PHP files which can be deployed straight-forward on the server. It is designed to monitor performance and system usage metrics. Monitors can be organized in a hierarchical way to improve scalability. Information is passed in XML files. It is possible to retrieve statistical performance data of different abstraction levels presented with graphs. Ganglia [9] uses same concept like SCMSWeb. It uses a multicast-based listen/announce protocol and a tree of point-to-point connections amongst representative cluster nodes to federate clusters and aggregate their state. It has a PHP front-end and uses RRDtool to visualise data. Ganglia uses XDR to transfer XML files. 3 Integration Strategy Integration is realized by an application that works as a kind of transformer and transmitter. It obtains the data from OCM-G through OMIS interface, extracts interesting information from messages and passes them to the MonALISA service. Below we briefly describe the used systems.

3 3.1 OCM-G as a Monitoring Data Supplier OCM-G is a system providing on-line information about clusters status. It is composed of the Service Managers which are distributed across the Grid Computation Elements. They are gathering the information from Local Monitor processes which are started on the Worker Nodes. The data is passed using Globus IO communication. The most interesting part of OCM-G from our point of view is the MainSM the principal Service Manager that is on top of the SMs tree and collects all the information. It is started first and is identified by an connection string which is then passed to monitored applications and every tool that wants to communicate with it through OMIS interface. This is what our application does. The moment after start of MainSM we can connect to it and query for information about the nodes. Furthermore we can monitor applications started on the Grid. The only requirement is that they must be compiled using the cg-ocmg-cc tool and run with the parameters defining MainSM and application name. To be able to get an arbitrary value from the application we need to define so called probe functions. When they are called in the application they return a value of type int or double to the SM. We can think of them as a kind of events that are triggering the action of sending some data to the registered monitoring tools. 3.2 MonALISA as a Publishing and Visualization System MonALISA (stands for for Monitoring Agents using a Large Integrated Services Architecture) is designed as a group of autonomous, agent-oriented services. The services can be deployed anywhere on the Internet. They only need a UDP connection to the LookUp Discovery Service in CERN, Switzerland. The services are automatically gathering information about the host they are running on and can be configured to receive UDP datagrams carrying data. MonALISA also defines Repositories - a kind of clients designed to gather specified data from particular services and store them in a relational database. There is a subgroup called Web Repositories which are using Servlet Engine to present historical and real time data in form of statistics and graphical charts. Sending data to the repository is made easy through the ApMon a library for most popular languages. It defines operations to form UDP datagrams with an arbitrary number of parameters of any type. We also define a list of recipients for the datagram, which can contain any number of services. There is also a mean of security service can be configured to accept only datagrams from given hosts or datagrams containing defined password. There is no reliability assurance of UDP datagrams if lost will not be retransmitted. This is whowever orking well because of the speed of this solution which is necessary for the amount of data transferred. 3.3 Making an Interconnector between Two Systems The idea of interconnecting OCM-G and MonALISA is shown in fig. 1.

4 Fig. 1: Architecture diagram. To integrate these two systems we developed an application which is acting as an interconnector. It is run on the same machine as MainSM and is bound to one application only. At startup we pass it the connection string to communicate with the MainSM and the name of the application that we want to monitor. The configuration file contains parameters we re interested in, both standard and probe-type ones. The application then registers itself as a listener for probe events and requests standard parameters from MainSM at a defined rate. The communication is done with OCM-G Java API which specifies some helper classes for OMIS messages. Nevertheless, every parameter needs a special parser to extract a single value for a given variable. At each timestep, standard parameters are requested from OCM-G and the values received are sent to MonALISA services through ApMon Java library. The addresses of services are defined in configuration files together with optional passwords to include in datagrams. Every time a probe event is triggered, the value of the variable returned is sent to services. 4 Sample Usage First of all MainSM is needed to get monitored parameters. After being started cg-ocmg-monitor prints Connection String. Tools have to be provide with this string to be able to connect to MainSM. $ cg-ocmg-monitor MainSM Connection String: 0a000001:a655 Now we are ready to start an application providing its name and MainSM s Connection String. $ app --ocmg-mainsm 0a000001:a655 --ocmg-appname monitored_application The application name may be any string, but must match the application name set in our tool s configuration.

5 To pass the parameters to be monitored to MonALISA we have to provide a configuration for our tool. Let us assume our cluster is named OurCluster and we use a MonALISA service located at example.com. We are going to monitor the load of nodes by sending queries once per two seconds. We also assume that the application is using a probe function called probe example and we are interested in its second argument 1. Thus we get the following configuration: <?xml version="1.0"?> <configuration> <cluster name="ourcluster" /> <monalisa> <service address="example.com" /> </monalisa> <parameters> <monitored name="load" frequency="2" /> <probe app="monitored_application" name="probe_example" arg="1" /> </parameters> </configuration> Upon having saved the configuration in a file, we can run our tool. We pass Connection String as first argument and the configuration file name as second 2. %$./run.sh 0a000001:a655 ourcluster_conf.xml The tool resumes the stopped application and passes the monitored parameters to MonALISA. We are now ready to view results in MonALISA. We can achieve it using MonALISA Interactive Client 3. It allows to view clusters in many ways including their presentation on a world map or list of groups. Each parameter can be plotted (fig. 2). Plots are updated in real-time when new data is arriving at the MonALISA Service. 5 Summary The integration of OCM-G and MonALISA is beneficial for both projects. It takes advantage of the fine grained parameter monitoring provided by OCM-G and a set of very convenient publication and visualisation services coming from MonALISA. On one hand, due to the design idea of OCM-G, which assumed that the interactive applications have much more rigorous requirements for monitoring than conventional batch-type ones, especially w.r.t the speed the monitoring data is delivered to a client, we have got really on-line monitoring data supply. On the other hand, MonALISA, which features very scalable publishing and visualisation capabilities, enables to gain a good deal of synergy with OCM-G. 1 Arguments are numbered starting from 0. 2 Configuration filename defaults to conf.xml, if not passed as an argument 3 Available at

6 Fig. 2: MonALISA Interactive Client. Acknowledgements. This research is partially supported by the POIG PL-Grid project and the AGH grant References 1. OCM-G: 2. OMIS: 3. R-GMA: 4. MonALISA: 5. Balaton, Z., Kacsuk, P., Podhorszki, N.; Application Monitoring in the Grid with GRM and PROVE. In Proc. ICCS 2001, Part I, pp , LNCS 2073, Springer, Ribler, R.L., Vetter, J.S., Simitci, H., and Reed, D.A.; Autopilot; Adaptive Control of Distributed Applications. In: Proc. 7th IEEE HPDC, Shaffer, E., Reed, D.A., Whitmore, S., Schaeffer, B.; Virtue: Performance Visualization of Parallel and Distributed Applications. In: Computer, vol. 32, no. 12, pp , Chalakornkosol, N., Uthayopas, P.; Monitoring the Dynamics of Grid Environment using Grid Observer, Poster Presentation in IEEE CCGRID2003, Toshi Center, Tokyo, May 12-15, Massie, M.L., Chun, B.N., Culler, D.E.; The ganglia distributed monitoring system: design, implementation, and experience. In: Parallel Computing, Volume 30, Issue 7, pp , 2004.