Grid Infrastructure Monitoring on the Service Oriented Computational Grids thesis proposal

Transcription

1 Grid Infrastructure Monitoring on the Service Oriented Computational Grids thesis proposal Ondřej Krajíček supervisor: doc. RNDr. Luděk Matýska, CSc. March 30, 2008

2 I certify that this text is adequate as a thesis proposal and agree with it. doc. RNDr. Luděk Matýska, CSc.

3 I would like to thank my supervisor, doc. Luděk Matýska for the opportunity of cooperation with him and studying under his guidance. Thanks also gratefully go to my colleagues at the Supercomputing Centre Brno and Laboratory of Advanced Networking Technologies for their cooperative support which I have always been proud to enjoy. Ondřej Krajíček

4 Abstract We propose a thesis studying problems related to an infrastructure monitoring of service oriented grids. The main goal is to define standard interfaces and guidelines which would provide standardised monitoring of grid services, based on the OGSA standard. Any defined solutions or architectures will then be put into context of large scale grid monitoring architectures, notably C-GMA, to allow seamless participation of the proposed solutions in heterogeneous environments. Keywords: service oriented grids, grid infrastructure monitoring, service monitoring, instrumentation

5 Contents I Infrastructure Monitoring in Service Oriented Grids 5 1 Introduction 5 2 Service Oriented Grid Infrastructure Web Services Open Grid Services Architecture 7 3 Grid Infrastructure Monitoring Monitoring OGSA Services Black Box Monitoring White Box Monitoring: Instrumentation User-oriented Monitoring Grid Monitoring Architecture Capability-based Grid Monitoring Architecture 14 II Research Plan 17 4 Research Goals A Preview of a Service Oriented Monitoring Architecture Simple Black Box Monitoring: Sampling Interactive Black Box Monitoring Asynchronous Monitoring White Box Monitoring: Service Oriented Instrumentation Preliminary Architecture Design Designing the Distributed Mediator for the C-GMA Distributed hash tables (DHTs) Content-based routing networks Peer-to-peer overlay networks 23 5 Conclusions 23 6 Research Summary Research Goals Research Schedule 24

6 Part I Infrastructure Monitoring in Service Oriented Grids 1 Introduction In this thesis proposal, we discuss the nature of service-oriented Grids as one of the kinds of grid infrastructures available in Grid Computing. We focus on the problem of implementing and providing monitoring support, with focus on identification of problems in this area and proposing research directions to address them. The proposal starts with an overview of the infrastructure of service oriented computational grid, along with discussion of existing approaches and tools, identification of possible open problems as research directions and in some case, the proposal provides preliminary research results in the respective areas. 2 Service Oriented Grid Infrastructure The service oriented computational grid has been originally defined in the works of Ian Foster and Karl Kesselman, namely [1]. The principal goal of this work has been to define foundations for interoperability of large distributed systems. Here, a grid (as in power grid) is a large scale data distribution network which connects resources of various kinds. These resources could be computer clusters, supercomputers, data warehouses, specialised hardware, etc. The key motivation behind the Grid Computing 1 is integration of "local" resources and applications into higher and large-scale architectures. We can safely assume, that the service oriented grid consists only of services, which encapsulate all participating elements. We will refer to these services as grid services. We may observe, that the grid services usually fall into one of the two basic domains: the application domain and the infrastructure domain. The application domain contains everything related to the applications deployed on a grid, these are services and resources accessed or leveraged by the end users. On the contrary, services and resources (not necessarily limited to software) which establish and support the grid infrastructure fall into the infrastructure domain. In both domains, we may identify several layers, with respect to the abstraction of underlying resources and encapsulation of their internal structure and mechanisms. These layers form a grid middleware, which provides grid-enabling functionality in some way. 1 Grid Computing is the research area devoted to the grid related research. 5

7 In this work, we focus on studying the properties of an infrastructure of a utility grid, with respect to monitoring its status, the status of resources deployed within and the implications for usage and maintenance of the grid. 2.1 Web Services Service Oriented Grids present an incarnation of a Service Oriented Architecture (SOA). SOA may be termed as engineering methodology describing how to model and build loosely coupled distributed systems. SOA presumes, that elements of such distributed system offer their functionalities encapsulated as services, with explicitly defined entry points. These entry points are used to exchange messages or more generally documents. The services thus involve three basic aspects (please note that these aspects do not involve the implementation of particular services): Service Description - the mechanisms required to describe the entry points of a particular service along with specification of messages/documents it is able to process. Then entry points are usually described using interfaces. These are similar to interfaces known from object oriented programming languages. Informally, we may consider interfaces to be sets of message types which together model some coherent functionality. By implementing an interface, the service guarantees that it understands the message types defined by this particular interface. Service Discovery - the process in which a required service is located. The party which is locating a service for later usage is usually referred to as the client 2 please note that the client may be a service itself. The criteria a client may specify to locate a service are not predefined and depend on a particular technology/implementation. Service Integration - the process of communication between a client and a service. This process is termed integration since the service is usually integrated into one or more complex applications (which may consist of more services). The beauty of the SOA approach is its simplicity. The SOA itself restrains from complex feature requirements that have been imposed on different distributed system technologies/approaches, such as distributed objects in CORBA, DOOM, various forms of remote procedure call, etc. This, however, has also a downside: SOA is not a standard and does not provide any foundation for building interoperable systems, which is however, its primary goal. SOA in this context offers merely a paradigm. In SOA a service is a unit of encapsulation. Service usually encapsulates some process, which essentially transforms input data to output data with possible hidden side 2 With the same meaning as in the original client/server paradigm. 6

8 effects. A service is informally defined as a software entity with explicitly defined communication protocol (i.e., the description of accepted input and produced output data) and communication interface (the entry point which receives the input data and provides the output data). The SOA requires, that the services have no implicit hidden state. The service state is either part of the input /output data or is explicitly defined. The most recent and also widely adopted implementation of the SOA architecture are the Web Services. Web Services provide basic aspects needed for service creation, deployment and interoperability: standardised language for describing services and standard communication protocol. The currently used communication protocol is SOAP [3], which is based on XML. There have also been attempts for different communication protocols suitable for web services, but in spite of great interoperability strength of SOAP (due to its XML nature), no solution achieved wider deployment. For more information refer to [5]. The language used for description of web services is called Web Services Definition Language (WSDL) [4]. Because of its strong reliance on XML-based technologies, the Web Services technology is sometimes referred to as XML Web Services. 2.2 Open Grid Services Architecture Open Grid Services Architecture (OGSA) is the original standard [2] which defines grid services and describes the infrastructure of a service oriented computational grid. OGSA may be seen as an extension of SOA. OGSA defines a grid in terms of standard services and standard interfaces. An OGSA service 3 is a service which implements the required standard interfaces. These interfaces facilitate behaviour required for the various management aspects of a grid infrastructure. The original OGSA proposal defines the following standard interfaces (porttypes in terms of the WSDL language): GridService - this is currently the only mandatory standard interface, all OGSA services are required to implement it. The interface supports querying the service for various kinds of information, both generally defined and service specific. It also supports basic service lifetime management. NotificationSource - by implementing this interface, the service indicates that it supports asynchronous topic-based change notifications. NotificationSink - this is the dual interface to the Notification- Source. NotificationSink supports operations for delivering messages 3 In the following text, we are going to use the term grid service and OGSA service interchangeably. 7

9 from the notification source to the subscribed service (the notification sink). Registry - implements registration of services. Registry interface is defined to support the service discovery concept. Factory - implements on-demand creation of OGSA services. HandleMap - implements service lookup. The OGSA defines a nontrivial naming scheme for grid services which facilitates features like service migration and location transparency. HandleMap interface defines operations for retrieving the exact location of a discovered service. Currently, several technologies which implement OGSA standard exist. The first implementation of the standard has been Open Grid Services Infrastructure (OGSI) [14], which leverages the original Web Services technology and provides straightforward implementation of the OGSA concepts. The key difference between a web service and grid service (according to the OGSI) is that the grid service is stateful in general. For the purpose of management of a stateful service, the OGSI introduced a notion of soft state. Each service begins its lifecycle with predefined lifetime and when the lifetime ends, the service is automatically destroyed. Service clients have the ability to extend the service lifetime. Based on the ideas introduced by OGSI, the academia and industry joined forces to create a new standard for stateful web services, called Web Services Resource Framework (WSRF) [13]. According to the WSRF standard, the original notion of stateful service is decoupled into two parts: a stateless service and one or more resources, which represent the components of a service/application state. Resources are created and manipulated by services and may be shared between them (they are identified by uniform identifiers, such as URIs). After the introduction of the WSRF standard, alternative standards have began to emerge, competing with the original WSRF specification in simplicity and implementation aspects. Also, based on the Web Services and WSRF technology stack, many extensions (collectively termed as WS-*) have appeared. These specification deal with various advanced aspects, such as security, asynchronous notification, capability negotiation, lifetime management and others. Many of these standards propose different treatment of the same aspects and are competing. 3 Grid Infrastructure Monitoring Monitoring is essential part of the management process of a grid infrastructure. Infrastructure monitoring produces data important for many aspects 8

10 of grid management and maintenance, most notably scheduling, fault tolerance and failover, security and others. Infrastructure monitoring focuses on resources which participate in a grid and software services which encapsulate them and which provide the basic encapsulation of grid infrastructure, these are consecutively called the grid middleware. Based on the abstractions offered by the service oriented grids, we may limit ourselves to the service monitoring. In this case, the monitoring of services is performed by their clients. We denote these special purpose clients as service monitors. The monitoring concept itself should be implemented and encapsulated by defining a monitoring extensions to OGSA in terms of standard services and interfaces. Such standard should by called OGSA-ServiceMonitoring (or OGSA-SM in short form) by convention. This is one of the principal topics of the proposed thesis. The distinction of the grid monitoring concept may be further refined by identification of its three aspects, which are: 1. discovery of monitoring peers (either services or service monitors) 2. production of monitoring information (denoted as monitoring data) 3. monitoring data transfer In this proposal and in the proposed thesis, we strongly focus on the first two aspects as there are well established systems and tools, which address the problems related to transferring monitoring data. Our preferred solution to the discovery problem is the C-GMA architecture. The production problem is independent from the C-GMA, nevertheless it is desirable that service which produce monitoring data could act directly as C-GMA producers. 3.1 Monitoring OGSA Services Considering the available mechanism in a service oriented architecture, we need to cope with the problem of the monitoring status and health of services without violating the "rules" imposed by the architecture. We have already defined the client, which performs monitoring as a service monitor, which itself can be a service. We may borrow an analogue from the software engineering and testing and identify two basic flavours of monitoring: Black Box Monitoring and White Box Monitoring. The distinction regards (as it does in software engineering) whether we are aware of internal structure and behaviour of the monitored service Black Box Monitoring We have informally defined black box monitoring as inspection of external service effects (side effects). Such effects may be log files, journal files, I/O 9

11 Operations on data files, database operations, etc. We may generalise this intuition of side effects as any interaction between the service software and its running environment which surpasses the service process address space. This means, that the application logic of the service is not involved, but its external manifestations are. Black box monitoring requires thorough understanding of the service behaviour, which can be gained either from the service author (which is not very common case) or by a systematic observation. Videlicet, such knowledge of service behaviour is used to evaluate the state of the service and to infer monitoring information from it. It is quite difficult to obtain the complete knowledge of the service external manifestations and it is even more difficult to automate the process. Usually, specifications from the end user are needed to facilitate this task. This makes black box monitoring quite suitable for monitoring legacy code applications, which cannot be enhanced with monitoring support for various reasons. Problems connected with integration of legacy applications to a grid infrastructure are not limited only to monitoring. Systems, which manage legacy code, provide integration capabilities for it and encapsulate legacy application as services exist. One of such systems is GEMLCA-R [27], which adopts techniques similar to black box monitoring for management of legacy code. Special case of the black box monitoring is monitoring service responses. The monitoring is realised by sending particular predefined requests to the service and checking the responses with pre-collected samples. From the behavioural deviations (such as fluctuations in response time, message size, etc.) and other factors, it is possible to infer the status of the service to some extent. Such approach is quite common in service oriented architectures and has been long ago adopted in many large scale projects White Box Monitoring: Instrumentation One of the historically successful techniques of monitoring status of software artifacts (e.g., grid services) has been instrumentation. Instrumentation is based on the knowledge of behaviour and often internal structure of the software. Assuming this, the software is enhanced instrumented with "hooks" which resemble various measurable characteristics of the software. These hooks are often specified in terms of metrics mostly informal specifications of value syntax and semantics (e.g., units of measurement, etc.). Depending on the particular tools used for instrumentation, the instrumented software provides means of retrieval of measured characteristics. Two basic flavours of instrumentation exist: static and dynamic instrumentation. The static instrumentation is further divided into source code and binary instrumentation. Static instrumentation means that the software artifact (be it a source code, bytecode or a native code executable) is instrumented statically, before compilation and/or execution; the set of in- 10

12 RegisterServer(serverName, serv e r Ad dress,...) Average memory consumption J ^ ^ ^^ y Figure 1: Distinction between metrics and service interface strumentation hooks is predefined, fixed and does not change without the recompilation/re-execution cycle to occur. Thus, it is up to the programmer to designate which parts of the software are instrumented and how. Dynamic instrumentation means that the software artifact is instrumented dynamically, it often means modification of the instruction flow of the executed processes or observation of behaviour of the executed process and its side effects (memory footprint, I/O operations, etc.) by some component of the operating system (such as specialised driver or kernel module). The advantage of this approach is that the system may make dynamic decisions when and how to instrumented a particular process, limiting the instrumentation overhead when suitable. The main disadvantage of this approach is that it is quite complex. Two basic approaches to dynamic instrumentation exist: a hardware based and a software-only approach. We focus on source level static instrumentation of grid infrastructure services. In this particular case, we believe that decision and design of the monitoring model lies in the hands of a programmer and possibly a maintainer of a grid infrastructure. The problem of static instrumentation is that it is difficult to implement the instrumentation hooks without affecting the clarity of the source code and also implementation must be done carefully not to impose unacceptable overhead on the resulting performance. When software engineering practises are applied, the clarity of the source code becomes important quality which has non-trivial consequences on maintenance and flexibility. Our goal is to explore approaches which would allow complete encapsulation (hiding of implementation) of instrumentation code in a way that would allow the programmer to focus on the application logic itself without increasing the overhead significantly. Currently, several tools exist which leverage instrumentation for the purpose of grid monitoring and problem diagnostics. Instrumentation (particularly dynamic instrumentation) is common technique used in application monitoring. Source-code instrumentation is used by various services to support metrics required by infrastructure monitoring tools. Source-code II

13 instrumentation is usually addressed by creating some kind of instrumentation programming interface which the programmer may use to manually capture entry/exit points of functions or data changes. With the help of a language preprocessor, such as in C language, it is possible to somewhat automate the process and ease the burden of the programmer. However, we are currently unaware of tools which use more sophisticated approaches to accomplish source code instrumentation. One of the interesting monitoring tools, which leverage instrumentation is OCM-G [12]. OCM-G is tailored for application monitoring, but its concepts may be leveraged to monitor the infrastructure as well. OCM-G stands for On-line Monitoring Interface Specification Compliant Monitor enhanced for the grid. OCM-G defines a set of low-level metrics which describe the behaviour of applications and allows definition of more complex metrics by composing the low-level ones. OCM-G provides library instrumentation tools, which monitor library function entry and exit points. The library is instrumented based on library instrumentation description files. This tool is able to instrument static libraries, for dynamic libraries a static wrapper must be created. The advantage of this approach is that arbitrary static library can be instrumented without need to modify the source code. However, the information on function prototypes and their signatures is maintained separately in two places and must be kept synchronised User-oriented Monitoring Infrastructure monitoring serves as an important source of useful information to resolve problems which occur in the infrastructure. Aside from external failures (such as hardware problems), these problems may be caused by configuration errors, broken communication links, desynchronisations (system time), etc. To make the problem even more complex, these errors usually occur only under certain conditions and their occurrence does not depend solely on the infrastructure/middleware of a particular system resource. User-oriented monitoring means that the monitoring system is able to take into account various contextual conditions, primarily tied to the user, under whose credentials the problem is exhibited (i.e., the owner of the faulting job, the client of the erroneous service). To accommodate useroriented techniques is a significant task for every robust monitoring system. User-oriented approach is tightly coupled with the problem of producing the monitoring data. The facility which produces the data must take into account not only the state of a monitored service, but also the context the running conditions. In this text, we do not propose specialised approaches to user-oriented monitoring. Instead, we assume that all proposed monitoring techniques are 12

14 Figure 2: Sample deployment of the GM A architecture context-aware, which encompasses also user orientation. 3.2 Grid Monitoring Architecture Independently from OGSA, the Global Grid Forum (GGF) has published a standard [16] (ranked as informational document) which describes a generic Grid Monitoring Architecture (GMA). The GMA is a model of grid monitoring implementation with the essential characteristics needed for scalable high performance and high throughput monitoring on a large distributed computational Grid. The GMA standard defines the architecture in terms of interacting components (in case of service oriented grid these would become standard interfaces). These components are: producer, consumer and directory service. The architecture also proposes how interaction occurs between components and features each component must implement. Basic requirements on monitoring architecture are defined as low latency, capable of high data rate, minimal overhead, security and scalability. The sample deployment is shown in Fig. 3.2 taken from [16]. The components of the GMA model have the following roles 4 : producer - a software entity (grid service in our particular case) which produces monitoring data related to monitored grid services; 4 It is worth noting that one grid service may act as multiple components, i.e., being producer and consumer at the same time, leading to implementation of intermediary data processing components (such as archivers). 13

15 the producer may be the monitored service itself or some intermediary component which further processes or archives the data; consumer - a monitoring agent which requests and processes the monitoring data; this may be some intermediary processing component, user interface or an administrative monitoring tool; directory service (interchangeably referred to as the registry) - a service which handles registrations and discovery of available producers and consumers; while the consumers and producers communicate with the registry to advertise their availability, the subsequent transfer of monitoring data between these two is completely independent from the directory service GMA model assumes that monitoring data are represented as (timestamped) events. Events are transfered only from producers to consumers (not vice versa), but communication can be initiated by both parties. Several communication models between producers and consumers have been identified: publish/subscribe, query/response, and notification. Operations supporting all types of communication are defined for producer and consumer. Basic operations for directory service have also been identified. In the all communication models, monitoring data and control messages are sent directly between a particular consumer/producer pair and directory service is involved only in location of corresponding sink. One of the key motivations behind GMA has been to endorse interoperability of various grid monitoring tools and systems. Interoperability is crucial concept on the grid, since it is a heterogeneous system and makes no or little assumptions about resources that are plugged in. It is difficult to imagine, that it would be possible to deploy a single particular monitoring tool on every site in a grid, which could span tens or hundreds geographically and politically scattered sites. The GMA implementation has proved to be general enough to accommodate the interoperability requirement, yet it is lacking any particular concepts which would endorse a real interoperability. Strictly speaking, GMA does not provide any foundations on which two monitoring systems can build to ensure or negotiate mutual interoperability. It does not communication paradigms (not at required level), communication protocols (only patterns), etc. The current state of the art is that the vast majority of available monitoring tools can be considered GMA-compliant, yet there is not standard way to achieve their interoperability. 3.3 Capability-based Grid Monitoring Architecture The interoperability problem described in 3.2 is tightly related to the discovery aspect of the grid monitoring. Each monitoring component is identified not only by the specification of the data it is willing to produce/consume, 14

16 but it should also be able to advertise its communication protocol, security requirements and any requirements related to the data itself. There is an ongoing effort 5 to design and implement grid monitoring architecture which would accommodate these features. It is called the Capability-based Grid Monitoring Architecture (C-GMA). Capability-based Grid Monitoring Architecture [18] is an extension to the original GMA. The C-GMA extends the GMA model to provide monitoring framework for true interoperability of various grid monitoring toolkits and infrastructures. One of the notable extension to the original GMA model is the introduction of additional component, which encapsulates the GMA directory called the mediator. The purpose of the mediator component is to proactively monitor component registrations and notify producers and consumers of interesting and potentially compatible counterparts. The component model of the C-GMA is illustrated in Fig. 3. It defines four basic components: Producer produces the monitoring data in the form of events. Consumer consumes the monitoring data. Individual consumers are connected directly to the appropriate producers. Registry (Directory Service) is an information service which stores information about available producers and consumers and also the data type schema. Mediator is used by consumers and producers to discover potential partners (producers and consumers, resp.). The actual discovery process is described later. Besides the component model of C-GMA, another model is necessary for the C-GMA to work. We denote this model as metadata model. The metadata model defines two metadata layers: the capability and attribute layer: describes properties, working conditions and requirements of components and data they exchange; metadata on this layer are further refined with scope, which specifies the range of metadata application (e.g., whether the metadata apply to the whole class of components or just a single one, etc.); the descriptions of capabilities and attributes are done in a particular capability language, we consider Classified Advertisements [21, 22] as a useful candidate; the data definition layer: typically adopted from an existing GMA implementation. In this layer, the metadata describes data types (e.g., B C-GMA is a joint project between Institute of Computer Science, Masaryk University, West-Bohemian University and CESNET z.s.p.o. 15

17 directory mediator producer consumer event (data) Figure 3: The C-GMA Component Model table names, metrics, etc.) of published and requested data, as well as further data specifications (possibly as constraints, predicates, conditions, etc.). This leads to multi-criteria matching of producers with consumers; By introducing new metadata layers and the mediator component, the C-GMA promises to aid in the process of discovering potentially compatible monitoring components. It is not difficult to observe, that the key point of the architecture is the mediator component and the functional and performance characteristics of the resulting solution would greatly depend on its design. It is one of the open problems in C-GMA how to design the distributed mediator and propose some implementation approaches. Some preliminary results are discussed in 4.2. The C-GMA architecture is highly relevant to the monitoring of a service oriented grids. We have already made the assumption, that C-GMA components are services, however this is not sufficient to enable true interoperability between them. However, the C-GMA capabilities can then be leveraged to advertise the standard interfaces the producers and consumers implement, the monitoring data they produce or consume and monitoring techniques they support. Enabling the C-GMA architecture for service monitoring is also one of the subjects of the proposed thesis. 16

18 Part II Research Plan 4 Research Goals The proposed thesis will focus on the production and discovery aspects of grid monitoring on the service oriented grids. The principal goal is to define OGSA-ServiceMonitoring as an extension to OGSA describing the standard services and interfaces required to facilitate monitoring. The complementary goal is to participate in the design and implementation of the C-GMA and integrate the service oriented concepts into it. In this chapter, we present preliminary results in the C-GMA architecture as well as in the implementation techniques of the grid service instrumentation. 4.1 A Preview of a Service Oriented Monitoring Architecture We provide a short preview of the design of a OGSA-based service oriented monitoring architecture. We try to identify the required standard services and standard interfaces and define them and also describe the overall design of the architecture. In previous chapter, we have described the basic concepts of grid monitoring in general and monitoring of a service oriented architecture in particular. Here, we describe a relatively straightforward application of these concepts into OGSA, the goal being to propose a feature complete monitoring extension to OGSA (OGSA-ServiceMonitoring). Defining a single rich standard for monitoring-aware service does not seem to be a feasible approach. Instead, we want to propose a different "levels" which we denote as monitoring modules, so that service authors can choose the monitoring model which best suits their service. Some of these modules are supersets of the others (in features), some are completely independent. Initially, we propose the following monitoring modules: Simple Black Box Monitoring, Interactive Black Box Monitoring, Asynchronous Monitoring and Service Oriented Instrumentation Simple Black Box Monitoring: Sampling Simple Black Box Monitoring means that we collect and observe the service responses to a particular request and collect response samples. These samples are used to "replay" requests, observe the responses and compare them to recorded sample responses. The resulting disproportion is processed accordingly to its severity, i.e., if the service responds with fault message to a request it should correctly process, a service failure should be reported to a monitoring system. 17

19 Simple Black Box Monitoring is performed by a specially implemented service monitor. The monitor may use the sample collection techniques or it may leverage a monitoring samples suite, i.e., a set of predefined samples published by the author of the monitored service, if such a sample suite exists. The main disadvantage of this approach, which is essentially a regression test (unit test) of a service, is that the samples used for monitoring should carefully cover all working aspects of the service. Moreover, the sample requests should only result in idempotent operations. Otherwise, monitoring may have unintended (and potentially destructive) effects on the grid and the participating resources. This simple approach does not require defining new standard interfaces in OGSA, i.e., this technique can be applied to all OGSA services which support requests/responses suitable for sampling. The service may provide any required information, such as the location of a monitoring samples suite via the GridService standard interface Interactive Black Box Monitoring Implementing Sampling may be difficult and complex tasks, moreover it does not provide accurate and complete information about the status of monitored services. To improve the situation, we may define a standard interface and guidelines to provide a standardised support for monitoring in services. The service monitoring than interacts with a monitored service using the standard interface, thus this approach is termed Interactive Black Box Monitoring. The interactive approach extends the basic black box approach in two ways: 1. Define standard operations which provide basic status and monitoring information about the service. These standard operations should be general enough to fit to all or at least to majority of existing services. 2. Define a standardised monitoring samples suite and provide an interface which may be used to send sample requests, retrieve sample responses and retrieve definition of these standardised samples. For this purpose, we define two new standard interfaces: ServiceStatus and Monitoring Samples Source. The ServiceStatus interface defines operations to retrieve basic status of the service along with a custom message describing the status, the timestamp which indicates the time of status retrieval and the status validity duration. The basic status is provided as a single word from the following list: OK, TemporaryProblem, PersistentProblem. How to handle the individual states is left to the service monitors. The MonitoringSamplesSource interface provides operations which retrieve the monitoring samples. On request, the service provides a sample request and 18

20 response pair along with their expiration time. Until the expiration time runs out, the service monitor may send the retrieve sample request to the monitored service, retrieve the response and compare it with the sample response. How to handle the differences is again left to implementation of a particular service monitor Asynchronous Monitoring Asynchronous monitoring is based on the notifications mechanism defined by the OGSA. If the service supports notifications (Notif icationsource standard interface), a service monitor may subscribe for status changes. The name of the topic, which must be used for subscription is retrieved from the service by means of the ServiceStatus standard interface defined in which must be extended accordingly White Box Monitoring: Service Oriented Instrumentation Service Oriented Instrumentation is OGSA-based incarnation of the white box monitoring approach, using metrics. Instrumentation is in general discussed in The instrumented service must be able to indicate which metrics does it support and provide their definitions (discovery of metrics), publish the metric values and provide these values on request (metric retrieval) and optionally it may provide asynchronous notifications, which occur when a metric value changes. Also, implementation of service instrumentation is a problem of its own (related to software engineering) preliminary results in this area are reviewed later in this chapter. The service-provided definition of a metric should at least include servicespecific metric name, a (possibly) unique metric identifier, a specification of metric units and a textual description of the metric. The interpretation of these values, particularly the unique metric identifier is left to the service monitors. We only suggest that the unique metric identifier is URI or URN, as is quite common in XML-related technologies. This would allow references to metric semantic definitions, such as OWL-S documents [29]. The metric value reported by a monitored service should contain the following information: metric name (see above) metric value metric units of measurement the timestamp of measurement - when the metric value has been obtained 19

21 validity duration - how long the value may be considered valid from the time of measurement As part of studying instrumentation and its implementation techniques, we have identified additional problem related to the implementation of instrumentation. It is difficult to identify and implement metrics in a service, in terms of a complexity and clarity of the source code. One of the possible approaches is to isolate specific parts of the source code (variables, values, functions) and turn them into metrics by suitable encapsulation. We have been working on a series of experimental instrumentation toolkits, based on various programming paradigms/languages and different techniques to examine this approach in various contexts. We are leveraging various software engineering guidelines and programming techniques to provide a service author with a simple to use mechanism, which would be able to easily identify instrumentation metrics in the source code and separate the monitoring and instrumentation concepts from the service application logic. We are currently experimenting with techniques from object oriented programming, generic constructs in imperative languages (templates), aspect oriented programming[9] and design by contract[10]. Preliminary results using generic constructs in C++ have been described in [17] Preliminary Architecture Design The service monitoring infrastructure of a service oriented grid consists of two kinds of basic elements: monitored services and their clients the service monitors. However, we need a facility which would implement large scale monitoring and provide a (semi-)complete 6 status overview of the entire grid. Such a facility would be able to combine monitoring data from various services and infer additional information from them. We denote this facility as Monitoring Policy Engine (MPE). The motivation behind MPE is to enable creation of large-scale monitoring policies, which would describe relationships and dependencies between grid services along with specifying constraints on the monitoring data available to an MPE. Violation of such constraints is then treated as a failure of a service or group of services. This allows to detect even problems related to simultaneous use of several services (such as in service oriented workflows, service orchestrations, etc.), even in case when all participating services indicate that they are working correctly. According to the policy, the MPE may than publish the "globalised" status information to the grid monitoring system, such as C-GMA, acting as a conventional producer. In this sense, the C-GMA architecture can be considered as a monitoring layer which encapsulates the MPE infrastructure, as demonstrated in Fig Usually, a complete overview of a grid infrastructure is not strictly required. We only need to supply information needed for particular management decisions. 20

22 Service Monitor Service Monitor Service Monitor Service Monitor C-GMA Producer Figure 4: Service Monitors and MPE interaction with C-GMA In the figure there are three services connected to the MPE, which acts as a C-GMA producer. The last service is not covered by the monitoring policy implemented by the MPE and it is connected directly to a C-GMA mediator acting as a C-GMA producer. 4.2 Designing the Distributed Mediator for the C-GMA The C-GMA is designed with the assumption of a single authority the mediator, which executes the matchmaking and initiates communication between various producers and consumers depending on their metadata. However, centralised implementation of the mediator may pose significant performance and scalability bottlenecks as well as a single point of failure. Considering these problems, we have began to evaluate various approaches to building mediator as a logically centralised, but physically distributed system. Such system needs to be able to process queries in distributed manner. It is yet to be seen whether a system with global (replicated) knowledge is needed or if a scalable system with distributed query processing would be better. Various infrastructures, such as peer to peer networks (JXTA [23]), distributed hash tables (Tapestry and Chimera [24], etc.) or publish/subscribe system (Siena [25]) may be ideal candidates to solve our problem Distributed hash tables (DHTs) Distributed hash tables with key spaces based on metric spaces offer interesting properties such as range and similarity searches. It is however difficult to define a metric space on C-GMA capabilities and attributes (expressed as Class Ads). Initial experiments focused on defining metrics on 21

23 capabilities and attributes have been conducted. It may be possible to define DHT hashing function by defining a total order on capabilities and attributes and using the following cluster hashing algorithm. 1. Order the capabilities and attributes in a Class Ad according to the defined total order. 2. Generate hash of each capability/attribute definition using cryptographically strong hash function with uniform distribution (such as SHA-256). 3. Take first n bits from each hash generated in step 2 and concatenate them into a resulting key. 4. Take first m bytes from the generated key. The mentioned algorithm is parametrised by two values: m and n. It may be used to generated keys of arbitrary length by combining m/n clusters, each of n-bits length. Elaboration of the optimal values of m, n is subject to further research. The proposed hashing algorithm generates keys which allow exact-match searches. Similarity search is not possible due to the nature of the uniformly distributed hash function. It is yet to be examined the practical potential of this hashing scheme. Another considered possibility is to define representation of ClassAds using minimised tree automata and devise an encoding function which encodes a particular automaton Content-based routing networks Content-based routing networks are structured message-passing overlay networks. The speciality of this overlay is that messages are routed based on criteria other than destination addresses. Content-based systems usually fall into one of the two categories: topic based (messages are routed according their designated "topic" to nodes which specifies interest in this topic) or content based (messages are routed according to their content based on evaluation criteria distributed among routers). Topic-based systems are not flexible enough to accommodate our needs but content-based system allows implementation of distributed matchmaking (as described in [20]). Building a distributed mediator using content-based publish/subscribe system (CBPS) requires to implement some kind of aggregation scheme on top of the required capability language, to accommodate the requirement of downstream replication / upstream evaluation. Downstream replication means that a notification should be routed in one copy as far as possible and duplicated only as close as possible along the paths leading to interested subscribers. Upstream evaluation implies that subscription filters are 22

24 applied on events as close as possible to publishers. The design goal underlying these requirements is to minimise the usage of network resources when routing events to large numbers of distributed subscribers. A prototype design of a distributed mediator for the C-GMA is described in [20] Peer-to-peer overlay networks Second generation peer-to-peer overlay networks offer decent scalability and maintainability required to be considered for usage in grid environment. Toolkits like JXTA offer services which make programming peer-to-peer applications much simpler. However, from the point of view of a distributed mediator design, these tools offer only relatively low-level functionality. For instance, the distributed query processing must be entirely implemented from the scratch. The advantage of using peer-to-peer overlay network without any advanced query processing capabilities (such as DHTs) is that it is not functionally limited and allows implementation of any query processing scheme without posing any requirements dictated by the network topology or management. 5 Conclusions The goal of this work is to define a standard architecture for monitoring services, while sustaining the interoperability which is part of the service oriented architectures. The chosen approach is to propose extensions to existing service oriented standards used in Grid Computing (OGSA, WSRF) in a seamless way and verify these standards using prototype implementations. The validity of the resulting solutions is verified by integrating it into a C-GMA architecture (which supports monitoring of large-scale heterogeneous grids) in a way that the services work as a natural elements of this architecture (i.e., define service oriented extensions and scalable components for the C-GMA). 23

25 Research Summary 1 Research Goals 1. Define OGSA-ServiceMonitoring extension of original OGSA in terms of standard interfaces. Validate the consistency of the extensions and possibilities of interoperability and coexistence with other OGSA extensions. 2. Examine existing and future Web Services standards and the suitable standards accommodate into OGSA-ServiceMonitoring architecture to maximise flexibility. 3. Define service oriented monitoring architecture based on the OGSA- ServiceMonitoring extension in terms of standard services and their interactions. 4. Implement a prototype of the proposed service oriented monitoring architecture. 5. Define the monitoring policy, its elements and a suitable definition language. 6. Design the Monitoring Policy Engine service and provide a prototype implementation. 7. Propose a integration mechanism of service oriented monitoring architecture into C-GMA, most importantly into C-GMA distributed mediator. Participate on a scalable distributed mediator design to accommodate the required changes. 2 Research Schedule 2007: Definition of the OGSA-ServiceMonitoring standard: standard services and standard interfaces, including description of roles of the standard services and their interactions (publication). 2007: Description of implementation approaches suitable for OGSA- ServiceMonitoring using the current standards at the time of writing (probably WSRF or different WS-* standards). 2008: Integration of OGSA services into C-GMA monitoring architecture, including definition of the required extension of the capability language (publication). 2008: Definition of the monitoring policy, including context-aware aspects (such as the user oriented monitoring). 24

26 2008: Design and of the Monitoring Policy Engine service (publication). 2009: Prototype implementation of the Monitoring Policy Engine, including integration with the C-GMA architecture. 2009: Dissertation thesis. 25

27 References I. Foster, C. Kesselman, J. M. Nick, S. Tuecke. The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration, papers/ogsa.pdf Global Grid Forum: Open Grid Services Architecture (informational document). GFD-I.030.pdf SOAP Protocol Specifications. Web Services Description Language Specification, org/tr/wsdl 0. Krajíček. GSIP - Invocation Architecture for Scalable and High Performance Web Services. Master thesis, Faculty of Informatics, Masaryk University, Foster, C. Kesselman. "The Grid 2: Blueprint for a New Computing Infrastructure." Morgan Kauffman Publishers Inc. San Francisco, USA B. Liskov and J. Wing. "Family Values: A Behavioral Notion of Subtyping." ACM Technical Report TR-562b Aspect J Project Website, aspect j.org Aspect Oriented Software Development Portal, "Building bug-free 0-0 software: Introduction to Design-by- Contract(TM)". http: //archive. eif f el. com/doc/manuals/ technology/contract/ "Choosing BOINC Projects". projects.php OCM-G User's Guide. userguide.pdf "The WS-Resource Framework (Globus Toolkit)", globus.org/wsrf/ S. Tuecke et al. "Open Grid Services Infrastructure (OGSI) Version 1.0", Global Grid Forum, GFD documents/gfd.15.pdf. "Globus Toolkit Website", 26