An Implementation of OGSI on Microsoft.NET

Transcription

1 An Implementation of OGSI on Microsoft.NET Daragh Byrne EPCC, University of Edinburgh 5 th August 2003 This white paper is available online at Copyright 2003 The University of Edinburgh. All rights reserved.

2 MS.NETGrid White Paper 2 Contents Contents Introduction Grid Services Grid Services on.net Design Goals Design Outline High-level Architecture Service Lifetime Maintaining State Developing Services Deploying Services OGSI.NET An alternative approach Limitations Glossary References... 14

3 MS.NETGrid White Paper 3 Introduction This paper describes an implementation of OGSI on.net. This is a portion of the work of the MS.NETGrid project undertaken at the National e-science Centre (NeSC) between March 2003 and March MS.NETGrid is a project to demonstrate the relevance of the Microsoft.NET suite of technologies to e-science and Grid computing. The project is described in detail on the project WWW site ( MS.NETGrid is carried out by EPCC representing NeSC in collaboration with Microsoft Research Limited ( The main goal of the MS.NETGrid project is to provide a practical demonstration to the UK e-science community of the applicability of Microsoft.NET technologies to the hosting, development and deployment of Grid Services. A complementary goal is to facilitate understanding of Grid technologies and e-science within Microsoft Research. These goals are realised by achieving the following project objectives: Developing an implementation of the Open Grid Services Infrastructure (OGSI) [OGSI-Spec] using.net technologies. Developing a suite of Grid Service demonstrators that can be deployed under this.net OGSI implementation. Developing training courses and materials to educate and inform the UK e-science community about.net and its applicability to Grid applications. These training courses will be based around the developed software. Delivering training courses to delegates from the UK e-science community. The remainder of this paper describes the technical aspects of the work carried out so far. We focus mainly on the implementation of OGSI called MS.NETGrid-OGSI that the project provides, and briefly discuss an alternative approach to this work being pursued by the University of Virginia [Virginia-Impl].

4 MS.NETGrid White Paper 4 1 Grid Services Grid computing is a way of organizing often heterogeneous computing and data resources so that they can be flexibly and dynamically allocated and accessed [OGSI-Primer, Anatomy, Physiology]. Often this sharing of resources occurs across organisational boundaries. The collection of organisations contributing resources in such a manner is often known as a virtual organisation [Anatomy]. The connection of heterogeneous resources requires an open, platform independent global architecture. Efforts to define this architecture have been made in [Physiology, OGSI-Spec]. In this picture, known as the Open Grid Services Architecture (OGSA), resources are represented by services. The term service is used to describe a network-aware entity (software component etc) that provides certain well-defined functionality, accessed via a known interface, which accepts and processes a certain set of messages. A service may also have a state associated with it. The behaviour of a service should not be strongly coupled to its implementation. An example of a common service is a File Transfer Protocol service. The Open Grid Services Infrastucture [OGSI-Spec] defines such a service-oriented view of Grids in terms of modern Web Services technologies. According to [OGSI-Spec], the capabilities and interfaces of Grid Services are defined in terms of Web Service Description Language (WSDL) [WSDL] documents. WSDL documents are used to describe the abstract interface to a service, in terms of porttypes, each of which supports a number of operations. WSDL also provides a means of defining the bindings to a service, i.e. the format and contents of messages it accepts. [OGSI-Spec] defines the GridService porttype, which all Grid Services must implement. A number of optional porttypes that deal with service creation, grouping, location and notification are also defined. OGSI extends the definition of WSDL to support features found only in Grid Services. The state of a service may be accessed via its servicedata, and operations for accessing the servicedata are defined on the GridService porttype. Transient service instances may be created using factory services, and lifetime management may be handled by client or server. Another difference between Grid Services and traditional Web Services is the multiple porttype implementation and porttype inheritance model of OGSI, although this will be brought into Web Services in revision 1.2 of WSDL. It is intended that the set of porttypes defined in [OGSI-Spec] will be built upon to provide high-level OGSA services. For more information about OGSI, see [OGSI-Spec, OGSI- Primer]. 1.1 Grid Services on.net There are a number of reasons for carrying out an implementation of OGSI on the Microsoft.NET platform. If Grid Services are to be platform agnostic, implementations of OGSI on all relevant platforms are necessary. The.NET platform is certainly relevant as it is Microsoft s platform of choice for software development for the foreseeable future. The enhanced Web Services capabilities offered by.net provide an ideal base on which to build an implementation of OGSI. Such an implementation may reveal unknown strengths and weaknesses both of OGSI and of.net. Known implementations at the time of writing include that described by this paper, and the work undertaken by [Virginia-Impl].

5 MS.NETGrid White Paper 5 2 Design Goals Since the MS.NETGrid project is concerned with demonstrating the applicability of Microsoft.NET to Grid services and to the broader e-science community, a core goal of the design of our OGSI container was to exploit and demonstrate as many relevant.net capabilities as is possible and sensible. Given that Grid services are essentially an extension of Web Services, a major goal was to exploit the Web Services functionality provided by.net, particularly ASP.NET, as far as possible. Additional design goals were as follows: To provide an implementation that exhibits a representation of the key aspects of OGSI, namely the functionality offered by the core porttypes, service state and servicedata, and lifetime management. Comment [DB1]: Was scalability a design goal too? MJJ yes it was. Should add in this. To support SOAP over HTTP as our communications protocol, offering no explicit support for other methods of invocation at this time. To aim for interoperability as far as possible, highlighting any interoperability issues that arose. To keep in mind performance-related issues during design, implementation and testing and to be prepared to identify the areas of our design and implementation that are performance-critical and options for improving these. To utilize any features of the target technology that contributed to rapid development time, as development was scheduled to occur over a short timescale (3 months from scratch). The design goals for the demonstrator services were as follows: To demonstrate the features of the MS.NETGrid OGSI container. Provide a selection of representative services that address real e-science problems.

6 MS.NETGrid White Paper 6 3 Design Outline Our design was inspired by features from both the Globus Toolkit 3 implementation of OGSI [GTk3] and the design of [Virginia-Impl]. Our resulting design is similar in spirit to both of these implementations, in that we solve the same basic problem of converting stateless Web Services into stateful Grid Services. We decided that it was best to exploit existing technology as much as possible to cut down the development time. A distinguishing feature of our design is in the use of an ASP.NET Web Application running under Internet Information Server (IIS) to act as a Grid Service container. Our container also mimics the ASP.NET service deployment model to a large extent, making it attractive for development by current developers of ASP.NET Web Services. 3.1 High-level Architecture Our container has the following high-level architecture: HTTP Request, SOAP Message ASP.NET Web Application GridServiceInstanceAspProxy Request handled by ASP.NET Web Service Service Skeleton Instance SOAP Response to client Selects Service Instance Calls method on Instance PortType We implement our OGSI container as an ASP.NET Web Application. Instances of a particular type of Grid service are each represented by an instance of a so-called skeleton object. The skeleton object contains references to objects that implement the operations of the porttypes the Grid service implements. Within the Web application are components for dealing with service instance management, managing communication with clients, providing OGSI porttype-related functionality, and allowing developers to deploy services. The following sections discuss the design in more detail. 3.2 Service Lifetime Our container allows two types of service lifetime. The first, known as transient service lifetime, applies to service instances that are spawned by other services, for example factory services. The service instance lives until its termination time has passed, when it then will no longer respond to operation invocation requests. However, clients may request extensions to the instance lifetime, as per [OGSI-Spec]. The second service lifetime type is known as persistent lifetime. In this case, the service instance is initialised when the container starts up and lives as long as the containing Web Application. Container-created (persistent) services are necessary to bootstrap essential services such as factories. The distinction of services in terms of their lifetimes persistent and transient is consistent with the implementations of [GTk3] and [Virginia-Impl], although it is not required by

7 MS.NETGrid White Paper 7 [OGSI-Spec]. 3.3 Maintaining State Consuming a traditional Web Service usually entails sending a SOAP message to a network endpoint somewhere. At the endpoint, the message is interpreted, for example by AXIS functionality [AXIS] in Globus Toolkit 3, or ASP.NET on Windows. The Web Service exposes a number of operations, and one of these is invoked, based on the content of the SOAP message. An object is instantiated, and the method called, with any de-serialized objects as parameters. Once the processing has been carried out, any return values are serialized into a SOAP message and sent back to the client. The processing object is destroyed, or left to be garbage collected. Essentially a traditional Web Service can be said to be inherently stateless. In ASP.NET, code for Web services is located in, or referenced by, files with the.asmx extension in the Web application virtual directory. When requests are received for files with this extension, ASP.NET processes the file with the Web services module, which translates the SOAP into a method call on an instance of an object, and returns the result to the client. A stateful Grid Service necessitates that a service object be maintained, so that the next call by the client is handled (conceptually) by the same object instance. We can do this either by Storing references to service objects and mapping client requests to objects directly, or Creating a new object for each call, and maintaining state between calls using some persistent storage e.g. serializing the object out to disk between calls. The first of these options was preferable as it avoided the overhead of saving and loading state data in response to each client request, and the associated development costs. The problem then becomes that of making transient ASP.NET Web Services act like persistent Grid Services. The solution we use is as follows: 1. A service instance is represented by an instance of a class derived from a GridServiceSkeleton class. A reference to each instance is stored in a table in the container application. Comment [m2]: Do we want the NeSC Flier communication diagram here? 2. Communication with the service instance is handled by a traditional ASP.NET Web- Service proxy. We provide a GridServiceInstanceAspProxy class (and derived classes), from which service proxies must derive. The proxies are located in.asmx files like normal ASP.NET Web Services. The proxy for a service class contains methods that map directly to the service methods. These methods correspond to the porttype operations exposed by the Grid service. 3. Clients interact directly with the proxies as if they were a normal Web Service. 4. A new instance of this proxy is created for every service invocation request as normal. Upon creation, the proxy uses information contained in the request (namely, the URL and query string) to obtain a reference to the GridServiceSkeleton instance that is supposed to deal with that request. 5. The proxy uses the request information to call a specific method on the service

8 MS.NETGrid White Paper 8 instance using reflection. 6. The result is returned by the proxy and automatically serialized back to the client by ASP.NET The following diagram illustrates the process: Return Result client Map Request to Instance (HttpApplication) Service Instance Method Call (Grid Service Instance) SOAP Request (Web Service Proxy) For every persistent service instance, there must be one unique proxy. For example, say we have a factory for a service CounterService. The proxy for this factory could be located at Note that under this scheme CounterServiceFactory1.asmx and CounterServiceFactory2.asmx would be proxies for two different persistent instances. Persistent service instances are identified in the container by the virtual path to the.asmx file. For every class of transient service, e.g. CounterService, there will be one proxy file. Thus, all CounterServices would be represented by the proxy located at We differentiate between different counter service instances by the use of an instanceid parameter in the querystring of the URL. So: ce1 and ce2 represent communication endpoints for two different CounterService instances accessed via the same.asmx proxy. 3.4 Developing Services The service author must carry out the following 3 steps when developing a service: Write the service implementation class. Write the proxy class. Deploy the service by editing a configuration file. Comment [DB3]: From the descritption is the text, it seems that this is largely boilerplate. Is there any way that the generation of proxy classes could be automated? MJJ could be but time does not yet permit.

9 MS.NETGrid White Paper 9 Full details of the development process are available in the user guide that is provided with the MS.NETGrid-OGSI software. The software and documentation may be downloaded from Deploying Services The.NET configuration API is exploited when deploying services. Information must be provided that links together the service implementation class, the service proxy class, the assembly in which the classes may be found, the lifetime type of the service, and the URL at which the proxy lives..net allows extensible configuration via the use of XML files, and ASP.NET provides a standard file called Web.config for this purpose. XML elements called Grid Service deployment descriptors are added to this file by the service developer.

10 MS.NETGrid White Paper 10 4 OGSI.NET An alternative approach The Grid Computing Group at the University of Virginia ( have also undertaken an implementation of OGSI on.net [Virginia-Impl], known as OGSI.NET. Work commenced in January 2003 and the result has been a stable and complete OGSI implementation. The design of OGSI.NET differs from the MS.NETGrid container in a number of ways. The primary difference is that OGSI.NET is implemented as a standalone Windows service. An ISAPI filter attached to IIS intercepts requests to the OGSA/Services virtual directory and redirects the request to the Windows service. Each Grid service instance resides in its own AppDomain (AppDomains are memory isolated regions within a process and are a fundamental concept in.net). A dispatcher within the Windows service process routes the request to the service AppDomain, where it is deserialised. Method invocation then proceeds in a manner similar to our container. Comment [DB4]: Presumably this is not to be confused with a Web Service? Comment [DB5]: Is this an instance of the Windows service or of the Grid Service? Or am I just confused? MJJ It s a Grid service should just qualify this. Architecturally the two implementations are quite similar. They both follow the following procedures when dealing with message requests: Intercept message. Use information in message or URL to select service instance. Reflect on service instance and invoke method. Return results to client. The difference between implementations lies in the level of control provided to the service developer. In OGSI.NET the level of control is substantially greater. It is possible to write custom message deserialisers for each service, allowing fine-grained security features or alternative message formats to be implemented, for example. Security is a topic also addressed in OGSI.NET where explicit allowance has not been made in the MS.NETGrid implementation. Comment [DB6]: This paragraph gives the strengths of OGSI.NET. Does MS.NETGrid have any strengths in return, e.g. ease of programming?

11 MS.NETGrid White Paper 11 5 Limitations The limitations of our design are as follows: Our container does not allow for strongly named assemblies yet. Assemblies are the.net equivalent of software libraries and provide explicit support for versioning and digital signing. Assemblies must be placed in the bin directory of the container application directory. We have made no explicit allowance for security. Fault handling is not done in a consistent manner. When an exception is thrown, it appears to the client as a SOAPException. Comment [DB7]: For each of these, it would be useful to say whether the limitation is inherent, or whether it is could be implemented given more development time. Comment [DB8]: This implies that there is some implicit support. Is it worth briefly describing that? Comment [DB9]: What should it do instead? We do not yet allow for accurate service description via the?wsdl query string convention. There are limitations relating to: o o Service data and service data descriptions. At the moment, the returned WSDL document is the document generated by ASP.NET for the service proxy class. This is pure WSDL1.1 and does not contain information about servicedata. Multiple or aggregated porttypes all operations are aggregated and appear to belong to one porttype, which is equivalent to the most-derived porttype which corresponds to a Grid service itself. Only support for the GridService and Factory porttypes are provided. Work is underway on the Notification porttypes, and it is anticipated that the ServiceGroup porttypes will also be developed. ServiceData and the XML format of certain operation messages do not match the OGSI specification exactly. Comment [DB10]: Will this then be all the porttypes in OGSI? MJJ there is also HandleResolver Comment [DB11]: I can t parse this. Persistence of service state between container restarts is not supported. Our design does not support arbitrary service names.

12 MS.NETGrid White Paper 12 6 Glossary For convenience of reference we include the following glossary..net The Microsoft.NET platform, including the.net SDK (Software Development Kit), associated development tools such as Visual Studio.NET [ and the Common Language Runtime. Comment [m12]: Add in a Section 7 Future Plans just a list (non-binding!) ~ogsanet/projman/plan/wp1/plan. txt ideas for example scalability, notification, service groups, correcting XML, faults. Common Language Runtime a runtime environment that executes Microsoft Intermediate Language (MSIL). MSIL The.NET bytecode, a low-level assembler-like language executed by the Common Language Runtime. Managed code Code that runs on the.net CLR. IIS Microsoft Internet Information Server, a web server offering communication over a variety of internet protocols. ISAPI An Application Programming Interface (API) that allows applications to use the network services provided by IIS. Commonly used to provide HTTP filters that respond to all HTTP requests on the server. AppDomain.NET allows the partitioning of a single Operating System process into a number of AppDomains, which are essentially memory safe areas within the process. If the code in an AppDomain crashes, other AppDomains within the process are unaffected. This concept has some performance advantages as using AppDomains avoids the overheads associated with starting a new process..net Remoting The mechanism by which.net allows Remote Procedure Call (RPC) between AppDomains and between remote machines. ASP Active Server Pages, a Microsoft technology that works in conjunction with IIS to host Web Applications. ASP.NET The.NET version of ASP, with extensions for Web Services and Enterprise Applications. C# - Pronounced C-Sharp, a new Microsoft programming language that takes advantage of.net platform features. Assembly A collection of managed code, which may form an executable or a library, and may be distributed across a number of physical files. Assemblies facilitate the separation of logical and physical resources. OGSA The Open Grid Services Architecture, which is a set of proposed architectures and protocols for Grid computing [Anatomy]. OGSI The Open Grid Services Infrastructure, a specification of behaviours and properties of Grid Services [Physiology, OGSI-Spec]. OGSA-DAI OGSA Data Access and Integration, Grid Services framework for seamless access and integration of data using the OGSA paradigm [OGSA-DAI].

13 MS.NETGrid White Paper 13 Globus The Globus toolkit. A standard toolkit of Grid software. Version 3 will contain an implementation of OGSI in Java [ WSDL The Web Services Description Language [ Web Services Refers to network-enabled software capabilities accessed using common Internet protocols such as HTTP and SOAP and described using WSDL. Grid Services Web Services conforming to the OGSI specification. Grid Service Instance A network accessible instance of an OGSI-compliant service. PortType A set of operations supported by a Web or Grid Service. ServiceData An XML-based mechanism that allows a client to query the state of a service instance in a flexible and extensible manner. ServiceDataElement A particular element of servicedata, identified by a name and a value Tomcat A container for Java-based Web Applications [ AXIS A Web Application running under Tomcat, which provides Web Services functionality. [ Programming Model A set of procedures and APIs used to develop an application in a given domain

14 MS.NETGrid White Paper 14 References Documents referenced in the text, and other related documents, include the following. [OGSI-Spec] Open Grid Services Infrastructure (Draft 29), S. Tuecke, K. Czajkowski, I. Foster, J. Frey, S. Graham, C. Kesselman, T. Maquire, T. Sandholm, D. Snelling, P. Vanderbilt, April 5th th [OGSI-Primer] Open Grid Services Infrastructure Primer (Draft), available from [Virginia-Impl] OGSI.NET: An OGSI-compliant Hosting Container for the.net Framework, Grid Computing Group, University of Virginia. WWW site: [GTk3] Globus Toolkit version 3 and OGSI, Available at [Physiology] The Physiology of the Grid (Draft 2.9), I. Foster, C. Kesselman, J.Nick, S. Tuecke, Available at [Anatomy] The Anatomy of the Grid, I. Foster, C. Kesselman, S. Tuecke, Available at [OGSA-DAI] Open Grid Service Architecture, Database Access and Integration [WSDL-Spec] Web Services Description Language 1.1, [AXIS] The Apache Axis SOAP Engine,