Web Services Development Guide: How to build EMBRACE compliant Web Services Version 2.0, 13 December 2006

Web Services Development Guide: How to build EMBRACE compliant Web Services Version 2.0, 13 December 2006 Jan Christian Bryne, Jean Salzemann, Vincent Breton, Heinz Stockinger, Marco Pagni 1. OBJECTIVE...2 2. GETTING STARTED WITH WEB SERVICES...2 2.1 WEB SERVICE DEVELOPMENT APPROACH...2 2.2 WEB SERVICE MECHANISM...3 3 LIST OF ACTIONS TO DEVELOP WEB SERVICES...4 3.1. EXPRESS DATA TYPES IN XML SCHEMA...4 3.2. DEFINE A WS-I COMPLIANT INTERFACE IN WSDL...5 3.3. GENERATE SERVER STUB FROM THE WSDL DOCUMENT...9 3.4. CONNECT BACKEND CODE WITH THE SERVER STUB...11 3.5. DEPLOY THE WEB SERVICE...12 4. WRITE THE DOCUMENTATION...12 4.1 MANDATORY WSDL DOCUMENTATION...12 4.2 MANDATORY API DOCUMENTATION...13 5 SUGGESTED TOOLS AND TOOLKITS FOR BEGINNERS...15 5.1 XML AND WSDL EDITORS...15 5.2 WEB SERVICE TOOLKIT...15 6 PITFALLS AND HINTS...16 ACKNOWLEDGEMENTS...16-1 -

1. Objective The goal of this document is to provide service developers a detailed step by step guide to build EMBRACE and WS-I compliant Web services. The intended audience is developers working first hand with the task of providing Web service interfaces to the resources that will be available within EMBRACE. A basic knowledge of programming and XML technologies including namespaces and XML Schema are assumed. This document is based on the appendix of deliverable D3.1.1, but here we will try to go deeply into the technical aspects. In the document we will explain the mechanism of Web services and how they can be used to interface an existing application. Even if you are already familiar with Web services, we suggest that you have a detailed look at Section 3.2 to check if your Web service is WS-I compliant and follows the document/literal wrapped style. 2. Getting Started with Web Services 2.1 Web service development approach There are a variety of different ways to develop a Web service. What they all have in common is that they can be described as using one out of two different approaches: Top-down development or bottom-up development. When making a Web service using the top-down approach a developer first designs the interface description and secondly implements the actual service. The bottom-up approach is the exact opposite where one first develops the service and then provides the interface description. The two approaches have different advantages and drawbacks. The bottom-up approach may provide a high level of automation, depending on the toolkit the developer uses. The most advanced toolkits can, provided with the code for a program, automatically generate both an interface description (a WSDL file) and the code necessary to implement a Web service interface to the program. This bottom-up approach can therefore be the most intuitive for developers, since it is code-centric. The top-down approach on the other hand is interface-centric since it requires that the developer designs the interface first. Some toolkits provide the possibility to automatically generate the necessary code to implement a Web service interface from a WSDL file. This code then has to be connected with the program that implements the functionality of the Web service. The biggest advantage of the top-down approach is that the developer has accurate control over the interface of the Web service. As EMBRACE progresses, there will be further standards defined on the data formats and interface design that will require a high level of control on the service interfaces. It is therefore recommended to use the top-down approach that starts with defining a WSDL file. Note that if the developer chooses a low-level toolkit that does not generate any code, the approach will not matter since the developer has accurate control over all parts of the service. However, the time to develop a Web service is greatly increased when using such toolkits. - 2 -

The bottom-up approach may initially be sufficient for some resources, but developers are referred to the documentation of the various toolkits for further instructions on this approach. This document will from now on focus on the top-down approach for making Web services. 2.2 Web Service Mechanism Since Web services are based on XML, it is necessary to possess knowledge in the basic XML concepts. The W3Schools website at http://www.w3schools.com/ provides an introduction to all the relevant topics. It is also recommended to use an XML editor that can perform syntax checking and XML validation, see Section 5.1. When integrating bioinformatics resources, the most relevant interaction pattern is the Remote Procedure Call (RPC) (This should not be confused with the RPC-style of making WSDL files which is in contrast to the document/literal style used in this document). The RPC interaction pattern means that a client submits a query to a Web service, and receives a response immediately from the Web service. The technology opens up for other interaction patters like notifications where there is only one message with no reply, or interaction patterns where there is no clear separation between what is a server and what is a client. In this guide we will focus only on the RPC interaction pattern (see figure). Serialization SOAP message Request Message SOAP message deserialization Client INTERNET Web Service deserialization SOAP message Response Message SOAP message Serialization A message is an XML document, with a header and a body: <envelope> <body> <mymethod> <x>5</x> <y>5.0</y> </mymethod> </body> </envelope> - 3 -

The exchange pattern is published by the server interface described in WSDL (Web Service Description Language). A given client only needs to know what is described in the WSDL of the service to be able to call it. A standard WSDL file has basically the following shape: <definitions> <types/> <message><part/></message> <porttype><operation/></porttype> <binding/> <service><port/></service> </definitions> types: This element contains a description of the data types message: This element contains a description of the message (requests and responses) that will be exchanged part: This element describes the content of the message porttype: This element is a description of the interface itself, and contains the various operations of the Web Service. operation: This element describes an operation of the service, including its name, the kind of messages it implies etc. (An operation can be compared to a method or a function) binding: This element defines how to access the Web service. It contains information about message format, protocol details for operations and messages defined by a particular porttype. service: Define a set of ports port: Define an endpoint (where the service is) for each binding. 3 List of Actions to Develop Web Services In this section, we identify and describe 5 actions for developing highly interoperable Web services: 1. Express data types in XML Schema 2. Define a WS-I compliant interface in WSDL 3. Generate server stub from the WSDL document 4. Connect backend code with the server stub 5. Deploy the Web service 3.1. Express Data Types in XML Schema One of the benefits of Web services is that a client can inspect the WSDL file to understand the format of the data it can retrieve from a Web service. This information can, for example, be used by the client to automatically generate native objects out of the received data. This would not be possible if the format of the data was simply defined as some text. Many bioinformatics resources do not have an internal data model. In this case, the task of designing a data model is a large fraction of the effort needed to make a Web service. In the case where there is a data model available, it can be necessary to define a subset of the data types to make publicly available. - 4 -

When a data model has been defined, it needs to be expressed in XML Schema. This requires the use of an XML editor. A general rule of making XML Schemas is to primarily use elements, and only limited use of attributes. As its name implies XML Schema is a document XML itself. Just like the DTD, it allows describing complex data structure and can be used for validation. Let s consider we have the following object: PERSON First Name Last Name Age It can be described this way in XML Schema: <element name="person"> <complextype> <sequence> <element name="firstname" type="string"/> <element name="lastname" type="string"/> <element name="age" type="int"/> </sequence> </complextype> An instance of this schema would look like this: <person> <firstname>john</firstname> <lastname>doe</lastname> <age>40</age> </person> 3.2. Define a WS-I compliant Interface in WSDL A WSDL document may look unnecessarily complex at first glance, but keep in mind that they are not primarily meant to be read by humans. As with all XML, having a good tool greatly simplifies understanding and handling of WSDL documents (see Section 5.1 for - 5 -

appropriate tools). It is recommended to start from a template/example WSDL that is known to be correct and helps you get started. One such template can be found at: http://api.bioinfo.no/wsdl/echoservice.wsdl While defining the WSDL file, please keep in mind the recommendations regarding interface design of statefull resources and databases in the Technology Recommendation (Section 5 in D3.1.1). While providing the necessary freedom to express any interface, WSDL documents must also ensure that different clients can interpret it unambiguously. In other words, the Web service interface must always be interpreted in the same way by clients accessing it in order to ensure interoperability with them. This can be achieved by restricting the WSDL language to a common set of rules which are described in more detail in the following two paragraphs: WS-I Compliance In order to achieve this the Web Service Interoperability (WS-I) organization has created a specification (also referred to as WS-I Basic Profile) that clarifies and restricts the usage of WSDL 1.2. As of writing this document there exists already a WS-I Basic Profile version 1.2 but we recommend that at least version 1.0 should be implemented. For further details on WS-I Basic Profile version 1.0 refer to: http://www.ws-i.org/profiles/basicprofile-1.0-2004-04-16.html Note that the above specification is meant to be read by WSDL experts and is therefore rather difficult to understand. Several compliance checking tools can be used to analyze your WSDL file and check for compliance. For instance, the WS-I organization provides such tools, but there are also several on-line checkers such as the one below: http://mindreef.net Document/literal wrapped style WSDL allows for several binding styles such as RPC or document based, encoded, not encoded etc. where only a certain combination is WS-I compliant. However, a best practice approach that is agreed among many Web service developers is the usage of a document/literal wrapped style. The following document referenced below gives an excellent overview of the different binding styles and provides examples on how to write WSDL files that use document/literal wrapped. We recommend that you read that document before you continue with our guide: http://www-128.ibm.com/developerworks/webservices/library/ws-whichwsdl/ In the remainder of this section we extract the most important parts of the above document to illustrate the usage of the document/literal wrapped style: - 6 -

1. All message elements should have only one part. The part should be named parameters. Example: <message name="mymethodrequestmessage"> <part name="parameters" element="mymethod"/> </message> 2. The element used in a message part should be wrapped. That means that everything that is sent in a message part should be placed inside a wrapper element containing a <complextype>. Example: <element name="mymethod"> <complextype> <sequence> <element name="x" type="int"/> <element name="y" type="float"/> </sequence> </complextype> 3. The name of a wrapper element inside an incoming message should have the same name as the operation that the message is sent to. Example (same as above): <element name="mymethod"> <complextype> <sequence> <element name="x" type="int"/> <element name="y" type="float"/> </sequence> </complextype> 4. The name of a wrapper element inside a return message should have the name as the operation that the message is returned from, appended with Response. Example: <element name="mymethodresponse"> <complextype/> 5. The binding style defined in the binding element of the WSDL file should be Document/Literal. Example: <binding name="myservicebinding" type="myservice"> <binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/> <operation name="mymethod"> <operation soapaction="http://mydomain.org/mymethod"/> - 7 -

<input> <body use="literal" encodingstyle="http://schemas.xmlsoap.org/soap/encoding/" /> </input> <output> <body use="literal" encodingstyle="http://schemas.xmlsoap.org/soap/encoding/"/> </output> </operation> </binding> This is the entire example WSDL file. This is not a fully functional WSDL file as it has been simplified to make the presentation clearer. <definitions> <types> <schema> <element name="mymethod"> <complextype> <sequence> <element name="x" type="int"/> <element name="y" type="float"/> </sequence> </complextype> <element name="mymethodresponse"> <complextype/> </schema> </types> <message name="mymethodrequestmessage"> <part name="parameters" element="mymethod"/> </message> <message name="empty"> <part name="parameters" element="mymethodresponse"/> </message> <porttype name="pt"> <operation name="mymethod"> <input message="mymethodrequestmessage"/> <output message="empty"/> </operation> </porttype> <binding name="myservicebinding" type="myservice"> <binding style="document" ransport="http://sch..."/> <operation name="mymethod"> <operation soapaction=" http://mydomain.org/mymethod "/> <input> <body use="literal" encodingstyle="http://sch..." /> </input> <output> <body use="literal" encodingstyle="http://sch..."/> </output> </operation> </binding> <service name="myservice"> - 8 -

<port name="myserviceport" binding="myservicebinding"> <address location="http://example.org/myserviceendpoint"/> </port> </service> </definitions> Here, we can see that it is a simple service with a single operation mymethod, which takes two parameters an integer and a float, and returns nothing (an empty message). A request message sent to the Web services described with the WSDL listed above would be like this: <envelope> <body> <mymethod> <x>12</x> <y>1.2</y> </mymethod> </body> </envelope> A response message would look like this: <envelope> <body> </mymethodresponse> </body> </envelope> Actually, here the mymethod and MyMethodResponse elements does not correspond to the operation, but to the wrappers, so it means that it is not possible to have overloaded operations because, you would need at this time to have two elements in the WSDL with the same name, which is not allowed. But as finally overloaded methods are forbidden by WS-I in the same porttype, it is not a real issue, even if WSDL allows it. 3.3. Generate Server Stub from the WSDL Document Stubs are pieces of code that acts as a representation of another computer system. This is accomplished by forwarding requests to the stub to the remote system, and forwarding the response to the local system. In the context of Web Services this means that a client stub acts as a local representation of the Web service, and a server stub acts as a representation of the client (see figure below). A stub can be used as though it was the remote system, and it will take care of all the communication details alone. - 9 -

Web service Client Stub XML Stub Method call Return value Backend Tools exist for several programming languages that can automatically generate client or server stub code based on a WSDL file. This means that all the code necessary to implement the Web service is automatically generated, and all you need is to connect the stub to the backend. The backend is the software system that will be provided with a Web service interface. This can be everything from a small script to a large application or a database. Note that not all toolkits have the capability to automatically generate stubs. In general it is a good idea to use a toolkit capable of this. However, there are more considerations to take into account when choosing a toolkit. Choosing a toolkit in the same programming language as the backend is implemented with can be a good idea, but it is not required. (Not that Web services are by design language and platform independent.) The toolkits available in the Java language are among the most mature. Example in Java With Axis1 for instance, it is possible to generate the stubs in java from the WSDL file with the class: org.apache.axis.wsdl.wsdl2java. To create a client stub, people have simply to run: % java org.apache.axis.wsdl.wsdl2java <wsdl file url> This will generate a class for each type defined in the WSDL, an interface for each porttype, a stub class for each binding and finally a service interface and locator implementation for each service of the WSDL. The stubs are useful on the client part to invoke the service. They are not to be instantiated directly, but instead a locator is created to get a new stub instance. A piece of code for invocation would look like this: public class client { public static void main(string [] args) throws Exception { myservice service = new myservicelocator(); myservicestub port = service.getmyservice(); port.mymethod( 12, 1.2 ); }} - 10 -

The service interface actually defines a get method for each port element of the WSDL and the locator is simply an implementation of this class. Wsdl2java can be also used to generate the server-side services classes with the following command: % java org.apache.axis.wsdl.wsdl2java --server-side --skeletondeploy true <wsdl-file-url> Two classes are generated at this stage: a skeleton and an implementation for the service. These classes are generated along with two WSDD files (web service deployment descriptor). Remarks By generating the code that implements the Web service directly from the WSDL file, it can be guaranteed to be a true implementation of the interface. The details of how this is performed depend on which toolkit is used. See the toolkit documentation. Note that some toolkits do not provide the functionalities to generate a server stub. In such cases the developer needs to implement this code manually. This may introduce significant extra effort for the developer compared to having it generated. It is important to validate Web service implementations that are completely manually written against the WSDL file, as mentioned at step 5 of this action list. 3.4. Connect Backend Code with the Server Stub At this stage you should have some code that is an implementation of the interface described in the WSDL file. However, it doesn t actually provide any functionality yet. To make the Web service able to provide the functionality it is supposed to have, you need to connect the server stub with the code of your existing resource (the backend). In the case when the server stub has been auto-generated, the developer is provided with a skeleton to implement. Example: If the Web service has an operation called mymethod that receives an integer and returns a string, the skeleton might look similar to this: mymethod(int x) { String y = null <Manual code goes here> return y } The manual code reads the x parameter, calls the backend using the parameter, and assigns the return value y (with the value returned from the backend). The resulting combination of backend code and server stub together forms the implementation of your Web service. It should be able to: Properly translate XML messages from a client into a request to the backend Execute the necessary logic of the backend - 11 -

Properly translate the return value of the backend into XML messages that will be returned to the client. 3.5. Deploy the Web Service The Web service you have developed might need to be deployed into an application server, depending on the toolkit used. Example in Java For instance with Axis1, two WSDD files were generated previously: deploy.wsdd and undeploy.wsdd These are Web service deployment descriptors (WSDD) that are useful to deploy easily or undeploy the service in an Axis container. The following command will deploy your service: % java org.apache.axis.client.adminclient deploy.wsdd and the following one, will undeploy it: % java org.apache.axis.client.adminclient undeploy.wsdd In the case of Axis2 this is done by first generating an Axis archive file (aar), then copying this file into the services directory of the Axis web application. This file is usually generated using an ant build script found at the Axis2 website. Other Web service toolkits might have very different deployment procedures. Remarks After deployment it is a good idea to test the service by either making a client or by using a tool such as SoapUI (http://www.soapui.org). The SOAP messages sent to and from the service should be validated against the WSDL definition. This can usually be done in an XML editor or a tool like SoapUI. 4. Write the Documentation 4.1 Mandatory WSDL Documentation It should be mandatory that people document their service for EMBRACE. The WSDL specification allows the inclusion of documentation in free text form within a service interface description file. It will help understanding what the service does. The parts that need to be specifically documented are the service, operation and data type elements in the schema section. All schema elements used in a way a user would perceive as an input or output type must be documented. People must use meaningful names for their parameters and operations, so that the users can understand more easily what the services are actually doing. - 12 -

4.2 Mandatory API Documentation All Web services must provide API documentation. This documentation must include a description of the service, of what it does, of the operations and their inputs and outputs. It should also give example inputs and anticipated outputs. The documentation is indeed a very important part, as it plays a key role in the semantic aspects of the services. When EMBRACE will be able to deliver common definitions and ontologies, it twill be easier to use the services and redefine them if necessary according to the Embrace ontologies. This is the example WSDL file provided with the necessary documentation: <definitions> <documentation>this element can give a overall description of the Web service. </documentation> <types> <schema> <element name="mymethod"> <annotation> <documentation>datatype elements can be documented in this way. If a wrapper element references other elements that are documented, documentation of the wrapper might not be necessary. </documentation> </annotation> <complextype> <sequence> <element name="x" type="int"> <annotation> <documentation>individual elements can also be documented. </documentation> </annotation> <element name="y" type="float"/> </sequence> </complextype> <element name="mymethodresponse"> <complextype/> </schema> </types> <message name="mymethodrequestmessage"> <part name="parameters" element="mymethod"/> </message> <message name="empty"> <part name="parameters" element="mymethodresponse"/> </message> <porttype name="pt"> <operation name="mymethod"> <documentation>operations can be documented in this way. </documentation> <input message="mymethodrequestmessage"/> - 13 -

<output message="empty"/> </operation> </porttype> <binding> </binding> <service> </service> </definitions> Note the difference in documentation for data type elements (in the schema part) and all other documentation. Data types need to use <annotation> </annotation> in addition to <documentation> </documentation>. Furthermore, WS-I compliance also requires that the documentation is always placed at the beginning within an element. Correct example: <element name="mymethod"> <annotation> <documentation>datatype elements can be documented in this way. If a wrapper element references other elements that are documented, documentation of the wrapper might not be necessary. </documentation> </annotation> <complextype> <sequence> <element name="x" type="int"> <annotation> <documentation>individual elements can also be documented. </documentation> </annotation> <element name="y" type="float"/> </sequence> </complextype> Wrong example: <element name="mymethod"> <complextype> <sequence> <element name="x" type="int"> <annotation> <documentation>individual elements can also be documented. </documentation> </annotation> <element name="y" type="float"/> </sequence> </complextype> <annotation> <documentation>datatype elements can be documented in this way. If a wrapper element references other elements that are documented, documentation of the wrapper might not be necessary. </documentation> - 14 -

</annotation> 5 Suggested Tools and Toolkits for Beginners In this section, we suggest tools and toolkits for those who wish to start developing Web services. As the technology is evolving extremely fast, these recommendations may become rapidly obsolete so they should be taken with caution and will be updated in the coming months. 5.1 XML and WSDL Editors Eclipse Web Tools Platform (Eclipse WTP) WTP is a vendor-neutral open development platform and application framework for building software which provides an XML editor and a WS-I validator. See http://www.eclipse.org/webtools/ for information and download. Altova XMLSpy XMLSpy is one of the leading tools for creating and manipulating XML documents. It contains many advanced features that are very useful when working with XML. See http://www.altova.com/products/xmlspy/xml_editor.html for further details. It is a commercial product and has a 30 day evaluation period. EMBRACE partners have the opportunity to become Altova educational partners that provide 30 free licenses to the enterprise version of all Altova XML products. The requirement is that a partner has to arrange an annual course where a tool from Altova is included. The shape of this course is not defined and may be held internal only. EMBRACE partners may contact Tina Eisinger on email: tina.eisinger@altova.com. 5.2 Web Service Toolkit The most important feature of Web services is that they are platform independent. Because of this platform independence it should not be very important which Web service toolkit is being used to develop a Web service. This is only partly true due to different levels of maturity of the various toolkits. This is a short list of the most mature toolkits for different platforms: Axis1 (Java) Axis1 is a relatively stable implementation that has been tested and tried for years. It provides all the advanced features described in this guide. Axis2 (Java) Axis2 is a new implementation of Axis and is therefore not as tried and tested as Axis1. However it is much easier to use, more efficient, and provides more advanced features than Axis1. Globus Toolkit 4 (GT4) (Java and Python) GT4 is based on Axis1, but adds additional features such as security and WSRF. - 15 -

gsoap (C/C++) gsoap provides many features, including stub code-generation. SOAP::Lite (Perl) SOAP::Lite has only limited support for automatic code generation. Zolera SOAP Infrastucture (ZSI) (Python) ZSI 2.0 supports client and server side stub generation, but has proven to be unstable. 6 Pitfalls and Hints Web services are still a rather new technology, and there is a rapid change in standards. For instance, if you take a text book that is 3-5 years old, it will most likely not give you correct examples with respect to WS-I compliance or document/literal wrapped styles. Many existing Web services actually use the more intuitive RPC style rather than document style. Whenever you look at WSDL examples, please have a careful look which style is used since mixing of the styles has implications on naming elements and types which can lead to unnecessary problems. We also suggest that you have a careful look at XML namespaces and the usage in WSDL. Acknowledgements We gratefully acknowledge comments and suggestions from Tom Oinn. - 16 -