Test and Validation of Web Services

Transcription

1 University Bordeaux 1 Doctoral school of Mathematics and Computer Science Thesis Test and Validation of Web Services presented by Ti ên Dũng Cao to receive the Doctoral Diploma of Computer Science December, 2010 Committee in charge: Charles Consel Professor - Institut Polytechnique de Bordeaux President Ana Cavalli Professor - Telecom SudParis Reviewer Manuel Núñez Professor - Universidad Complutense de Madrid Reviewer Richard Castanet Professor - Institut Polytechnique de Bordeaux Examiner Patrick Félix Assistant Professor - University Bordeaux 1 Examiner Fatiha Zaidi Assistant Professor - University Paris-Sud XI Examiner Supervisors: Prof. Richard Castanet - Assistant Prof. Patrick Félix Thesis n o :

2

3 Acknowledgments First and foremost, I would like to express my gratitude to my advisors: Mr. Richard Castanet, professor of Institute Polytechnic of Bordeaux, and Mr. Patrick Félix, assistant professor of University Bordeaux 1, for their patience, support, and encouragement throughout my graduate studies. This dissertation would not have been possible without their invaluable advice and guidance I also want to thank: Mrs. Ana Cavalli, professor of Telecom SudParis, and Mr. Manuel Nunez, professor of Universidad Complutense de Madrid (Spain), for their acceptance as the reviewers of my thesis. Mr. Charles Consel, professor of Institute Polytechnic of Bordeaux, and Mrs. Fatiha Zaidi, assistant professor of University Paris-Sud XI, for their acceptance as the members of committee in charge. I have had the great pleasure to work with my partners and friends in WebMov project. I would like to thank Nguyen Thi Kim Dung (PUF-HCM), Julien Borderie, Zouhair El Hilali, Guillaume Laborde, Guillaume Lameyre (University Bordeaux 1), who help me during development of prototype tools. I do not forget to thank my friends in LaBRI, my vietnamese friends in Bordeaux, for their help during my living time in Bordeaux. I wish to extend my deepest gratitude to my parents, for their years of hard work and dedicatation. Lastly, but most importantly, no words can express my gratitude to my wife for her unrelenting support, understanding and love. And the last words are reserved to my son.

4

5 Abstract In this thesis, we propose the testing approaches for web service composition. We focus on unit, integrated testing of an orchestration of web services and also the runtime verification aspect. We defined an unit testing framework for an orchestration that is composed of a test architecture, a conformance relation and two proposed testing approaches based on Timed Extended Finite State Machine (TEFSM) model: offline which test activities as timed test case generation, test execution and verdict assignment are applied in sequential, and online which test activities are applied in parallel. For integrated testing of an orchestration, we combines of two approaches: active and passive. Firstly, active approach is used to start a new session of the orchestration by sending a SOAP request. Then all communicating messages among services are collected and analyzed by a passive approach. On the runtime verification aspect, we are interested in the correctness of an execution trace with a set of defined constraints, called rules. We have proposed to extend the Nomad language, by defining the constraints on each atomic action (fixed conditions) and a set of data correlations between the actions to define the rules for web services. This language allows us to define a rule with future and past time, and to use the operations: NOT, AND, OR to combines some conditions into a context of the rule. Afterwards, we proposed an algorithm to check correctness of a message sequence in parallel with the trace collection engine. Specifically, this algorithm verifies message by message without storing them. Key-words: Web service orchestration, Timed Extended Finite State Machine, Online/Offline testing, Active/Passive testing, Runtime verification, Test case generation.

6 Résumé Nous proposons dans cette thèse les approches de test pour la composition de services web. Nous nous intéressons aux test unitaire et d intégration d une orchestration de services web. L aspect de vérification d exécution en-ligne est aussi consideré. Nous définissons une plateforme de test unitaire pour l orchestration de services web qui compose une architecture de test, une relation de conformité et deux approches de test basés sur le modèle de machine à l états finis étendues temporisés: l approche offline où les activités de test comme la génération de cas de test temporisé, l exécution de test et l assignment de verdict sont appliquées en séquentielle tandis que ces activités sont appliquées en parallel dans l approche online. Pour le test d intégration d une orchestration, nous combinons deux approches: active et passive. Au debut, l approche active est utilisée pour activer une nouvelle session d orchestration par l envoi d un message de requête SOAP. Après, tous les messages d entré et de sortie de l orchestration sont collectés et analysés par l approche passive. Pour l aspect de vérification d exécution en-ligne, nous nous intéressons à la vérification d une trace qui respecte un ensemble des constraintes, noté règles, ou pas. Nous avons proposé extendre le langage Nomad en définissant des constraintes sur chaque action atomique et un ensemble de corrélation de données entre les actions pour définir des règles pour le service web. Ce langage nous permet de définir des règles avec le temps futur et passé, et d utiliser des opérations NOT, AND, OR pour combiner quelque conditions dans le contexte de la règle. Ensuite, nous proposons un algorithme pour vérifier l exactitude d une séquence des messages en parallèle avec le moteur de collecte de trace. Mots-clés: L orchestration de services web, Machine à états finis étendues temporisés, Test en-ligne, Test actif/passif, Vérification d exécution en-ligne, Génération de cas de test temporisé.

7 Contents List of Figures ix List of Tables xi List of Algorithms xii 1 Introduction Context Web services Composite of web services Example of web services Motivations Web services development The necessaries of test phases Contributions Outline Publications of thesis Preliminaries Web service standards WSDL SOAP UDDI BPEL Introduction Based activities Structural Activities Scope Event handlers Fault handlers Software testing Conformance testing Test generation Passive testing Test tools Modelling orchestration of web service Automata Petri Nets Other formals v

8 2.4.4 Discussions Web services testing Testing from WSDL-based Testing from BPEL specification Passive testing Other works Conclusion Testing Approaches for Web Services Introduction Formal model Timed Extended Finite State Machine Unit testing Test architectures Conformance relation Offline approach Online approach Integrated testing Methodology Checking algorithm Conclusion Runtime Verification Introduction Methodology Test architecture The rules Checking algorithm Conclusion Implementation A case study: Product Retriever WSOTF tool Introduction Experimentations and results RV4WS tool Introduction Experimentations and results Conclusion Conclusions and perspectives Synthesis and results Perspectives Bibliography 97

9 A The tutorial of the tools 107 A.1 WSOTF A.1.1 TEFSM designer A.2 RV4WS A.2.1 Graphical interface A.2.2 Proxy A.3 TGSE A.4 BPELUnit B Specification of xloan web service 118 B.1 WSDL file B.2 BPEL code Index 123

10

11 List of Figures 1.1 Web Service Standards Stack General architecture of web services xloan - use cases xloan - BPMN specification WS development process and testing phases WSDL document structure SOAP message structure A BPEL structure of xloan example Classification of test type Examples of test cases An example of TEFSM An abstract model of SUT and its test architecture An example of xtioco relation of D-TEFSMs Offline testing approach for web services Test purpose example (t is a clock) Examples of abstract timed test cases Out reach time intervals Synchronous product and abstract test case selection A CSUT model Data generation example An example of test execution scenario Online testing approach illustration An overview of integrated testing approach Trace collection architecture for web services ProductRetriever - BPMN specification Architecture of the WSOTF engine Input format of WSOTF TEFSM of ProductRetriever Architecture of the RV4WS tool ParseData Interface of RV4WS Rule format example Checking analysis of RV4WS tool Testbed architecture Checking analysis of Product Retriever ix

12 5.11 Trace collection of Product Retriever A choreography example and the part under test A.1 The main GUI of TEFSM Designer A.2 The visualization of the test results by the tree A.3 The graphic visualization of the test results A.4 The main GUI of RV4WS A.5 Dialog to define a rule A.6 Dialog to define the properties (data correlation) A.7 Illustration of BPELUnit s test activities A.8 The main GUI of BPELUnit A.9 Edition of Send/Receive Synchronous of BPELUnit A.10 Test case execution of BPELUnit

13 List of Tables 4.1 An example of runtime verification Test results of ProductRetriever by WSOTF tool xi

14 List of Algorithms 1 Make the coverage tree from TEFSM Test execution Online testing algorithm Get outgoing transitions from the current state Checking algorithm from TEFSM Runtime verification algorithm verify_future(rule, msg, t, result) verify_past(rule, msg, t, result) xii

15 Chapter 1 Introduction Contents 1.1 Context Web services Composite of web services Example of web services Motivations Web services development The necessaries of test phases Contributions Outline Publications of thesis Context This thesis is effectuated in the WebMov 1 project that is supported by the French National Agency of Research within the part of Software Technology. This project is composed of two industrial partners (Softeam 2 and Montimage 3 ) and four academic partners (LaBRI, LRI- Orsay 4, GET-INT 5, Unicamp 6 Brazil). The main objective of WebMov is to contribute to the design, composition and validation of Web services through a high level of abstraction view and a Service Oriented Architecture based on a logical architecture vision (i.e., UML profile), from which BPEL orchestration of service composition is generated. Next, this orchestration specification is modeled by the formal models based on a variant of Timed Extended Finite State Machine [94]. From this model, many testing approaches were proposed for web service composition. Concerning the test and validation research part of this project at LaBRI, our works were effectuated in the Langage Système et Réseaux (in English: Language, System 1 Web Services Modeling and Validation

16 2 Chapter 1. Introduction and Network) team. This is a team which has extensive experience in the field of test with twenties thesis were supported in Bordeaux. 1.2 Web services Web services [5] are designed to overcome the challenge of automating business process interactions. Following definition of W3C 7, and specifically the group involved in the Web Service Activity: Web service is: a software application identified by a URI (i.e., Uniform Resource Identifier), whose interfaces and bindings are capable of being defined, described, and discovered as XML (i.e., Extensible Markup Language) artifacts. A Web service supports direct interactions with other software agents using XML-based messages exchanged via Internet-based protocols. Web services use XML format to wrap the data transmission. This format allows services that are implemented on different platforms, for example: Microsoft.NET, Sun J2EE, run on Window or Linux, can communicate using a common format (i.e., XML format). XML Schema provides the type system for XML documents as an approach to define the data structure, and SOAP [146] (i.e., Simple Object Access Protocol) is a framework to further standardize the structured type declaration for XML messages. An Internet protocol as HTTP (i.e., Hypertext Transfer Protocol) is usually used to exchange these XML-based messages from/to another service. Web services themselves are described by using public standards. Each web service has to publish its invocation interface, e.g., network address, ports, functions provided, and the expected XML message format to invoke the service, using the WSDL [143] (i.e., Web Services Description Language) standard. To describe the behavior, it means control flows, data manipulation semantics and service qualities, we can use such languages as BPEL [132], WS-CDL [144], OWL-S [145]. A stack of standardized protocols which were used by Web services is shown in the figure 1.1. Figure 1.1: Web Service Standards Stack After providing a service, its specification and functionality description, in the file WSDL, are registered in a UDDI registry, which allows each web service to be discovered by other services. If a client or service consumer wants to use this service, it will find the file WSDL in the UDDI [118] registry, and can analyse it to get information about the service. After 7 World Wide Web consortium

17 1.2. Web services 3 that, it will make a SOAP request message and send it to a port indicated by the WSDL file. A SOAP response message will be returned on either the same port or not depending on the support of service provider is synchronous or asynchronous. The synchronous services are characterized by the client invoking a service and then waiting for a response to the request on the same port. Otherwise, with asynchronous services, the client invokes the service but does not (or cannot) wait for the response. Often, with these services, the client does not want to wait for the response because it may take a significant amount of time for the service to process the request. The client can continue with some other processing rather than wait for a response. Later, when it does receive the response, it resumes whatever processing initiated the service request. We present here two patterns that are usually used to solve the problem of asynchronous services: One-way and notification operations (or callback): the request and the response are two messages defined within separate operations. The request is modeled as an inbound one-way operation and the response is modeled as an outbound notification operation. Each message is sent as a separate transport-level transmission; Request/reply operations: In this pattern, a request and a response are two messages defined within a single request/reply operation and sent as two separate and unrelated transport-level transmissions; The general architecture of web service is shown in the figure 1.2. Figure 1.2: General architecture of web services Composite of web services Web services are the elements based on a SOA application. A web service can be a SOA application. We can reuse, integrate other web services to have a new service. The latter is executed by interacting with other services, called partner services, to accomplish its workflow. We called this service is a composite of web services if we know the interaction between services of a composite. We need to distinguish two problems of a composite of web services:

18 4 Chapter 1. Introduction Orchestration: is the way that allows us to describe a new web service at message level, including the order of message executing and logical behavior. An orchestration exposes the internal logic of a single component by specifying the control flow structure and the data flow dependencies of that particular component. In an orchestration, there is only one main process (named orchestrator) that controls its partner services using a central architecture. BPEL Executable Process [132] is a standard language to define an orchestration specification. We can also use this language as the execution code of an orchestration by using an BPEL engine as Active-BPEL [4], Sun BPEL engine, Oracle etc. Choreography: describes a collaboration of services in a composite to accomplish one or several objects. A choreography is aimed at exposing the whole flow of interaction among all parties involved in the composite. In this context, there is no a main process as the orchestration, all participants are aware of the fact that they are taking part of the composition and, thus, of the way they should interact with each other. WS-CDL [144] and BPEL Abstract Process [132] are the languages to describe a choreography for a composite of web services Example of web services We present here a running example of web services: xloan. It is an extension of Loan approval example presented in the BPEL standard [132]. This service is built by using two web service partners: assessment service and bank service. It receives a request message from the client with: identification, name, income by month, amount, payment by month and month number for payment. Using this information, the Loan service executes a simple process resulting in either a "loan accept" or "loan reject". If the amount requested is greater than $10000, a risk assessment service is used to obtain a quick evaluation of the risk associated with the customer. If this risk is "high", a "loan reject" will be returned to client. In a case where the amount requested is less then or equal $10000, or low risk assessment, the bank service is used to approve the request of customer. Afterwards, loan assessment will be sent to client. Finally, if loan assessment is "accept", this process will wait for one minute to receive a confirm message from the client and forward this message to the bank service. If not, a cancel message will be sent to bank service as a notification. The figure 1.3 and 1.4 show the use cases and the flow specification by BPMN language. Its WSDL specification is described in the appendix B.1. Figure 1.3: xloan - use cases

19 1.2. Web services 5 Figure 1.4: xloan - BPMN specification

20 6 Chapter 1. Introduction 1.3 Motivations The Service Oriented Architecture (SOA) is an architectural paradigm which has gained great popularity in recent years. The main idea of SOA is: reuse and integrate the services, which are loosely coupled and mostly distributed. The interface of a service is described in an abstract interface language and can be invoked without knowledge of the underlying implementation. Services can be dynamically discovered and used. Nowadays, the SOA is the latest approach to building, integrating and maintaining complex enterprise software systems. Web services are the applications-based to build these SOA applications. A web service can be built as an SOA application by reusing and integrating the others web services. The Quality Of these Services (QoS) effects to the quality of application, and test is a method to assess the QoS. We introduce in section the web service development process based on SOA and the necessaries of test phases when we develop a web service in the section Web services development The web service (WS) development process based on SOA and the testing techniques that apply on each phase of the development are showed in the figure 1.5. Firstly, the WS specification is written in a WS specification language as WSDL. A decision needs to be made in the first step to decompose a WS into multiple services 8. If no decomposition is necessary, the WS will consist of only one service, otherwise, it will consist of multiple services. For each service, WS search will be performed (step 2) and the WS found will be tested if it allows to be tested. However, there are some running services that do not allow us to test directly (active testing) because the test may effect its database, so some of the located WSs may be tested after being composing into a composite by using passive testing technique. At this phase, the unit testing is applied on WS-based or a composite of WS depends on the available information from its specification. If there is no existing WS that satisfies our criterion, a new service will be developed (step 3). Of course, this service is tested before composing to the new composite. After all partner services are ready, we compose them to have a new service (step 4) and publish it (step 5). As said earlier, if these partners allow us to test, the integrated testing technique is applied before publishing. Finally, passive testing is used on the running service The necessaries of test phases In the figure 1.5, we propose four phases to test a web service: unit testing is applied to each partner service and to a new composite, integrated testing is an option phase that is used to test the interaction between the services on the real environment and finally, passive testing is used for monitoring. Unit testing: In a composite of web services, if the quality of one service is not good, that can effect other partners and the composite. For instance, a composite executes in parallel with some partners, if there is an error from any partners while the others have been updated. This traces the mistake of the partners if the rollback is not supported. On the other hand, if a composite does not conform to its specification, all its partner services may 8 In the example of xloan (section 1.2.2), this web service have been decomposed into two services: assessment and bank

21 1.3. Motivations 7 Figure 1.5: WS development process and testing phases

22 8 Chapter 1. Introduction be effected. So, unit testing is necessary to find the bugs on each single partner service and on the isolated composite without the interaction to its partners by simulating them. There are some characteristics of unit testing. In the context of this thesis, we focus on the conformance testing of composite of web services. Moreover, the time constraints must be considered in the test approach because web services are the real-time applications. Time may be the deciding condition for a composite either to continue the process or not. Integrated testing: web services are real-time applications and their quality depends on the condition of the real environment. All partners and their composite behavior can be correct but the quality of the back-end service is not guaranteed if the speed of the network is not satisfied or the capacity of the server is limited etc. This explains why integrated testing is necessary to check the interaction between services in the real environment before publishing to the back-end users. Passive testing: is an approach that is applied on the running services to verify some properties. Finding all faults of the services through testing is impossible, so supervising them after publishing is necessary. On the other hand, we cannot apply the active testing method to test some running systems, in many cases. For example, if we use the active method to test the function create_new_account of a bank service, this will make a mistake on the database of the service. A composite of web services is a system that is integrated at runtime and its result depends on its partners or the real environment. Monitoring technique is sometimes used to monitor some properties of the services that have the high capability of fault. But the conclusions made about these properties are not given. Passive testing is a technique that can monitor some properties of the services and give the verdict (true/false) by collecting the execution traces and analyzing them. Therefore, we are interested in using this approach to continue testing a web service after its publishing to the back-end users because this approach does not effect the running services. A method that can check real-time with the running service is very important because it allows us to immediately find the faults whenever they happen. Then, a solution can be proposed to solve these faults and avoid unnecessary damage. 1.4 Contributions This manuscript presents some our contributions to test a web service orchestration where some following aspects of test as active, passive, unit, integrated are focused. Most of our works focuses on the TEFSM (Timed Extended Finite State Machine) model of Lallali et al [94]. Unit testing framework: Unit testing is used to find bugs on a single web service. With a web service orchestration, unit testing means that the main process is tested without interaction with its real partners. We proposed in chapter 3 a framework for unit testing of an orchestration implementation that conforms to its specification. We focus on a gray-box testing type where the interactions between the composite of web services and its partners are available. This framework is composed of a test architecture, a conformance relation (i.e., xtioco) and a testing approach that is itself composed of 5 steps: (i) modeling the composite of web service by a TEFSM, (ii) generating all possible test purposes (iii) generating the abstract timed test case, (iv) deriving the concrete timed test case, (v) executing the timed

23 1.4. Contributions 9 test case and producing the verdict. Automatic timed test case generation: Based on the xtioco conformance relation, we proposed an automatic timed test case generation method (see section of chapter 3). This generation is guided by a test purpose which describes some needing verification properties on implementation of service under test. Using the TEFSM model as a formal specification, our method computes the synchronous product of the specification and a test purpose. While calculating this product, other feasible paths are considered in an attempt to cut the unsatisfying path. Finally, the timed test case is generated by selecting a trace leading to Accept state on the product. Online testing approach: Online testing is an approach that combines test generation and execution: only a single test primitive (input event or time delay) is generated from the model at a time which is then immediately executed on the implementation under test (IUT). Then the output produced by the IUT, as well as its occurrence time, is checked against the specification [113]. At that time, a new test primitive is produced, based on the values of previous events or random selection if there is some acceptable options and so forth until we arrive at a final state. Using this approach, the complete test scenario (test case suite and data) is built during test execution. This approach does not use the test purposes. We applied the latter for unit testing of web service orchestration (see section of chapter 3). Integrated testing of web services: Integration testing is aimed at exercising the interaction among components and not just single units. We propose in chapter 3 (section 3.4) an integrated testing approach to test the interaction of an orchestration with its real partners. This approach includes two steps: firstly, an active approach is used to start a new session of composite by sending a SOAP request to the main process. The communicating messages among components are then collected, including its occurrence time, to analyze and produce the verdict. We also proposed an algorithm that verifies an observable trace to be correct to a TEFSM specification. It means the passive approach is applied at this step. Automated runtime verification: We propose in chapter 4 a new approach to verify a running system by applying a set of constraints. This approach collects the observable traces of the system by installing a probe and analyzes it to produce a verdict. We extended the Nomad language to define a constraint, called rule, and proposed an algorithm to check whether an observable trace satisfies a set of rules or not. The rule is defined based on the order of messages, interval time between them, including future (a then b) and past (b before a) time. Specifically, our algorithm can check in parallel with the trace collection engine, so the faults may be found immediately and, we can stop the system to avoid any damage. Implementation and case study: We have developed two tools that support the complete testing of a web service composition. The first tool, named WSOTF (i.e., Web Services, Online Testing Framework) supports the complete active testing that is composed of timed test case generation, test execution and verdict assignment. This tool also allows us to simulate all partner services of an orchestration when we apply the unit testing technique. The second tool, named RV4WS (i.e., Runtime Verification For Web Services) focuses on the problem of passive testing, it means that the verification of some properties on a running service. These tools are going to introduce in the chapter 5.

24 10 Chapter 1. Introduction 1.5 Outline The structure of this thesis is organized as follows: the chapter 2 shows an overview of the state of the art. Chapters 3 and 4 are the main body of thesis that shows the results of our research. Chapter 5 presents some implementations of these theories and experience on a case study. Conclusions and perspectives are discussed in chapter 6. Chapter 2 presents the details of web service standards and the BPEL language that is used to compose some web services in order to have a new services. This is called a composite of web services. This chapter also discusses the conformance testing methods and the existing tools for the software systems. Some formal models and the testing approach for web services are introduced in this chapter as the starting point of the thesis. Chapter 3 introduces our testing approaches for web services. Firstly, we present a formal model TEFSM (Timed Extended Finite State Machine) that is used to model an orchestration of web services. In this chapter, two approaches (offline and online) for unit testing and an approach for integrated testing of a web service are introduced. For unit testing, we focus on conformance testing, so we defined a conformance relation, named xtioco (Extended Timed Input/Output Conformance), a test architecture, an algorithm that can generate all possible abstract timed test cases and online testing algorithm that generates and executes in parallel the timed test cases. For integrated testing, we propose a method that is composed of two approaches: active testing and passive testing. Firstly, active testing is used by generating and sending a SOAP request to SUT to start a new session. Then passive testing is applied by collecting the communicating messages among services and analyzing them. A new passive testing (runtime verification) approach is introduced in chapter 4. We discuss how to extend the Nomad language, by adding the constraints on each message and adding data correlation to group the messages, to define the rules for passive testing of web services. An algorithm that verifies message by message without storing them is proposed in this chapter. A mechanism of trace collection is also discussed in this chapter. Two implementation tools (WSOTF and RV4WS) of this thesis are presented in the chapter 5. WSOTF is a tool that is implemented based on the online testing approach for unit testing a web service or an orchestration of web services. RV4WS focuses on the aspect of verification of the running services. For each tool, we show an experience of a real-life case study of WebMov project, named Product Retriever. Finally, chapter 6, the conclusions and perspectives of our thesis are discussed. 1.6 Publications of thesis This thesis is the result of 2 years and 6 months of research spent at the LaBRI laboratory within the WebMov Project, under the supervision of Professor Castanet and Assistant Professor Felix. In the context of the WebMov Project, some international publications and technical reports have been produced. We shown below are five international publications under proceeding of conference with the reviews where: I am the main author of papers [1,3,4,5]

25 1.6. Publications of thesis 11 and paper 2 is a collaboration work with the partners of the WebMov project. 1. Tien-Dung Cao, Trung-Tien Phan-Quang, Patrick Felix and Richard Castanet. Automated Runtime Verification for Web Services. In the IEEE International Conference on Web Services, pages 76-82, Miami, Florida, USA, July 5-10, (Research Track, acceptance rate 17.6%) 2. Ana Cavalli, Tien-Dung Cao, Wissam Mallouli, Elilan Martins, Andrey Sadovykh, Sebastien Salva and Fatiha Zaidi. WebMov: An dedicated framework for the modeling and testing of Web services composition. In the IEEE International Conference on Web Services, pages , Miami, Florida, USA, July 5-10, (Applications and Industry Track, acceptance rate 17.5%) 3. Tien-Dung Cao, Patrick Felix and Richard Castanet. WSOTF: An automatic testing tool for web services composition. In the Fifth International Conference on Internet and Web Applications and Services, pages 7-12, Barcelona, Spain, May 9-15, IEEE Computer Society (acceptance rate 31%) 4. Tien-Dung Cao, Patrick Felix, Richard Castanet and Ismail Berrada. Online Testing Framework for Web services. In third IEEE International Conference on Software Testing, Verification and Validation, pages , Paris, France, April 6-9, (acceptance rate 26.5%) 5. Tien-Dung Cao, Patrick Felix, Richard Castanet and Ismail Berrada. Testing Web Services Composition Using the TGSE Tool, In SERVICES 09: Proceedings of the 2009 IEEE Congress on Services - I, pages , Los Angeles, CA, USA, July 6-10, 2009.

26 12 Chapter 1. Introduction

27 Chapter 2 Preliminaries Contents 2.1 Web service standards WSDL SOAP UDDI BPEL Introduction Based activities Structural Activities Scope Event handlers Fault handlers Software testing Conformance testing Test generation Passive testing Test tools Modelling orchestration of web service Automata Petri Nets Other formals Discussions Web services testing Testing from WSDL-based Testing from BPEL specification Passive testing Other works Conclusion

28 14 Chapter 2. Preliminaries We present in this chapter an overview of web service standards (section 2.1) and WS- BPEL or BPEL for short (i.e., Web Services Business Process Execution Language) that is an XML-based language designed to compose Web services in order to implement business processes (section 2.2). The existing approaches of software testing as black-box testing, white-box testing, unit testing, integrated testing, conformance testing will be discussed in section 2.3. Some existing models of web service orchestration will be introduced in section 2.4. Section 2.5 will present the test approaches from different specifications of web services. Finally, section 2.6 concludes this chapter. 2.1 Web service standards In section 1.2 of chapter 1, we show an overview of web services. Also some web service standards that are based on XML-based language are introduced. This section will discuss about these standards WSDL WSDL [143] (i.e., Web Service Description Language) is an XML-based language for describing Web services and how to access them. A WSDL document defines a service as a set of network endpoints (or ports). In WSDL, the abstract definition of endpoints and messages is separated from their concrete network deployment or data format bindings. This allows the reuse of abstract definitions: messages, which are abstract descriptions of the data being exchanged, and port types which are abstract collections of operations. The concrete protocol and data format specifications for a particular port type constitutes a reusable binding. A port is defined by associating a network address with a reusable binding, and a collection of ports define a service. The figure 2.1 presents the structure of a WSDL document where: Types: data type definitions using XSD schema; Message: an abstract, typed definition of the data being communicated; Operation: an abstract description of a supported action; PortType: an abstract set of operations by one or more endpoints; Binding: a concrete protocol and data format specification for a particular porttype; Port: a single endpoint defined as a combination of a binding and a network address; Service: a collection of related endpoints SOAP SOAP [146] is a transport protocol that provides the definition of the XML-based information which can be used for exchanging structured and typed information between applications in a decentralized, distributed environment using the network protocol as HTTP, SMTP etc. The figure 2.2 shows the structure of a SOAP message that contains two SOAP-specific subelements within the overall Envelope, namely a Header and a Body:

29 2.1. Web service standards 15 Figure 2.1: WSDL document structure Header element: is a optional part that is used to declare the informations of authentification, session management. Body element: is the mandatory element which implies that this is where the main end-to-end information conveyed in a SOAP message must be carried. Within the body element, two types of message exchange are supported: Conversational Message Exchanges and Remote Procedure Calls (RPC). A Conversational Message Exchanges can be modeled simply as XML-based content exchanged where the semantics are at the level of the sending and receiving applications. While a message with RPC type, the data and its semantics, for example: the address of the target SOAP node, the procedure or method name, the identities and values of any arguments to be passed to the procedure, are modeled at itseft UDDI Universal Description, Discovery and Integration (UDDI) is a platform independent, Extensible Markup Language (XML)-based registry for businesses worldwide to list themselves on the Internet. In the case of web services, it is composed of a set of web service description files using WSDL language. The communication between the client and the web service is firstly passed by a step of discovery and localization of service from this directory.

30 16 Chapter 2. Preliminaries 2.2 BPEL Introduction Figure 2.2: SOAP message structure Web Services Business Process Execution Language (WS-BPEL or BPEL) is a standard language that allows us to define a composite of web services in two ways: Abstract or Executable. A BPEL Abstract Process declares an interface of process behavior that is not intended to be executed and that must be explicitly declared as abstract. Whereas Executable Processes are fully specified and thus can be executed by an BPEL engine. Here, we introduce an overview of BPEL Executable Process (from here, called BPEL process) that is usually used to define a specification (or execution) of an orchestration of web service. BPEL defines the syntax based on XML format to describe complex business processes that can interact synchronously or asynchronously with their partners. A BPEL process is considered a composite of web services via a WSDL interface. This process always starts with the process element (i.e. the root of the BPEL document). It is composed of the following children: partnerlinks, variables, activities and the optional children: faulthandlers, eventhandlers, correlationsets. These children are concurrent. Like any programming language, BPEL provides the based activities as basic commands to interact with its partner or internal interaction, and the typical structural activities to control the message flow based on message values or timing constraints. A correlationset property is used in BPEL as a sessionid of the partners upon the value of data variables. The figure 2.3 shows a BPEL structure of xloan example 1. In this example, a correlation set is declared, named CS1, to correlate the client identification between two client requests (request function and conf irm function). 1 The full BPEL source is shown in the appendix B.2

31 2.2. BPEL 17 Figure 2.3: A BPEL structure of xloan example

32 18 Chapter 2. Preliminaries Based activities The based activities are the basic command of BPEL language. We can classify these activities into two types: communicating activities (receive, reply, invoke): A business process provides services to its partners through inbound message activities (<receive>, <onmessage> and <onevent>) and corresponding <reply> activities. <onmessage> and <onevent> are fasten in the structural activities <pick> and <eventhandler> that will be introduce in the next section. The <invoke> activity is used to call an operation on service providers. This activity can enclose other activities, inlined in compensation handler and fault handlers. The operation in this latter can be request-response or one-way operations, corresponding to WSDL operation definitions. and internal activities (assign, exit, empty, throw, wait): The <assign> activity can be used to copy data from one variable to another. The <exit> activity is used to immediately end the business process instance. The <empty> activity that does nothing is used to provide a synchronization point to wait other activities. The <throw> activity is used when a business process needs to signal an internal fault explicitly. The <wait> activity specifies a delay for a certain period of time or until a certain deadline is reached Structural Activities The structural activities control the execution order of its sub-activities by a sequence as <sequence>, <if>, <while> and <repeatuntil>, by a concurrence as <flow> or by the arrival of events as <pick>. <sequence> allows us to define an activity collection that are executed in sequential. <if> defines an ordered list of conditional choices that allow to chose only one activity to execute. <while> provides for repeated execution of sub-activities as long as the boolean condition evaluates to true at the beginning of each iteration. <repeatuntil> provides for repeated execution of sub-activities until the given boolean condition becomes true. <pick> waits for the occurrence of exactly one <onmessage> event from a set of <on- Message> events, then executes the activity associated with that event. The <onmessage> is similar to a <receive> activity, in that it waits for the receipt of an inbound message. After an event has been selected, the other events are no longer accepted. If it does not receive an event after a duration in <for> or a specified deadline in <until>, an <onalarm> event will be raised as a timeout and the activity associated with this event will be executed.

33 2.3. Software testing 19 <flow> provides concurrency or synchronization of a set of activities. The synchronization among activities are provided by means of links. Each link can have a source activity and a target activity. Furthermore, a transition condition, which is a boolean expression, is associated with each link and is evaluated (true or false) when the source activity terminates. For each activity of a flow, a join condition may exist, which consists of incoming links of the activity combined by boolean operators. Only when all the values of its incoming links are defined (true) and its join condition is evaluated as true, an activity is enabled and can start. If the join condition evaluates to false, the activity will not be executed Scope BPEL provides the <scope> like the sub-function of any programming language. The <scope> activities can be nested hierarchically, while the "root" context is provided by the <process>. A <scope> includes also variables, partner links, message exchanges, correlation sets, event handlers, fault handlers, and specially a termination handler and a compensation handler, which can not be attached to the <process>. In a <scope>, the errors may appear while we invoke a partner and this partner has been updated by our transaction. So a rollback mechanism must be supported to restore the previous state of the partner before we want to stop our <scope> or continue other works. A <compensationhandler> allows us to solve this problem Event handlers Each scope, including the process scope, can have a set of event handlers that are similar to a <pick> activity with the <onevent> to wait for the receipt of an inbound message. But the latter can be run concurrently and invoked when the corresponding event occurs. The child activity within an <onevent> or an <onalarm> must be a <scope> activity Fault handlers While a <process> or a <scope> is running, the faults may occur. In many cases, these faults need to be captured to process. Explicit fault handlers, if used, attached to a scope provide a way to define a set of custom fault-handling activities, defined by <catch> and <catchall> constructs. Each <catch> construct is defined to intercept a specific kind of fault, defined by a fault QName. A <catchall> clause can be added to catch any fault not caught by a more specific fault that are not defined explicitly. 2.3 Software testing Testing is an important step to verify and assess the quality of applications. To test, there are several approaches. According to the characteristics of the application, the phases of the development, the available information of specification and the capability of application control, An appropriate type of test can be applied for each concrete case. Following Jan Tretmans [135], we can categorize the types of test into four properties above and use a schema with four axes to show these types (Fig 2.4).

34 20 Chapter 2. Preliminaries Figure 2.4: Classification of test type The characteristics: Conformance: this kind is firstly considered because it is used to test the conformance of an implement under test with its specification. Robustness: is used to test the capability to deal with the unexpected data. Performance: can refer to the assessment of the performance of an application in the different cases. It is usually used to determine the speed or effectiveness of an application. Security: is a process to determine that an information system protects data and maintains functionality as intended. There are some security concepts that need to be covered by security testing are: Authentication: the process of establishing the identity of the user. Authorization: the process of determining that a requester is allowed to receive a service or perform an operation. Availability: assuring information and communications services will be ready for use when expected or the information must be kept available to authorized persons when they need it. Integrity: a measure intended to allow the receiver to determine that the information which it is providing is correct. etc.

35 2.3. Software testing 21 Reliability: to assess the good function of the different conditions, for example: timing constraints, speed of network, etc. The phases of the development: Unit testing: to verify the operation of only one component or one module in isolation to the rest of system. Integrated testing: to test the interactions between components or integrated system. It means that a system is verified at the interface level of each component. System testing: to verify the global behavior of the system. The accessibility: Black-box: the tester does not know the internal structure of system. Only its specification is used to generate the test cases for functional testing. Gray-box: some informations of the internal structure are available for tester. White-box: the tester knows the internal structure of the system (i.e., the code) and it verifies the structure by testing different paths in the code. The controllability: Active: the tester can interact directly with the system under test by sending the requests and receiving the results to analyze them. Passive: the tester assesses a system from input/output events or log file without interacting with the system under test Conformance testing Conformance testing means to verify if the behavior of the component, the delay and the exchanged messages, in a single system implementation corresponds to its specification. This is a kind of active testing and depending on the available informations, we can apply black-box, gray-box or white box technique to conclude the conformance. There are three main activities of a conformance testing method: test case generation, test execution and verdict assignment. To give a verdict (pass, fail or inconclusive), we need a concept that defines "how is the conformance?". Tretmans [136] defined a conformance relation, called ioco, that is usually used for black-box testing of a system using the input/output events. Definition 2.1. (ioco conformance relation): An implementation I conforms to a specification S by ioco conformance relation if for all possible evolution of S, the outputs of the implementation I may perform after a given input must be a subset of those for the specification.

36 22 Chapter 2. Preliminaries This ioco conformance relation allows us to know that an implementation conforms to its specification using only input/output event of implementation. For a timed system, ioco is not enough to say that I conforms to S because it does not define when these events happen. An extension of ioco, called tioco, is defined by M. Krichen and S. Tripakis [92]. In the latter, the timing constraints have been considered. It says that a system may perform a given action but also record a necessary duration that the system need to do. If output of I is slower than output of S, then I does not conform to S. That means the time delays are included in the set of observable outputs. A test represents a sequence of one or several inputs applied to an implementation under test. Once an output is received for a duration, we check whether it belongs to the set of expected outputs or not. In the latter case, a fail verdict is produced. In the former case, either a pass verdict is emitted or the testing process continues by applying another input. Each sequence of test likes that is a test case. If the time constraints are indicated in these test cases, we call them a timed test case. In this case, we want to interrupt the testing process while we do not yet find a pass or a fail verdict, an inconclusive verdict can be produced. In the other cases, an inconclusive verdict is also produced if we apply a test case from a test purpose. In [117], a test case is defined as following: Definition 2.2. (Test case): A test case is a tuple T = (S, I, O, T r, s 0, S I, S O, S F, S P, C) where S is the set of states, I and O are disjoint sets of input and output actions, respectively, T r S I O S is the transition relation, s0 S is the initial state,and the sets S I, S O, S F, S P S are a partition of S. The transition relation and the sets of states fulfill the following conditions: S I is a set of input states, s 0 S I and s S I,!t = (s a s ) T r such that a I and s S O. S O is a set of output states, and s S O, o O,!s such that s s / S O. Moreover, i I and s S such that s i s T r. o s T r and S F, S P are the set of fail and pass states that are terminal. Besides, s S F S P, a I O and s S such that s a s T r. C: S P T ime is a function associating time stamps with passing states. In Figure 2.5, we present some examples of test cases (time stamps are admitted at pass state) where i i represents the inputs (resp o i are the outputs) Test generation We have discussed two problems of test that are test execution and verdict assignment. The rest of the problem is how to test case generation? manually or automatically? This is the biggest problem of automatic testing. Many methods have been proposed to automatically generate the test case (or timed test case). Earlier methods such as TT, W-method, UIO... are based on the generation of paths in automata modelling the interactive system to be tested. In this section, we introduce the idea of some methods. Some test tools that use these methods or other methods will be introduce in section Most of these methods are based on the generation (global or partial) of the reachability tree of all the behaviours.

37 2.3. Software testing 23 Figure 2.5: Examples of test cases Hit-or-Jump [48] is an algorithm to generate the test case for an embedded system that can be modeled by communicating extended finite state machines. It is a generalization and unification of both the search technique and random walks. The idea of the latter is: at any moment, a local search (depth first or breadth first) from the current state in the neibourhood of the reachability of the graph is executed until find a transition (noted t) that does not yet consider or reach a search depth (or space) limit (Hit step). In the latter case, the Jump step will be carry out by randomly selecting a leaf node from the tree, that is generated by local search, and a new local search is started from this node. In the former case, the current state will be updated by target state of t and a new local search continues. The algorithm terminates when all transitions are considered at least once. A modification of this strategy is used by Lallali et al [96] to generate a timed test case from an IF (Intermediate Format) model using a timed test purpose. Model checking is a technique of formal verification that allows us to automatically analyze a complex system and check if it satisfies certain properties or not. In model checking, the properties that are used to verify the system, are usually specified by the Linear Temporal Logic(LTL) formula or other logics. The model checker reaches all the possible states in the model and checks whether the properties hold. On the other hand, if the properties do not hold, a trace of the steps illustrating the violation of the property is given, which is called a counterexample. The works in [33, 67, 75, 83, 157] are used in this technique to generate a test case from different model checker tools such as SPIN [81], NuSMV [1], Uppaal [18]. In these approaches, test purposes are used as the properties to verify a system by the syntax: "the transition X is never executed".

38 24 Chapter 2. Preliminaries Building a coverage tree from a formal model of Extended Finite State Machine (EFSM) that can cover all possible paths, all states and all transitions through an EFSM to generate a set of test cases based on symbolic transformations makes it possible to exploit a particular analysis technique using breadth or depth first searches, called Symbolic Execution Tree (SET), is proposed by Kim et al [91]. This approach is also used in [23, 68] to generate a set of test cases from an Input/Output Symbolic Transition System (IOSTS). But the test purposes are defined as some particular subtrees of this symbolic execution tree and the length of tree is used to cover test cases. A test sequence is generated by simply applying random inputs constructing a random walk over the graph representing a formal model until the stop condition is satisfied. This is the simplest method for generating test sequence. It is extended by executing immediately a system under test and verifying the correct output. The next input will be composed of the random walk and the value of the previous input/output. It is called the online approach [19, 32, 60, 97]. The biggest problem in the automatic test generation and model checking is the state explosion of the reachability tree. To limit this explosion, some methods that are based onthe-fly technique have been proposed. Castanet et al [44] proposed a method to generate the test case by using test purpose. Firstly, the authors define a set of rules to compute the synchronous product between the model specification and the test purpose. Next, all clocks are synchronized to one clock and an interval intersect between them is computed for each transition. Finally, for each visible trace of the product which there is no the transitions where its interval time intersect is empty, a pass verdict will be assigned if it leading to Accept states. Else, an inconclusive verdict is assigned. The rest cases will be assigned a fail verdict. Jard and Jéron [87] also used this technique, but test cases are directly selected on the visible behaviors of the product (it performs a selection of traces leading to Accept states) because in the formal model of [87], there are no constraints on the clocks Passive testing Passive testing is a process to collect a sequence of message from the system under test and analyse it to give the verdict. In passive testing the tester does not need to interact with the system under test. Passive testing is usually used as a monitoring technique to detect and report errors when we can not use an active testing method because it may be cause a negative effect in the system, for instance, a mistake on database. Another area of application is in network management to detect configuration problems, fault identification, or resource provisioning. This section will discuss some passive testing approaches. Bayse et al [17] and Cavalli et al [47] proposed a passive testing approach based on invariants of a Finite State Machine (FSM). For a FSM M = (S, s in, I, O, T ) where S is a set of finite states, s in is an initial state, I is the set of input actions, O is the set of output actions and T is the set of transitions. The authors define two types of invariants: Simple invariant: a trace such as i 1 /o 1, i 1 /o 1,..., i n 1 /o n 1, i n /O is a simple invariant of M if each time that the trace i 1 /o 1, i 1 /o 1,..., i n 1 /o n 1 is observed if we obtain the input i n then we necessarily get an output belonging to O,where O O.

39 2.3. Software testing 25 Obligation invariant: to express properties such as "if y happens then we must have that x had happened before". Next, the authors present two algorithms to check from left-to-right and right-to-left a finite trace to give a verdict. This approach does not consider the time constraints on the traces. TIPS [49](Test Invariant of Protocols and Services) is an implementation tool of this approach. An extention of simple invariant with time constraints, called Timed Invariant which allows us to express temporal properties is introduced in the works of C. Andrés et al [8, 9, 10]. There are some limitations of the Timed Invariant model: Supports only the future time, the past time was not admit. Its semantic is: if a pair input/output event (the interval time between an input and an output is also considered) or some event pairs have been happened and we continue obtain an input, then an output must be happened after a duration; Not support the operations as NOT, AND, OR to combine some conditions to a Timed Invariant; Not consider the constraints on the content of each event, so the data correlation problem between the events is also not considered; Finally, the tool PasTe [9] that is implemented to check the correctness of a log with respect to a set of time invariant does not allow us to verify an execution trace in parallel with the trace collection engine (it means the runtime verification or the online checking). Mallouli et al [108] proposed the security rule using the Nomad language to express the constraints on the traces with obligations, prohibitions and permissions. A prohibition or a permission rule is granted and it applies immediately on the trace, an obligation rule needs a deadline and the works are not completed before this deadline. This approach solves the time constraints of invariant approach. An algorithm to check the correction of the trace following these security rules is introduced. This approach does not consider the correlation of messages by its data values, an important problem of passive testing. M. Tabourier and A. Cavalli [133] proposed an approach to verify the traces actually belong to the accepted specification that is provided by a finite state machine. This method is composed of two stages: Firstly, passive homing sequence is applied to determine the current state. Initially, all states are the candidates. When an input/output arrives, these current states will be updated by the destination state of corresponding transition if it is the source state of transition. If not, it is removed from the candidate list. After a number of iterations, either a single current state is obtained and we move to the second step to detect the fault or an input/output pair is not accepted by any candidate state. In the latter case, a fault has been detected.

40 26 Chapter 2. Preliminaries Secondly, fault detection is used by applying the search technique from the current state and the current input/output pair. If a state which does not accept the following transition is reached, there is an error. If not, the end of the trace is reached, no error was detected. In the case where the trace is collected from the execution of multi-sessions that run in parallel, we can not used this approach. Moreover, this method does not consider the time constraints on the traces Test tools A number of tools have been developed for test including test case derivation, test execution and verification. Reviewing some of them in this thesis is our practical motivation to develop the new tools for web service testing. TGSE TGSE [27] (Test Generation, Simulation and Emulation) is a toolkit developed by LaBRI within the RNRT Averroes project and the European project Marie Curie RTN TAROT (MCRTN ). It implements a generic test generation algorithm to generate the test cases by simulating a Communicating Systems (CS). A CS declares a set of shared resources (parameters and variables), a set of automata, a set of rules describing the different possible synchronizations between the entities, and a test purpose modeled by an automaton. This tool supports the passive and active testing (with test purpose) of one or several components with data and temporal constraints. For active testing, test cases generation is based on simulation where the exploration is guided by test purposes. For passive testing, the TGSE tool is used to check that a trace of an implementation is either a valid execution or not. This trace is also modeled as an ETIOA (Extended Timed Input Output Automata) and it is a component of the CS. This tool will be used in our approach to generate a test case for web services by using the test purposes (section 3.3.3, chapter 3). TGV TGV [37, 87] (Test Generation with Verification technology) is a tool that was developed by Irisa Rennes and Verimag Grenoble with the support of the Vasy team of Inria Rhones- Alpes, integrated into the toolset CADP, for the generation of test cases based on a system s specification and a test purpose. This tool implements the ioco implementation relation [136]. From the specification and the test purpose, a synchronous product is computed, in which the states are marked as Accept and Refuse using the information of the test purpose. Next, test cases are generated by selecting accepted on the visible behaviors, i.e., it performs a selection of traces leading to Accept states. The Pass verdicts are based on traces that reach "Accept". Traces not leading to an "Accept" state are truncated and the Inconclusive verdict is added. Finally, the Fail verdict is generated from observations not explicitly presented in all test cases corresponding to the test purpose. This tool does not generate the timed test cases because the time constraints do not consider in the input model and the test purposes.

41 2.3. Software testing 27 TestGen_IF TestGen_IF [49, 93] tool is based on active testing technique to generate a timed test case from a timed automata model IF [142]. This tool implements an automated test generation algorithm based on a Hit-or-Jump exploration strategy [48] and is guided by a set of timed test purposes. The timed test case that defined in this tool is a timed observable trace which can valid the requirements of the test purposes including time constraints. At any moment it conducts a local search from the current state in a neighborhood of the reach ability graph. If a state is reached and one or more test purposes are satisfied (a Hit), the set of test purposes is updated and a new partial search is conducted from this state. Otherwise, a partial search is performed from a random graph leaf(a Jump). In this tool, the Hit-or-Jump exploration strategy will stop when all test purposes are satisfied or reached a deadlock (no transition to explore). TorX TorX [32, 134] is a prototype testing tool based on on-the-fly (or online) approach using the ioco [136] or tioco quiescence conformance relation depending on the input format. The tioco quiescence admits a time delay at each state, but there is no fixed a concrete duration. It was developed at Formal Methods and Tool Research group at University of Twente in the Netherlands, in collaboration with Eindhoven University of Technology, Philips Research Laboratories and Lucent Technologies R&D center Twente. This tool implements an online testing algorithm that is a loop of two steps: execute an input to SUT, wait for an output from SUT for a duration and verify it. After each input/output action, the next action is computed. A pass verdict is given if the loop reaches a stop condition. If not, a fail verdict is produced. The basic formal model of TorX is Labelled Transition System (LTS). But it also allows us to use some inputs format as: LOTOS, Promela, Aldebaran, FSP (Finite State Processes). JTorX [19] is a version of TorX,which is wrote in Java, that allows us to easily use, install and configure because it can be done via the JTorX Graphical User Interface(GUI). We do not use this tool to test a web services because: All these formats do not allow us to model all properties of a specification of web service composition, for example BPEL, and specifically data type. This tool does not consider the timestamps transition and time constraints on each state (time invariant) which is very important with the asynchronous web services. T-Uppaal T-Uppaal [97, 114] (or Uppaal-TRON) is a tool integrated into the Uppaal tool environment. It performs model based black-box conformance testing of the real time constraints of embedded systems. T-Uppaal is an online testing tool which means that it, at the same time, both generates and executes tests event-by-event in real time. This tool uses timed automata [6] as the formal model. Instead of simulation of environment as TorX, it needs an assumed environment modeling that is also a timed automata and it will check that an implementation conforms to this environment following tioco [117] theory. From a current symbolic state, T-Uppaal randomly chooses between one of three basic actions: either send a randomly selected relevant input to the IUT (Implementation Under Test), letting time pass by some amount and observe the IUT for outputs, or reset the IUT and restart. Of course,

42 28 Chapter 2. Preliminaries the current state will be updated after each action. As the TorX, the system under test is attached to T-Uppaal via a test-adapter and considered as a black-box since its states cannot be directly observed; only communication events via input/output channels. But we can not use them for unit testing of a web service orchestration because: The input format of T-Uppaal is a timed automata, so its data types are poor. We can not use them to model the SOAP messages. Web service works follow the session by sending a request to enable a new session and, in general this session finishes when a response is returned. So a test needs to start from its initial state to a final state before we can restart, while the algorithm of this tool is randomly chosen at each current state between sending an input, waiting for an output or restarting. Therefore, restarting may be chosen when we do not arrive at a final state. Finally, T-Uppaal does not simulate the environment. It needs an environment model as the inputs. 2.4 Modelling orchestration of web service Many formal methods are used for the validation, verification and test of an orchestration of web services [2]. In recent years, many formal models have been proposed to model an orchestration specification of web services as the Automata, Petri Net, Symbolic Transition System, Guard Automata, Web service Timed State Transition Systems etc. In this section, we discuss an overview about some recent formal models before selecting one for our formal methods of this thesis. We can classify them into some following family: Automata, Petri Net, Symbolic Transition System. On each model, we also assess its advantages and disadvantages Automata Automata (or Finite State Machine) is an abstract machine that takes a symbol as an event and jumps or transitions, from one state to another according to a transition function. The transition function tells the automata which state to go to next given a current state and a current symbol. From this model-based, many extended versions are proposed to adapt with different objects. Here, we present some extended versions to model an orchestration of web services that is described by BPEL language. Fu et al [64] use a Guard Automata (GA) to model a BPEL process and added to the transition a guard that is directly queried from the messages or local variables by using XPath. There are three transition types: local-transition t = (s 1, g, s 2 ) where g is a guard, receive-transition t = (s 1,?a, s 2 ) where a is an incoming message and send-transition t = (s 1, (!b, g), s 2 ) where b is an outgoing message and g is a guard. This works focuses on analysis of interacting between web services more than internal logic of one service. So data flow or time constraints are not yet considered. For instance, the guard on transition of <while> or <repeat> activities depends on the data update functions that are not used in this model. The translation from BPEL specification to adfa (i.e., annotated Deterministic Finite State Automata) is proposed by Wombacher et al [147]. This formalism does not consider a

43 2.4. Modelling orchestration of web service 29 set of variables on data and on clocks. The guard of the transition if it exists, is used from the set of messages. So, the internal logics as while, for and the data correlation were omitted because we do not have the data variables to control this. Moreover, we can not control a delay action or a deadline for an action because the clocks were not admitted in this model. Kazhamiakin et al [89] introduce a formalism, called Web Service Timed State Transition Systems (WSTTS), to capture the timed behavior of the composite web services. This formalism considers the time constraints but the problem of data flow were omitted. It means that the data variables are not defined, so the data correlation management are not considered. In this work, the authors only focus on check BPEL compositions against time-related requirements more than all problems of conformance testing. Zheng et al. [154, 155, 156, 157] give the formal definition for the static semantics and briefly describe the dynamic semantics of a Web Service Automaton (WSA) that is an extension of Mealy machines (one in two types of Finite State Machine), to fulfill the formal model requirements of the web service domain. In this model, the update function and guard condition are admitted to the data flow. But the time constraints or time delays are omitted, so that the timed activities of BPEL as <wait>, <onalarm> are not considered. This model is used as the intermediate format and was coded by XML to be translated into the input format of Model-Checker SPIN or NuSMV. A timed model, called Timed Extended Finite State Machine (TEFSM) that can model most of BPEL activities as time constraints, time delays, data correlation, the property on each action etc, was proposed by Lallali et al [93, 94]. This formalism is closely related to timed automata [6] and permits carring out timing constraints, clocks, state invariants on clocks, property on transition, data variables and its domain. The authors also define a set of rules to translate each BPEL activities based into a partial machine and composes of them by the partial machine of the structural activities. An Intermediate Format (IF) [142] based on the communicating timed automata is proposed as the input format of the test or verification tools [95]. BPEL2IF [93, 96] is a tool to translate the BPEL specification into IF Petri Nets Petri nets [151] provide an elegant and useful mathematical formalism for modelling concurrent systems and their behaviors. A Petri net is a directed bipartite graph, in which the nodes represent transitions (i.e., events that may occur, signified by bars) and places (i.e., conditions, signified by circles). The directed arcs describe which places are pre- and/or postconditions for which transitions (signified by arrows). A number of extended Petri nets have been introduced to enhance its expressive capabilities. Among them are colored Petri nets, timed Petri nets, High-level Petri nets, more. This section shows some overview works of modelling BPEL-based web service composition using the Petri nets or its extension. Hinz et al [79] and Ouyang et al [120] defined a set of patterns translating a BPEL process to Petri nets. While Stahl [131] and Dong et al [53] use the High-level Petri nets (HPN) to model a BPEL process for analysis and testing. In these works, each BPEL activity, including basic activities, structural activities and also exceptional activities as faults, events, compensation. BPEL2PN [130] is a tool to translate a BPEL process to the input format of the LoLA [127] tool that has been implemented for the validation of reduction techniques for place/transi-

44 30 Chapter 2. Preliminaries tion net reachability graphs. Yang et al [149, 150] used the colored Petri nets to model a web service composition (orchestration or choreography), then used a CPNtools [50] to verify some properties of web service composition as reachability, boundness, dead transitions, dead marking, liveness, fairness, etc Other formals Bentakouk et al [23] translated a BPEL process to a Symbolic Transition System (STS) where two main problems of BPEL are interested: input/output direction of the messages on the different channels and its data variable domain while the guard on the data variable of the transition is protected. Foster et al [59] presented an approach to model and validate a web service composition. The BPEL semantic is described by the Finite State Process (FSP) notation that represents a Labelled Transition System (LTS). A BPEL process is translated to a FSP which is used as an input format of the model checker LTSA (Labelled Transition System Analyser) to check the composition correctness. These models (STS and FSP) are only interest in the flow control of service composition without considering its temporal aspect. Moreover, FSP does not describe all BPEL structure, for example, the correlation of the message and the exceptions. A new model, named BFG (BPEL Flow Graph) which is an extension of CFG (Control Flow Graph) is proposed by Yuan et al [152] to represent a BPEL process in a graphical model. Then concurrent test paths can be generated by traversing the BFG model, and test data for each path can be generated using a constraint solving method. Finally test paths and data are combined into complete test cases Discussions In this section ( 2.4), we have discussed many formal models for the orchestration of web services. Each model has different advantage and disadvantage because they focus on the different problems of testing. The works that focus on the automata model as adfa, GA, WSTTS, WSA, TEFSM, allows us to verify the behaviour of an orchestration by the time and data variable constraints. The Petri nets allows us to model the concurrent actions of an orchestration, for example the activities in a flow activity of BPEL. While the Symbolic Transition System (STS) focuses on input/output direction of the messages on the different channels and its data variable domain. But the STS and Petri nets models do not consider the time constraints. In our works, we use TEFSM of Lallali et al [93, 94] by adding a set of correspondent variable domain to model an orchestration of web service. 2.5 Web services testing As said earlier, web service is the application based to build the SOA application. In the last years, the community of software testing was interested in this domain. This section discusses some methods and tools that have been proposed to test a web service based (use only WSDL-based specification) or a composite of web service using BPEL specification Testing from WSDL-based Testing of web service from WSDL-based is a kind of black-box testing because only specifications of input/output messages of the operations are available. There is no information of

45 2.5. Web services testing 31 the internal logic, the relation between operations or data correlation between input/output messages of the operators etc. The methods that use this specification usually focus on the problems: operation flow, existence checking of operations, the validation of input/output message types, fault management, robustness etc. In the method of X. Bai et al [11], the test case generation method from WSDL-based includes four levels: test data generation, test operation, operation flow and test specification. Test data is generated from the data type of the message by examining the XML schema if it is a complex type and generating a correspondent value for a simple data type. The data generation may be used from a set of default values, a pattern list or random. Next these messages are used to test the operations. The test of operation flow is generated based on dependency analysis between operations by input/output parameters. Finally, the generated test cases are encoded in XML called Service Test Specification (STS) to execute to Service Under Test. STS takes the concepts of WSDL including operations, part, message, input and output, reflecting a nature mapping between WSDL elements and test elements. From this WSDL-based, Salva et al [125, 126] are interested in the existence checking of operations, the correction of output message types, fault management and robustness. For operation existence, each operation is called with random parameter values respecting the WSDL file. An operation exists if it returns either a response type as described in the WSDL file or a SOAP fault whose cause is different from the client and the endpoint reference not found. This first cause means the operation is called with bad parameter types, the second cause means that the operation name does not exist. Then the operation robustness is tested if it exists. This technique is implemented in the tool WS-AT (Web Service Automatic Testing). C. Keum et al [90] generate the test cases which are operation flows using the Extended Finite State Machine that is constructed from the pre-conditions and post-conditions of parameter values. J. Offutt and W. Xu [119] presented a new approach to testing the interaction of Web services based on data perturbation. Existing XML messages are modified based on rules defined on the message grammars, and then used as tests. Data perturbation uses two methods to test Web services: data value perturbation and interaction perturbation. Test tools for web services-based Jambition [60] is a tool which automatically tests web service-based based on STS (Symbolic Transition System) specification. It is integrated in the framework PLASTIC [28]. This tool focus on the same problem as X. Bai et al [11]. But the testing approach of Jambition is random, on-the-fly and uses ioco relation. SOAPUI [58] is a basic tool used to execute a web service, verify the syntax and semantic of WSDL file, and also verify the SOAP messages. It generates automatically the SOAP template for input messages from XML schema. It allows us to simulate the partner services in the case of a composite of web services via the MockServices. WS-TAXI [16] is a tool used to generate random values for SOAP template of SOAPUI, while TestMaker [122] is a platform for functional testing, regression, load and performance testing, and business service monitoring.

46 32 Chapter 2. Preliminaries Testing from BPEL specification Nowadays, BPEL is a popular language for specification and development of web service composition. So many testing approaches for web service composition use BPEL as an input format before translating to a formal model. This section discusses the test approaches: structural testing and functional testing. In the case of functional testing of web service composition, we called gray-box to distinguish with black-box of web service-based because we know the interaction between services in the composite and we can capture the communicating messages between them. This approach is usually used when we do not know the internal actions via its specification as Abstract BPEL, BPMN, UML etc. The structural testing means that we verify the structure of application using the source code, for example BPEL executable process. There are two types of this approach: static analysis via verification (called analysis and verification) and dynamic analysis via test execution (called white-box testing). Analysis and verification Model-Checking is a technique of formal verification that allows us to automatically analyze whether a complex system satisfies certain properties or not. In web service composition, this technique is also used in many methods to analyze a BPEL specification. To use this approach, firstly BPEL is translated directly or via an intermediate format before translating into an input format of the Model-Checking. Next, a set of constraints are defined to verify some properties. Finally Model-Checker is used to analyze these properties. The EA4B [72] (Execution Analysis tool for BPEL) is a analysis tool for BPEL process, integrating of the WSAT [65] tool and the Model-Checker SPIN [81]. The WSAT is a tool that generate the Promela language from the BPEL specification by using the Guard Finite State Automata as intermediate format. This tool also analyses the synchronizability before generating the Promela. Many works also use the Model-Checker for verification of web service composition described in BPEL as Foster et al [59], Nakajima [116]. Yang et al [149, 150] firstly translated the BPEL specification to the colored Petri nets (CPNs) then used a CPNtools [50] to verify some properties of BPEL as reachability, boundness, dead transitions, dead marking, liveness, fairness, etc. The complete transformation from BPEL to Petri nets is given by Ouyang et al [120]. Hinz et al [79] describe the tool BPEL2PN that implements the transformation when abstracting from data. The resulting Petri net can be verified using the tool LoLA which stands for a Low Level Analyzer, supports the verification of standard properties of Petri nets, like, for example, determining if a Petri net contains a deadlock, and the verification of properties expressed in the logic CTL. Ouyang et al introduce the WofBPEL [121] tool for analysis: Unreachable activities; Potentially conflicting "message receipt" actions; Determining, for each action in a BPEL process definition, which messages might eventually be consumed by the process after this action has been performed. This information can be used by a BPEL engine in order to detect which messages in the inbound queue may be discarded and thus optimize resource consumption.

47 2.5. Web services testing 33 White-box testing When we test a service, we receive only the input and output message. Using the white-box testing technique, we can verify the correction of output data because we know its internal actions via source code. While the gray-box testing technique does not allow us to do this. Several works [102, 103, 110, 111, 152] focus on verify the structure of BPEL via test execution: Li et al [102] proposed a framework for unit testing of a BPEL process. This framework includes: a test architecture based on simulating the partners, a lifecycle management schema and a test design outline. For the test architecture, the authors proposed two types: centralized and distributed. The centralized architecture is used if the Process Under Test (PUT) has only one partner, or if we choose to simulate the joint behavior of all the partner processes by a single test process. The distributed architecture is used if we want each test process to simulate a PUT partner. In this case, the communication (or synchronization) between test processes can be used depending on a sequential or parallel execution logic of PUT. Mayer [110, 111] developed a framework, named BPELUnit [34], that also focuses on unit testing of BPEL process and uses Li s distributed architecture [102]. Each partner is simulated by one test process, but there is no synchronization between these test processes. This framework allows us to execute, it means send the input (a SOAP message or a fault message), receive and verify the output, one or several test cases described by XML format. The interaction with BPEL process is synchronous or asynchronous. It supported verify the time constraints of the outputs, time delay of the inputs but it does not generate automatically the test case. Yuan et al [152] proposed a graph-search based approach to BPEL test case generation, which effectively deals with BPEL concurrency semantics. Firstly, BPEL is translated to BPEL Flow Graph (BFG) which is an extension of CFG (Control Flow Graph) to represent a BPEL program in a graphical model. Next, traverse the BFG to generate test paths. A BPEL test path is a partially-ordered list of basic activities that are executed during a specific test run. BPEL activities in a test path are classified into three types: one is "input-type" (receive, the receiving direction of 2-way invoke, etc); another is "output-type" (the sending direction of 2-way invoke, reply, 1-way invoke, etc); the third is "datahandling-type" (assignment, etc). After test paths are found, test data of "input-type" should be generated based on the conditions of boolean expressions on the path or random data generation to make the test path executable. At this step, "datahandling-type" is removed and output-type logic should be manually prepared in a form of either exact data values or invariable assertions, which are commonly called "test oracles". The authors called this is the abstract test case because they must be transformed to an input of a test execution tool. Yan et al [148] method of BPEL test case generation, which is based on concurrent path analysis. This method first uses an Extended Control Flow Graph (XCFG) to represent a BPEL process, and generates all these sequential test paths from XCFG using either the basic path coverage proposed by the authors or the test coverage criterion defined by the user. These sequential test paths are then combined to form concurrent test paths. The authors also provide a technique to process constraints to generate test data for test paths. This approach solves not only the test case generation, but also the test case for concurrent

48 34 Chapter 2. Preliminaries flow, but the time constraints are not yet considered. Zheng et al [156, 157] proposed a framework to test a composite of web services described in BPEL. The BPEL process is modeled by an intermediate formal model WSA (Web Service Automata) (see 2.4.1). Next, this model is translated to SMV (the input format of Model- Checker NuSMV) or Promela (the input format of Model-Checker SPIN). NuSMV or SPIN is used as the test case generation tool where the test coverages (i.e., all-states, all-transitions or all-du-paths) are described as the temporal logic (liked LTL or CTL). Garcia-Fanjul et al [66, 67] use also the Model-Checker SPIN to generate the test cases, but the BPEL process is translated directly to Promela instead of using an intermediate formal model. Gray-box testing Gray-box testing of a composite of web services means that we test only the interactions of the web services via the communicating messages without the internal actions. Depending on the available information specification, either the interaction of all web services is tested or only the interaction of the client and service under test is tested. Lallali [93] proposed a framework to unit testing, it means that only one process in a composite is tested isolated with its partners. The author used BPEL as a specification (not code source) of composite, the IF is used (see 2.4.1) to model this specification and used TestGen_IF [49, 93] tool to generate the timed test cases based on the test purposes. Finally, these timed test cases are executed using the BPELUnit [34] framework by simulating all partner services. Bentakouk et al [23] have proposed a framework to automated test a web service orchestration described by BPEL. The authors firstly translated BPEL into a formal model, named Symbolic Transition System (STS), then the Symbolic Execution Tree (SET) is computed from this STS. While the SET is computed, the feasible paths are considered as a condition to limit the state explosion by evaluating the existing variables and cut off the path if the guard on transition does not satisfy. The path length is used as a criterion to cover a set of execution paths which are finally run by a test oracle against the orchestration implementation. The test cases that are generated by this approach can be used by a gray-box testing approach because these test cases include a sequence of communicating input/output messages among client and SUT, or SUT and its partners. But the authors have used a black-box testing architecture to run these test cases against the orchestration implementation by skinning the communicating messages between SUT and its partners. Moreover, this method does not consider the time constraints Passive testing In recent years, many methods and tools are proposed and developed for passive testing of a web service (including a composite of web services) [54, 14, 13, 12, 45, 101]. These works focus on either checking the order of messages and/or its occurrence time on a trace file to give a verdict [45, 101, 100] or proposing a method for dynamic statistics [14, 12] of some properties of web services. Dranidis et al [54] propose the utilization of Stream X-machines for constructing formal behavioral specifications of Web services. The authors also present a runtime monitoring and

49 2.6. Conclusion 35 verification architecture and discuss how it can be integrated into different types of serviceoriented infrastructures. But the authors do not present an algorithm or a tool to verify an execution trace using the Stream X-machines specification of web services. Baresi et al [14, 13] present a monitoring framework for BPEL orchestration which is obtained by integrating two approaches namely Dynamo and Astro. These approaches are used for dynamic statistics of some properties of BPEL process from single instance or multi instances. These works focus on the behavioral properties of composition processes expressed in BPEL rather than on individual Web services. Moreover, an assessment (a verdict true/f alse) about service is not considered in this work. Cavalli et al [45] propose a trace collection mechanism for SOA by integrating modules within BPEL engine and a tool [45, 108] that checks offline an execution trace. This approach uses the Nomad [51] language to define the security rule. But it does not allow us to check real-time (i.e., "online") whenever a message happened. Moreover, this work does not consider the data correlation between the messages in the rules. The works of Li et al [101, 100] present the pattern and scope operators as the rulebased to define the interaction constraints of Web services. The authors use the finite state automata (FSA) as semantic representation of interaction constraints. In this approach, the validation process runs in parallel with the trace collection. This approach is limited by the pattern number. Moreover, this work does not consider the time constraints Other works There are several works that focus on the robustness testing. Looker et al [104, 106, 107] interested in the domain of assessing the dependability of web services by fault injection. The author also provided a fault injector tool, named WS-FIT [105] (Web Service Fault Injection Technology) that allows us to inject the faults at network level to be used to test the web services. The communicating faults that are injected into a web service are: duplication, omission, extra, delay and ordering of SOAP messages. Bessayah et al [31] introduced a fault injection tool for testing web service composition, named WSInject. This tool allows us to inject the fault into not only the communicating messages between client and SUT, but also the messages between SUT and its partners. WSInject supports to inject the fault at two levels: SOAP interface faults and communication faults. At SOAP interface level, the fault is injected by modifying the message contents, for instance, changing the values, deleting a field in the message, etc. At communicating faults level, the authors consider the following type of fault injections: corruption of parameter values or of message structure, delaying requests/responses, message deletion and message replication. 2.6 Conclusion At the begining of this chapter, we presented an overview of web service standards as WSDL, SOAP, UDDI, and a popular language (i.e., BPEL) that is usually used to define a

50 36 Chapter 2. Preliminaries composite of web services in two ways: abstract (choreography) and executable (orchestration). Atfer, we introduced an overview of software testing classification types, some testing approaches and test tools for conformance testing and for passive testing. This chapter also discussed the existing works on web service testing as the start point of our thesis. The different formal models of web service orchestration is discussed as Guard Automata, WSA, WSTTS, STS, TEFSM, Petri Nets, BPEL Flow Graph etc. Many testing methods of web services that focus on different types of test as black-box, white-box, gray-box or verification techniques are presented. We also introduced in this chapter some works on passive testing, monitoring and fault injection. In the next chapter, we will propose some conformance testing approaches for web services, including unit and integrated testing approaches. For unit testing, we propose two methods: offline where the test activities as test case generation, test execution and verdict assignment are executed in sequential and online where the test activities are executed in parallel.

51 Chapter 3 Testing Approaches for Web Services Contents 3.1 Introduction Formal model Timed Extended Finite State Machine Unit testing Test architectures Conformance relation Offline approach Online approach Integrated testing Methodology Checking algorithm Conclusion This chapter presents the main contributions on conformance testing. It is composed of the approaches for unit testing and integrated testing. For unit testing, we have proposed two approaches: offline and online that can apply to test a web service based or an orchestration. Offline testing generates firstly abstract timed test cases from a formal model. Then the concrete timed test cases are generated and executed against the implementation under test, while online testing generates, executes in parallel and randomly selects the timed test case. For integrated testing, we focus only on an orchestration. All our approaches are based on the TEFSM model defined by Lallali [94]. Some works of this chapter have been published in the ICST 2010 [42], ICIW 2010 [40] and Service Congress 2009-I [41]. 3.1 Introduction Web services are the elements based on a SOA application. A web service can be a SOA application. We can reuse, integrate other web services to have a new service, called web 37

52 38 Chapter 3. Testing Approaches for Web Services service composition. The latter is executed by interacting with other services, called partner services, to accomplish its workflow. The Quality Of these Services (QoS) effects the quality of the SOA application. Nowadays, the software testing community are interesting in proposing the test method for web services, including web service composition. Many testing methods for web services were analyzed in section 2.5, chapter 2. The following characteristics make the difference between a web service composition and other applications or systems: (i) the interactions are synchronous or asynchronous, (ii) the complexity of communicating message type, (iii) faults and events management, (iv) the data correlation between messages, (v) session management, etc. It is important to consider all these characteristics when testing this composition. A formal model of web service composition must exist with at least: (i) the time constraints to validate the synchronous or asynchronous interactions, (ii) set of variables to model the communicating messages, faults, events, (iii) a finite set of final state to indicate that a session finishes. The formal model of approaches or tools that were analyzed in sections or is not composed of at least three problems or the data type of variables does not allow us to model the complexity of communicating message. Moreover, most of the test generation methods that were applied on these models do not consider the problem of update data variable while generating test case. So that, these methods can not generate the exact test cases if the model exists the loops that are controlled by the variables. In general, web services-based testing is functional or black-box testing where test cases are designed based on the interface specification of the web services under test, not on its implementation. In this case, the internal logic does not need to be known, whether it s coded in Java, C# or BPEL. Problems such as the operation flow, the existence checking of operations, the validation of input/output message types, fault management, robustness, are usually considered, while most of the testing approaches for a web service orchestration focus on white-box testing or static analysis from a BPEL source code [59, 102, 110, 116, 149, 152, 156]. But the source code of a web service orchestration is not alway available for tester. In many case, we test a web service orchestration using only its specification and input/output messages that are generated by its implementation. But there is a difference between a web services-based testing and a web service orchestration testing. With orchestration testing, we not only have the input/output messages between tester and implementation under test (IUT) as web services-based testing, but also the communicating messages between IUT and its partners. We called this approach gray-box testing to distinguish it from black-box testing of web services-based. The approaches in [23, 93] focus on gray-box testing of a web service composition. But, [23] does not consider the time constraints while [93], the authors do not consider that the time occurrence of an output must belong to an allowed interval time. Only the time delay before sending an input and action delay of SUT are considered. 3.2 Formal model Many formal models have been proposed to model a web service orchestration, as analyzed in section 2.4. There are also some timed models that are proposed at LaBRI as ETIOA [24, 26] (Extended Timed Input/Output Automaton) to model a communicating system. TEFSM (i.e., Timed Extended Finite State Machine) is a formal specification that is

53 3.2. Formal model 39 proposed by Lallali et al [94] to model a web service orchestration described by BPEL language. This formalism is closely related to timed automata [6] and permits to carry out timing constraints, clocks, state invariants on clocks, property on transition and data variables. This formalism allows us to model the complete activities of BPEL as basic and structural activities, <faulthandler>, even though: time activities as <wait> and <onalarm>, data correlation. Specially, a set of rules that are defined to translated a BPEL process to TEFSM was introduced [93, 94]. In the context of WebMov project, we use this model as input for our approach Timed Extended Finite State Machine Definition 3.1. (TEFSM): A TEFSM M is a tuple, M = (S, s 0, F, V, D V V, E τ, C, Inv, T) where: S = {s 0, s 1,..., s n }, is a finite set of states; s 0 S is an initial state; F S is a set of final state; V is a finite set of data variables; D V V is the data variable domain of V; E τ is a finite set of events. E τ is partitioned into: Input event?a (E I ); Output event!b (E O ); τ is the internal event. C is a finite set of clocks including a global clock gc (never reset); Inv: S Φ(C) is a mapping that assigns a time invariant to states; T S E τ P (V ) Φ(C) 2 C µ S is a set of transitions where: The clocks P ( v)&φ( c): are guard conditions on data variables and clocks; µ( v): Data variable update function; X 2 C : Set of clocks to be reset; A clock is a variable that allows to record the passage of time. It can be set to a certain value and inspected at any moment to see how much time has passed. Clocks increase at the same rate, they are ranged over R +, and the only assignments allowed are clock resets R C in the form c:=0 (It is denoted by R 0). A set of clocks (c 0, c 1,..., c n ) will be denoted by c. Definition 3.2. (Timed constraint): For a set C of clocks, the set of timed constraints Φ(C) is defined on C by the grammar: Φ := Φ 1 Φ 2 Φ 1 Φ 2, Φ 1 := c m, Φ 2 := n c where c is a clock of C, (n, m) are two natural numbers, and {<,, >,, =}.

54 40 Chapter 3. Testing Approaches for Web Services The variables For a set V of variables, The data domain of each concrete variable v V can be the simple type (R: real, Z: integer, B: boolean, etc) or complex type (structure, interval, enumeration array, etc). An universal domain D V V is used to represent the abstract data types of V. A set of variables (v 0, v 1,..., v m ) will be denoted by v. Definition 3.3. (Variable constraint): For a set V of variables, the set of variable constraints P(V) that is defined on V is a recursive constraint expressions following the grammar: 1. all propositions that are defined on variables, true or false are a constraint expression; 2. v i d v is a constraint expression where v i V, d v D v is a value belonging to the domain D v, and {<,, >,, =, }; 3. if p and q are the constraint expressions, then p, p q, p q are also the constraint expressions; 4. all expressions are built by (1), (2) and (3); Definition 3.4. (Variables update function): The update of data variables is represented by the action v := µ( v). It is denoted by [ v := x]. Transitions Each transition is annotated that represents an edge from state s i to state s j with the set of guards, actions, data variable updates and clock resets. Definition 3.5. (Transition): A transition t = (s l s ) T is associated to a triple l =< e, cond, [ v := x; R 0] > where: cond = P ( v)&φ( c) is a guard over clocks and data variables; e E τ is an event; v := x is a set of data update function; R 0 is a set of clocks to be reset. The semantics of TEFSM The semantic of TEFSM is defined by labeled transition system (LTS). Two transition types are considered: delay transition and action transition. A delay transition does not change the state and the machine does not execute any action, but increments the current value of the clock. The priority of the delay transition is lowest, it means that if any other transitions going out from the same source state, these transitions will be executed first.

55 3.2. Formal model 41 An action transition indicates that it will be executed immediately (in this case, its priority is higher more than the priority of the delay transition) if an event (e E τ ) arrives and the condition is valid. Then the machine follows the transition by executing action a, changing the current values of the data variables by the action [ v := x], resetting the subset clocks R, and moving in the next state. Definition 3.6. (The semantics of TEFSM): For a TEFSM M = (S, s 0, F, V, D V V, E τ, C, Inv, T), the semantic of M is defined by a Labeled Transition System as following: Sem M = (L, l 0, Γ, ) where: L S R C + D V V is a set of semantic states (s, u, v) where: s is a state of the machine M; u is an assignment that satisfies one invariant of the state s (i.e., u Inv(s)); v is a data value represented by data variable valuation at state s. l 0 = (s 0, u 0, v 0 ) is the initial state; Γ = E τ {d d R + } is the labels set where d corresponds to the elapse of time; L Γ L is the transition relation defined by: action transition: Let (s, u, v) and (s, u, v ) be two states. Then (s, u, v) (s, u, v ) iff t = s <e,cond,[ v:= x;r 0]> s T such that u cond, u = u[r 0], u Inv(s ); v = v[ v := x]; a delay transition: Then (s, u, v) d (s, u d, v) iff 0 d d, u d Inv(s); Let M = (S, s 0, F, V, D V V, E τ, C, Inv, T), be a TEFSM, Sem M = (L, l 0, Γ, ) be a semantics of M, we have some following definitions: Definition 3.7. (Deterministic): M is deterministic (noted D-TEFSM) if l = (s, u, v), α E τ {d d R + }, (l α l l α l ) (l = l ) Definition 3.8. (Timed event): A timed event over E τ is a pair a/d such that a E τ and d R +. Let Σ = (E R + ) as the set of timed events and Σ τ = (E τ R + ) as the set of timed events including internal actions (i.e. E τ = E {τ}). Definition 3.9. (Timed sequence): A timed sequence over Σ τ σ = a 1 /d 1, a 2 /d 2,..., a n /d n is a number of timed observable events. Seq(Σ τ ) denotes the set of timed sequences.

56 42 Chapter 3. Testing Approaches for Web Services Definition (Run): A run r of machine M over σ is a finite sequence of the form (s 0, u 0, v 0 ) a 1/d 1 =...(s n 1, u n 1, v n 1 ) an/dn = (s n, u n, v n ). This execution is noted by Run σ = (s 0, u 0, v 0 ) σ (s n, u n, v n ) and Run(M) denotes a set of run of M. Let Σ Σ τ and σ Seq(Σ τ ) be a timed sequence. π Σ (σ) denotes the projection of σ to Σ obtained by deleting from σ all actions not present in Σ. Definition (Timed traces): The observable timed traces of M are defined by: T races(m) = {π Σ (σ) σ Seq(Σ τ ) (s 0, u 0, v 0 ) σ (s n, u n, v n ). Definition (Reachability): A state s is reachable in M if and only if (s 0, u 0, v 0 ) σ (s n, u n, v n ) Run(M) such that s = s n. Definition (The correct TEFSM): M is called a correct machine if and only if s S M, s is reachable. Figure 3.1: An example of TEFSM Example 3.1. The figure 3.1 illustrates a simple example of TEFSM where s 0 is the initial state, s 5 is the final state, t is a local clock and: S = {s 0, s 1, s 2, s 3, s 4, s 5 }, the time invariant of s 3 is t <= 5 and the rest of states are undef ined;

57 3.3. Unit testing 43 V = {x, y, z}; D V V = {int, int, int}; E = {?a,!b,!c,?d,?e,!f}; T = {t 0, t 1, t 2, t 3, t 4, t 5, t 6, t 7 } where ( the symbol _ denotes the internal event): t 0 = (s 0, <?a, [], {x := a; t := 0} >, s 1 ); t 1 = (s 1, < _, [x > 10], {} >, s 0 ); t 2 = (s 1, <!c, [5 x 10], {} >, s 2 ); t 3 = (s 1, <!b, [x < 5], {y := x + 2; t := 0} >, s 3 ); t 4 = (s 2, <?d, [], {z := x + d} >, s 4 ); t 5 = (s 3, <?e, [t < 5], {} >, s 4 ); t 6 = (s 3, < _, [t == 5], {} >, s 5 ); t 7 = (s 4, <!f, [], {z := z + y} >, s 5 ); 3.3 Unit testing Unit testing is used to find bugs on a single web service based or an orchestration. In the case of an orchestration, it means that we test only the main process without interaction with its real partners to guarantee the correct input condition. Conformance testing establishes that the implementation respects either its specification or not. It is a type of functional testing of black-box (in the case of web service based) or gray-box (in the case of the orchestration) where the internal behaviour is not available. The conformance of SUT is tested using only messages communicating between SUT and its environment. We know that testing conceptually consists of three activities: test case generation, test execution and verdict assignment (or test analysis). Depending on the test approach, these activities can be applied in parallel (called online approach) or in sequential (called offline approach). Both will be discussed in this section. Before discussing them, we introduce the test architecture and the conformance relation which we use for our approaches Test architectures A web services orchestration is a process that reuses and integrates existing web services. It cannot run without its partners (i.e., its environment), even though we want to test only it. So we must simulate all its partners when we want to test this process. A test architecture describes how a service under test (SUT) communicates with its environment and what information is available for tester? Test architectures for unit testing of web service orchestration that were proposed by Li et al [102] are composed of centralized architecture and distributed architecture. In our works, we used a centralized test architecture that was composed of many tester processes and one controller process to co-ordinate these testers. In these testers, there is one that plays the role of the client (or the service consumer), others play the role of simulating the partner services. In this architecture, the roles of the controller are: generating the test cases (the input event and its data), co-ordinating the tester to send the inputs to SUT and to receive the outputs from tester, verifying the outputs and giving

58 44 Chapter 3. Testing Approaches for Web Services the verdict. The role of each tester is very simple, sending a message to SUT when it receive a request from the controller and forwarding the response of SUT to the controller. This architecture is also used to test a web service based (there are no the interactions between SUT and its partners or we do not know them) by existing only one tester that represents the client. From here, a SUT means a web service orchestration and gray-box is called instead of both. The figure 3.2 shows an abstract model of SUT with two partners and one client (top), and the correspondent test architecture (bottom). Figure 3.2: An abstract model of SUT and its test architecture Conformance relation Conformance here defines a relation between an implementation model and its specification model. In the context of gray-box testing of a timed system, the term, the timed observable traces are used instead of the implementation model and the conformance relation defined by this term. In our conformance testing approach of web service orchestration modeling by a D-TEFSM, we propose a timed conformance relation for D-TEFSM that is denoted by xtioco (extended timed input output conformance relation). This notation is extended by

59 3.3. Unit testing 45 ioco [136] with data variables and time invariant on state. In order to define conformance relation, we define a number of operators. Let M = (S, s 0, F, V, D V V, E τ, C, Inv, T), be a D-TEFSM, Sem M = (L, l 0, Γ, ) be a semantics of M and σ Run(M), M after σ is a state s that is reachable in M. Formally: M after σ = {(s, u, v) L SemM σ Seq(Σ τ ) such that (s 0, u 0, v 0 ) σ (s, u, v)} Let l = (s, u, v) L SemM, out(l) is a set of all observable "output events" that can occur when system is at state s. Formally: out((s, u, v)) = {a E O s a } The extended timed input output conformance relation, denoted xtioco, is defined by the following definition. Definition (xtioco): Let M S be a D-TEFSM that is a specification model of system and M I be its implementation: M I xtioco M S σ T races(m S ) such that out(m I after σ) out(m S after σ) and inv(m I after σ) inv(m S after σ) Figure 3.3: An example of xtioco relation of D-TEFSMs Example 3.2. The figure 3.3 shows an example of xtioco and non-xtioco between two implementations I 1, I 2 and its specification S. It is easy to recognize that I 1 does not conform to S following xtioco because: out(i 1 after {(?a, a = 6, 0) (!c, c = 1, 4) (?d, d = 10, 2)}) = {!f} out(s after {(?a, a = 6, 0) (!c, c = 1, 4) (?d, d = 10, 2)}) = {!g} While I 2 conforms to S following xtioco because the difference is allowed, v is the data variables where v a 5: out(i 2 after {(?a, v, 0) (!c, v, 4) (x = 3, v, 3)}) = {!e} = out(s after {(?a, v, 0) (!c, v, 4) (x = 3, v, 3)}) = {!e} and {1 < x 2} {x 2}

60 46 Chapter 3. Testing Approaches for Web Services Offline approach Offline testing is the approach which test activities (test case generation, test execution and verdict assignment) are applied in sequential. Firstly, the abstract test cases are generated by a tool. Next, the concrete test cases are derived by generating the inputs data of the abstract test cases using its data type. Finally, these concrete test cases are executed using a test executor by sending the inputs, collecting the outputs and analyzing it to produce a verdict. Because TEFSM is a timed extended model with a set of data variables and time constraints, there are some problems that we must consider when we test: control flow, data flow and time constraints between inputs and outputs. In this section, we discuss what is a test case for TEFSM (called timed test case) and how to generate the timed test case from a TEFSM model. Finally, how these timed test cases are executed against the SUT to check the conformance of an implementation with its specification. Figure 3.4 resumes our testing approach for web services. Figure 3.4: Offline testing approach for web services

61 3.3. Unit testing 47 Timed test case A test (or test case) is an experiment performed on the implementation by a tester. There are different types of tests, depending on the specification of system under test. Here, we focus on tests that can measure precisely the delay between two observed actions, and can emit exactly an output at any point in time after sending one or several inputs. A (abstract) timed test case of TEFSM must indicate if we send an input with some conditions on data fields or do a synchronous time delay then which possible outputs that we will receive after a duration satisfying a set of time constraints. Executing a test case allows us to produce one of three verdicts: pass, f ail or inconclusive. Because some outputs may be generated after sending an input, so we need to indicate which one is the expected output, it means what is our test purpose. We call a sequence of transitions on the path from the initial state to final state of TEFSM be a test purpose. Our abstract timed test case will be generated based on this test purpose. Definition (Test purpose): Let M be a TEFSM, a test purpose of M, say Tp is deterministic TEFSM, over E I E O, with a distinguished non empty of states. This set of states is denoted by Accept(Tp), Accept(T p) S(T p) LastAction(T p) LastAction(M). Figure 3.5: Test purpose example (t is a clock) Definition (Abstract timed test case): An abstract timed test case is a tuple T C = (S, I, O, D, T r, s 0, S I, S O, S D, S U, S P, C, Inv) where S is the set of finite states, I, O and D are disjoint sets of input, output actions and time delays, is respectively, T r S I O D P (I) Φ(C) 2 C S is the transition relation, s0 S is the initial state, and the sets S I, S O, S D, S U, S P S are a partition of S. C is a finite set of clocks. The transition relation and the sets of states fulfill the following conditions: S I is a set of input states, s 0 S I and s S I,!t = (s a s ) T r such that a I. Moreover, d D and s S such that s d s T r. S O is a set of output states, and s S O, o O,!s such that s o s T r. Moreover, i I, d D and s S such that s i s T r or s d s T r. S D is a set of time delay states, and s S D,!t = (s d s ) T r such that d D and s / S D. Moreover, i I and s S such that s i s T r. S P S U are two sets of pass states and inconclusive states that are terminal. Besides, s S P S U, a I O, d D and s S such that s a s T r or s d s T r. Inv: S O Φ(C) is a mapping that assigns a time invariant to output states. P (I) Φ(C) are the constraints on the data value and on the clocks of the input actions. X 2 C are set of clocks to be reset.

62 48 Chapter 3. Testing Approaches for Web Services Example 3.3. The figure 3.6 shows two abstract timed test cases whose the semantics of the case on the left are: if we send an input i 1 to SUT, then the expected output is o 1. If the output is o 1, there is no conclusion (inconclusive verdict), if the ouput is anything (i.e., not o 1 or o 1 ) a fail verdict is produced. Next, if o 1 arrives, we continue to send another input i 2 and at this moment the clock t is reset (t ). Finally, if we receive the output o 2 within a duration less than or equal to 5, a pass verdict is assigned. If anything else is received, this test case will return a fail verdict. Figure 3.6: Examples of abstract timed test cases Definition (Timed test case): A timed test case is an abstract timed test case where each input action, its data values and its time delay are fixed. Test case generation Many test case generation methods and tools were introduced in sections and Each approach focused on different models and used the different input conditions depending on the test purpose to generate the different types of test case or timed test case. A test purpose that is defined by testers describes a special function of SUT. It is used effectively to derive a test case in particular for the large models or test in context. Using the test purposes sometimes allows us to limit the problem of state explosion because we only consider one part of the model. It demands that the testers must indicate which cases they want to test. But it is not easy to know how many cases where we must test to conclude that an IUT conforms with its specification. In the case of web service orchestration, using the sequence of input/output actions and the constraints of a TEFSM s path as a test purpose allows us to obtain all possible abstract timed test cases over a TEFSM and, after executing this set of test cases, we can affirm that an implementation under test (IUT) either conforms with its specification or not. We introduce here our method to generate the timed test cases. It is composed of three steps: 1. Building a coverage tree from TEFSM model to obtain a set of possible paths (each path is started from the initial state to a final state). Then, we pick the input/output

63 3.3. Unit testing 49 transitions (i.e., the visible actions), including its constraints on the data variables and on the clocks, of each path to build a test purpose. 2. These test purposes are applied against the specification to generate the abstract timed test cases. It means that we verify the reachability of each state on the path and add the inconclusive states. 3. Finally, using the abstract timed test case and WSDLs file, the concrete timed test case is derived. Step 1: generating all possible paths: We propose to use the path coverage criteria to cover all possible paths, where each path is started from the initial state to a final state, through TEFSM. The paths satisfying this coverage can be constructed in terms of simple breadth or depth first search over TEFSM. In [91], a coverage tree is built by using all state coverage or all transition coverage. The paths obtained in this case sometimes do not satisfy the condition: each path is started from the initial state to a final state. Algorithm 1 shows a method for global path coverage by traversing TEFSM in the breadth first order. The algorithm will stop when all leaf nodes of the tree belong to a set of final state of TEFSM. To do this, for each current state (noted s), if s is not visited, we continue to build the tree from the descendant states of s. On the contrary, if s has been visited and s is not a final state, the tree continues to be built by doubling the rest of TEFSM from s (see else condition in the algorithm). Moreover, the loops can exist in a TEFSM followed by a defined condition and the internal variables are usually used to control these loops. To guarantee the algorithm finishes, we must update the data value of these variables to the condition becomes true after the n step. Finally, for each path, we only pick the input/output actions and these actions will be decorated by the constraints from the conditions on this path (guard condition on transition) to have a test purpose.

64 50 Chapter 3. Testing Approaches for Web Services Algorithm 1: Make the coverage tree from TEFSM Input : aut = (S, s 0, F, V, D V V, E τ, C, Inv, T) TEFSM specification Output: A tree queue {s 0 }; tree ; while queue do s queue.pop(); tlist getouttransition(s) a //{t=(s <e,cond,[ v:= x;r 0]> s ) T cond false}; foreach t =(s <e,cond,[ v:= x;r 0]> s ) in tlist do update V by data update function of t; if s is not visited then tree.add(t); queue.push(s ); mark s in visit; else //double the rest part of TEFSM from s, it means that a new set of transitions are added into T ; create a new state nstate; update target state of t by nstate; tree.add(t); queue.push(nstate); mark nstate in visit; tlist getouttransition(s ); foreach t in tlist do t copy of t //create a new transition; update source state of t by nstate //modify the reference from s to nstate; add t into T ; return tree; a At each moment, the condition of transition has three values: true, false and undefined when the condition depends on input parameters. The getoutt ransition(s) function will return a set of transitions that have source state be s and condition evaluation be true or undefined. Step 2: generating the abstract timed test case: We present here our approach to generate the timed test case from a TEFSM using a test purpose. Our approach firstly computes the synchronous product between a TEFSM and a test purpose. While computing the synchronous product, we also consider the feasible paths (based on constraints on data variables and on clocks) on this product to cut the unsatisfying path. Secondly, the test case is generated by selecting a trace leading to the Accept state and with each output transition (noted t) on this trace, if other output transitions exist, that go out from the same state with t, and the intersection between the constraints of t with the constraints of these transitions is not empty, the inconslusive states will be assigned to the target state of these transitions.

65 3.3. Unit testing 51 Castanet et al [44] defined a synchronous product as following: Definition (Synchronous product): Let M and Tp be a TEFSM and a test purpose. The synchronous product of M and Tp is a TEFSM SP defined as follows. C SP = C M C T p ; V SP = V M ; E SP E M ; S SP S M S T p ; S SP, T SP and Inv SP are the smallest relations defined by the following rules: Rule R 0 : Rule R 1 : s SP 0 = (s M 0, s T p 0 ) SM S T p Inv(s SP 0 ) = Inv(s M 0 ) Inv(s T p 0 ) (s 1, s 2 ) S SP s 1 <e 1,cond 1,u 1 > s 1 T M s 2 <e 2,cond 2,u 2 > s 2 / T T p (s 1, s 2 ) S SP (s 1, s 2 ) <e 1,cond 1,u 1 > (s 1, s 2 ) T SP Inv(s 1, s 2 ) = Inv(s 1) Rule R 2 : (s 1, s 2 ) S SP s 1 <e,cond 1,u 1 > s 1 T M s 2 <e,cond 2,u 2 > s 2 T T p (s 1, s 2) S SP (s 1, s 2 ) <e,cond 1 cond 1,u 1 u 2 > (s 1, s 2) T SP Inv(s 1, s 2) = Inv(s 1) Inv(s 2) When we test a SUT, a global clock alway exists that is real time. So, we can add a global clock that is not reset while we execute a test case. The initial value of this clock at the start moment of the test case is zero. Next, we need to map all local clocks to this global clock to compute an out reach time interval for each transition on real time. Let s first define an operator on intervals such that: if I = [a, b] and J = [c, d] are intervals, then the sum of I and J, noted I J is defined as [a+c,b+d]. Berrada et al [25] define the out reach time interval over a global clock h as follows: Definition (Out reach time interval) Given a TEFSM M and a path ρ = t 1, t 2,..., t n of M from the initial state. The out reach interval is the time interval outside of which the transition must happen according to the global clock, noted h, in order to be in agreement with the ρ time constraints. If a transition happens in this interval, we know that at least one of the constraints of ρ is violated. Formally we define: h(t 0 ) = [0, 0] h(t i ) = x C(ti )h(t kx ) SC(t i, x) where x is last reset in t kx (k x < i), h(t i ) the complementary of h(t i ) is noted h(t i ), and SC(t i, x) the constraints over x.

66 52 Chapter 3. Testing Approaches for Web Services For the computation of this interval of a transition t, for every clock x of t, we look for a transition t where x is last reset. If we find such t, we know that x is reset during h(t ). Then, we add 1 time constraint of x to h(t ). Finally, the complementary of the intersection of all these intervals is the wanted interval. Figure 3.7: Out reach time intervals Example 3.4. Let ρ be the path as the figure 3.7, with c 1 and c 2 are the local clocks. Firstly we add a new transition t 0 in which all clock are reset: h(t 0 ) = [0, 0]. For t 1, the clocks c 1, c 2 are reset in t 0 then: h(t 1 ) = h(t 0 ) [0, 2] = [0, 2]. For t 2, the clocks c 2 is reset in t 1 then: h(t 2 ) = h(t 1 ) [3, 5] = [3, 7]. For t 3, the clocks c 1 is reset in t 2 then: h(t 3 ) = h(t 2 ) [0, 3] = [3, 10]. For t 4, the clocks c 1 is reset in t 3 and c 2 in t 1 then:. h(t 2 ) = (h(t 3 ) [0, 3]) (h(t 1 ) [7, 10]) = [7, 12] Because the behaviour of TEFSM depends on the constraints of time and data variables. So we continue to consider the constraints on data variables here. Similarly with the clock, we also calculate the out reach data domain of each variable 2 at each transition that it appears. Definition (Out reach data domain) Given a TEFSM M and a path ρ = t 1, t 2,..., t n of M from the initial state. The out reach data domain of variable v at transition t i, i [1, n] is the intersection of the out reach data domain of variable v at the previous transition (noted 1 using 2 We only consider the variables that depend on the input parameters

67 3.3. Unit testing 53 t j, j < i) where the variable v appears and the intersection of the constraints on t i. Formally we define: D v (t i ) = D v (t j ) ( CS v (t i )) where CS v (t i ) is a constraint of variable v on transition t i and D v (t 0 ) is defined in the TEFSM. Definition (Feasible path) Let ρ = t 1, t 2,..., t n is a path. ρ is called a feasible path iff t i, i [1, n]: 1. h(t i ) is not empty. 2. v CS(t i ), D v (t i ) is not empty. Figure 3.8: Synchronous product and abstract test case selection Example 3.5. The figure 3.8 presents a synchronous product of TEFSM and a test purpose where we applied the feasible path technique with variable v 1 to cut the unsatisfying path at state (4,2). We stop at state (2,2) because the input d is define one time in the test purpose and it has appeared on the current path. The abstract test case is the path from initial state (0,0) to accept state (6,3). While we select this path, at state (1,1), there are two output

68 54 Chapter 3. Testing Approaches for Web Services actions that may be produced after giving the input a. But the intersection between the constraints of b and c is an empty set, so the inconclusive state is not added after output action c. Like the introduction in section of chapter 2, TGSE generates the timed test cases by simulating a Communicating System Under Test (CSUT) that is modeled by one test purpose and some ETIOAs [27, 26] (i.e., Extended Timed Input/Output Automaton). Moreover, there is no supported tool for our method, so we used the TGSE as a supported tool for our second step. But this tool does not allow us to add the inconclusive states into the test case, it verifies only the reachability of each state on the test purpose. Before discussing how to generate a test case by using the TGSE tool, we present the following definitions. Definition (ETIOA): An Extended Timed Input/Output Automaton (ETIOA) is a 10-tuple M = (S, L, C, P, V, V 0, P red, Ass, s 0, ) where: S is a finite set of states. s 0 is the initial state. L is a finite alphabet of actions. C is a finite set of clocks. P is a finite set of parameters. V is a finite set of variables. V 0 is a finite set of the initial values for variables of V. Pred = Φ(C,P,V) P [P,V], where P [P,V] is a set of linear in equalities on V and P. Ass = x :=0 x C v := f(p,v ) v V is a set of updates on clocks and variables. S L P red Ass S is a set of transitions. Clocks and Constraints of ETIOA: For a set C of clocks, a set P of parameters, and a set V of variables, the set of clock constraints Φ(C, P, V ) is defined by the grammar: Φ := Φ 1 Φ 2 Φ 1 Φ 2, Φ 1 := x f(p, V ), Φ 2 := f(p, V ) x where x is a clock of C, and f(p, V ) is a linear expression of P and V. Definition (CSUT): A Communicating System Under Test (CSUT) is a 5-tuple CS = (SP, SV, R, M i,1 i n, TP) where: SP is a finite set of shared parameters; SV is a finite set of shared variables; M i,1 i n is a ETIOA ; TP is a ETIOA representing the test purpose;

69 3.3. Unit testing 55 R is a finite set of synchronization rules where each rule r is a vector of n+1 elements; A CSUT declares a set of shared resources (parameters and variables), a set of ETIOAs, a set of rules describing the different possible synchronizations between the entities, and a test purpose modeled by an ETIOA. The figure 3.9 shows an example of CSUT. Figure 3.9: A CSUT model In order to generate test cases for web services using the TGSE tool, firstly the test purpose must be defined and model its by an ETIOA (the ETIOA of test purpose does not include a set of finite parameters). Secondly, transforming the TEFSM specification into the ETIOA by deleting the set of time invariants. For each variable that is updated by data value of input event and is not shared with variables of test purpose, a correspondent parameter will be declared. Thirdly, we build a CSUT from the ETIOA specification of web service under test and the ETIOA representing test purpose by declaring a set of synchronization rules, shared variables and shared parameters. Finally, using TGSE to simulate this CSUT, a test case in the XML format satisfying the test purpose will be returned if it is found. Else, there is nothing. The values of parameters are randomly generated and saved in the OutputLp.out file if they are declared. These values are used as the conditions of input events when we generate its data value on the next step. An experience on a case study is showed in the appendix part (see A.3) as the tutorial of this tool. Step 3: deriving the concrete test cases: after abstract timed test cases are generated, the data and time delay are generated for each input message to build the concrete test cases. The test data generation based on data type (XML Schema) analysis of input message is defined in WSDL file of web services. The XML schema syntax defines two message types: simple type and complex type. Complex data type defines a composition of simple and/or complex data types. It defines the "sequence", "choice", and "all" relationship among member data types. Different types can be combined to aggregated data types. Here, we discuss how

70 56 Chapter 3. Testing Approaches for Web Services to generate the data of the input messages using its data type and its set of constraints. With each simple type, we have a set of default configurations, for example type "int" is declared belong to [min, max] or default values belong to an enumeration {1,3,5,8,10}. The basic algorithm to generate the data is as follows: 1. Simple type: randomly generate a value corresponding its data type definition, using either default configuration or constraints on the abstract test case. 2. Complex type: the generator recursively analyzes the structure of the data type until it reaches the simple type: (a) Analyze the hierarchical tree structure of a complex data type. (b) Traverse the tree, and at each tree node: If it is a simple data type, perform simple data type analysis. If it is a "sequence" structure node, generate a set of test cases with each test case corresponding to an ordered combination of the child nodes. If it is a "choice" structure node, generate a set of test cases with each test case corresponding to one child node. If it is an "all" structure node, generate a set of test cases with each test case corresponding to a random combination of the child nodes. The time delay for the input messages is randomly generated from a region of constraint on clock. If there is no time constraints, the message will be sent immediately. Figure 3.10: Data generation example Example 3.6. The figure 3.10 shows an example of automatic data generation for the reserverequest message that has data type on the left side. The cusname field is picked from an enumeration and other fields are random. This message is generated using the SOAP document binding type.

71 3.3. Unit testing 57 Test execution We present in this section an algorithm to execute a timed test case for web services using a centralized test architecture (section 3.3.1). This algorithm will be executed by the controller and uses xtioco to produce the verdict. The functions of send_to_su T are: indicate an event belonging to which tester and sends it to the correspondent tester. When the tester receives an output from SUT, it sets the latter into a global queue. The function receive in this algorithm will either return the first message of global queue or raises a timeout if this queue is empty after a duration. A supported tool, named BPELUnit, that can use to execute a timed test cases is introduced in section A.4. But this tool does not allow us to synchronize between partner services. Algorithm 2: Test execution Require: max: is synchronous request timeout; Input : T C: a timed test case Output : pass/fail or inconclusive state s T 0 C ; while state / SP T C do switch state belong to do case SI T C tran {t = (s... s ) T r T C state = s}; //delay a duration before sending to SUT if time delay is indicated (d>0); delay d=time(tran) units of time; send_to_sut(tran.action()); case SD T C tran {t = (s... s ) T r T C state = s}; delay d=time(tran) units of time; case SO T C try d (invariant(state) = undef ined)? max : invariant(state); om sg receive(d); tran {t = (s... s ) T r T C state = s omsg = t.action()}; if tran = then return fail; else if tran.target() SU T C then return inconclusive; catch timeout after d units of time return fail; state tran.target(); return pass;

72 58 Chapter 3. Testing Approaches for Web Services Example 3.7. The figure 3.11 shows an example of test execution scenario for xloan service. The test architecture has one controller and three tester (a client and two partners). A timed test case t = {?request!approvereq?approveres!response delay = 60s!cancelReq} where?request and!response are request/response message from client,!approvereq and?approveres are request/response from SUT to bank service and!cancelreq request message from SUT to bank service. The test execution scenario of this test case is executed as follows: Firstly, the controller sends an input?request to Tester1 (client), then it receives a request!approvereq from Tester2 (bank service) after 2 units of time. The controller continues to send a response?approveres to Tester2 and receives a response!response from Tester1 (client) after 3 units of time. Afterwards, it delays 60 seconds to wait a request from Tester2 (bank service). When the request!cancelreq arrives, the verdict pass is produced for this test case and the test process finishes. In this test case, the Tester3 (assessment service) is not used. Figure 3.11: An example of test execution scenario Online approach The problem of the approach that we introduced in section is the explosion of state because we must pass all states of TEFSM while generating a set of test cases. Online testing is an approach in which test case generation and test execution are run in parallel. This approach solves the problem of offline approach by randomly selecting the test case. But this approach does not guarantee that all abstract test cases are found. From an initial state, only a single test primitive (input event) is generated from the model at a time which is then immediately executed on the system under test (SUT). Then the output produced by the SUT, as well as its time of occurrence, is checked against the specification. At a time, a new test primitive is produced based on values of previous events or random selection, if there is some acceptable options and so on forth until we arrive a final state. Some tools were developed from this approach as Jambition, T-Uppaal, TorX.

73 3.3. Unit testing 59 Online testing algorithm In this section, we proposed an online testing algorithm for web services based on TEFSM model. This algorithm is executed by the controller in the test architecture (Fig. 3.2) to generate timed test case, execute this timed test case and assign the verdict using the conformance relation xtioco (section 3.3.2). The main idea of this algorithm is: from a current state (at the start of algorithm, the current state is the initial state), a list of following actions is found based on the current data of variables. In the case that this list is empty, it means that we have arrived at a final state, a trace was found and there is no fault. We reset the current state to be the initial state and continue to execute another trace. On the other hand, this list is composed of either the input actions (noted I), delay action (noted T ) or the output actions (noted O). If the list of input actions and delay actions is not empty (I T ), we randomly choose an action (noted t I T ) from this list 3. If t I, we generate randomly the data for this action using the algorithm in the section 3.3.3, send it to SUT, update data variables and add this action to current trace. Else (t T ), we make a delay to synchronize the time with SUT, and add this time delay to the current trace. In the case of I T = (it means that O ), we wait for a duration of time to receive the outputs from SUT and check it against the specification. If there is no output after a duration of time, a timeout fault is assigned to the current trace. Finally, after each action (input, output or delay), the current state is updated by the target state of the correspondent transition of action. The algorithm will be finished when it generates a number of trace (input parameter) or finds a fault. The following figure 3.12 shows an illustration of this approach. 3 In our model, we consider a timestamps transition as an action that is controlled by tester, so at the any current state, (I T ) O =

74 60 Chapter 3. Testing Approaches for Web Services Figure 3.12: Online testing approach illustration

75 3.3. Unit testing 61 Algorithm 3: Online testing algorithm Require: max: is synchronous request timeout; Input : aut: TEFSM specification, nb is the trace number. Output : List of traces and corresponding verdict state s 0 ; counter 0; tracelist //traces list; trace //the current trace; queue //the queue to store the outcoming messages of SUT; while counter < nb do nactions getnextaction(state) //see algorithm 4; //arrive a final state; if nactions = then if trace then tracelist.add(trace) //the pass verdict is assigned to current trace; trace ; state s 0 ; counter + +; else inputlist getinputtrans(nactions); timelist getelapsetrans(nactions); if (itlist inputlist timelist) then randomly choose t itlist; if t inputlist then im sg generate_data(t.action); send_to_sut(imsg); update_variable(imsg); trace.add(imsg); else delay d time(t) times units; trace.add(delay=d); state t.target // move to new state; else d (inv(state) = undefined)? max : inv(state); sleep for d time units or wake up if queue is not empty at d d; if (omsg queue.pop()) is not null then if verify(omsg)=true then update_variable(omsg); trace.add(omsg, d ) //add output and its time delay into trace; state t.target //move to new state; else exit() //receive the unexpected output (fail); else exit() //time constraint does not satisfy (fail); return tracelist;

76 62 Chapter 3. Testing Approaches for Web Services Algorithm 4: Get outgoing transitions from the current state function getnextaction(state) begin result ; // transitions list; queue {t=(s <e,cond,[ v:= x;r 0]> s ) T s = state cond = true}; while queue do trans queue.pop(); if trans is an input/output transition or a timestamps transition then result.add(trans); else update the variables by data update function of trans; temp {t=(s <e,cond,[ v:= x;r 0]> s ) T s = trans.target cond = true}; queue.push(temp); return rerult; 3.4 Integrated testing A composite of web services is a system integrated at runtime with its results depending on its partners or the condition of the real environment. Unit testing on each service is not enough to access about quality of a composite while the works on integrated testing either are not many or focus on WSDL specification. Integrated testing can be performed to verify timing constraints on transmission messages between services under test and its partners on the real environment, this service is either ready to use or not, etc. In general, we can use passive testing by verifying an execution trace of a web service orchestration. But we know that the passive testing is a method which tester does not interact directly to service under test (SUT). This makes us spend a long time to collect the execution trace because we do not know when a session is started and when it finishes. To improve this problem, we can interact directly with SUT by sending the input requests to start a new session (active), then collecting the trace to analyze (passive) Methodology The figure 3.13 shows an overview of our proposed method. It is composed of three steps: first of all, the orchestration specifications (for example: UML, BPMN, BPEL, etc.) must be translated into a formal model, (i.e., TEFSM). This model will be used to check the correctness of a trace that is generated by step 2. In step 2, a SOAP request will be automatically generated and sent to SUT to enable a new session. The content of this message will be randomly generated based on XML schema of message type, ( see the derivation of concrete test cases part of section for more detail). Afterwards, all communicated messages between SUT and its partners (including tester) will be immediately collected by installing a probe to build an execution trace. In the figure 3.13, the notion Point of Control and Observation (PCO), and Point of Observation (PO) are introduced as a probe. The time occurrence of each message is also saved. Finally, this trace is verified by a checking engine to produce the verdict (step 3).

77 3.4. Integrated testing 63 Figure 3.13: An overview of integrated testing approach Checking algorithm This section presents an algorithm to check the correctness of an execution trace with its specification (i.e., a TEFSM). This algorithm checks message by message and produces the verdict for each message. We know that each session has a correspondent specification. In the case of multi-sessions are executed in parallel, we must check on multi-specifications. At each specification, its behaviour depends on data variables. To check a message conform to its specification, we need a current state, the time occurrence of last message and a list of variables with current values. To start a new session, the current state is the initial state of TEFSM, the time occurrence of the last message is undef ined and data variables receive the initial values. We call each trio: current state, the time occurrence of last message and list of variables is a ConfigState C = (s c, t c, V c ). To verify multi-session, we will have a global set of configuration SC = C 1, C 2,..., C n where each C i is correspondence with a session. Its values will be updated whenever a message arrives. The algorithm 5 receives a single message including time occurrence, verifies it and produces a verdict true/f alse. This allows us to run the checking engine in parallel with trace collection engine because the algorithm checks the correctness of the message without needing the correlation between the messages at any moment.

78 64 Chapter 3. Testing Approaches for Web Services Before introducing the detail of algorithm, we discuss some necessary functions that are used by the algorithm: getnextevent(configstate C = (s c, t c, V c )): this function returns either a set of next input/output transitions of current state s c based on current values of V c or null if we arrive a final state. An input/output transition is a transition which its event belongs to E I E O. While we query the next input/output transitions, the internal variables may be updated by the data update function of transition. getm essagen ame(string msgcontent): this function returns message name from its content by mapping the structure of message with XML schema that is defined in WSDLs. update_variable(v ector V, String msgn ame, String msgcontent): this function will find a variable which its type is msgname from the list of variables and updates its value by msgcontent.

79 3.4. Integrated testing 65 Algorithm 5: Checking algorithm from TEFSM Require: CS: is the global list of StateConfig aut: is a TEFSM specification Input : message msg, arrival time at. Output : true/f alse //get message name from its content; msgn ame getmessagename(msg); foreach C (s, t, V ) in CS do //get next input/output action from current state; translist getnextevent(c); //check: we arrive either a final state or not; if translist = null then remove C from CS //a session finishes; else foreach trans in translist do if trans.eventname() = msgname at-t satisfies guard condition on clock of trans then updatevariable(v, msgname, msg); update current state of C by trans.gettarget(); update time occurrence of last message by at; return true; //check a new session; C (aut.getinitialstate(), undef ined, aut.getvariables()); translist getnextevent(c); foreach trans in translist do //a new session is enabled; if trans.eventname() = msgname then updatevariable(v, msgname, msg); update current state of C by trans.gettarget(); update time occurrence of last message by at; add C into CS; return true; return f alse;

80 66 Chapter 3. Testing Approaches for Web Services 3.5 Conclusion We have presented in this chapter some testing approaches for web service orchestration. We have proposed an unit testing framework that composes of a centralized test architecture, a conformance relation and two testing methods: offline and online. Our testing approach composes of five steps: (i) modelling the web service orchestration by TEFSM, (ii) generating test purpose, (iii) generating the abstract timed test case, (iv) deriving the concrete timed test case, (v) executing the timed test case against implementation under test to produce the verdict. For offline method, these activities are applied in sequential while online method, they are applied in parallel. For offline approach: we presented in this chapter a method to generate the abstract timed test case by computing the synchronous product (SP) between a TEFSM and a test purpose. While computing the SP, the feasible path technique is applied to cut the unsatisfying path of SP. Finally, the abstract timed test case is selected by tracing a path to accept state and decorating it by the constraints on the path. For online approach, the input and its data are randomly selected from a current state and randomly generated to send to SUT. Afterwards, we wait for the output and verify its correctness using the input data and the sending time moment of input. If the output is correct, we continue with other input until arriving at a final state, otherwise we stop and a fail verdict is produced. An integrated testing approach is also proposed to test an orchestration with its real partners. For this approach, the data of request from client is randomly generated and sends to SUT to enabled a new session. Then all input/output messages of SUT is collected to build a execution trace. Finally an algorithm is proposed to check this trace. In the next chapter, we will propose a new approach to check the correctness of a sequence of messages. It composes of the rule syntax, a trace collection architecture and an checking algorithm that checks message-by-message without storing them.

81 Chapter 4 Runtime Verification Contents 4.1 Introduction Methodology Test architecture The rules Checking algorithm Conclusion This chapter presents a new methodology to perform passive testing of behavioural conformance of a system based on a set of constraints of input/output messages (noted rules). Firstly, we define the passive testing architectures for web services. Secondly, the Nomad language is extended to define the rules. Finally, an algorithm is proposed to verify the correctness of service under test using the execution trace. The proposed algorithm can be used either to check a trace (offline checking) or to runtime verification (online checking) with timing constraints, including future and past time. The problem of data correlation between messages is also considered. The works of this chapter have been published in the ICWS 2010 [43]. 4.1 Introduction The activity of conformance testing is focused on verifying the conformity of a given implementation to its specification. In most cases testing is based on the ability of a tester that interacts directly with the implementation under test and checks the correction of the answers provided by the implementation (called: active testing). Several active testing methods for web services are proposed in the chapter 3. However, we cannot apply this method to test a running system, in many cases. For example, if we use the active method to test the function create_new_account of a bank service, this will make a mistake in the database of the service. With a composite web service, we can only use the method of active testing for unit testing by simulating its partners to guarantee that does not effect to its partners while testing. But the composite web service is a system that is integrated at runtime and its result depends on its partners or the real environment. In this case, the passive testing method is 67

82 68 Chapter 4. Runtime Verification used to verify the result of partner services or the interval time between a message request and a message response. Passive testing is a method that collects the observable traces of the system by installing a probe and analyzes it to produce a verdict. This method does not effect running system. There are two approaches of passive testing, online and of f line. The online approach immediately checks an execution trace whenever an input/output event occurs. The advantages of this approach are: the faults may be found immediately and, we can stop the system to avoid the damage. On the contrary, the offline approach checks an execution trace after it is collected for a period of time, meaning the error is not found immediately if it occurs. This approach does not require added resources such as CPU, RAM or another computer to run the trace collection engine and checking engine in parallel. Depending on the concrete case, we can apply the online approach or offline approach to verify the conformance of the system. The rule is the constraints on the order of messages and/or on data of these messages. We can understand a rule on a natural language as follow: if a message M1 occurs (may be including the constraints on data) then a message M2 (or a suite of message SM2) must occur before/after M1 for a period of time. A generic temporal logic (like LTL) is usually used to define the constraints on the order of messages for model checking engine. Modeling constraint is required to specify permissions and prohibitions. However, this is generally not sufficient to express constraint properties such as availability and obligations must be also considered. In contrast to permissions and prohibitions, obligations are often associated with deadlines to specify bounded time availability requirements. In this case, a violation only occurs if the obliged action is not performed before the deadline. These timing constraints do not consider the LTL. Cuppens et al [51] have defined a formal security model with deadlines called Nomad that helps us to solve this problem. In many cases, for each rule, we do not check all messages of a trace. We only pick some messages of a trace that satisfy some fixed conditions (for example, msg.username= xyz ) or exist a correlation between these messages by its data values, then our rule is applied on this set of messages. For example, there are many sessions that are executed in parallel and each message has a sessionid field. Afterwards, we need to group the messages belonging to a session by using sessionid field of the messages before applying our rule to check the correctness. This is called data correlation. In our works, we have proposed to extend the Nomad language, by defining the constraints on each atomic action (fix conditions) and a set of data correlations between the actions, which is more convenient to use than LTL (for our objective) to define the rules for web services. We chose this language because it provides a way to describe permissions, prohibitions that are granted (they are applied immediately) and obligations (needing a time duration to complete) related to non-atomic actions within contexts that take time constraints. Moreover, its syntax and our natural language are quite similar. But we only support permissions and prohibitions by adding the timing constraint that is bounded by a duration with time min and time max. We also consider the problem of data correlation between messages of a rule.

83 4.2. Methodology Methodology Passive testing (or Runtime verification) method is composed of three steps: 1. define the passive testing architecture to collect the execution traces on a running system. 2. define the rules that are applied to verify the correctness of the execution traces. 3. analyze (online or offline) the execution traces to produce the verdict. This verdict is pass if the system trace respects the specified constraints and fail if it does not. In previous works, the inconclusive verdict is possible when the tester cannot extract the necessary information from the execution trace because it is too short. But in our works, this verdict is not admitted because we return the verdict pass or fail at each message. The methodology which is presented in this section has been fully implemented in the tool RV4WS. A detailed description of this tool is introduced in section 5.3 of chapter Test architecture In this section, we introduce two architectures for the trace collection based on the notion Point of Observation (PO) of ISO standard 9646 [3], one for web service based and another for web service composition. Our trace collection architectures are shown in figure 4.1. With a web service based, testing approach is black-box. It means that we can only collect the communicating SOAP messages between the client and the service under test by setting a PO between them. With a web service composition, we can also test the communication between a web service and its partners. So that, to collect all input/output messages of a web service composition, each connection between the service and its partner will be set a PO. All messages that are collected by the POs and its correspondent occurrence time will be sent to a checking engine to analyze and produce the verdict. Figure 4.1: Trace collection architecture for web services

84 70 Chapter 4. Runtime Verification The rules In our works, we consider each message as an atomic action. We use one or several messages to define a formula. When we define a formula, the constraint on message parameters value may be considered. Finally, from these formulas, the rule is defined in two parts: supposition (or condition) and context. The set of data correlations are included as an optional. Definition 4.1. (Atomic action): We define an atomic action as one of following actions: an input message, an output message. Formally: where: AA := Event(Const) Event represents an input/output message name; Const := P V Const Const Const Const where: P are the parameters. These parameters represent the relevant fields in the message. V are the possible parameters values. {=,, <, >,, }. Definition 4.2. (Formula): A formula is defined recursively as following: F := start(a) done(a) F F F F F O d [m,n] F where A is the atomic action. start(a): A is being started. done(a): A has been finished. O d [m,n] F : F was true d units of time ago if m > n, F will be true d units of time if m < n, where m, n are two natural numbers. Definition 4.3. (Rule): If α and β are formula then R(α β) is a rule where R {P: permission; F: Forbidden;}. The constraint P(α β) (resp. F) means that it is permitted (resp. prohibited) to have α true when context β holds. Example 4.1. Rule defined examples: 1. We only allow to create a new account on the services if we have had successfully login within maximum one day ago and have not logged out. P(start(createAccountReq) O d [1,0]D done(loginres) done(logoutreq))

85 4.2. Methodology In the case of web service composition, we can define a rule to verify the interval time (i.e. 10 seconds) between a request message and a response message from a partner service, to assess about the successful rate, if this partner is installed on a far host. P(start(msgReq) O d [0,10]S done(msgres)) In some cases, we need to check the correctness of the messages from a message sequence by the groups. For example, many sessions are executed in parallel, the data correlations between messages are necessary to validate the messages belonging to a session by its data values. In this case, a rule is also extended with a set of data correlations to verify this problem. Definition 4.4. (Data correlation): A data correlation is a set of parameters that have the same data type where each different parameter represents a relevant field in a different message and the operator = (equal) is used to compare a parameter with others. A data correlation is considered as a property on data. Example 4.2. Let A(p A 0,pA 1 ), B(pB 0,pB 1,pB 2 ) and C(pC 0 ) are messages with p i are the parameters where p A 0,pB 0,pC 0 have the same type. A data correlation set that is defined based on A, B and C is: {p A 0,pB 0,pC 0 } {pa 0 = pb 0 = pc 0 } Definition 4.5. (Rule with data correlation): Let α and β are formula, CS is a set of data correlations based on α and β (CS is defined based on the messages of α and β). A rule with data correlation is defined as: R(α β)/cs where R {P: permission; F: Forbidden;}. The constraint P(α β) (resp. F) means that it is permitted (resp. prohibited) to have α true when context β holds within the conditions of CS. Example 4.3. We can re-define the rules in example 4.1 with the data correlation as follows: 1. we use sessionid to indicate the messages belonging to a session. P(start(createAccountReq) O d [1,0]D done(loginres) done(logoutreq))/ {{createaccountreq.sessionid, loginres.sessionid, logoutreq.sessionid}} 2. we validate the send/receive messages on the same machine (msgreq.sourceip = msgres.destip). P(start(msgReq) O d [0,10]S done(msgres))/ {{msgreq.sourceip, msgres.destip}} Checking algorithm In this section, we briefly outline the computation mechanism used to determine whether a rule holds for some given input/output sequence of events. Our algorithm determines message-by-message the conformity with each rule without storing the message sequence. Here, we use two global variables: currlist is a list of current rules that were enabled and rulelist is a list of rules that are defined to verify the system. Before introducing the detail of algorithm, we present some functions to compute on the context of each rule:

86 72 Chapter 4. Runtime Verification update: this function updates the value of context whenever a message arrives and this message exists in the context. For example, the context of a rule is loginresponse logoutrequest. When the loginresponse message arrives, this context is updated as true logoutrequest. evaluate: this function evaluates whether a context of rule is holds (true) or not. This function returns one of three values: true, false or undefined if a message that is not updated exists. While the evaluation, a message with the function not will be assigned provisionally is true. For example: at the time of evaluation, the expression true logoutrequest will be evaluated as true true = true. correlation: this function will return one in three values: undefined, true and false. undefined when a message msg is not defined in the set of data correlations of the rule. In the case that the msg is defined in the set of data correlations of rule, this function will query the correspondent value and compare it with the value of previous messages to return true/f alse. contain: to find a message in the context of a rule. This function returns true if the message msg is found in in the context of a rule and its condition is validated. For example, the context of the rule is the following expression: msga[msga.id = 5]&msgB. When the msga (with its value id=4) arrives, the contain function will be returned as false because the message name is found but its condition does not satisfy. In the case of msgb arrives, this function returns true. As said earlier, there is two types of rule: future time and past time. To make this more clear, we will analyze the checking algorithm for each type. Rule with future time We know that each rule has two parts: the supposition part and the context part. The rule will be validated if its supposition was enabled and its context is hold (true). In a rule with future time, the context part will happen after its supposition was enabled. Our algorithm has two steps: Step 1) Each time that a message (called msg) arrives, we have a list of current rules (currlist) that have been enabled to wait the validation of its context. Therefore, we will firstly update the context of the current rule (noted rule) in this list (currlist) if msg appears in the context of rule and data correlation of msg satisfies if it is defined. Secondly, we will evaluate the context of each rule. If the context is true and the time constraint is satisfying, a verdict pass/f ail, depending on the permission/prohibition of the rule, will be given in time msg arrives, and will remove this rule from current list (currlist). If we cannot evaluate the context, we will wait for the next message to complete the context. In this case, a pass verdict is given. Step 2) we will examine all rules in rulelist and enable it (add into currlist) if its supposition part contains the message msg and condition of supposition part is valid with the data of msg. When we enable a new rule, the properties of data correlation set will be assigned by the values that are queried from msg.

87 4.2. Methodology 73 Rule with past time In a rule with past time, the context part will happen before its supposition is enabled. It means that the context part must be completed and the evaluate function must return true or false, when its supposition is enabled. As the future time, we have also two steps: Step 1) we check firstly in the list of active rules (currlist). If its supposition part contains the message msg and the condition of the supposition part is valid with the data of the msg and the data correlation of the msg satisfies if it is defined. We will evaluate its context to give a verdict. On the contrary, we will check the time constraints on the rules to remove it from the list (currlist) if the time constraints do not satisfy. If the context of the rule contains this message (msg), we update the context to wait the next message. Step 2) we will examine all rules in the rulelist and enable it (add into currlist to wait the message in the supposition part) if its context contains the message msg. Like the future rule, the properties of data correlation set will also be assigned by the queried values of msg. Finally, we combine it to have a complete algorithm. The detail of main checking algorithm is shown in algorithm 6. This algorithm verifies message-by-message and returns the verdict at a time of arrival message.

88 74 Chapter 4. Runtime Verification Algorithm 6: Runtime verification algorithm Require: currlist is the list of current rules that were enabled, rulelist is list of rules that are defined to verify the system. Input : message msg, occurrence time t. Output : true/f alse res true; list ; //a list; //step 1: check in currlist to give a verdict; foreach rule in currlist do //if a rule is enabled many times, we consider only one time (i.e. one session); if rule.id / list then if rule is future time then res verify_future(rule, msg, t, res); else res verify_past(rule, msg, t, res); list.add(rule.id); //step 2: check in rulelist to enable new rule; foreach rule in rulelist do if msg rule.supposition() rule.condition(msg)= true then if rule is future time then r1 rule; //create a new rule; r1.active_time t; // set active time; r1.assignvalue4properties(msg); currlist.add(r1 ); //add into enabled list; else if rule.correlation(msg) f alse rule.evaluate()! = true rule.id / list then res false; else if rule is past time rule.id / list rule.context.contain(msg) then r1 rule; //create a new rule; r1.active_time t; // set active time; r1.update(msg) //update context; r1.assignvalue4properties(msg); currlist.add(r1 ); //add into actived list; return res;

89 4.2. Methodology 75 Algorithm 7: verify_future(rule, msg, t, result) Require: currlist: is a global variable Input : rule: a rule, msg: a message, t: occurrence time Output : true/f alse if verifytime(t, rule.active_time) = false rule.type = P then result false; currlist.remove(rule); else if r.context.contain(msg) rule.correlation(msg) f alse then rule.update(msg) //update context; if rule.evaluate() = true then currlist.remove(rule); if rule.type = F verifytime(t, rule.active_time) = true then result false ; else if rule.evaluate()=false then currlist.remove(rule); if rule.type = P then result false; return result; Algorithm 8: verify_past(rule, msg, t, result) Require: currlist: is a global variable Input : rule: a rule, msg: a message, t: occurrence time Output : true/f alse if msg rule.supposition() rule.condition(msg)=true rule.correlation(msg) f alse then currlist.remove(rule); if rule.evaluate() = true then if rule.type = F verifytime(t, rule.active_time) = true then result false; else if rule.type = P then result false; else if verifytime(t, rule.active_time) = false then currlist.remove(rule); else if rule.context.contain(msg) rule.correlation(msg) f alse then rule.update(msg); return result;

90 76 Chapter 4. Runtime Verification Example 4.4. For example, we have an execution trace with the message name and its time occurrence: MS ={(a 0,0), (a 1,2), (a 0,3), (b 1,8), (b 0,9), (a 1,12), (b 2,15), (c 0,16)}. The rules that are defined to assess the system are: r 1 = P(start(a 0 ) O d [0,10] done(b 0 ) done(c 0 )), r 2 = P(start(b 1 ) O d [+,0] done(a 1 ) done(c 1 )) and r 3 = P(start(a 2 ) O d [0,10] done(b 2 ). The table 4.1 shows result of the algorithm when we applied the set of rules {r 1, r 3, r 3 } to verify the execution trace MS. message enabled rule list verdict add/remove (+/-) (a 0, 0) {r 1 + = P(true Od [0,10] done(b 0 ) done(c 0 ))} true +r 1 (a 1, 2) {r 1 = P(true O d [0,10] done(b 0 ) done(c 0 )); true +r 2 r 2 + = P(start(b 1) O d [+,0] true done(c 1 ))} (a 0, 3) {r 1 = P(true O d [0,10] done(b 0 ) done(c 0 )); r 2 = P(start(b 1 ) O d [+,0] true done(c 1 )); true +r 1 r 1 + = P(true Od [0,10] done(b 0 ) done(c 0 ))} (b 1, 8) {r 1 = P(true O d [0,10] done(b 0 ) done(c 0 )); true -r 2 r 1 = P(true O d [0,10] done(b 0 ) done(c 0 ))} (b 0, 9) {r 1 = P(true O d [0,10] done(b 0 ) done(c 0 ))} true -r 1 (a 1, 12) {r 1 = P(true O d [0,10] done(b 0 ) done(c 0 )); true +r 2 r 2 + = P(start(b 1) O d [+,0] true done(c 1 ))} (b 2, 15) {r 2 = P(start(b 1 ) O d [+,0] true done(c 1 ))} false* -r 1 (c 0, 16) {r 2 = P(start(b 1 ) O d [+,0] true done(c 1 ))} true Table 4.1: An example of runtime verification *at the message (b 2, 15), we receive a false verdict because rule r 1, which has the last enabled message is (a 0, 3) and valid for a duration 10 unit of the time, is fail at the time Conclusion In this chapter, we proposed a new methodology to perform passive testing of the web services based on the rule specification. We defined the trace collection architectures and introduced an algorithm that checks the correctness of a sequence of messages (i.e., SOAP messages) with a set of rules. Importantly, this algorithm allows us to check a sequence of message in parallel with a trace collection engine by verifying message by message without storing them. The time constraints, including future and past time, are also supported on the rules to verify the interval time between messages. In our approach, we allow to apply the rule on only some messages that satisfy some fix conditions or on the groups of messages that have the data correlation. The implementation of this approach, the tool RV4WS, the online testing approach of chapter 3 and the tool WSOTF, are presented in the next chapter.

91 Chapter 5 Implementation Contents 5.1 A case study: Product Retriever WSOTF tool Introduction Experimentations and results RV4WS tool Introduction Experimentations and results Conclusion In the context of the WebMov project, some tools are developed. This is also the solution to demonstrate these theories presented in chapter 3 and 4. In this chapter, we will introduce two tools (WSOTF: Web Service, Online Testing Framework and RV4WS: Runtime Verification for Web Services). WSOTF is implemented based on the online approach (section 3.3.4) that can be used to test a web service based or to unit test an orchestration. RV4WS uses the theories in chapter 4 that allowed us to verify a web service at runtime. For each tool, we show an experimentation result based on a real-life case study. 5.1 A case study: Product Retriever In this section, we present a real-life case study, named Product Retriever [115], of Web- Mov project. This case study is a BPEL process that allows users to automate part of the purchasing process. It enables you to retrieve one searched product sold by a preauthorised provider. The search is limited by specifying a budget range and one or more keywords characterizing the product. The searched product is done through the operation getp roduct and the parameter RequestP roductt ype that is composed of information about the user (firstname, lastname and department) and searched product (keyword, max price, category). This process has 4 partner services, named AmazonF R, AmazonU K, CurrencyExchange and P urchaseservice that are developed by Montimage 1 and available

92 78 Chapter 5. Implementation at and its overview behaviour, illustrated in figure 5.1, is described by the following: 1. Receives a message from the client with the product and keywords of the characteristics of the product. 2. Contacts the P urchaseservice partner to obtain the list of authorized providers for that product. In a case where there is no authorized provider, an announcement will be sent to the client by a fault message response. 3. Depending on the authorized provider result, the process contacts either the AmazonFR or the AmazonUK service to search a product that matches the price limit by Euro and the keywords. 4. Sends back to the client the product information and the name of the provider where the product was found and a link to where it can be ordered. If a matching product is not found, a response with unsatisfying product will be sent back to the client. 5. After receiving the product information, the client can send an authorization request to confirm the purchase of the product within a certain duration (i.e., one minute) of time. The Product Retriever service is built in Netbeans and deployed by a Sun-Bpelengine within a Glassfish 2.1 web server.

93 5.1. A case study: Product Retriever 79 Figure 5.1: ProductRetriever - BPMN specification

94 80 Chapter 5. Implementation 5.2 WSOTF tool Introduction WSOTF (Web Service, Online Testing Framework) is developed based on algorithm 3 (section 3.3.4). We can use this to test a web service based (WSDL specification) or a web service orchestration. It is implemented by Java and its detailed architecture is shown in figure 5.2 that consists of two parts: the controller and the adapter. The controller is composed of five main components: a loader, an interface to process data, a data function library, an interface to send/receive the messages to/from SUT and an executer. Figure 5.2: Architecture of the WSOTF engine 1. Loader: loads the input format and analyses it; 2. DataProcess Interface: used to generate the data content, get the name of the message and query the value of the message. To implement this interface, we reuse the code of SoapUI [58] to generate a SOAP format. The data for each field of a SOAP message is randomly generated or a default value is used. This depends on the configuration file;

95 5.2. WSOTF tool Data function library: defines a list of functions that are used to update the variables or evaluate a boolean expression. The current version, only variables with data types int, boolean or string are supported to control the internal behaviour; 4. Executer: implements online testing algorithm to generate a test case, controls test execution and assigns the verdict. It uses the data process interface and the data function library to generate SOAP message. It updates the value of variables and evaluates the constraints; 5. Test Execution Interface: used to send and receive the message to/from SUT. To implement it, we used a http client to invoke the request into SUT (the client request or partner callback in the case of asynchronous services, the result is returned on a different port) and http server to receive and return the message from SUT on the same port. When test execution receives a SOAP message from SUT, it sets this message into a queue to wait for a processing of the executer. It receives directly SOAP message from the executer and sends it to SUT. Figure 5.3 shows an input format example of WSOTF. This format consists of a partner section that declares the partners name and location of wsdl specification, variables list (variable types are: int, boolean or message type of SOAP that is defined in WSDL), local clock, initial state, and a list of transitions. Each transition consists of seven fields: source state, target state, event name, guard on variable, guard on clock, data update function and local clock to be reset. In WSOTF, we use the form?pl.pt.op.msg to represent a format of an input action that means the reception of the message (msg) for the operator (op) of the porttyte pt from the partner (pl) and the form!pl.pt.op.msg represents a format of an output action (resp. the emission of the message (msg) for the operator (op) of the porttyte pt to the partner (pl)). The result of WSOTF (traces including interval time between two actions and its correspondent verdict) is saved in a xml file. This tool enables the declaration of the value of some fields of SOAP message in an enumeration file that can be used to test by purpose (fix the condition for each branch), correlation data (current version not support yet the correlation data functions) or to debug. At each execution time, WSOTF requests a number N (integer) and it repeats N times to generate N traces if there is no error and the corresponding verdict is P ASS. WSOTF will stop immediately if it finds an error (message receive incorrect, or timeout). This tool returns a XML traces file with the following format: <?xml v e r s i o n ="1.0" e n c o d i n g ="UTF 8" s t a n d a l o n e ="no"?> <t r a c e s > <t r a c e i d =" t 0 "> <a c t i o n name=" x l o a n. xloanpt. r e q u e s t. r e q u e s t R e q u e s t " t i m e d e l a y ="0.0" type=" i n p u t "> <data> <x l o : r e q u e s t I n f o xmlns : x l o =" h t t p : / f r. webmov. l a b r i / x l o a n /"> <id >1</id > <name>honorem t a l i a flammato </name> <income >9994</ income> <amount>9997</amount> <maxpayment>10005</maxpayment> <maxmonth>10007</maxmonth> </ x l o : r e q u e s t I n f o > </data>

96 82 Chapter 5. Implementation Figure 5.3: Input format of WSOTF

97 5.2. WSOTF tool 83 </ a c t i o n > <a c t i o n name="bank. bankpt. approve. approverequest " t i m e d e l a y ="0.05" type="output "> <data> <a e t g t : a p p r o v e I n f o xmlns : a e t g t =" h t t p : / f r. webmov. l a b r i / bank /" xmlns : ns1=" h t t p : / f r. webmov. l a b r i / x l o a n /"> <id >1</id > <name>honorem t a l i a flammato </name> <income >9994</ income> <amount>9997</amount> <maxpayment>10005</maxpayment> <maxmonth>10007</maxmonth> </ a e t g t : a p p r o v e I n f o > </data> </ a c t i o n > <a c t i o n name="bank. bankpt. approve. approveresponse " t i m e d e l a y ="0.0" type=" i n p u t "> <data> <bank : approveres xmlns : bank=" h t t p : / f r. webmov. l a b r i / bank/"> a c c e p t </bank : approveres> </data> </ a c t i o n > <a c t i o n name=" x l o a n. xloanpt. r e q u e s t. r e q u e s t R e s p o n s e " t i m e d e l a y ="0.01" type="output "> <data> <a e t g t : r e q u e s t R e s xmlns : a e t g t =" h t t p : / f r. webmov. l a b r i / x l o a n /" xmlns : ns1=" h t t p : / f r. webmov. l a b r i / bank/"> a c c e p t </ a e t g t : requestres > </data> </ a c t i o n > <a c t i o n type=" d e l a y "> <data >60</ data> </ a c t i o n > <a c t i o n name="bank. bankpt. c a n c e l. c a n c e l R e q u e s t " t i m e d e l a y ="0.0" type="output "> <data> <a e t g t : c a n c e l I n xmlns : a e t g t =" h t t p : / f r. webmov. l a b r i / bank /" xmlns : ns1=" h t t p : / f r. webmov. l a b r i / x l o a n /">1</ a e t g t : c a n c e l I n > </data> </ a c t i o n > </ t r a c e >... </ t r a c e s > We have developed a graphic user interface (named TEFSM Designer) to design the input for WSOTF and to visualize the test results (appendix A.1.1). The test results are shown by two types: sequence of messages and path on the graph of TEFSM Experimentations and results In this section, we introduce how to test an orchestration of web service using the WSOTF tool. The tested orchestration is Product Retriever in section 5.1. Because this tool can simulate all partners of this orchestration, its partners will be simulated by WSOTF instead of using the existing its partners (i.e., AmazonUK, AmazonFR, Purchase, CurrencyExchange). To do this, before deploying this orchestration, we modified the endpoints of its partners to the server handle of WSOFT (for example:

98 84 Chapter 5. Implementation Modeling the Product Retriever orchestration by TEFSM Lallali et al [94] define many rules to translate a BPEL specification into a TEFSM. They use a local clock and a global clock is optional to verify the time constraints. The value of input/output message will be handled by a variable. Here, we use these rules to model the Product Retriever orchestration by a TEFSM. But the assign activity is not considered because we are only interested in the time constraints and the communicating activities. The figure 5.4 presents a graph of ProductRetriever s TEFSM. Figure 5.4: TEFSM of ProductRetriever

99 5.3. RV4WS tool 85 Test results and analysis In this section, we introduce the test results of Product Retriever service by applying our tool. Because this case study has the data correlation when the client sends the message getauthorisationrequest (i.e., firstname, lastname and userid). Our tool now does not support this problem when it generates the SOAP messages. But we can declare an enumeration for each field of SOAP messages in an enum file. To limit the repeat of coincident cases, we declared a list of fix values for the condition fields, for example: provider=amazonfr;amazonuk or maxprice=1000;900;800 etc. The following table 5.1 resumes the test results of Product Retriever service using WSOTF. In these test results, we found a fault that is not admitted in the specification when the product price by Euro of the partner is greater than the maxprice. Traces Verdict?getProductRequest(maxPrice=1000)!getProviderRequest() 1?getProviderResponse(provider=AmazonFR)!searchItemFRRequest() Pass?searchItemFRResponse(price=500)!getProductResponse(price=500) delay=60?getproductrequest()!getproviderrequest()?getproviderfault() 2!getProductFault() Pass?getProductRequest(maxPrice=800)!getProviderRequest()?getProviderResponse(provider=AmazonUK)!searchItemUKRequest() 3?searchItemUKResponse(price=500)!getCurrencyRateRequest() Pass?getCurrencyRateResponse(rate=1.5)!getProductResponse(price=750) delay=60?getproductrequest(maxprice=1000)!getproviderrequest()?getproviderresponse(provider=amazonuk)!searchitemukrequest() 4?searchItemUKResponse(price=500)!getCurrencyRateRequest() Pass?getCurrencyRateResponse(rate=2)!getProductResponse(price=1000)?getAuthorizationRequest()!purchaseAuthorizationRequest()?purchaseAuthorizationResponse()!getAuthorizationResponse()?getProductRequest(maxPrice=1000)!getProviderRequest()?getProviderResponse(provider=AmazonUK)!searchItemUKRequest() 5?searchItemUKResponse(price=500)!getCurrencyRateRequest() Fail?getCurrencyRateResponse(rate=2.5) FAULT Table 5.1: Test results of ProductRetriever by WSOTF tool 5.3 RV4WS tool Introduction RV4WS (Runtime Verification for Web services) is implemented to verify a web service at runtime based on a set of constraints that are declared by the defined syntax in section

100 86 Chapter 5. Implementation This tool receives a sequence of messages (message content and its occurrence time) via a TCP/IP port, then verifies the correctness of this sequence. The detail of architecture is shown in figure 5.5. Figure 5.5: Architecture of the RV4WS tool One of the most interesting components in this architecture is the checking engine component that implemented the runtime verification algorithm 6. The engine allows us to verify each incoming message without any constraint of order dependencies, so we can apply this approach to both of online and offline testing. Also, this algorithm verifies the validation of current message without using any storage memory. In order to use this engine for the other systems, there is a difference between the systems is the data structure of input/output messages, we define an interface (i.e., IP arsedata, shown in the figure 5.6) as an adapter to parse the incoming data of RV4WS. The methods in IP arsedata are for gathering information from incoming message. getm esssagen ame() returns the message name from its content and querydata() allows us to query a data value from a field of message content. In each concrete case, we will implement this interface. For example, in the case of Web services, its implementation is the class P arsesoapimpl. This engine has been designed as a java library and is controlled by a component called Controller, which received a data stream coming from TCP/IP port. Figure 5.6: ParseData Interface of RV4WS

101 5.3. RV4WS tool 87 The input format for this tool is a xml file that has been defined in figure 5.7. A rule with a true verdict represents a permission and a f alse verdict represents a prohibition. A context of rule will be expressed as an expression with two operators AND and OR. Each data correlation is defined as a property with some query expressions from the different SOAP messages. In the case of web services, we have developed a Graphic User Interface (GUI) that allows us to easily define a set of rules from WSDL files (appendix A.2). Figure 5.7: Rule format example If a rule is found to be not satisfying, the checking algorithm returns a fail verdict. This rule may be not applied to the current message. To know which rule has failed at an arrival message, we have also presented a Graphic User Interface (GUI) that is used to visualize some statistical properties, calculated at any moment of testing process. Whenever a rule is activated, this means that its conditions have been satisfied and a statistical property such as type counter will be used to compute the percentage of un-satisfying time when applying the rule on the input data stream. If the rule was satisfied, we need to know the time duration from the activating moment to its context s holding moment. We have three statistical properties about time (time-min, time-max and time-average) for each rule. Now we need to know the values of these statistical properties and also visualize the relationships between them. For example, one rule executing shows its fail percentage in proportion to its duration time or to others properties. If we had used a histogram view and applied it for each, we would not have been able to get this information because of the different scales of these properties. We built a visual interface which is based on the idea of parallel coordinates scheme, introduced by Inselberg [86]. In information visualization, parallel coordinates view is used to show the relationships between items in a multidimensional dataset. Each axes in this view parallel to each other and a point in n-dimensional space is represented as a polyline with vertices on these axes. Considering that list of statistical properties of our testing process as a multivariate/multi dimension data, we have applied

102 88 Chapter 5. Implementation this visualization to RV4WS tool and made it possible to explore the result of our checking algorithms. As said earlier, we have implemented the checking algorithms inside RV4WS tool which enables a user-tester to verify these conditions defined in rules. Then the user-tester discovers that rule s properties change over time and he or she often needs a complete view of these traces of testing process. There are the parallel coordinates views correspondent to rules. In the figure 5.8, each scheme of parallel coordinates represents a time-log of statistical values as these polylines crossing properties axes. Within each view, there is a single polyline per time instance. The lines of current time are always highlighted. So this view enables the tester to visualize rapidly if these changes of executing rule s properties are interesting or not. Because of this problem, this visualization is refreshed after a duration. It means that it does not run in real-time. Figure 5.8: Checking analysis of RV4WS tool Experimentations and results In this section, we present some preliminary results we got after conducting our first experimentations on the Product Retriever case study (section 5.1) using RV4WS tool. SoapUI [58] is a well known test tool for web services based. We used it in our experiments as a client of Product Retriever service, sending requests to activate the web service (i.e., BPEL process). To collect the communicating messages between the Product Retriever service and its partners (including SoapUI), we have developed a proxy that allows us to forward a message to a specified destination. This allows us to receive and forward from/to some sources and destinations. Each connection is handled on a different port. Afterwards, this message and its time occurrence are also sent to our tool (i.e., RV4WS) to check its correctness. SoapUI and Product Retriever service were configured to make connections through the proxy.

103 5.3. RV4WS tool 89 The connection information (service name) is also sent to RV4WS to help this tool to easily identify which message belongs to which service. Figure 5.9 shows our testbed architecture. Figure 5.9: Testbed architecture Rules define We can define many test purposes to verify the interaction order with partner services. Here we introduce three test purposes: 1. During the execution of service, if the client receives a P roductf ault message, so before the Purchase service must return a P roviderf ault message. The time constraint of this test purpose is less important, so we define the maximum interval time between two messages as 10 seconds. P(start(P roductf ault) O d [10,0]s done(p roviderf ault)) 2. If the Purchase service introduces the provider service AmazonUK then the orchestration must contact the CurrencyExchange service within maximum of 10 seconds. P(start(getP roviderresponse[provider = AmazonUK]) O d [0,10]s done(getcurrencyraterequest))

104 90 Chapter 5. Implementation 3. When the client sends an authorization request message to confirm the purchase of a product, so it must receive a product response message with the EmptyResponseP roduct field be null within maximum one minute ago. In this rule, the data correlation is used by userid. Checking results P(start(getAuthorizationRequest) O d [1,0]m done(getp roductresponse[emptyresponsep roduct = null]))/ (getauthorizationrequest.userid, getp roductresponse.userid) Figure 5.10: Checking analysis of Product Retriever Figure 5.10 presents the checking analysis of the Product Retriever. This figure indicates: 1) the fault messages that are defined in rule 1 do not occur. 2) the getp roviderresponse message with provider = AmazonU K appeared two times, but the tool did not found a message getcurrencyraterequest within 10 seconds from the occurrence time of the message getp roviderresponse. See figure 5.11, we found the interval time between them is 11 seconds for the first case and 25 seconds for the second case. 3) the message getauthorizationrequest

105 5.4. Conclusion 91 appeared three times. Before that, the getp roductresponse message also appeared with the field EmptyResponseP roduct is empty and the interval time between them is less than one minute. Figure 5.11 returns the f alse verdict when the itemsearchresponse arrives because at the occurrence time of itemsearchresponse, the time constraint for the second rule (i.e., 10 seconds) does not satify. Figure 5.11: Trace collection of Product Retriever 5.4 Conclusion This chapter introduced two implementation tools for web services testing. One, named WSOTF, is developed based on online testing approach for unit testing. This tool purports the complete a test scenario (generating the abstract test case, deriving the concrete test case, executing the test, announcing the verdict), time constraints, and time delay to synchronize the time with the service under test. It allows us to simulate all the partner services to avoid the unnecessary damage of the running partners or to reduce the interaction scenarios with less effort etc. There is a limit of current version where the data correlation between input messages is not supported. The other, named RV4WS, focuses on runtime verification. It means that this tool is used to verify a trace execution as either correct or incorrect to a set of rules with timing constraints, including future and past time. This tool can check a trace execution in parallel with trace collection engine. Specifically, it checks message by message of a trace sequence without storing them.

106 92 Chapter 5. Implementation

107 Chapter 6 Conclusions and perspectives Contents 6.1 Synthesis and results Perspectives This thesis allows me to update knowledge about web services, its composition, the different formal models of web service orchestration and some testing approaches applied based on these models. The purpose of this thesis is to consider some aspects of web service testing as unit testing, integrated testing, passive testing, etc and propose the test method for each aspect. In the context of finding the test approach for web services that allows me to explore many interested testing approaches. In the suite, we resume the effectuated works, obtaine results and discuss some remaining problems for future works. 6.1 Synthesis and results In the context of this thesis and WebMov project, beside updating our knowledge about web services, its composition, the different formal models of web service orchestration, some testing approach applied on these models, and many interested testing approaches, we also obtained the following results: Unit testing framework. In chapter 3, we introduced an unit testing framework for web service compostion. This framework focuses on gray-box testing approach and is composed of a centralized test architecture, a conformane relation (i.e., xtioco) and an offline testing approach that is composed of five steps: (i) modelling the composite of web service by a TEFSM, (ii) generating all test purposes (iii) generating the abstract timed test case, (iv) deriving the concrete timed test case, (v) executing the timed test case and producing the verdict. We also presented the tutorial about two existing tools, named TGSE and BPELUnit, that were used as the support tools for our framework. Automatic timed test case generation. Test case generation is alway the biggest problem of all test approaches. In chapter 3, we presented a new method of automatic timed test case generation from a TEFSM specification using test purpose. This method generates 93

108 94 Chapter 6. Conclusions and perspectives the timed test case by computing the synchronous product of a TEFSM specification and a test purpose. While computing the synchronous product, we also consider a path be either feasible or not to cut the path that does not satisfy the test purpose. Finally, the test case is generated by selecting a trace leading to an accept state. Online testing approach. We have also proposed an online testing approach based on TEFSM specification of a web service orchestration (section of chapter 3). This approach likes a debugger, from a current state, an input or a delay action is selected based the current data value of the variables and immediately send to Service Under Test (SUT). Then if an expected output is returned within an expected duration, we update the current state and repeat until we arrive at a final state. Else a fault verdict is produced. This approach also uses the centralized test architecture and the xtioco conformane relation. We have implemented this approach in the WSOTF tool that is introduced in chapter 5. A method for integrated testing. A composite of web services is a system integrated at runtime and its result depends on its partners or the condition of the real environment. Integrated testing can be performed to verify timing constraints on transmission messages between service under test and its partners in the real environment, this service is either ready to use or not etc. Passive testing may be used to verify an execution trace of a web service orchestration. But we know that passive testing is a method in which tester does not interact directly with the service under test (SUT). Maybe this forces us to spend a long time collecting the execution trace because we do not know when a session has started and when it finishes. To improve this problem, we proposed an approach for integrated testing of a web service orchestration in section 3.4 of chapter 3. This approach interacts directly with SUT by sending the input requests to start a new session, then collecting the trace to analyze. An algorithm that checks the conformance of an execution trace to a TEFSM is also proposed. A new runtime verification method. The runtime verification or passive testing is a method that collects the observable traces of the system by installing a probe and analyzes it to produce a verdict. This method is applied to the running service to verify that a service either satisfies some properties on the real environment or not. In chapter 4, we proposed a new approach to perform passive testing of the web services based on a set of rules. This approach consider time constraints, including future and past time, and data correlation problems. The algorithm can be used to check either online or offline. Implementation of the proposed methods. In the context of WebMov project, we implemented two tools: WSOTF and RV4WS that are introduced in chapter 5. The first tool is implemented from the online testing approach that allows us to generate and simultaneously execute the timed test cases. Using this tool, the complete test scenario (test case and its data) is built during test execution. It also supports the time constraints and time delay. To support design the input (i.e., TEFSM) and visualize the results, a graphic user interface is also developed, named TEFSM designer. The second tool focuses on the runtime verification problem which allows us to check the constraints of the messages that are happening in the real environment.

109 6.2. Perspectives Perspectives In this section, we present some existing problems and discuss some open solutions. Supporting data correlation of the WSOTF tool. The correlation of messages by its data content is a popular problem of web services. But our active testing tool (i.e., WSOTF) does not support this problem. Its current version allows us to fix the content of the based fields (the fields with data type are: int, string, boolean, etc) in a message by declaring its content within an enumeration file. We can use this to correlate the messages. In a case where two messages are correlated by a complex structure, this tool cannot solve. In the next version, we will study to support the problem of data correlation. Improving online testing algorithm. In the web service orchestration, a part of orchestration where some actions (or sequence of actions) may be run in parallel while the rest are executed in sequential. For example, the activities in the flow activity of BPEL language. In this case, it is not easy to generate the test cases if the offline testing approach is applied. But with the online approach, in future works, we can extend TEFSM with a set of synchronous states Syn : S {on, off} is a mapping that assigns the property to states. The state has property on (resp. of f) represents that its following actions will be executed simultaneously (resp. finishing the concurrence part). Then, we improve our algorithm to process simultaneously the activities of a flow activity. When we arrive at a state that has the property be on, all following actions (or sequence of actions) will be simultaneously processed where each action (or sequence of actions) has a correspondent current state. When we arrive at a state that has the property be off, these current states will be removed and only one is keeped to continue the rest part of specification. Test of web service choreography. Most of the works that are presented in this thesis focus on testing of a web service orchestration which describes the interaction of a main process to other services. A choreography is a composite of web services that describes the collaboration among services at the communicating message. This collaboration is distributed where each service is responsible for a part of the workflow. In the future, we will study how to model a choreography by TEFSM from its specification, for example WS-CDL [144] to apply our approach for timed test case generation. Moreover, we define the architectures to test a choreography or a part of choreography then applying our tools as WSOTF or RV4WS to test them. Figure 6.1 presents an example of choreography and of a part under test.

110 96 Chapter 6. Conclusions and perspectives Figure 6.1: A choreography example and the part under test

111 Bibliography [1] NuSMV: a new symbolic model checker. [2] International Workshop on Web Services and Formal Methods, WS-FM Lecture Notes in Computer Science, [3] International Standard ISO Information technology, open systems interconnection, conformance testing methodology and framework, [4] Active-BPEL. [5] G. Alonso, F. Casati, H. Kuno, and V. Machiraju. Web Services - Concepts, Architectures and Applications. Data-Centric Systems and Applications. Springer Verlag, [6] Rajeev Alur and David L. Dill. A theory of timed automata. Theoretical computer science, 126(2): , [7] Rajeev Alur and David L. Dill. Automata-theoretic verification of real-time systems. In Formal Methods for Real-Time Computing, [8] César Andrés, Mercedes G. Merayo, and Manuel Núnez. Passive testing of timed systems. In International Symposium on Automated Technology for Verification and Analysis, volume 5311, pages LNCS, [9] César Andrés, Mercedes G. Merayo, and Manuel Núnez. Formal correctness of a passive testing approach for timed systems. In IEEE International Conference on Software Testing, Verification, and Validation Workshops, pages 67 76, Denver, Colorado, USA, April [10] César Andrés, Mercedes G. Merayo, and Manuel Núnez. Passive testing of stochastic timed systems. In International Conference on Software Testing Verification and Validation, pages 71 80, Denver, Colorado, USA, April [11] Xiaoying Bai, Wenli Dong, Wei-Tek Tsai, and Yinong Chen. WSDL-based automatic test case generation for web services testing. In International Workshop on Service-Oriented System Engineering (SOSE 05), pages , Beijing, October IEEE Computer Society. [12] Luciano Baresi and Sam Guinea. Towards dynamic monitoring of ws-bpel processes. In Third International Conference on Service-Oriented Computing, pages , Amsterdam, The Netherlands, Dec [13] Luciano Baresi, Sam Guinea, Raman Kazhamiakin, and Marco Pistore. An integrated approach for the Run-Time monitoring of BPEL orchestrations. In 1st European Conference on Towards a Service-Based Internet, pages 1 12, Madrid, Spain, Springer-Verlag. [14] Luciano Baresi, Sam Guinea, Marco Pistore, and Michele Trainotti. Dynamo + Astro: An integrated approach for BPEL monitoring. In 2009 IEEE International Conference on Web Service, pages , Los Angeles, CA, USA, July [15] Howard Barringer, Allen Goldberg, Klaus Havelund, and Koushik Sen. Rule-based runtime verification. In 5th International Conference on Verification, Model Checking, and Abstract Interpretation, pages , Venice, Italy, Jan

112 98 Bibliography [16] Cesare Bartolini, Antonia Bertolino, Eda Marchetti, and Andrea Polini. WS-TAXI: A WSDLbased testing tool for web services. In 2009 International Conference on Software Testing Verification and Validation, pages , Denver, Colorado - USA, April IEEE Computer Society. [17] Emmanuel Bayse, Ana Cavalli, Manuel Nunez, and Fatiha Zaidi. A passive testing approach based on invariants: application to the WAP. Computer Networks, 48: , [18] Gerd Behrmann, Alexandre David, and Kim G. Larsen. A Tutorial on Uppaal. Aalborg University, Denmark, November [19] Axel Belinfante. JTorX: A tool for on-line model-driven test derivation and execution. In Tools and Algorithms for the Construction and Analysis of Systems, volume 6015, pages Springer Berlin, [20] Johan Bengtsson and Wang Yi. Timed automata: Semantics, algorithms and tools. Springer Berlin : Lectures on Concurrency and Petri Nets, 3098/2004:87 124, [21] A. Benharref, R. Glitho, and R. Dssouli. A web service based-architecture for detecting faults in web services. In IFIP/IEEE International Symposium on Intergrated Network Management, Nice - Acropoli, France, May IEEE. [22] Abdelghani Benharref, Rachida Dssouli, Mohamed Adel Serhani, Abdeslam En-Nouaary, and Roch Glitho. New approach for EFSM-based passive testing of web services. Testing of Software and Communicating Systems, 4581:13 27, [23] Lina Bentakouk, Pascal Poizat, and Fatiha Zaïdi. A formal framework for service orchestration testing based on symbolic transition systems. In TestCom/FATES, pages 16 32, [24] Ismail Berrada. Modélisation, Analyse et Test des systems communicants à contraintes temporelles : Vers une approche ouverte du test. PhD thesis, University Bordeaux 1, [25] Ismail Berrada, Richard Castanet, and Patrick Felix. A formal approach for real-time test generation. In Workshop On Testing Real-Time and Embedded Systems (WTRTES), Satellite Workshop of Formal Methods (FM), pages 15 30, Pisa, Italy, September [26] Ismail Berrada, Richard Castanet, and Patrick Felix. Testing communicating systems: a model, a methodology, and a tool. In TestCom 2005, volume LNCS 3502, pages , [27] Ismail Berrada and Patrick Felix. TGSE : Un outil générique pour le test. In Proc. of CFIP 2005, pages Hermès, [28] Antonia Bertolino, Guglielmo De Angelisand Lars Frantzen, and Andrea Polini. The PLASTIC framework and tools for testing service-oriented applications. In International Summer School on Software Engineering, volume 5413, pages LNCS, [29] Antonia Bertolino, Lars Frantzen, Andrea Polini, and JanTretmans. Audition of web services for testing conformance to open specified protocols. Architecting Systems with Trustworthy Components, 3938/2006:1 25, [30] Antonia Bertolino and Andrea Polini. The audition framework for testing web services interoperability. In 31st EUROMICRO Conference on Software Engineering and Advanced Applications, pages IEEE Computer Society, 30 Aug - 3 Sept [31] Faycal Bessayah, Ana Cavalli, Willian Maja, Eliane Martins, and Andre Willik Valenti. A fault injection tool for testing web services composition. In TAIC PART 2010, Windsor, UK, September [32] Henrik Bohnenkamp and Axel Belinfante. Timed testing with TorX. In FM 2005: Formal Methods, pages , Newcastle, UK, July 18-22, Lecture Notes in Computer Science.

113 Bibliography 99 [33] Adilson Luiz Bonifácio, Arnaldo Vieira Moura, Adenilso da Silva Simao, and José Carlos Maldonado. Towards deriving test sequences by model checking. Electronic Notes Theoretical Computer Science, 195:21 40, [34] BPELUnit. The open source unit testing framework for BPEL. [35] Antonio Bucchiarone and Stefania Gnesi. A survey on service composition languages and models. In International Workshop on Web Services Modeling and Testing (WsMaTe 2006), Palermo, Italy, June [36] Antonio Bucchiarone, Hernan Melgratti, and Francesco Severoni. Testing service composition. In 8th Argentine Symposium on Software Engineering, Mar del Plata, Argentina, August [37] CADP. TGV manual page. [38] Gerardo Canfora and Massimiliano Di Penta. SOA: Testing and self-checking. In International Workshop on Web Services - Modeling and Testing, [39] Honghua Cao, Shi Ying, and Dehui Du. Towards model-based verification of BPEL with model checking. In Proc. Sixth IEEE International Conference on Computer and Information Technology CIT 06, pages , Sept [40] Tien-Dung Cao, Patrick Felix, and Richard Castanet. WSOTF: An automatic testing tool for web services composition. In The Fifth International Conference on Internet and Web Applications and Services, pages 7 12, Barcelona, Spain, May 9-15, [41] Tien-Dung Cao, Patrick Felix, Richard Castanet, and Ismail Berrada. Testing web services composition using the TGSE tool. In SERVICES 09: Proceedings of the 2009 Congress on Services - I, pages , Los Angeles, CA, USA, IEEE Computer Society. [42] Tien-Dung Cao, Patrick Felix, Richard Castanet, and Ismail Berrada. Online testing framework for web services. In Third International Conference on Software Testing, Verification and Validation, pages , Paris, France., April 6-9, [43] Tien-Dung Cao, Trung-Tien Phan-Quang, Patrick Felix, and Richard Castanet. Automated runtime verification for web services. In 2010 The IEEE International Conference on Web Services., pages 76 82, Miami, Florida, USA, July 5-10, [44] R. Castanet, O. Kone, and P. Laurencot. On the fly test generation for real time protocols. In 7th International Conference on Computer Communications and Networks,, pages , Lafayette, LA, USA, Oct, [45] Ana Cavalli, Azzedine Benameur, Wissam Mallouli, and Keqin Li. A passive testing approach for security checking and its pratical usage for web services monitoring. In NOTERE 2009, Montreal, Canada, [46] Ana Cavalli, Tien-Dung Cao, Wissam Mallouli, Elilan Martins, Andrey Sadovykh, Sebastien Salva, and Fatiha Zaidi. WebMov: An dedicated framework for the modelling and testing of web services composition. In 2010 The IEEE International Conference on Web Services., pages , Miami, Florida, USA, July 5-10, [47] Ana Cavalli, Caroline Gervy, and Svetlana Prokopenko. New approaches for passive testing using an extended finite state machine specification. Information ans software technology, 45: , [48] Ana Cavalli, David Lee, Christian Rinderknecht, and Fatiha Zaidi. Hit- or- jump an algorithm for embedded testing with applications to in services. In IFIP International conference FORTE/PSTV 99, Beijing, China, 5-8 October 1999.

114 100 Bibliography [49] Ana Cavalli, Edgardo Montes De Oca, Wissam Mallouli, and Mounir Lallali. Two complementary tools for the formal testing of distributed systems with time constraints. In 12th IEEE/ACM International Symposium on Distributed Simulation and Real-Time Applications, pages IEEE Computer Society, [50] Denmark CPN Group, University of Aarhus. CPN Tools: Computer tool for coloured petri nets. [51] Frederic Cuppens, Nora Cuppens-Boulahia, and Thierry Sans. Nomad: A security model with non atomic actions and deadlines. In 18th IEEE workshop on Computer Security Foundations, pages IEEE Computer Society, [52] Elisângela de Araújo Rodrigues Vieira. Automated Model-based Test Generation for Timed Systems. PhD thesis, University of Evry-Val d Essonne, [53] Wen-Li Dong, Hang Yu, and Yu-Bing Zhang. Testing BPEL-based web service composition using high-level petri nets. In International Enterprise Distributed Object Computing Conference (EDOC 06). IEEE Computer Society, [54] D Dranidis, E Ramollari, and D. Kourtesis. Run-time verification of behavioural conformance for conversational web services. In 2009 Seventh IEEE European Conference on Web Services, pages , Eindhoven, The Netherlands, Nov [55] Abdeslam En-Nouaary and Rachida Dssouli. A guided method for testing timed input output automata. In TestCom2003, pages , [56] Abdeslam En-Nouaary, Rachide Dssouli, Ferhat Khendek, and Abdelkader Elqortobi. Timed test cases generation based on state characterization technique. In Real-Time Systems Symposium, Madrid, Spain, IEEE Computer Society. [57] Martin Evan, Basu Suranjana, and Xie Tao. WebSob: A tool for robustness testing of web services. In 29th International Conference on Companion Software Engineering ICSE2007, pages 65 66, May [58] Eviware. [59] H. Foster, S. Uchitel, J. Magee, and J. Kramer. Model-based verification of web service compositions. In Proc. 18th IEEE International Conference on Automated Software Engineering, pages , 6-10 Oct [60] Lars Frantzen, Maria de las Nieves Huerta, Zsolt Gere Kiss, and Thomas Wallet. On-the-fly model-based testing of web services with Jambition. In Web Services and Formal Methods, volume 5387 of LNCS, pages Springer Berlin, [61] Lars Frantzen, Jan Tretmans, and Rene de Vries. Towards model-based testing of web services. In International Workshop on Web Services - Modeling and Testing - WS-MaTe2006, pages 67 82, Palermo, Italy, [62] Chen Fu, Barbara Ryder, Ana Milanova, and David Wonnacott. Testing of java web services for robustness. In ACM SIGSOFT Software Engineering Notes, volume 29, pages ACM New York, NY, USA, July [63] Xiang Fu. Formal Specification and Verification of Asynchronously Communicating Web Services. PhD thesis, University of California, September [64] Xiang Fu, Tevfik Bultan, and Jianwen Su. Analysis of interacting BPEL web services. In International Conference on World Wide Web (WWW04), [65] Xiang Fu, Tevfik Bultan, and Jianwen Su. WSAT: A tool for formal analysis ofweb services. In The 16th International Conference on Computer Aided Verification, 2004.

115 Bibliography 101 [66] José García-Fanjul, Claudio de la Riva, and Javier Tuya. Generation of conformance test suites for compositions of web services using model checking. In Proceedings of the Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART 06), [67] José Garcia-Fanjul, Javier Tuya, and Claudio de la Riva. Generating test case specifications for BPEL compositions of web services using SPIN. In International Wrokshop on Web Services Modeling and testing (WS-MaTe 06), [68] Christophe Gaston, Pascale Le Gall, Nicolas Rapin, and Assia Touil. Symbolic execution techniques for test purpose definition. In TestCom2006, volume 3964, pages LNCS, [69] Arthur Gill. Introduction to the Theory of Finite-State Machines. New York : McGraw-Hill, [70] Stefania Gnesi, Diego Latella, and Mieke Massink. Formal test-case generation for UML statecharts. In Proceedings. Ninth IEEE International Conference on Engineering Complex Computer Systems, pages 75 84, April [71] A. Goldberg and K. Havelund. Automated runtime verification with eagle. In Verification and Validation of Enterprise Information Systems, Miami, USA, May [72] Ariane Gravel, Xiang Fu, and Jianwen Su. An analysis tool for execution of BPEL services. In The 9th IEEE International Conference on E-Commerce Technology and the 4th IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services, CEC/EEE, pages IEEE Computer Society, July [73] R. Heckel and L. Mariani. Automatic conformance testing of web services. In Fundamental Approaches to Software Engineering, volume 3442/2005, pages 34 48, Edinburgh, Scotland, 2-10 April Springer. [74] Anders Hessel, Kim G. Larsen, Marius Mikucionis, Brian Nielsen, Paul Pettersson, and Arne Skou. Formal Methods and Testing, chapter Testing Real-time systems using UPPAAL, pages Springer Berlin / Heidelberg, [75] Anders Hessel, Kim G. Larsen, Brian Nielsen, Paul Pettersson, and Arne Skou. Time-optimal real-time test case generation using UPPAAL. In 3rd International Workshop on Formal Approches to Testing of Software (FATES 03), [76] Anders Hessel and Paul Pettersson. A test case generation algorithm for real-time systems. In International Conference on Quality Software QSIC 04, pages , Germany, 8-9 September IEEE Computer Society. [77] Anders Hessel and Paul Pettersson. A global algorithm for model-based test suite generation. In Third Workshop on Model-Based Testing, March 31 - April [78] Teruo Higashinoy, Akio Nakatayy, Kenichi Taniguchi, and Ana R. Cavalli. Generating test cases for a timed i/o automaton model. In IFIP IWTCS 99, Budapest Hungary, 1-3 September [79] Sebastian Hinz, Karsten Schmidt, and Christian Stahl. Transforming bpel to petri nets. In The 3rd International Conference on Business Process Management, volume 3649, pages , Nancy, France, September LNCS. [80] Ian Ho and Jin-Cherng Lin. Generating test cases for real-time software by time Petri Nets model. In 8th Asian Test Symposium, pages IEEE Computer Society, [81] Gerard J. Holzmann. The model checker SPIN. IEEE Transactions on software engineering, 23(5): , MAY [82] Hyoung Seok Hong, Yong Rae Kwon, and Sung Deok Cha. Testing of object-oriented programs based on finite state machines. In In the Second Asia Pacific Software Engineering Conference, 1995.

116 102 Bibliography [83] Hyoung Seok Hong, Insup Lee, Oleg Sokolsky, and Sung Deok Cha. Automatic test generation from statecharts using model checking. Technical report, University of Pennsylvalia, [84] Jun Hou, Baowen Xu, Lei Xu, Di Wang, and Junling Xu. A testing method for web services composition based on data-flow. Wuhan University Journal of Natural Sciences, 13(4): , August [85] H. Huang, W. T. Tsai, R. Paul, and Y. Chen. Automated model checking and testing for composite web services. In International Symposium on Object-oriented Real-time distributed Computing (ISORC), pages , Seattle, May IEEE. [86] Alfred Inselberg. The plane with parallel coordinates. The Visual Computer, 1(2):69 91, [87] Claude Jard and Thierry Jéron. TGV: theory, principles and algorithms. International Journal on Software Tools for Technology Transfer, 7(4): , [88] T Jéron. Génération de tests pour les systèmes réactifs et temporisés. Ecole d Eté Temps-Réel, Télécom ParisTech, Paris, September [89] R. Kazhamiakin, P. Pandya, and M. Pistore. Timed modeling and analysis in web service compositions. In The First International Conference on Availability, Reliability and Security, pages , [90] ChangSup Keum, Sungwon Kang, In-Young Ko, Jongmoon Baik, and Young-Il Choi. Generating test cases for web services using extended finite state machine. In IFIP International Conference on Testing Communicating Systems (TESTCOM2006), volume 3964/2006, pages , New York, 6-18 May Springer Berlin. [91] Y.G. Kim, H.S. Hong, D.H. Bae, and S.D Cha. Test cases generation from UML state diagrams. IEE Proceedings Software, 146(4): , August [92] Moez Krichen and Stavros Tripakis. Black-box conformance testing for real-time systems. In S. Graf and L. Mounier, editors, SPIN2004, volume LNCS 2989, pages Springer-Verlag Berlin Heidelberg, [93] Mounir Lallali. Modélisation et test fonctionnel de l orchestration de services Web. PhD thesis, Telecom Sud Paris, [94] Mounir Lallali, Fatiha Zaidi, and Ana Cavalli. Timed modeling of web services composition for automatic testing. In SITIS 07: Proceedings of the 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System, pages , Shangai, China, December 16-19, IEEE Computer Society. [95] Mounir Lallali, Fatiha Zaidi, and Ana Cavalli. Transforming BPEL into intermediate format language for web services composition testing. In International Conference on Next Generation Web Services Practices, Seoul, October [96] Mounir Lallali, Fatiha Zaidi, Ana Cavalli, and Iksoon Hwang. Automatic timed test case generation for web services composition. In European Conference on Web Services (ECOWS 08), Dublin, Ireland, November [97] Kim G. Larsen, Marius Mikucionis, and Brian Nielsen. Online testing of real-time systems using UPPAAL. In Formal Approaches to Software Testing, pages 79 94, Linz, Austria, September Springer Berlin. [98] David Lee and Mihalis Yannakakis. Principles and methods of testing finite state machines. A survey, 84: , [99] J. Jenny Li and W. Eric Wong. Automatic test generation from communicating extended finite state machine (cefsm)-based models. In International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 02). IEEE Computer Society, 2002.

117 Bibliography 103 [100] Zheng Li, Jun Han, and Yan Jin. Pattern-based specification and validation of web services interaction properties. In 3rd International Conference on Service Oriented Computing, pages 73 86, Amsterdam, The Netherlands, Dec [101] Zheng Li, Yan Jin, and Jun Han. A runtime monitoring and validation framework for web service interactions. In The Australian Software Engineering Conference ASWEC 06, pages 70 79, Washington, DC, USA, IEEE Computer Society. [102] Zhong Jie Li, Wei Sun, Zhong Bo Jiang, and Xin Zhang. BPEL4WS unit testing: Framework and implementation. In IEEE International Conference on Web Service, pages , Orlando, FL, USA, July 11-15, [103] Chien-Hung Liu, Shu-Ling Chen, and Xue-Yuan Li. A ws-bpel based structural testing approach for web service compositions. In IEEE International Workshop on Service-Oriented System Engineering,, pages , Jhongli, Taiwan, December IEEE Computer Society. [104] Nik Looker, Malcolm Munro, and Jie Xu. Simulating errors in web services. International Journal of Simulation Systems, Science & Technology, 5(5):29 37, December [105] Nik Looker, Malcolm Munro, and Jie Xu. WS-FIT: A tool for dependability analysis of web services. In Annual International Computer Software and Applications Conference (COMPSAC 04), [106] Nik Looker, Malcolm Munro, and Jie Xu. A comparison of network level fault injection with code insertion. In Annual International Computer Software and Applications Conference (COMP- SAC 04), pages , [107] Nik Looker and Jie Xu. Assessing the dependability of soap rpc-based web services by fault injection. In IEEE International Workshop on Object-Oriented Real-Time Dependable Systems, pages , Italy, October [108] Wissam Mallouli, Faycal Bessayah, Ana Cavalli, and Azzedine Benameur. Security rules specification and analysis based on passive testing. In 2008 IEEE Global Telecommunications Conference, pages 1 6, New Orleans, LA, USA, Nov 30 - Dec [109] Wissam Mallouli, Amel Mammar, and Ana R. Cavalli. A formal framework to integrate timed security rules within a TEFSM-based system specification. In 16th Asia-Pacific Software Engineering Conference, pages IEEE Computer Society, [110] Philip Mayer. Design and implementation of a framework for testing BPEL compositions. Master s thesis, Leibniz, Hannover, Germany September [111] Philip Mayer and Daniel Lubke. Towards a bpel unit testing framework. In Workshop on Testing, Analysis and Verification of Web Software, pages 33 42, Portland, Maine, USA, July [112] Mercedes G. Merayo, Manuel Núnez, and Ismael Rodríguez. Formal testing from timed finite state machines. Computer Networks, 52(2): , [113] Marius Mikucionis, Kim G. Larsen, and Brian Nielsen. Online on-the-fly testing of real-time systems. Technical Report ISSN , Basic Research In Computer Science, Aalborg, Denmark, December [114] Marius Mikucionis, Kim G. Larsen, and Brian Nielsen. T-UPPAAL: Online model-based testing of real-time systems. In 19th IEEE international conference on Automated software engineering, pages , Linz, Austria, Sept IEEE Computer Society. [115] Montimage. Webmov case studies: definition of functional requirements and test purposes. Technical Report WEBMOV-FC-D5.1/T5.1, WebMov, [116] S. Nakajima. Verification of web service flows with model-checking techniques. In First International Symposium on Cyber Worlds, pages , Tokyo, Japan, November

118 104 Bibliography [117] Manuel Núnez and Ismael Rodríguez. Conformance testing relations for timed systems. In FATES 2005, pages Springer Berlin, [118] OASIS. Uddi spec technical committee. [119] Jeff Offutt and Wuzhi Xu. Generating test cases for web services using data perturbation. In Workshop on Testing, Analysis and Verification of Web Services, July [120] Chun Ouyang, Eric Verbeek, Wil M. P. van der Aalst, Stephan Breutel, Marlon Dumas, and Arthur. Formal semantics and analysis of control flow in WS-BPEL. Sci. Comput. Program., 67(2-3): , [121] Chun Ouyang, Eric Verbeek, Wil M.P. van der Aalst, Stephan W. Breutel, Marlon Dumas, and Arthur H.M ter Hofstede. WofBPEL: A tool for automated analysis of BPEL processes. In Third International Conference on Service Oriented Computing (ICSOC 2005), volume 3826, pages , Amsterdam, The Netherlands, December LNCS. [122] PushToTest. [123] Sylvain Rampacek. Sémantique, interactions et langages de description des services web complexes. PhD thesis, University of Reims Champagne-Ardenne, November [124] Vlad Rusu, Lydie du Bousquet, and Thierry Jéron. An approach to symbolic test generation. In International Conference on Integrating Formal Methods, LNCS 1945, pages Springer Verlag, November [125] Sébastien Salva and Issam Rabhi. Automatic web service robustness testing from wsdl descriptions. In 12th European Workshop on Dependable Computing, Toulouse, France, 14-15, May [126] Sébastien Salva and Antoine Rollet. Testabilité des services web. In Ingénierie des Systemes d Information 13, pages 35 58, [127] Karsten Schmidt. LoLA: a low level petri net analyzer. de/top/lola/lola.html. [128] Oleg Sokolsky, Usa Sammapun, Insup Lee, and Jesung Kim. Run-time checking of dynamic properties. Electron. Notes Theor. Comput. Sci., 144(4):91 108, [129] Jan Springintveld, Frits Vaandrager, and Pedro R. D Argenio. Testing timed automata. Theoretical Computer Science, 254(1-2): , March [130] Christian Stahl. BPEL2PN overview. bpel2pn/. [131] Christian Stahl. A petri net semantics for bpel. Technical report, Humboldt-Universitat zu Berlin, [132] OASIS Standard. Web services business process execution language version 2.0, April [133] M. Tabourier and A. Cavalli. Passive testing and application to the GSM-MAP protocol. Information ans software technology, 41: , [134] G. J. Tretmans and H. Brinksma. TorX: Automated model-based testing. In First European Conference on Model-Driven Software Engineering, pages 31 43, Nuremberg, Germany, December [135] Jan Tretmans. A Formal Approach to Conformance Testing. PhD thesis, University of Twente, 1992.

119 Bibliography 105 [136] Jan Tretmans. Testing concurrent systems: A formal approach. In CONCUR 99-10th International Conference on Concurrency Theory, volume 1664, pages 46 65, Eindhoven, The Netherlands, Springer Verlag. [137] W. T. Tsai, X. Wei, Y. Chen, and R. Paul. A robust testing framework for verifying web services by completeness and consistency analysis. In International Workshop on Service-Oriented System Engineering (SOSE), pages , Beijing, October IEEE. [138] W. T. Tsai, X. Wei, Y. Chen, B. Xiao, R. Paul, and H. Huang. Developing and assuring trustworthy web services. In Proceedings of 7th International Symposium on Autonomous Decentralized Systems (ISADS), pages 43 50, Chengdu, China, April [139] Wei-Tek Tsai, Yinong Chen, and Ray Paul. Specification-based verification and validation of web services and service-oriented operating systems. In Proceedings of the 10th IEEE International Workshop on Object-Oriented Real-Time Dependable Systems (WORDS 05), pages IEEE Computer Society, 2-4 Feb [140] W.T. Tsai, Ray Paul, Yamin Wang, Chun Fan, and Dong Wang. Extending WSDL to facilitate web services testing. In International Symposium on High Assurance Systems Engineering (HASE 02). IEEE Computer Society, [141] W.T. Tsai, X. Wei, Y. Chen, R. Paul, and B. Xiao. Swiss cheese test case generation for web service testing. IEICE Transactions on Information and Systems, E88-D(12): , December [142] Verimag. IF tutorials. [143] W3C. WSDL Version 1.1, March [144] W3C. Web services choreography description language version 1.0, TR/2004/WD-ws-cdl /. [145] W3C. Owl-s: Semantic markup for web services, S/. [146] W3C. SOAP Version 1.2, April [147] A. Wombacher, P. Fankhauser, and E. Neuhold. Transforming bpel into annotated deterministic finite state automata for service discovery. In Procs of ICWS 04, [148] Jun Yan, Zhongjie Li, Yuan Yuan, Wei Sun, and Jian Zhang. BPEL4WS unit testing: Test case generation using a concurrent path analysis approach. In 17th International Symposium on Software Reliability Engineering, pages 75 84, Raleigh, North Carolina, USA, November [149] YanPing Yang, QingPing Tan, and Yong Xiao. Verifying web services composition based on hierarchical colored petri nets. In IHIS 05: Proceedings of the first international workshop on Interoperability of heterogeneous information systems, pages 47 54, New York, NY, USA, ACM. [150] YanPing Yang, QingPing Tan, JinShan Yu, and Feng Liu. Transformation bpel to cp-nets for verifying web services composition. International Conference on Next Generation Web Services Practices, 0: , [151] Hsu-Chun Yen. Introduction to Petri Net Theory, volume 25 of Studies in Computational Intelligence. Springer Berlin / Heidelberg, [152] Yuan Yuan, Zhongjie Li, and Wei Sun. A graph-search based approach to bpel4ws test generation. In International Conference on Software Engineering Advances, page 14, Tahiti, French Polynesia, USA, October 29-November IEEE Computer Society.

120 106 Bibliography [153] Fatiha Zaidi. Contribution à la génération de tests pour les composants de service. Application aux services de Réseau Intélligent. PhD thesis, University of Evry Val d Essonne, [154] Yongyan Zheng and Paul Krause. Automata semantics and analysis of BPEL. In International Conference on Digital Ecosystems and Technologies, (DEST 07). IEEE Computer Society, [155] Yongyan Zheng, Jiong Zhou, and Paul Krause. Analysis of BPEL data dependencies. In 33rd EUROMICRO Conference on Software Engineering and Advanced Applications., pages IEEE Computer Society, August [156] Yongyan Zheng, Jiong Zhou, and Paul Krause. An automatic test case generation framework for web services. University of Surrey, Journal of software, 2(3):64 77, September [157] Yongyan Zheng, Jiong Zhou, and Paul Krause. A model checking based test case generation framework for web services. In International Conference on Information Technology, (ITNG 07). IEEE Computer Society, 2007.

121 Appendix A The tutorial of the tools A.1 WSOTF To test a web service orchestration using WSOTF, before deploying the SUT, we must change the port of the partners by the ports handle of WSOTF to simulate as follows: /wsoft/partnername where machinename (localhost), port number (8888) and URL base (/wsotf) are the configurations in the wsotf.properties file. The partnername is the name that is defined in the correspondent WSDL file. <wsdl : d e f i n i t i o n s name="bank "... >... <wsdl : s e r v i c e name=" b a n k S e r v i c e "> <wsdl : p o r t b i n d i n g ="impl : bankportsoapbinding " name="bankport"> <w sdlsoap : a d d r e s s l o c a t i o n =" h t t p : / / l o c a l h o s t : / w s o t f / bank"/> </wsdl : port > </wsdl : s e r v i c e > </wsdl : d e f i n i t i o n s > The current version, the binding message type is SOAP document [146]. A SOAP message that is formatted with RPC type is not supported. <wsdl : b i n d i n g name="xloanbinding " type="impl : xloanpt"> <w sdlsoap : b i n d i n g s t y l e ="document " t r a n s p o r t =" h t t p : / / schemas. xmlsoap. org / soap / h t t p " xmlns : wsdlsoap=" h t t p : / / schemas. xmlsoap. org / wsdl / soap /"/>... </wsdl : b i n d i n g > A.1.1 The command line to run independently the WSOTF: java -jar wsotf.jar -i inputfile -o outputfile -n tracenumber TEFSM designer The TEFSM designer is developed to create the input format for WSOTF engine by a graphic interface. This tool allows us to visualize the test results by the trees or the paths on the graph format of TEFSM. Because this tool is developed to design the TEFSM for the web services, so we can only add the properties on the transition after indicating which is the SUT and its partners via its WSDLs. Figure A.1 shows the global interface of this tool, figures A.2 and A.3 illustrate how to show the test results by the tree and the paths on the graph. 107

122 108 Appendix A. The tutorial of the tools Figure A.1: The main GUI of TEFSM Designer

123 A.1. WSOTF 109 Figure A.2: The visualization of the test results by the tree Figure A.3: The graphic visualization of the test results

124 110 Appendix A. The tutorial of the tools A.2 RV4WS To easily define the rules and visualize the checking analysis, we have been developed a graphic user interface (GUI). This GUI allows us to create a new project, define the rules of project, visualize the results, run and pause the checking analysis. Before running this tool, we need to configure the port that its TCP server component will handle to receive the messages from the trace collection engine (i.e., the proxy in our cases). This configuration is saved in the rv4ws.properties file. A.2.1 Graphical interface We present here some main dialogs of the tool. Figure A.4 shows the main GUI of RV4WS where it is composed of three parts: a project explorer, a checking analysis, a test results that is composed of a verdict, an occurrence time, message name and the extract of its content.

125 A.2. RV4WS 111 Figure A.4: The main GUI of RV4WS

126 112 Appendix A. The tutorial of the tools Figure A.5: Dialog to define a rule Figure A.6: Dialog to define the properties (data correlation)

127 A.3. TGSE 113 A.2.2 Proxy The command line to run the proxy is: java -jar JProxy.jar -p port -fs fwdserver -fp fwdport -cf config, where the config is a xml file with format as following: <?xml v e r s i o n ="1.0" e n c o d i n g ="UTF 8"?> <c o n f i g > <! l i s t e n d p o i n t s mapping t o s e r v i c e name > <e n d p o i n t s > <e n d p o i n t s e r v i c e n a m e = " amazonuk"> h t t p : / / : / AmazonUKService/AmazonUK </ endpoint > <e n d p o i n t s e r v i c e n a m e = " c u r r e n c y e x c h a n g e "> h t t p : / / : / CurrencyExchangeService / CurrencyExchange </ endpoint > <e n d p o i n t s e r v i c e n a m e = " amazonfr"> h t t p : / / : / AmazonFRService /AmazonFR </ endpoint > <e n d p o i n t s e r v i c e n a m e = " p u r c h a s e "> h t t p : / / : / P u r c h a s e S e r v i c e S e r v i c e / P u r c h a s e S e r v i c e </ endpoint > </endpoints > <! p o r t and s e r v e r name o f RV4WS s TCP s e r v e r component > <c h e c k i n g e n g i n e > <c h e c k i n g e n g i n e p o r t >9090</ c h e c k i n g e n g i n e p o r t > <checkingenginename >l o c a l h o s t </checkingenginename > </ c h e c k i n g e n g i n e > </ c o n f i g > A.3 TGSE The TGSE only supports integer and boolean data types. So all variables that appear on transition conditions will be mapped to integer to evaluate. An ETIOA in TGSE is described by: number of state, an initial state, a list of clock variables and a list of transition. Each transition t is composed of: 1. source_state(id, name); 2. target_state(id, name); 3. event (nop denotes an internal event); 4. guard condition on clocks (# denotes true); 5. guard condition on variable (# denotes true); 6. reset clocks (# denotes empty); 7. update variables (# denotes empty); Following the ETIOA of xloan that is modified from TEFSM in figure A.1 where the risk and response variables are mapped to integer type instead of the string type. (risk = 1 risk = high, risk = 0 risk!=high, response = 1 response = accept and response = 0 response!= accept ). Because the variables req_amount, risk and response are evaluated by the transition conditions and they depend on input data, we declare three correspondent parameters: p_req_amount, p_check_resp and p_xloan_resp. P_AUTO xloan { n b _ s t a t e s : 15

128 114 Appendix A. The tutorial of the tools i n i t i a l _ s t a t e : 0 ( 0, i n i t ), ( 1, s 1 ),? xloan_request, #, #, #, req_amount=p_req_amount ( 1, s 1 ), ( 3, s 3 ), nop, #, req_amount [ 1, ], #, # ( 1, s 1 ), ( 2, s 2 ), nop, #, req_amount ]10000,+ i n f ], #, # ( 2, s 2 ), ( 5, s 5 ),! a s s e s s _ c h e c k _ r e q u e s t, #, #, #, # ( 3, s 3 ), ( 4, s 4 ),! bank_approve_request, #, #, #, # ( 4, s 4 ), ( 7, s 7 ),? bank_approve_response, #, #, #, # ( 5, s 5 ), ( 6, s 6 ),? a s s e s s _ c h e c k _ r e s p o n s e, #, #, #, r i s k=p_check_resp ( 6, s 6 ), ( 7, s 7 ), nop, #, r i s k [ 1, 1 ], #, # ( 6, s 6 ), ( 3, s 3 ), nop, #, r i s k [ 0, 0 ], #, # ( 7, s 7 ), ( 8, s 8 ),! xloan_response, #, #, #, r e s p o n s e=p_xloan_resp ( 8, s 8 ), ( 9, s 9 ), nop, #, r e s p o n s e [ 1, 1 ], h:= t, # ( 8, s 8 ), ( 1 4, s14 ), nop, #, r e s p o n s e [ 0, 0 ], #, # ( 9, s 9 ), ( 1 0, s10 ), nop, t [ 6 0, 6 0 ], #, #, # ( 9, s 9 ), ( 1 1, s11 ),? xloan_confirm, t [ 0, 6 0 [, #, #, # ( 1 0, s10 ), ( 1 2, s 12 ),! bank_cancel, #, #, #, # ( 1 1, s11 ), ( 1 3, s 13 ),! bank_confirm, #, #, #, # ( 1 3, s13 ), ( 1 4, s 14 ), nop, #, #, #, # ( 1 2, s12 ), ( 1 4, s 14 ), nop, #, #, #, # } Test purpose: The xloan process is initiated by receiving an input request from the client. Next, it sends an approval request to the bank service. The client continues send a confirm message to xloan process at time t=20. Finally, the bank service receives a confirm request from the xloan process. The test purpose for this scenario is formulated in TGSE as following: TESTER t e s t { n b _ s t a t e s = 5 i n i t i a l _ s t a t e = 1 f i n a l _ s t a t e = 5 ( 1, i n i t ), ( 2, s 2 ),! xloan_request, #, #, #, # ( 2, s 2 ), ( 3, s 3 ),? bank_approve_request, #, #, #, # ( 3, s 3 ), ( 4, s 4 ),! xloan_confirm, t [ 2 0, 2 0 ], #, #, # ( 4, s 4 ), ( 5, f i n i s h ),? bank_confirm, #, #, #, # } A Communicating System Under Test (CSUT) is declared as following: SYSTEM xloan nb_automatons : 1 nb_rules : 4 parameters : p_req_amount p_check_resp p_xloan_resp c l o c k s : t a u t o m a t o n _ f i l e s : [ xloan / x l o a n. aut ] t e s t e r _ f i l e : xloan / t e s t e r. aut RULES <? xloan_ request,! xloan_ request > <! bank_approve_request,? bank_approve_request> <? xloan_ confirm,! xloan_ confirm > <! bank_confirm,? bank_confirm>

129 A.4. BPELUnit 115 Using TGSE to simulate this CSUT, we have the following time test case. In our case, we focus on gray-box testing, it means that we cover only the input/output events or delay action. We are not interested in the internal events. So, from TGSE result, we will drop internal actions. 1.? x l o a n _ r e q u e s t ( p_req_amount=1) 2.! bank_approve_request 3.? bank_approve_response 4.! x l o a n _ r e s p o n s e ( p_xloan_resp =1) 5.? xloan_confirm 6.! bank_confirm A.4 BPELUnit BPELUnit [34] is an open source tool to execute the concrete timed test cases for white-box testing. It allows us to: send a request with time delay and receive a response; verify the content of response message; simulate the partner services; This tool uses a distributed test architecture where there is no the synchronization between the simulation track partners to verify the order of messages or data correlation between them. To execute a test case, the client track and all partner tracks will be started to perform its definition activities. If a track that reports unsuccessfully exists, a fail verdict will be assigned. Else, a pass verdict is produced. For each partner track, it supports six activity types: (i) send asynchronous, (ii) receive asynchronous, (iii) send/receive synchronous, (iv) receive/send synchronous, (v) send/receive asynchronous, (vi) receive/send asynchronous. Figure A.7: Illustration of BPELUnit s test activities

130 116 Appendix A. The tutorial of the tools Figure A.7 shows an example of BPELUnit s test activities. This tool does not allows us to automatically derive the test case. So, it supports a graphic user interface (GUI) to manually define the test cases. Figure A.8 shows the main GUI of BPELUnit where we can edit a test suite, select a SUT, add the partners, create a new test case and define all activities for each partner. Figure A.8: The main GUI of BPELUnit Figure A.9 presents an edition interface of Send/Receive Synchronous of the client track where we can select a port name, an operation and define the literal XML data (the SOAP template is not automatically generated) to send to SUT. And figure A.10 presents the test execution result with a pass verdict. Like the WSOTF, we need to change the port of the partners by BPELUnit s port handle (Base URL) before deploying the SUT.

131 A.4. BPELUnit 117 Figure A.9: Edition of Send/Receive Synchronous of BPELUnit Figure A.10: Test case execution of BPELUnit