ICT IP Project. Deliverable D4.3. Final release of V&V tools and infrastructure. LAT E X template v. 1.16

Transcription

1 ICT IP Project Deliverable D4.3 Final release of V&V tools and infrastructure implementation LAT E X template v. 1.16

2

3 Project Number : FP Project Title : Large Scale Choreographies for the Future Internet Deliverable Number : D4.3 Title of Deliverable : Final release of V&V tools and infrastructure implementation Nature of Deliverable : Report + Prototype Dissemination level : Public Licence : Creative Commons Attribution 3.0 License Version : 3.0 Contractual Delivery Date : M30 31/03/2013 Actual Delivery Date : 19/04/2013 Contributing WP : WP4 Editor(s) : Antonia Bertolino, Guglielmo De Angelis, Andrea Polini Author(s) : Midhat Ali (Unicam), Cesare Bartolini (CNR), Amira Ben Hamida (Linagora), Antonia Bertolino (CNR), Antonello Calabrò (CNR), Flavio Corradini (Unicam), Francesco De Angelis (Unicam), Guglielmo De Angelis (CNR), Felicita Di Giandomenico (CNR), Roberto Gagliardi (Unicam), Fabio Kon (USP), Nelson Lago (USP), Paulo Moura (USP), Julien Lesbegueries (Linagora), Leonardo Mostarda (Unicam), Andrea Polini (Unicam), Alberto Ribolini (CNR) Reviewer(s) : Animesh Pathak - Inria, Valerie Issarny - Inria, Sebastien Keller - Thales Abstract Workpackage 4 (WP4) focuses on the definition and implementation of tools and approaches supporting Governance and V&V activities for ULS FI choreographies. In this deliverable we provide the final versions for the developed tools, whose preliminary versions were delivered in D4.2.1 and D The released prototypes are made available from the official repository. For each tool we provide a brief overview, a report of improvements made in the last reporting period, and a discussion of related scalability issues. In particular, the Governance and V&V infrastructure includes: the Rehearsal framework, supporting development-time testing of choreographies following a Test-Driven Development paradigm and including scalability testing; the Q4BPMN profile for annotating choreography specifications with non-functional requirements; ServicePot and the Resource Lifecycle Manager registries, enabling choreography governance by supporting on-line testing and SLA management, respectively; the ParTes tool supporting model-based testing of services and choreographies; the CRank framework, which implements algorithms for ranking of services and choreographies: a multi-source monitoring framework, supporting the monitoring of resources and events at infrastructure, business and event levels. FP III

4 Keyword List Choreography, Future Internet, Governance, Monitoring, Policy, Quality of Service, Ranking, Registry, Role, Service Level Agreement, Test Driven Development, Testing, Ultra Large Scale, Verification and Validation FP IV

5 Document History Version Changes Author(s) Antonia Bertolino, 1.0 Draft ToC Guglielmo De Angelis, Andrea Polini Antonia Bertolino, 1.1 ToC refinement and initial contents Guglielmo De Angelis 1.2 Contributions from all the partners all Antonia Bertolino, 2.0 Draft for internal review Guglielmo De Angelis, Andrea Polini 2.1 Applied changes from internal review all 3.0 Revised Final Version all A.0 Final EU submitted Version implementing modifications suggested by the PTC all A.1 Antonia Bertolino, Final EU submitted Version implementing modifications suggested by the PTC Guglielmo De Angelis, Andrea Polini Document Reviews Review Date Ver. Reviewers Comments Outline 13/03/ WP4 team N.A. Draft 5/04/ Animesh Pathak Annotated.pdf with comments is available on the internal Wiki here: org/pages/viewpage. action?pageid= Some advices for integrating QA 10/04/ Valerie Issarny with other deliverables and about scalability issues, see to: and cc-ed to PTC 19/04/2013 A.1 N.A. N.A. FP V

6 FP VI

7 Glossary, acronyms & abbreviations Item ACRB AoSBM API BPMN BSM CEP CRUD DoW EMF FI IDRE LGPL MDE MID OMG PL PMT PTC PMM Q4BPMN QoS REST SLA SOA SOAP TDD UL ULS URI V&V WS WP WPL WSDL XML XSD SL Description Adaptive Customer Relationship Booster Abstraction oriented Service Base Management Application Programming Interface Business Process Model and Notation Business Service Monitor Complex Event Processor Create, Read, Update, Delete Description of Work Eclipse Modeling Framework Future Internet Integrated Development and Run-time Environment GNU Lesser General Public License Model Driven Engineering Mobile Internet Device Object Management Group Project Leader Project Management Committee Project Technical Committee Property Meta Model Quality for BPMN Quality of Service REpresentational State Transfer Service Level Agreement Service Oriented Architecture Simple Object Access Protocol Test-Driven Development Ultra Large Ultra Large Scale Uniform Resource Identifier Verification and Validation Web Service Work Package Work Package Leader Web Services Description Language extensible Markup Language extensible Service Discovery Scientific Leader FP VII

8 FP VIII

9 Table Of Contents List Of Tables XI List Of Figures XIV List of Listings XV 1 Introduction Conceptual architecture Advances in the Reporting Period Roadmap Development-Time Analysis and Testing Tools Rehearsal Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies Q4BPMN Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies Run-Time Testing Tools ServicePot Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies CRank Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies ParTes Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies Resource Lifecycle Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies FP IX

10 4 Multi-source Monitoring Infrastructure Monitoring Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies Business Service Monitoring Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies Event Monitoring Overall Description Enhancements in the Final Release Scalability Dimensions, Issues, and Strategies Conclusions Bibliography A ParTes API FP X

11 List Of Tables Table 2.1: ULS dimensions and Rehearsal Table 2.2: ULS dimensions and Q4BPMN Table 3.1: ULS dimensions and ServicePot Table 3.2: ULS dimensions and CRank Table 3.3: ULS dimensions and ParTes Table 3.4: ULS dimensions and Resource Life Cycle Framework Table 4.1: ULS dimensions and Infrastructure Monitoring Table 4.2: ULS dimensions and BSM Table 4.3: ULS dimensions and Event Monitoring FP XI

12 FP XII

13 List Of Figures Figure 1.1: Conceptual Overview of Governance and V&V Framework Figure 2.1: Scalability Test Execution Flow Figure 2.2: Q4BPMN: Generation of Warning Rules Figure 3.1: ServicePot: High Level Architecture Figure 3.2: ServicePot: Mapping of BPMN2 concepts to UDDI Figure 3.3: ServicePot: The ChorDDI Component Figure 3.4: ServicePot: Plugin Interfaces and Events Figure 3.5: ServicePot: Distributed Publishing Figure 3.6: ServicePot: Distributed Lookup Figure 3.7: ParTes: Simple Parallel of ChoreographyTasks without Dependencies Figure 3.8: ParTes: receiver-sender Pattern Example Figure 3.9: ParTes: same-receiver Pattern Example Figure 3.10: ParTes: closed chain Pattern Example Figure 3.11: ParTes: Example of Complex Dependencies in a Choreography Figure 3.12: ParTes: Example of a Workflow Graph Figure 3.13: ParTes: Example of a RPST Figure 3.14: ParTes: Refactoring rules to Refactor a Workflow Graph Figure 3.15: ParTes: Example of Refactoring Figure 3.16: ParTes: Example of an Affection Graph Figure 3.17: ParTes: Example of Affection Graph exploration and Interleaving Tree creation Figure 3.18: ParTes: Violated Structural Transitions Figure 3.19: ParTes: Equivalent Traces Figure 3.20: ParTes: Equivalent Traces from Neutral Nodes Figure 3.21: ParTes: Not Valid Traces FP XIII

14 Figure 3.22: ParTes: Modification in the Architecture Diagram Figure 3.23: ParTes: Modification in the Class Diagram Figure 4.1: Multi-source Monitoring Figure 4.2: Platform Monitoring Deployment Figure 4.3: The Glimpse Architecture Figure 4.4: Main Components of the CEP for the Adaptive Monitor Figure 4.5: Diagram of Interactions during Rule Synthesis Figure 4.6: A Deployment Configuration of the Monitoring Infrastructure FP XIV

15 List of Listings 4.1 Business Service Monitor WS Notification Subscription Business Service Monitor- Connecting EasierBSM to EasyESB node The Complex Event Rule XSD Meta-Rule Example Generated Rule : SLA violation due to the overload of the hosting node Generated Rule : SLA violation due to an anomaly at Business Service level FP XV

16 FP XVI

17 1 Introduction Deliverable D4.3 is meant as a final self-contained description of the tools and the overall infrastructure supporting the governance and Verification and Validation (V&V) of ULS choreographies within the IDRE. This document is the accompanying documentation of the respective prototypes released as D4.3P and available from the official repository hosted by OW2. D4.3 follows Deliverables D4.2.1 (released at M18) [10], and Deliverable D4.2.2 (released at M24) [11], both sharing the title: V&V tools and infrastructure Strategies, architecture and implementation. Those deliverables provided a detailed description of the goals, motivations and research contributions behind the development of the various tools produced in WP4. We provide here a concise overview of the tools: this overview is a short summary of the extensive descriptions reported in D4.2.1 and D4.2.2 [10, 11], to which we refer for more detailed information. The tools are here described individually, and indeed each of them could be considered as one standalone utility satisfying different needs. Of course, they are better exploited when used in combined way, because the respective offered supports are complementary. The integration of the tools into the IDRE is performed within WP5, and we refer to the deliverables from WP5 for the user manuals. In this document, after a short overall description, for each tool we report explicitly what are the advances made in the last reporting period following the M24 release. Finally, given the emphasis of on ULS, we had included in D4.2.2 a discussion in tabular form of what scalability issues apply to each proposed technique and how we address them. We reinclude here, for the sake of completeness, those tables, either verbatim or with little revisions where needed Conceptual architecture As in D4.2.2, to introduce and contextualize the tools we refer to a schematic overview of the developed components given in Figure 1.1, which provides a sort of conceptual architecture of the WP4 framework. We reflect in this abstract architecture the organization of the components according to the different stages of a choreography lifecycle to which they apply. In particular, the components belong to three main categories and are accordingly packaged into three main subsystems: 1) the package Development-time V&V infrastructure collects the components devoted to support the testing activities in an off-line setting at design time; 2) the package Run-time Governance and V&V infrastructure collects the components supporting the extension of the verification phase along run-time activities. Run-time V&V is pursued both through testing and monitoring, and can concern both functional and non-functional aspects. We remark that we distinguish between on-line testing, that actively launches test cases and observes outcomes, from monitoring, that passively observes how the services spontaneously interact; 3) the package V&V Enabling Components collects those components implementing methods and notations that support the above activities, including test case generation and non-functional requirements specification. FP

18 Figure 1.1: Conceptual Overview of Governance and V&V Framework 1.2. Advances in the Reporting Period Although the framework released at M24 (see D4.2.2 [11]) was already at an advanced state, several specific improvements, both on technical and research aspects, have been achieved in the elapsed six months, involving all released components. Most advances have been related to improve the scalability and efficiency of the components, and include: more powerful analyses available from the Scalability Explorer component of the Rehearsal framework; so-said apprehensive version of Q4BPMN that is capable to raise warnings of violations for implicit non-functional constraints; more scalable version of ServicePot, based on distributed registration and lookup; technical enhancements to CRank, supporting its integration with both ParTes and ServicePot; completely renewed algorithms behind ParTes, to make it more efficient; technical enhancements to Resource Lifecycle Manager, in order to make it more stable and standard-compliant; several technical improvements to the implementation of the Multi-Source Monitoring infrastructure, including also an enhanced adaptive version of the Event Monitor part. Overall, effort has been devoted to improve integration of these tools within the IDRE. This is part of the collaboration between WP4 and WP Roadmap The deliverable is structured as follows: FP

19 the next chapter covers tools and approaches to be applied at development-time of choreographies, namely the final versions of Rehearsal (which includes Scalability Explorer) and of the Q4BPMN profile. Chapter 3 describes the tools and approaches supporting (active) testing of choreographies at run-time, including enhanced versions of ServicePot, CRank, ParTes and Resource Lifecycle manager. Still at run-time, but in passive mode, we support choreography behaviour validation via monitoring. In Chapter 4 we present the refined version of the Multi-source Monitoring Framework. Finally, in Chapter 5 we draw conclusions, by performing a brief qualitative self-assessment of achieved results with respect to DOW objectives and hint at future work towards the next and final WP4 deliverable at M36. The APIs for the new classes of ParTes are reported in Appendix A. FP

20 FP

21 2 Development-Time Analysis and Testing Tools The main components of the Development-time V&V infrastructure include the Rehearsal framework and the Q4BPMN profile. The former provides an environment supporting the automation of correctness and scalability testing, that can be applied during the development process of choreographies. It is described in Section 2.1. The latter is a UML profile enhancing the standard BPMN2.0 notation, which is adopted in for choreography specification, to also specify non-functional requirements. Q4BPMN is presented in Section Rehearsal Developing complex distributed compositions of web services is a difficult task, and creating scalable choreographies is an even more challenging endeavor. Thus, effective tools for testing the correctness and scalability of choreographies are required for the development of robust large-scale systems. Rehearsal aims at applying Test-Driven Development (TDD) to choreographies to facilitate their development and widen their adoption. It also supports scalability testing. Using the framework, the developer can manipulate service instances in a cloud environment, execute the defined scenarios, and assess their behavior and scalability Overall Description Rehearsal is a framework for automated testing of web service choreographies. It supports correctness and scalability tests. Concerning correctness tests, Rehearsal supports unit and acceptance testing of Soap/WSDL and RESTful services, through dynamic clients. The other functionalities are restricted to Soap/WSDL services. They are the message interceptor, compliance testing and service mocking, used in integration tests, as well as the abstraction of choreography, that delivers Java object to represent the choreography elements, such as services and roles, allowing tests to be written regardless of service endpoint information, even before implementing the choreography. These components are finished and have been described in Deliverable D4.2.1 [10]. Additionally, a TDD based development methodology was proposed an evaluated in an exploratory study, presented in Deliverable D4.2.2 [11]. Rehearsal also includes Scalability Explorer, which assists the verification of a choreography scalability characteristics. It encapsulates the skeleton of a scalability test and includes hot spots that can be used to specify how to scale workload and system architecture during the testing, how to distribute the requests over time, and how to handle the metrics the framework collect for analysis. A small set of implementations for these hot spots are included in the Scalability Explorer, but its extensible architecture allows for customization according to user needs. Using the provided implementations, the user can write a scalability test with very little code specific to the integration with the system to be tested. In the following section, we present the enhancements made in the Scalability Explorer in the final release. FP

22 Enhancements in the Final Release The main issue addressed for this release of Scalability Explorer was related to the analysis of data collected during a scalability test. Previously, the only assessment allowed by Scalability Explorer was analyzing an aggregated performance chart that could be plotted at the end on an experiment. The framework was made more flexible by creating a new hot spot where the user can set an analysis component. With this improvement, a small set of analyzers was implemented that allows for three common types of scalability analysis: Speedup: used when an experiment varies only the system architectural capacity, for instance, increasing the number of service instances, or using nodes with greater computational power, while the workload keeps the same. The goal is to verify how the performance improves as the architectural capacity grows. Degradation: used when the workload increases while the architectural capacity remain unchanged. This is used to verify how the system performance is affected by the workload variation. Aggregate performance: used when workload and architectural capacity grow in parallel. The performance metric measurements collected in each iteration of the experiment are aggregated in a single value (e.g mean or percentile) and plotted in a chart. Although commonly used, a drawback of these analysis techniques is that the decision whether the tests succeeded is left to the user. To overcome this issue, we implemented an analysis of variance (ANOVA) [12] component, used to test, when we have several populations, if the means of the populations are equivalent. Then, after executing an experiment with a sequence of iterations where several requests are executed and measured, this component can verify if the mean performance obtained in each iteration is equivalent. If so, we can consider that the test succeeded. Otherwise, it failed. The workflow of a scalability test is shown in Figure 2.1. To use the framework, a programmer must extend an Experiment and implement methods to specify the interaction with the system been tested. The ScaleCaster is used to increase the scalability dimensions before each iteration. The other items presented in the diagram are interfaces or abstract classes for components that can be connected to the experiment to define its behavior. The Scalability Explorer includes at least one implementation for each, and they can also be implemented according to specific needs. Additionally, the Scalability Explorer is now integrated with the Enactment Engine, allowing to write scalability tests that automatically deploy services in a cloud. It is made by extending a Deployer implementation that encapsulates an Enactment Engine client. The programmer must overwrite a method that sets the choreography, called in the beginning of the experiment, and another that scales it, called before each iteration Scalability Dimensions, Issues, and Strategies The Rehearsal Framework has two facets, including functionalities for correctness tests, as well as scalability tests. The functionalities for correctness test will mostly not be used in a Ultra-Large-Scale (ULS) scenario. Nevertheless, in some cases it may be desired to perform integration tests involving a higher amount of services. On the other hand, the Scalability Explorer can be more commonly used in ULS scenarios, to validate if a choreography can handle a large frequency of messages. Table 2.1 summarizes how Rehearsal relates to these scalability dimensions Q4BPMN Some of the effort of the project has been centered around the Q4BPMN concept and profile. The idea is based on the underlying assumption (which has received a lot of attention over FP

23 Figure 2.1: Scalability Test Execution Flow ULS Dimension Number of different kinds of services in a single choreography Frequency of messages (requests, operations or transactions) exchanged by the services in time period Consideration about Rehearsal It must be possible to generate mocks and message interceptors for the number of different kinds of available services in the choreography. Since Rehearsal s standard behavior is to deploy mocks and interceptors locally, we must assess the impact of having a large number of these services running on a single machine. However, they were developed to be light and common test scenarios require just a few instances. It must be possible to generate ULS frequency of messages in order to support testing this dimension. The load generator was updated to a multi-threaded implementation that also supports new request distributions over time, including Poisson, Gaussian and Random. While this provides good scalability properties, if necessary the implementation may be changed to support distributed load generation. Table 2.1: ULS dimensions and Rehearsal the past years) that a service, to comply with the needs of users, must not only provide the requested functionality, but also satisfy some requirements relating to its efficiency. Such requirements might concern the service s security, its performance, and the guarantees it offers about its availability and correctness. These are collectively called non-functional requirements, and are the issues addressed by Q4BPMN. Non-functional requirements are generally expressed in some form of contract between the client and the service provider, generally as a Service-Level Agreement (SLA) specification. The reason behind Q4BPMN is that BPMN, the mainstream language used for specifying service choreographies and also adopted in, revolves around the interactions between the partners from a functional perspective, and it does not allow, apart from a very minor extent, to add non-functional requirements to the specification. But it is clear that non-functional requirements are part of the service composition no less than the functional specification, and the overall description of the service composition must cover both families of requirements. FP

24 Overall Description The basic idea was to add non-functional requirements to the BPMN specification language. Clearly, this purpose could be achieved by way of several different means. The chosen solution was based on the definition of a UML profile (also following advice of the project reviewers). Normally, BPMN tools define BPMN choreography diagrams as a profile of UML activity diagrams, and the chosen tool for the development, MagicDraw by NoMagic, made no exception to this. This suggested to implement the specification of non-functional requirements as an additional profile, with stereotypes to describe the type and content of each requirement. This solution is very efficient because it integrates smoothly into existing BPMN tools, without the need to define an extension of the BPMN language (which would require the tools to be modified accordingly). Q4BPMN is not a language for expressing the quality requirements, but rather provides a placeholder within the BPMN diagrams for expressing the specific quality requirement into some specific language. Specifically, Q4BPMN implements the concepts defined by PMM [14] as a UML profile. Thus, Q4BPMN covers the definition of three classes of non-functional requirements: performance; security; dependability. Each non-functional requirement can be associated to a single choreography task, to a role within a task, or to an entire choreography. In other words, the BPMN specification is enhanced with nonfunctional annotations which provide the information on the non-functional requirements. Non-functional annotations, in short, are a representation of the contents of the SLA, expressed in the same diagram as the functional specification. The benefits of this added expressiveness have been already expressed in D4.2.2 [11]. Of course, the purpose is not merely declarative, otherwise there would be very little added value with respect to the SLA. The combined diagram containing the functional specification and the nonfunctional annotations can be used for the purpose of validating the service composition and verifying its compatibility with the non-functional requirements. Some analysis can be carried out to detect whether the requirements are too strict or incompatible, and what should be expected by the actors playing the roles in the choreography Enhancements in the Final Release The most recent updates to the research on Q4BPMN [6] are centered on the generation of runtime monitoring warnings descending from unspecified non-functional requirements. In a nutshell, nonfunctional requirements can be expressed using the Q4BPMN profile, but due to the desired loose coupling between the services and the providers, the choreography designer might opt for a relaxed specification, in which not all choreography elements are annotated with non-functional requirements. But the lack of annotations on a choreography task does not mean that task has no constraints from a non-functional perspective: there might well be some non-functional requirements on the overall choreography that are inherited by the unannotated task. The respect of the overall requirements will of course depend on the runtime behaviour of both annotated and not annotated tasks. Our proposed solution is therefore to create warning rules for the monitor, rules that will issue a warning to the relevant stakeholder (e.g., a choreography governing board) in case the runtime execution of some services is viable to cause the whole choreography to fail its (expressed) requirements. This is achieved using the KLAPER language [16], which is a pivot language between modeling and analysis tools. The language is implemented in the KlaperSuite, a tool which is capable of having FP

25 as an input (among other formalisms) the Q4BPMN-annotated BPMN choreography, and translating it into several languages used in analysis tools. Some algorithms implemented within the KlaperSuite (described in detail in the publication [6]) generate the warnings for the monitor computing them from the annotated non-functional requirements. The whole process is summarized in Figure 2.2. Figure 2.2: Q4BPMN: Generation of Warning Rules Currently, the methodology has been finalized and the KlaperSuite tool is enabled to translate Q4BPMN non-functional annotations into monitoring rules for the monitor. Work is in progress to implement the methodology into a full-fledged use case, using the airport choreography from the deliverable D6.2 [13] as a test bed. The results will be reported in the next D4.4 deliverable Scalability Dimensions, Issues, and Strategies The Q4BPMN approach fits well into a scaling ULS environment, but additional care and effort is needed. It is assumed that the non-functional requirements are not created in the Q4BPMN approach; rather, the approach only requires converting existing non-functional requirements into a PMM format and annotating them on the model of the service choreography. The main issue is therefore the conversion, but this process can be automated. If the requirements are specified in a formal non-functional requirement language, or an MDE technique, a transformation script or model transformation tool can be applied to automatically introduce the annotations into the service choreography model. In other words, the first step of the Q4BPMN approach could be a major manual effort when the service choreography grows to a ULS environment, but with an automatic transformation no difference should emerge in scaling. The second step again has no difficulty in scaling. There is already a tool for transforming the nonfunctional requirements from the Q4BPMN format into the KLAPER pivot model, and again from the latter to the formats supported by the various analysis tools. This transformation is immediate (the computation time may be a bit longer, but it is only a translation), independently of the size of the service choreography. The third step does not change qualitatively, but it does change quantitatively. Some computations need to be performed to check the compatibility of different non-functional requirements, and in a ULS FP

26 environment this might be intensive. However, since all the computation is carried out prior to the deployment of the service choreography, it just lengthens the deployment time and does not affect the QoS at run-time. The last step changes significantly. As the choreography scales towards a ULS environment, more non-functional requirements will be expressed, but many others will be omitted as well. So the number of implicit non-functional requirements will tend to grow exponentially as there are more constraints which are not expressed in SLAs. This has several consequences: the computation can become much more intensive, since an exponentially-growing number of possible combinations of explicit non-functional requirements must be accounted for to compute the implicit ones; the precision of the implicit non-functional requirements will be more and more approximate. The gray areas left by the unexpressed non-functional requirements will increase, thus making the overall run-time execution of the service choreography less predictable; for the same reason, the number of possible warning rules generated by the tool will increase strongly, probably giving way to a large number of redundant warnings, and it would be hard for the service monitor to separate the useful ones from those that should be omitted. Table 2.2 summarizes the estimated effects of each aspect of scalability on the efficiency of Q4BPMN. FP

27 ULS Dimension Number of different kind of available services Number of different kind of services in a single choreography Number of service consumer instances in a single choreography (i.e., for a given kind of service consumer, we may have a UL number of instances) Frequency of messages (requests, operations or transactions) exchanged by the services in time period Number of atomic services (actuators and sensors) aggregated in services Change of the above dimensions over time Consideration about Q4BPMN This dimension represents the amplitude of the scale of the whole ULS system and affects the amount and value of nonfunctional requirements involved. However, it does not involve the processing of those requirements, which would be carried out independently of their values. The increase in the number of services has a major effect on the number of non-functional requirements of different classes expressed in the service choreography. A very large scale along this dimension would make implicit non-functional requirements and service monitoring rules much more complex and approximate. Since the Q4BPMN methodology is not affected by consumers invoking the services choreography, the approach can be applied independently of the growth of consumers. Q4BPMN does not involve the communication among services (apart from the constraints that can be expressed related to them), so this dimension is not an issue. This dimension is not an issue for the Q4BPMN methodology, since only the non-functional requirements expressed in those services are accounted for. Although the Q4BPMN methodology in itself does not change when the service choreography is reconfigured, still it needs to be run all over again, with possible effects on performance, especially in a ULS environment. Also, during the transition phase, some warnings which are not significant anymore might be issued by the service monitor, and others might be missed because they are in the process of being formulated. An extensive study on the dynamic evolution of the service choreography subject to the Q4BPMN methodology has not been faced as of yet. Table 2.2: ULS dimensions and Q4BPMN FP

28 FP

29 3 Run-Time Testing Tools In this chapter we report about the set of tools developed in WP4 to support the on-line testing of choreographies. We recall that on-line testing is meant as the activity of launching test cases on enacted choreographies, and on the composing services, and is meant as a proactive verification activity different from the plain passive monitoring of their spontaneously emerging behaviour. The latter is described in the next chapter. On-line testing tools include: the ServicePot component, described in Section 3.1, an enhanced registry allowing for managing governance and testing of logged services and choreographies; the CRank component, described in Section 3.2, which implements mechanisms for rating both services and service choreographies, using both objective and subjective indicators; the ParTes component, described in Section 3.3, which is the proposed framework for test derivation and execution of choreography model-based testing; the Resource Lifecycle component, described in Section 3.4, for the efficient publication and discovery of heterogeneous resources such as services, SLAs, and choreographies. It is worth to recall that the two components ServicePot and the Resource Lifecycle Manager, which provide extensions to the canonical functionality of a service registry, are complementary and both part of the Abstraction Oriented Service Base Management (See Deliverable D3.2.2 [8]) providing pluginoriented structure ServicePot ServicePot is a Registry for Web Service Choreographies and Web Services, relying on an increased information model of a classical service registry. In particular, ServicePot enables the storage and the retrieval of choreography specifications. Additionally, it provides an extensible architecture which can be used to enhance the functionality offered by a canonical registry, and include important features like testing, analytics and scalibility Overall Description This section describes the overall architecture of ServicePot. The framework relies on an increased information model of a classical service registry. In particular, ServicePot enables the storage and the retrieval of choreography specifications. It assumes that a service willing to be indexed in the registry specifies which roles it is able to play within one or more choreographies. In this first implementation we decided to take this simple path; nevertheless more complex functionality can be implemented as extensions to the basic one. Therefore the implementation we are considering is somehow limited by the following characteristics: it provides an API that permits to directly associate a role to a service implementation. FP

30 Figure 3.1: ServicePot: High Level Architecture indexing of choreographies does not use any grouping technique and lookup mechanisms are mainly based on those provided by the UDDI implementation. Both limitations can be addressed thanks to the ServicePot extension mechanism described in the following sections. In particular it is possible to include mechanisms for dynamic and automatic mapping of services to roles and complex indexing functionality possibly using metadata structures. Figure 3.1 depicts the overall architecture of ServicePot, which is described in detail below. ServicePot ServicePot is a UDDI compliant SOA Registry, enhanced with additional information to store Web Service Choreographies. ServicePot was first introduced in D4.2.1 [10] as a tool to enable run-time Verification and Validation of Web Services and Choreographies. UDDI Registry At its core, ServicePot is based on a UDDIv3 registry UDDI Proxy ServicePot contains a proxy for that UDDI registry that would actually store the data. This proxy receives standard calls from external callers and forwards it to the UDDI Registry ChorDDI ServicePot provides primitives for publishing a service choreography: store and drop a choreography; attach and drop a specific role to a specific service (i.e. participant of a choreography). Furthermore, it also exposes primitives for inquiring the registry looking for choreographies, roles in a choreography, or participants (i.e. services) that can play a specific role within a choreography. All this functionality is provided by the component ChorDDI. Governance Pool As Service Oriented Architectures are inherently fluid and subject to change, we anticipate that new requirements will emerge in response to those changes. To allow handling all such requirements, we need to make ServicePot extensible. FP

31 The motivation behind Governance Pool is the need to implement such extensibility. At the core of Governance Pool is an OSGi framework implementation [21]. Governance Pool features the architecture by supporting configurable extensions that can enforce governance activities over the life-cycle of the registered items. For instance, The augmented registration process described in D4.2.1 can be implemented using an extension that intercepts the conventional service registration procedure. ServicePot uses an event-driven architecture for extensibility. It raises events on both entering and exiting each operation of its exposed API. Extension to the ServicePot functionality can be done by listening and responding to these events. ServicePot provides events for all invocations on the UDDI Proxy and the ChorDDi components. To allow event handling and enumeration of the event listeners, GovernancePool also includes interface definitions for all the events raised by ServicePot. ServicePot events can be handled by importing and implementing the interfaces for event handlers provided by GovernancePool. Later, the extension handling the events can register with the bundled OSGi implementation. To raise an event, ServicePot looks up from the GovernancePool all the extensions registered as event handler of the currently executing method. It then calls all the event handlers sequentially, passing the data received as input, or the processed data about to be returned to the ServicePot caller. This puts a constraint on the ServicePot extensions to provided the PluginInterface required by ServicePot and to register with Governance Pool Using GovernancePool, we have supported a number of plugins that perform important governance functions. for example, the Partes Plugin invokes the ParTes component to perform conformance tests of choreography participants, CRank Plugin uses the CRank component to rank the services. An extensibility plugin provides support for distributed partitioning of the registry. Abstract Plugin Servicepot Plugins import and implement the event handler interfaces exported by GovernancePool. These events are raised during UDDI and ChorDDI operations, GovernancePool maintains a collection of registered plugins, and the registered plugins are retrieved and invoked for each event. Choreography storage and retrieval ChorDDI is the component responsible for Storage and retrieval of Choreographies. UDDIv3 does not natively support the storage of Web Service Choreographies. However it does allow expression of new concepts using an Object called TModel. We use TModels to store information about choreographies and the roles in the choreographies. Figure 3.2 shows this mapping between BPMN 2.0 concepts and UDDI concepts The overall structure of ChorDDI is expressed in Figure 3.3. Extensibility in ServicePot As discussed above, GovernancePool is the component providing extensibility for servicepot. The various interfaces and events available to ServicePot plugins are expressed in Figure 3.4. Notice that in the figure, most UDDI events are hidden for simplicity. Only one pair of UDDI events is shown as an example. All events raised by ChorDDI are shown Enhancements in the Final Release In the final release we developed the plugin proposed for scalability in ServicePot, by making the registrations and lookups distributed using a global data store. This plugin was proposed as future work in D4.2.2 [11] As discussed earlier [11], we use the approach mentioned by [5] to federate the registries at the publication and query level. Similar to what is described by the authors, we use a global distributed FP

32 Figure 3.2: ServicePot: Mapping of BPMN2 concepts to UDDI Figure 3.3: ServicePot: The ChorDDI Component FP

33 Figure 3.4: ServicePot: Plugin Interfaces and Events data store to index the entities in the UDDI registry. To implement this federation we utilized the extension mechanism provided by ServicePot (see details in D4.2.2 [11]). This plugin implements the approach described in [5]. The flow of the Publish Choreography and Find Choreographies use cases, when the plugin is installed, is described in the sequence diagrams in Figure 3.5 and Figure Scalability Dimensions, Issues, and Strategies ServicePot faces the same scalability challenges as any other web applications. We solve the scalability problem by distributing the ServicePot application and data over multiple nodes. Table 3.1 summarizes the main dimensions of scalability as highlighted in D1.4 (see [18]) expanded in relation to ServicePot CRank The V&V framework provided by the project implements a comprehensive strategy for managing both choreographies and services. Specifically, as introduced in Section 3.1, the framework put a special emphasis on governance aspects related to choreography-oriented V&V activities supporting an on-line testing techniques when a service/choreography asks for registration within a registry. In deliverable D4.1 [9], we elaborated on a potential set of governance policies and rules relevant during the definition and the implementation of V&V framework with on-line capabilities. Among the others, D4.1 identified which aspects have to be considered for rating the quality of both choreographies, and services. In this sense, CRank is a ranking framework implementing the rules and the metrics for computing FP

34 Figure 3.5: ServicePot: Distributed Publishing quality parameters for both services, and choreographies. As reported in the following, it supporting several strategies for calculating such ranking functions Overall Description CRank is a ranking framework for both services and choreographies. Specifically, it supports two main computational ranking models : objective ranking indicators (i.e. testing scores from on-line testing session launched from ParTes (see Section 3.3); and ranking indicators based on subjective evaluation (i.e. feedbacks form the users). As discussed in [4], both of those ranking models can be used as a criteria for selection of choreographies for a particular requirement, or services to play a choreography role. In addition, they can influence SOA governance decisions leveraging the definition of those (minimum) criteria granting for some QoS requirements according to which systems and their owners should behave to ensure a mutual and sustainable cooperation. Furthermore it can be used in order to deal with properties on the whole choreography (e.g. determining if a choreography is enactable ). A first release of this component was given in Deliverable D4.2.1 [10] and Deliverable D5.2 [7] as Reputation Center. D4.2.2 [11] further developed those concepts by proposing a refined implementation of the main elements implementing CRank (i.e. sub-components, interfaces, and interconnections), by also providing a detailed description of its API Enhancements in the Final Release The main enhancements in this final release were aiming at improving the integration level between ServicePot and CRank, and between CRank and ParTes (see 3.3). FP

35 Figure 3.6: ServicePot: Distributed Lookup FP

36 ULS Dimension 1. Number of different kind of available services 2. Number of different kind of services in a single choreography 3. Number of service consumer instances in a single choreography (i.e., for a given kind of service consumer, we may have a UL number of instances) 4. Frequency of messages (requests, operations or transactions) exchanged by the services in time period 5. Number of atomic services (actuators and sensors) aggregated in services 6. Change of the above dimensions over time Consideration about ServicePot This dimension represents the amplitude of the scale of the whole ULS system. This is the main motivation of distributing the data storage in ServicePot This dimension affects the use case of listing all possible participants for a role. This can cause a large communication overhead in this usecase if the participants are distributed. We will need to minimize the distribution of the services performing a role whenever possible As this dimension increases, we will see an increase in the number of lookup requests in ServicePot The message exchange taking place in the execution of the services is irrelevant for ServicePot ServicePot currently does not distinguish between aggregate and atomic services, so this dimension is irrelevant at the moment Depending on the capabilities of ServicePot nodes, a scaling by 1000 could necessiate introducing new nodes and partitioning existing data. Table 3.1: ULS dimensions and ServicePot Most of the integration takes place when ServicePot stores either a new choreography, or a new service. Specifically, in the first case, the plug-in wrapping CRank to ServicePot (see Section 3.1, and [4]) implements the following steps: create the Dependency Matrix allocate resources in CRank for the reputation-based ranking of the choreography allocate resources in CRank for the testing-based ranking of the choreography Then, in the second case, when a new service subscribes in ServicePot, the plug-in includes it the procedure calculating the reputation/testing-based ranking. The computation is activated for all the roles the service referred in the category bag (i.e. the structure ServiceDetail [11]) from the registration process. With respect to the integration between CRank, and ParTes, basically the integration is realided within the CRank plugin for ServicePot by querying the ParTes component dedicated to the execution of the test cases (i.e. the Test Execution component) [11]. Specifically, this component exposes its offered functionalities by menas of the PartesTestRunner interface, and it relies on a library from soapui [15] in order to actually launch the test suites. Assessment about the cooperation of these three components is reported in [4] Scalability Dimensions, Issues, and Strategies CRank faces the same scalability challenges as most of the other solutions based on the Web Services technologies. Specifically, CRank tackles some of the ULS dimensions addressed in by FP

37 clustering the implementation over multiple nodes; for example in order to provide both high availability, and scalability of the service. Specifically, high availability refers to the ability to serve client requests by tolerating failures; while scalability is the ability to serve a large number of clients sending a large number of requests without any degradation to the performance. In this sense, it is difficult or in some cases impossible to support massive level of scalability and high availability from a single server, despite how powerful the server hardware or how efficient the server software is. Web services clustering allows to deploy and manage several instances of identical web services across multiple web services servers running on different server machines. Then, client requests are distributed among these machines using a suitable load balancing system. As introduced above, clustering is an asset that is usually provided by any platforms for Web Services. Thus most of the solutions among the several platforms are similar. Nevertheless, as the current prototype is developed for Axis2 1, CRank relies on the features offered by this specific platform. Specifically, to achieve scalability, an Axis2 cluster will be managed by a load balancer. This module is configured in order to spread the load across the cluster according to some load balancing algorithm. In this sense, it would be possible to rely on canonical configurations available from the Axis2 community. For example, usually, the round-robin load balancing algorithm is one such popular and simple algorithm. However, the ULS dimensions identified within CRank (see Table 3.2) ask for more specific approaches; for example the computations calculating the overall rank of such roles are based on matrix multiplications involving the Dependency Matrix. Indeed, the size of the Dependency Matrix derived from the choreography specification impacts on the computational cost for each specific choreography referred. Thus both the Axis2 load balancer, and its other clustering mechanisms will be configured aiming at allocating and distributing the ranking calculation of choreography on a specific instantiation of the CRank service in the cluster by the size of the Dependency Matrix. In such a way, it is possible to allocate the computation of the choreography ranks according to the specific needs that any enactment may dynamically require at run-time. Also, the number of nodes in the cluster is a function of the number of choreographies currently available (e.g. because they have been registered on registries like ServicePot see Section 3.1). According to its official documentation, Axis2 supports several membership schemata for the node binding a cluster. Specifically, those schemata includes : the static membership only a defined set of members are in the cluster, and they are the only ones that can be dynamically activated/deactivated; the dynamic membership does not define any predefined set of cluster members. Nodes can join or leave a cluster by querying a specifics entity within Axis2 specifying the appropriate group name; the hybrid membership where there are a set of static members (i.e. well-known members ), but external nodes can join this group by notifying one of the well-known members. This last schema looks the most interesting solutions facing the dimensions of ULS defined by CRank within the context of the project. The Table 3.2 summarizes the main dimensions of scalability in relation to the CRank ParTes In this section we describe the status of the implementation of the ParTes tool. The aim of the tool is the derivation of a test skeleton for the participants involved in a choreography. The derived tests aim at discover possible mismatches of the service behaviour with respect to the behavior foreseen by a choreography role. Mismatches can prevent the integration in the choreography bringing to runtime errors. Within integration mismatches are also addressed by the synthesis approach, nevertheless ParTes complements the synthesis of choreography delegates for two different aspects: 1 FP

38 ULS Dimension 1. Number of different kind of available services for a specific role 2. Size of the Dependency Matrix derived from the choreography specification 3. Number of service consumer instances in a single choreography (i.e., for a given kind of service consumer, we may have a UL number of instances) 4. Frequency of messages (requests, operations or transactions) exchanged by the services enacting the choreography in time period 5. Change of the above dimensions over time Consideration about CRank Each participant defined in a service choreography represents a specific role that many services may subscribe for implementing it. Thus, if on the one hand we do not necessary refer to choreographies composed by a large number of tasks, on the other hand we see that this dimension is the amplitude of the scale of the whole ULS system. This is the main motivation of clustering and distributing the resources implementing by the CRank service. The Dependency Matrix used by CRank represents the relations among the roles specified in the choreography. Thus, most of the computations calculating the overall rank of such roles are based on matrix multiplications involving the Dependency Matrix. Indeed, a choreography that specifies many roles (i.e. roles not services bind a role) may require appropriate strategies for dealing with scalability issues. The current version of the prototype CRank does not explicitly support this aspect yet. Nevertheless, for further improvements of the approach it could be possible to refer/exploit the scalable solutions (e.g. matrix reduction techniques) provided by the communities specifically working either on Linear Algebra Optimization, or on Scientific Calculations. This dimension concerns the number of services binding a role of choreography. Simplifying, CRank represents a role as an array of the rank of the services. Indeed, at the moment, this dimension is considered irrelevant, as the computational costs associated to the calculation of the rank of a role is linear with respect to the number of services binding it. The message exchange taking place in the execution of the services is irrelevant for CRank Depending on the capabilities of the nodes available for the instantiation of the cluster of implementing the CRank service, a scaling by 1000 could necessitate introducing additional nodes. As described above, for this specific aspect, the prototype we developed relies on the flexible clustering features offered by the Axis2 2 framework. Table 3.2: ULS dimensions and CRank Permits to identify issues related to wrong specification of models used for synthesis, with respect to real code Provides skeletons useful to highlight possible mistakes in the usage of data structures As pointed out in Deliverable D.4.2.2, the tool no longer uses a model checking based technique in order to select interesting executions traces that could better highlight integration issues in BPMN2 choreographies. The JPFRunner component has been replaced by a component based on an ad-hoc algorithm conceived within. While using model checking techniques to check the possible interactions can seem a good solution, it easily suffers by the state-explosion problem. Our objective FP

39 has been to derive a new testing strategy that, taking in input a choreography specification, is able to derive skeleton test cases for the different participants, or subset of them, in order to check if a given service could play a role within a choreography enactment. The element of novelty of our approach is considering dependencies we defined among interleavings to select just traces we consider "good" to create tests, instead of generating all as a standard model-checking approach would do. Next sections will explain this new strategy adopted Overall Description the new strategy A service that takes part in a choreography plays a role that should be tested considering all possible scenarios of integration allowed by the choreography considering also the other roles involved. The new algorithm deals directly with the BPMN2 concepts instead of exploring the JPFModel at java code level. In particular the focus is on managing the parallelism between ChoreographyTask s that can affect communication between roles. A way to know how an interaction could change the internal state of a service is to derive the flow of messages exchanged with the other participants according to the choreography specification. Assuming that, in general, messages received from a sender can alter the internal behavior of the receiver, we say that there is a potential message dependency between them. Our approach intends to identify those interaction traces which are foreseen by the choreography specification, and that can lead to different behaviour of the involved services. In this respect, the algorithm does not explore those traces which can identify a different interleaving of the exchanged messages but nevertheless does not lead to a possible different behaviour for the involved services. Figure 3.7 shows a simple example of a parallel gateway that generates two branches in the choreography. The ChoreographyTask s within each branch can be executed in parallel and we are potentially in a situation where interactions that take place in a ChoreographyTask may affect other interactions. In such situations it is important to consider these potential interleavings that can represent hazards to a proper execution of the choreography. Conversely, if the ChoreographyTasks performed in parallel do not have hazardous conditions is not necessary to generate all the interleavings as these will produce equivalent final results (i.e. equivalent service status). In Figure 3.7 there are no dangerous conditions. A classic model checker will produce all possible interleavings but not all are useful for the test generation because ChoreographyTask s describe independent interactions. Our algorithm instead select just one trace without considering all possible interleavings. the data dependency We must consider interleavings when the dependencies can cause problems in the execution of the choreography. There are different patterns of how dependency between parallel ChoreographyTask s ("CT" for short) may occur and lead to possible issues which we want to test. Each pattern, taken individually, produces a different number of execution traces: receiver-sender pattern: this is the case in which a participant is the "receiver" in a CT and the sender in another CT performed in parallel. The execution order is potentially important because the received data could alter the internal state of the service and hence lead to a different interaction in the other CT. This situation is depicted in Figure 3.8. same-receiver pattern: this is the case in which a participant is the "receiver" in more CTs performed in parallel. The execution order is potentially important because the received data could alter the internal state of the final service. This situation is depicted in Figure 3.9. closed chain pattern (or cyclic dependency): this is the case in which the order of CT execution may be significant because of the fact that some participant are "sender" and then "receiver" in FP

40 No dependency + P1 CT1 P2 P1 CT3 P6 P2 CT2 P4 P6 CT4 P8 MC traces reproduce all possible interleavings, but not all are useful for testing if CT are independent. In this case our algorithm consider all the traces as equivalent for testing. - --> CT1 --> CT2 --> CT3 --> CT4 - --> - --> CT1 --> CT3 --> CT4 --> CT2 - --> - --> CT1 --> CT3 --> CT2 --> CT4 - --> - --> CT3 --> CT4 --> CT1 --> CT2 - --> - --> CT3 --> CT1 --> CT2 --> CT4 - --> - --> CT3 --> CT1 --> CT4 --> CT2 - --> Figure 3.7: ParTes: Simple Parallel of ChoreographyTasks without Dependencies "receiver-sender" pattern + P1 CT1 P2 P2 CT2 P3 Data dependencies are detected and only the relevant interleavings are reproduced in traces. We need to check different hazards. - --> CT1 --> CT2 - --> - --> CT2 --> CT1 - --> Figure 3.8: ParTes: receiver-sender Pattern Example other CT executed in parallel (and vice versa) in a cyclic manner. This situation is depicted in Figure Clearly, the dependencies can arise in more complex ways mixing the patterns across several nested parallel gateways. An example is showed in Figure The new algorithm works building a graph ables to describe all the structural constraints (due to sequences of CTs) and dependencies found in the BPMN2 specification of a choreography. Exploring this graph, the algorithm generates all the traces that will be used to generate test skeleton the tester can use to identify ordering issues about message exchange. Tests will be generated for services willing to play a role within a choreography Enhancements in the Final Release This section analyze the details of the new algorithm used to generate traces suitable to build skeleton test cases. The algorithm handles the BPMN2 specification as a graph, refactoring and reducing it to FP

41 "same-receiver" pattern + P1 CT1 P2 P9 CT2 P2 P2 CT5 P10 We need to check how the receiver P2 could change its internal state, so we try different sequences of received messages. CT5-CT1-CT2 and CT5-CT2-CT1 are considered equivalent. - --> CT1 --> CT2 --> CT5 - --> - --> CT1 --> CT5 --> CT2 - --> - --> CT5 --> CT1 --> CT2 - --> - --> CT2 --> CT1 --> CT5 - --> - --> CT2 --> CT5 --> CT1 - --> Figure 3.9: ParTes: same-receiver Pattern Example "closed chain" pattern + P1 CT1 P2 P2 CT2 P3 P3 CT3 P1 - --> CT1 --> CT2 --> CT3 - --> - --> CT2 --> CT3 --> CT1 - --> - --> CT3 --> CT1 --> CT2 - --> Figure 3.10: ParTes: closed chain Pattern Example eliminate any parallelism. From BPMN to Workflow Graph The core part of the ParTes strategy aims to reduce BPMN2 ParallelGateway statements to BPMN2 ExclusiveGateway statements which include just CTs execution traces considered not equivalent according to the message dependency defined in the previous section. These traces will be re-organized in a tree data structure (interaction tree) in which each root-leaf path will represent a possible complete choreography execution trace used to derive test skeletons. Our reduction strategy considers FP

42 + P1 CT1 P7 P1 CT3 P4 P7 CT2 P8 P4 CT4 + P3 P4 CT5 P7 P3 CT6 P4 + + Figure 3.11: ParTes: Example of Complex Dependencies in a Choreography the choreography specification in a formal and technologic-independent way as a Workflow Graph G = (N,T). A Workflow Graph is a simple directed acyclic graph (DAG) where N is a finite set of nodes and T is a finite set of transitions, representing directed edges between two nodes. In this structure for each transition t T we define the two mappings fromnode[t] and tonode[t] which return the incoming node and the outgoing node of the transition t respectively. Moreover, for each node n N we have: nodet ype[n] {T ASK, OR, P ARALLEL} represents the type of a node. T ASK stands for a ChoreographyTask, while OR and P ARALLEL stand respectively for ExclusiveGateways (decision point) and ParallelGateways. OutT rans[n] = {t : t T fromnode[t] = n} and InT rans[n] = {t : t T tonode[t] = n} are the sets of outgoing and incoming transitions to/from n OutNodes[n] = {m : m N t T s.t. (fromnode[t] = n tonode[t] = m)} and innodes[n] = {m : m N t T s.t. (tonode[t] = n fromnode[t] = m)} are the sets of succeeding/preceding nodes that are adjacent to n. The workflow graphs we consider in this paper are well-structured, i.e. there are matching pairs of nodes that split and join the flow. Moreover, they do not contain cycles. Their presence in the original choreography will be reduced exploring the cycle for a bounded number of iteration. By default our approach makes a single unfolding of each cycle, nevertheless the tester could decide to unfold each cycle with higher values. An example of a Workflow Graph is shown in Figure The refactoring of a Workflow Graph Given a workflow graph, we start a reduction procedure to exert the CT execution traces that are allowed by the relative choreography. To do this we need to isolate fragments of the choreography characterized by a Single Entry and a Single Exit (SESE) boundary nodes that can be easily replaced with other equivalent fragments without parallel processes. This decomposition in fragments can be performed building a Refined Process Structure Tree (RPST) that consists in a hierarchy of sub-workflows. The hierarchy represents an ordering in the fragments to be used in our reduction procedure and we call reduction set the set of reducible fragments (that are leaf in the hierarchy) that have as entry and exit point a ParallelGateway. Figure 3.13 shows the RPST relative to the workflow graph of Figure The refactoring of the Workflow Graph is performed applying a set of rules on the deepest fragments of the RPST until the Workflow Graph has just one Exclusive Gateway containing all the relevant traces to test. Indeed, the refactoring process is recursive and each iteration lower the height of the RPST. In the specific, if a fragment is characterized by a Parallel Gateway, it is reduced replacing it with an Exclusive Gateway as stated before. The rules used for refactoring are shown in Figure For the sake of comprehension we reported the rules using a BPMN2 notation. FP

43 BPMN2 specification relative Workflow Graph Figure 3.12: ParTes: Example of a Workflow Graph P1 B0 CT1-> AND P0 P2 AND -> CT3 CT5 -> AND AND -> CT2 CT3 -> CT5 CT4 -> AND CT2 -> CT4 The set of leaves is the fragment to reduce Figure 3.13: ParTes: Example of a RPST While the major of the refactoring rules consist on reorganizing the Workflow Graph moving nodes, the reduction from a Parallel Gateway to an Exclusive Gateway is more complex. The reduction must take into consideration the message dependencies among participants, handling the reduction in an efficient way so to consider only those execution traces that can be relevant for testing purpose. The Figure 3.15 shows an example of how to use the refactoring rules. From a Parallel Gateway to an Affection Graph So far we considered the result of the reduction of a Parallel Gateway in the form of an Exclusive Gateway but technically the algorithms returns execution traces organized as a tree structure (Interleavings Tree) made of nodes labeled by the names of explored CTs. Anyway, the tree can be viewed as a set of nested Exclusive Gateways. To obtain the tree, we need to formalize dependencies as relations and a new structure for them. We define: Definition 1 A structural relation is a relation over CTs that appear sequentially in a choreography. Two nodes a, b N corresponding to CTs, are in structural relation a b if b OutT rans[a]. In other word a b if b is the next CT after a in the same branch of a Parallel Gateway. This relation must be respected in all the execution traces. Definition 2 An affect relation is a relation over CTs that appear in different branches of a parallel FP

44 Figure 3.14: ParTes: Refactoring rules to Refactor a Workflow Graph statement. Two nodes a, b N with nodet ype[a] == nodet ype[b] == T ASK are in affect relation a b if they appear in a "receiver-sender", "same-receiver", or "chain" pattern as introduced previously. We state that a affects b and b is affected by a. We call a the affecter node, and b the affected node. These relations are used to build a new structure from the workflow graph, called affections graph, which contains all the information to manage message dependencies. We will explore it to found all the execution traces to store in the interleavings tree. From now, we refer to the following definitions and notations. Definition 3 An affections graph AG = (CT set, RT set) is characterized by: CT set which is a finite set of ChoreographyTasks that are used as nodes of the graph; RT set = ST AT which is a finite set of transitions where ST and AT respectively represent structural and affect relations between two nodes; Given the AG, for each node n CT set: StOut[n] = {m : m CT set t ST s.t. (f romn ode[t] = n ton ode[t] = m)} are the nodes pointed by outgoing ST transitions from n and StIn[n] = {m : m CT set t ST s.t. (fromnode[t] = m tonode[t] = n)} are the nodes pointed by incoming ST transitions to n. Given that we have max( StOut[n] ) = max( StIn[n] ) = 1, these sets represent the preceding node and the successor node of n considering only transitions in ST. StIn[n] = StIn[n] StIn[m] where m CT set t ST s.t. (fromnode[t] = m tonode[t] = n) is the set of all the preceding node of n reached by structural transitions. FP

45 BPMN2 specification Workflow Graph. refactor-iteration 0 x >= 0 x < 0 Workflow Graph. refactor-iteration 1 x < 0 x >= 0 Workflow Graph. refactor-iteration 2 x >= 0 x < 0 Workflow Graph. refactor-final iteration Figure 3.15: ParTes: Example of Refactoring AtOut[n] = {m : m CT set t AT s.t. (fromnode[t] = n tonode[t] = m)} are the nodes pointed by outgoing AT transitions from n and AtIn[n] = {m : m CT set t AT s.t.(fromnode[t] = m tonode[t] = n)} are the nodes pointed by incoming AT transitions to n. AtIn[n] = AtIn[n] AtIn[m] where m CT set t AT s.t. (fromnode[t] = m tonode[t] = n) is the set of all the preceding node of n reached by affect transitions. Definition 4 An interleavings tree IT = (T N, T E) is a tree where T N is a finite set of nodes each one labeled with one or more CTs and T E is a set of edges. For each node n T N we will use the FP

46 notations child[n], parent[n], with the usual meaning over tree structures. Moreover, we use: ct[n] the list of all the CTs that forms the label of n; graph[n] used in the next algorithms to indicate a copy of AG associated with n. An Affection Graph AG considers as nodes each CT inside the branches of a ParallelGateway. The affect relations among CTs are represented by affect transitions (solid arrows) among nodes. The structural relations instead, are represented by structural transitions ( dashed arrows) among nodes. To add the affect transitions to the AG, an adjacency map is used. We consider this map as an hash table where we can store tuples in the form sender, [(ct, receiver[ct]]) where sender is a key. The value stored for a key = s is the list of pairs (ct, receiver[ct]) such that sender[ct] = s. The addition of the transitions proceeds according to the map. These are added recalling that if two nodes are in the same branch they can not affect each other. In particular, given a pair (ct, receiver[ct]), the ct has affect transitions to all ct i such that sender(ct i ) = receiver(ct). This construction is shown in Figure Fragment of a Workflow Graph XOR CT2 CT4 CT3 CT7 CT5 XOR relative Affection Graph relative Adjacency Map for affect transitions P3 [(P1,CT3) ; (P2,CT4)] P4 (P3,CT5) P1 (P2,CT2) P6 (P3,CT7) Figure 3.16: ParTes: Example of an Affection Graph From Parallel to Exclusive Gateway exploring a Workflow Graph Once we have AG, we need to explore the graph to derive execution traces that replace parallelism. To this aim, the affection graph is explored starting each time from a different node. In the exploration each visited node contributes to an execution trace to be added to the interleavings tree. Each time a graph node gn CT set has been explored following the affect transitions, a corresponding tree node tn is created. The tn relative to the first gn of an exploration will be appended to the treeõs root. When a gn has an incoming structural transition, the sequence of all the StIn[gn] preceding nodes must be used as label of the relative tn node. After that, gn itself is added to the execution trace appending its name to the label of tn. This leads to the label ct[tn] = StIn[gn] gn. When a node is used to label a tn, it is removed from the AG. At this point, the exploration proceeds towards AtOut[gn] nodes. Indeed, if gn is not the starting node and has many affecters, it could receive messages from one, none, all or a subset of them. All these cases should be analyzed to cover all the possibilities of interleaving and the exploration should cover all the affecting nodes of gn, i.e. AtIn[gn], in turn AtIn[AtIn[gn]] and so on. FP

47 The interleavings tree provided after the reduction of a single ParallelGateway, constitutes a different view of the nested ExclusiveGateway s we expected. These will be refactored applying the refactoring rules viewed before. The Figure 3.17 shows an example of exploration of an Affection Graph. Affection Graph 1) Exploration starts from CT7, respecting structural transitions TreeRoot CT2 CT3 CT7 CT4 CT3 CT7 CT5 2) proceeds through affected CT2 CT4 CT3 CT7 CT5 TreeRoot CT3 CT7 3) all the possible state of the affected should be explored CT2 TreeRoot CT3 CT4 CT3 CT7 CT7 CT5 CT5 CT4 CT4 4) always respecting the structural transitions CT2 TreeRoot CT3 CT4 CT3 CT7 CT7 CT5 CT5 CT2 CT4 CT2 CT4 5) if there are not out affect transition, the remaining nodes are appended CT2 TreeRoot CT3 CT4 CT3 CT7 CT7 CT5 CT5 CT2 CT4 CT2 CT4 CT5 6) and starts a new exploration from another node TreeRoot CT2 CT3 CT4 CT3 CT7 CT3 CT7 CT5 CT5 CT2 CT4 CT2 CT4 CT5 equivalence between Interleaving Tree and nested XORs TreeRoot CT3 CT7 CT5 CT2 CT4 CT5 XOR CT3 CT7 XOR XOR XOR CT2 CT4 CT2 CT4 CT2 CT4 CT5 CT5 Figure 3.17: ParTes: Example of Affection Graph exploration and Interleaving Tree creation Particular cases during Affection Graph exploration During the exploration of an Affection Graph, particular cases could raise. For instance, during the exploration of all the possible states of a node, the algorithm could cover a node that could not be added due to its structural dependencies. In this case the trace are marked as invalid and deleted from the Interleaving Tree. En example is showed in Figure In some other situations, the exploration could generate equivalent traces. This occurs especially when a node has structural and affection transitions related, as showed in Figure In this case the algorithms smartly avoid to generate same traces several times. Equivalent traces could be generated also when there are nodes that are not affected neither affecters FP

48 trace not valid. not respect structural dependencies F B G C C G B H F G B B H H H F all the possible states of H CGB_H GB_H B_H _H not possible because to explore C, B must be already explored this issue can not rise, since the exploration algorithm doesn't allow such situation Figure 3.18: ParTes: Violated Structural Transitions avoid duplication A C E E E A E C B D B D [A]B E C D B D B C C B E B C B C C C E C C If a node 'B' affects a node 'C' that is affected in turn by a node 'A' such that 'A' is in StIn[B]*, when the exploration starts from B, is not necessary to explore B->C because it has been already explored by the exploration starting from 'A' Figure 3.19: ParTes: Equivalent Traces (neutral nodes). In this case the algorithm generates just one trace for them. Figure 3.20 shows an example. [equivalent] traces from neutral nodes A B C D traces given by the exploration starting from A, are equivalent to those given by exploration starting from C. If there are not affect transitions in the workflow graph, all the traces are equivalent, so just one trace starting from any node is enough. Figure 3.20: ParTes: Equivalent Traces from Neutral Nodes When there is a cycle of affect transitions, the exploration could generate not valid traces when trying to explore the possible states of an affected node. An example is showed by Figure To avoid this, nodes are marked for "being added in the trace" before explore its states. FP

49 cycle of affect transitions A C C B A E C E B C B E C B C A all the possible states of C CEB_C EB_C B_C _C not valid To avoid situations like this, when the algorithm explores all the possible states of a node 'gn', this 'gn' is "marked". Figure 3.21: ParTes: Not Valid Traces From Workflow Graph refactoring to Test Skeleton ParTes takes as input a BPMN2 Choreography specification, handles it like a Workflow Graph and, after parallelism reduction and graph refactoring, provides all the execution traces the tester can stress to identify ordering issues about message exchange. If the initial BPMN2 specification contains Exclusive Gateways, final traces provided by PARTES are grouped by same condition expressions as showed by Figure 3.15 earlier in the text. For each condition there may be several different traces due to reduced parallelism. After the traces generation, the tester will be asked to choose which conditions to test so that only the traces that satisfy them will be considered for testing. However this does not guarantee that provided data for testing will effectively satisfy conditions chosen: the tester should provide right data to test skeletons. Test skeletons will be generated for services willing to play a role within a choreography and we need to isolate the local behavior of a single participant from the tree. We call this operation projection in which we isolate local traces of executions, related to a single participant, from the traces selected by the tester choosing the conditions to test. The local traces are called trace snippet. These traces are the ones we consider relevant for what concerns the detection of interoperability threats and are used to generate test skeletons for each participant. The tester will implement the missing pieces of the generated skeleton adding data information, not available in the choreography diagram or in other BPMN2 diagrams. Refinements in diagrams and API The adoption of the new technique to generate traces to test resulted in a refinement of the ParTes component. Taking into consideration the Figure 3.12 ("ParTes Architecture" pag. 32) in D4.2.2, the new Architecture Diagram showed by Figure 3.22 uses JBPT API instead of Java Path Finder API. The Java Business Process Technologies (JBPT) API are used to build the Workflow Graph and the Refined Process Structure Tree (RPST). This leads to a modification also in the Class Diagram. Referring to the old Class Diagram (Figure D.1: ParTes Class Diagram, page 96), the JPFRunner class has been replaced by the InteractionTreeBuilder class, as showed in Figure The API of the new classes are contained in the Appendix A. FP

50 Figure 3.22: ParTes: Modification in the Architecture Diagram Figure 3.23: ParTes: Modification in the Class Diagram Scalability Dimensions, Issues, and Strategies The scalability of the tests generation and execution functionality offered by ParTes is addressed through replication of the service itself (integrated as a ServicePot plugin) in multiple nodes, in this sense it can benefit from deployments within cloud based infrastructures. Moreover, also the execution of the tests will be made scalable via the replication of a testing service (based on the soapui API, also integrated as a plugin in ServicePot) to meet the needs of the ULS context. It is worth noting that replication is a simple effective approach for ParTes, in particular in order to address scalability issues, given its purely functional behavior with no state-based consequence among different invocations. In this sense the functionality of tests generation offered by ParTes, and the functionality of tests execution built in ServicePot will be replicated in the context of ULS systems ensuring the operational independence of the elements which constitute the run-time testing support of the choreography. This approach of "division of labor" in the basic components brings advantages in the ULS context. Due to the large number of users, components and requirements in the presence of a ULS system there will FP

51 ULS Dimension 1. Number of different kind of available services 2. Number of different kind of services in a single choreography 3. Number of service consumer instances in a single choreography (i.e., for a given kind of service consumer, we may have a UL number of instances) 4. Frequency of messages (requests, operations or transactions) exchanged by the services in time period 5. Number of atomic services (actuators and sensors) aggregated in services 6. Change of the above dimensions over time Consideration about ParTes This dimension represents the amplitude of the scale of the whole ULS system and affects the need for replication of the services for generation and execution of tests. The derivation strategy itself is not affected by this dimension. This is a sensible dimension for ParTes, with the increase in the number of services the complexity of the generation increases. The bottleneck is given by the use of model checking and the related explosion of states in the presence of interleaving. The work on the algorithm for BPMN exploration was on the elimination of this problem to make exploration scalable with respect to the number of ChoreographyTasks contained in the BPMN specification of the choreography. This dimension is not an issue for the derivation strategy nor for the deployment of services for generation and execution of tests. This dimension is not an issue for the derivation strategy nor for the deployment of services for generation and execution of tests. This dimension is not an issue for the derivation strategy nor for the deployment of services for generation and execution of tests. This dimension can affect the need for replication of services for test generation and execution. The strategy itself is not affected by this. Table 3.3: ULS dimensions and ParTes be several configuration options to support testing at run-time. Integrating different capabilities of test generation (other than ParTes), different modes of execution or - in the case where users prefer fast response of the registry - disabling the testing support at a cost of having an increased error rate. The Table 3.3 summarizes the main dimensions of scalability already highlighted in the project (deliverable D1.4) in relation to the ParTes tool. The main ULS dimension that affect ParTes is the "Number of different kind of services in a single choreography" Resource Lifecycle Overall Description The Resource Lifecycle architecture enables the discovery, and registration of several resources. It is complementary to the Extensible Service Discovery middleware achieved in WP3. Actually, thanks to a plugin mechanism the Abstraction Service Base Management (AoSBM) is populated by heterogeneous protocols. One of these plugins is called the Governance Plugin and integrates the Resource Lifecycle to the whole discovery infrastructure. It enables the population of the AoSBM with business services abstractions. FP

52 In D4.2.1 [10] and D4.2.2 [11], we have detailed the architectural style as well as the implementation details of the Resource Lifecycle management tool. The tool is structured into Resources, a Core Manager and a set of specific Managers. We consider Resources such as Services, Endpoints, Interfaces, and Service Level Agreements (SLA). The Core Manager incorporates a Resource Manager, a Configuration and a Connection Manager. Furthermore, a set of Managers handle the creation, adding, and retrieval of the modeled resources. Thanks to the Connection Manager, the Resource Lifecycle tool is synchronized with the EasyESB Nodes ( Middleware) and enables the retrieval of more specific resources inherent to the running services such as Interfaces, and Endpoints. The Resource Lifecycle tool is integrated into the Governance and V&V Framework as well as into the Extensible Service Discovery Middleware sketched in WP Enhancements in the Final Release In the final release of the Resource Lifecycle tool, we have achieved a more stable and advanced implementation of the components proposed in D4.2.1 and D It is now possible to register a Service description respecting the WSDL specification, and to retrieve a given description. Moreover, it is also possible to register and look for a Service Level Description of a given service. We have realized the needed development efforts to achieve these functionalities. Furthermore, we have also achieved the discovery of running services from the middleware. Indeed, the Connection Manager enables the retrieval of all the endpoints in the targeted nodes. Consequently, it is now possible thanks to the Resource Lifecycle Tool to find the interface and endpoint of a service exposed on any bus nodes, provided that the governance tool is connected and synchronized with the bus nodes. The details of the implemented APIs were given in deliverable D The code of the Resource Lifecycle tool will be released with the Governance and V&V Framework. The user manual of the Resource Lifecycle tool is provided in deliverable D5.3.1 and D Scalability Dimensions, Issues, and Strategies Table 4.2 lists the ULS challenges that the resource registry needs to address as well as the envisaged solutions that we implement to achieve this. FP

53 ULS Dimension Consideration about the Resource Registry 1. Heterogeneity of resources This dimension represents the number of governed resources discovered in the ULS System. We classify these resources as Services, Interfaces, Endpoints, Choreographies, SLAs, Roles, and Policies. We rely on an extensible multi level modeling approach including abstract classes and interfaces able to generate instances of each resources when needed. The specificity of each resource is considered while common criteria are gathered into a main ResourceType. 2. Number of Resources The resources need to be stored for publication and discovery; in ULS Systems an important number of resources may be discovered. In order to address possible issues, we implement a storage mechanism that classifies and stores the resources based on their types. In order to leverage the storage capability of the resource registry, we will further study new ways for exploiting the cloud storage mechanisms for instance. 3. Number of run-time services The resource registry is synchronized with the runtime distributed middleware which is an enterprise service bus (EasyESB). The latter is distributed over several nodes and is deployed on top of large cloud infrastructure. The resource registry is able to collect all the resources that are running on top of the middleware and exposes them in the registry. This synchronization is valuable as it enables exploiting the runtime evolution of managed resources. 4. Frequency and time response for the resources discovery Considering the large amount of governed resources, we adopt an analysis mechanism that parses the resources schemas and eases the retrieval and discovery of services. We believe this is a valuable functionality that faces the large scale challenges, mainly with respect to the lookup functions of resources. Table 3.4: ULS dimensions and Resource Life Cycle Framework FP

54 FP

55 4 Multi-source Monitoring Within the context of SOA, in order to effectively detect unexpected or undesirable behaviors of services, locate the origin of the issue, or even predict potential failures it is generally necessary to track, combine, and analyze events occurring at different abstraction levels. Therefore, in contrast with the use of more monitors operating in separate contexts, a promising strategy that is pursued in is to architect SLA monitoring solutions able to reveal or predict run-time anomalies due to the combination of phenomena originated from sources operating at different levels [17]. In this chapter we describe the Multi-source Monitoring Framework that we have developed, which can correlate the messages monitored at business-service level with the observations captured by the infrastructure monitoring the low level resources. In the following we describe the tool infrastructure implementing multi-source monitoring at three levels, namely: infrastructure layer, reported in Section 4.1; business layer, reported in Section 4.2; and finally at level of the Complex Event Processor, reported in Section 4.3. which is able to correlate heterogenous events. The above three different tools are loosely integrated by means of the Distributed Service Bus (DSB). The high-level architecture of our Multi-source Monitoring Framework is depicted in Figure 4.1. The DSB distinguishes between a set of channels dedicated to the monitoring activities (i.e., Control Plane), and other channels where both coordination and application messages can flow (i.e. Data Plane). The CEP correlates the data passing through the Control Plane Infrastructure Monitoring As typical when using cloud computing, resource usage in is expected to vary according to demand, and the middleware should accomodate this by creating or removing service instances as well as nodes (virtual machines) accordingly. The middleware should also be able to work around failures in the infrastructure, such as network outages or crashed nodes. All these tasks involve knowledge on the status of the environment where is running and, therefore, needs to support the monitoring of resources, both in terms of utilization and health status. In this section, we present the subsystem responsible for monitoring this kind of resources Overall Description As described in Deliverable D4.2.1 [10], the resource monitor in leverages previous works by borrowing heavily from Ganglia [22] and has three main components: 1) A set of data collectors that gather local information such as load average, I/O rates, and network utilization. These collectors run on every active node of the cloud. Data for each node is both FP

56 Figure 4.1: Multi-source Monitoring made available on demand over TCP/IP and, at the same time, periodically pushed over UDP to be replicated in nearby nodes. These data collectors are simply instances of the Ganglia gmond component. 2) A set of aggregators that periodically pull data from the collectors, summarize it, and keep a historical record of the values received. Each aggregator may be responsible for a subset of the nodes in the system and may also be replicated. Finally, aggregators may be organized hierarchically, so that higher-level ones provide a broader view of the system. Data stored in each aggregator is made available on demand over TCP/IP. These aggregators are instances of the Ganglia gmetad component. 3) A notification mechanism that detects potentially relevant events, such as exceptional load average or too little available disk space, and notifies the higher-level event monitor system described in Section 4.3 for further analysis. It is up to this higher layer (according to the parameters defined by the middleware) to assess whether such events indeed warrant special attention or not. The notification daemon is based on simple hard-coded rules that define what constitutes a potentially relevant event. The gmetad component is deployed to every node and data is available for the system administrator if needed, but we have not identified a strong use case for the data it provides in, and it is consequently unused. The other components should be deployed in a similar fashion to Figure 4.2. More detailed information about the implementation of the notification daemon is available in Deliverable D4.2.2 [11] Enhancements in the Final Release The material presented in M24 is already the final version of the infrastructure monitoring module, and therefore no new functionality has been added to it. The current focus is on integration with the middleware and the use cases. FP

57 Figure 4.2: Platform Monitoring Deployment Scalability Dimensions, Issues, and Strategies As can be seen in Table 4.1, the number of virtual nodes in a cluster 1 is the only scale variable that affects the platform monitoring subsystem: for every new virtual node, there is an additional set of monitoring components deployed onto this new virtual node. This means the resource usage of these components (memory for local data, disk space, CPU usage) on each node is always constant. Since the overhead of the gmond data collectors is widely acknowledged to be very low, we may consider it negligible. Memory consumption on each node for the data collected from neighboring nodes grows linearly with the number of nodes in the cluster; and the volume of data exchanged among nodes for replication grows quadratically with the number of nodes in the cluster. However, they are still low and, therefore, it is not uncommon for Ganglia deployments to feature thousands of nodes. Since it would be unwise to deploy any exceptionally large environment without partitioning the network, planning the cloud infrastructure to partition the Ganglia replication cliques accordingly is highly desirable. 1 A cluster is a set of (virtual or actual) machines that are grouped together, usually because they belong to a single organization and are close in terms of network topology. FP