How to Manage Faults With Web Services Fault Management

Transcription

1 Recovery Policies for Enhancing Web Services Reliability Abdelkarim Erradi 1, Piyush Maheshwari 2, Vladimir Tosic 1 1 School of Computer Science and Engineering University of New South Wales, Sydney, Australia {aerradi,vtosic}@cse.unsw.edu.au 2 IBM India Research Lab New Delhi, India pimahesh@in.ibm.com Abstract Web services are gaining acceptance as a standards-based approach for integrating loosely coupled services often distributed over a network. Hence, achieving high levels of reliability and availability in spite of service or infrastructure failures poses unique set of challenges. However, current Web services middleware provide limited constructs for specifying faults detection and recovery actions. Additionally, faults-handling logic often gets scattered and tangled with the service logic. Consequently, this negatively impacts maintainability and adaptability. To address these requirements for reliable and fault tolerant Web services execution, we propose a set extensible recovery policies to declaratively specify how to handle and recover from typical faults in Web services composition. The identified constructs were incorporated into a lightweight service management middleware named MASC (Manageable and Adaptive Service Composition) to transparently enact the fault management policies and facilitate the monitoring, configuration and control of managed services. Several experimental results with a service based supply chain management system illustrate the effectiveness of our approach to providing reliable and uninterrupted services. 1. Introduction Web services (WS) are rapidly becoming a predominant approach for building open distributed systems, via assembling a set of self-contained, discoverable and Internet-accessible software components. Services imply well-defined boundaries that communicate with each other using well-defined messages. Hence, Web services reliability is strongly dependant on the fault-handling mechanisms of the communication protocols and the messaging infrastructure mediating their interactions. Reliability can be defined as the continuity of correct service delivery. This implies zero or, at worst, relatively few failures and rapid recovery time [2]. In contrast to traditional client-server applications, a Web Services can compose a number of Web Services possibly hosted and managed by different providers across the Internet. Because the composed services are often geographically distributed without centralized control, no guarantees can be made about the reliability of the resulting composed service. The unreliability of any of the constituent Web Services could lead to the failure or QoS degradation of the composed service, even if other constituent services happen to be reliable. Addressing reliability and availability of Web services is a challenging task, particularly detecting and handling faults. This is because of the autonomy of the systems involved (i.e., service consumer has no influence over the factors affecting QoS provision and services can spontaneously appear and disappear on the Internet), the cross-administrative domains interactions and the Internet latency and its unmanaged nature. Moreover the dynamic discovery and late binding to continuously evolving Web services interfaces and offerings makes the problem of reliability more challenging than conventional client-server systems. These challenges call for further improvements in the Web services containers and the mediation infrastructure to support dependable composition of Web Services while taking into account their inherent autonomy and heterogeneity. To address the requirements for reliable and fault tolerant Web service interactions, we propose a policydriven middleware which intercepts the execution of composite services and transparently provides recovery services. We define an extensible set of crucial recovery policies (e.g., retry, skip, use equivalent service), with a well-defined behavior, to declaratively specify the handling and recovery from typical faults in service-centric business processes. The proposed policies are defined in a way that allows their automated enforcement by the proposed service management middleware. Our approach cleanly modularizes and externalizes the faults-handling logic and hence keeps the service composition simple and uncluttered. Furthermore, the recovery policies can evolve independently while allowing potential reuse. We have incorporated runtime support for the identified recovery constructs into a lightweight service management middleware named Manageable and Adaptive Service Composition (MASC 1 ). MASC is an extensible mediation layer that intercepts exchanged messages and transparently enforces the faults handling policies. MASC is an evolution of our 1

2 previous messaging platform presented in [5, 6]. In order to demonstrate the merits of our techniques, MASC was deployed in a supply chain management (SCM) system. The preliminary measurements confirmed the improved availability and reliability of the whole application at the acceptable increase in latency. The rest of this paper is structured as follows. Section 2 describes our solution approach and discusses the proposed MASC fault management policies. Section 3 then presents the architecture and features of MASC. In section 4, we present experimental results that demonstrate the capabilities of the proposed approach. Finally, we discuss related work in Section 5 while Section 6 draws some conclusions and presents some directions for future work. 2. Recovery strategies for reliable Web services Traditionally, distributed systems programming models rely on structured exceptions handling to trap and handle errors and timeouts. For Web services, exceptions cannot be propagated across service boundaries without being serialized and wrapped in a SOAP fault message in the format declared in the service contract. The service WSDL document can explicitly include one or more fault message descriptions for the entire service or for individual operations. Web services fault management requires first to identity and classify the typical faults that might occur during service execution and then specify a set of strategies to handle them Classification of main faults in distributed service-based applications Faults that can occur during Web services execution can be classified into three main categories according to their causalities: functional faults, operational faults, and faults linked to the violation of Service Level Agreements (SLAs) and collaboration policies. (1) Functional and behavioral faults refer to the scenario where a constituent service cannot complete a task execution or the service delivers incorrect results due to computational/logic errors, erroneous data flows or semantic incompatibility of the exchange messages. Additionally, behavioral failures can be caused by conversation exceptions such as improper invocation order of service operations, lost messages when processing fails and interrelated messages processed individually. Moreover conversation protocol errors can introduce inconsistent states between participating services or they can lead to circular wait deadlocks that could prevent the process enactment to compete as services could be waiting for input from each other. (2) Operational faults refer to communication infrastructure exceptions and middleware failures of the hosting servers and the database servers. Examples of such faults could be network unavailability causing disconnections, network congestion causing message loss, and overloaded application server causing excessive delays and timeouts. Operational failures can also result from configuration errors or security breaches such as Denial of Service attacks. Operational faults can render the service either entirely or partially unavailable or unable to accept new requests. (3) Violations of agreed upon SLAs and policies with regards to functional (e.g., price limits or delivery deadlines) and non-functional requirements (e.g. service response time, service availability and security). In this case the service execution might be completed but the results are not conforming to the negotiated SLAs and collaboration policies. The first step to handle the above faults is to detect them and recognize their type. Various techniques can be used to achieve this. First, the service may return fault messages as specified by its WSDL interface. Also, the service invocation infrastructure can use timers to raise timeout exceptions when the service does not respond within the set timeout interval. Moreover, the exchanged messages between composed services need to be validated against the interface contract provided by the source service, in case of a mismatch a fault should be raised. Additionally, monitoring probes can be used to detect violations of execution constraints (e.g. service response time), as stipulated by pre-established SLAs. Once a fault is detected, an exception event is raised to trigger the necessary recovery actions to handle the faults in an attempt to reach a successful process completion while satisfying the QoS constraints to the largest extent possible. Service composition languages, like WS-BPEL, provide the ability to scope activities and specify basic fault handlers or compensation handlers for scopes to deal with certain class of faults by compensating the effects of the service invocation. However, most approaches lack flexible faults handling mechanisms to cover all possible exceptions including QoS constraints violations. Moreover, attempting to describe all possible execution paths including exceptions using just fault and compensation handlers becomes either awkward or impractical. Hence, we advocate a policy-based approach to externalize faultshandling.

3 2.2. MASC policy language for faults management Our proposed approach uses a policy-driven model to faults management to simplify Web services development through the separation of the business logic and the faults handling policies. The essence of our approach is to augment the Web services descriptions with XML-based declarative policy assertions to specify the recovery and repair actions and the configuration parameters required by the middleware services to enact these actions. An example of such configuration parameters is the number of times a service invocation should be retried. These assertions are attached to the service contract but still evolve independently. We claim that this approach offers more effective and easier to change faults- handling. Our recovery policies define responses to each fault type in terms of high-level abstraction of management operations. The latter are enforced via mapping the policies into runtime monitoring and middleware configuration adjustments. The proposed adaptation policy language encompasses two distinct parts. The first part is used by the Events Monitors to detect faults and to trigger an update to the service Health State. The second part of the policy triggers the required management action(s) to restore a healthy state. The adaptation policies take as an input the health states that are designated as erroneous and decide on the appropriate management actions without being concerned with how that state was reached or what was the cause. Adaptation policies are associated with health states and not events. The Events Monitors are used to decouple the adaptation policies from particular system events, in order to enable generic policy based management. Policies control and respond to changes in the service state by prescribing corrective state transitions. Leaving events out of policies enables policy reuse in different contexts. Generally, one state may be changed as a result of many different events in a system. Instead of specifying a separate policy for every possible event that may cause the change, MASC simply specifies only one policy to change the state of the service and one or more detection policies to trigger it. The format the events monitoring policy is as follows: ON Event(s) WHEN Constraints Assertions TRIGGER Health state transition Events may signal indications that some significant aspect of the service operational conditions may be changed (e.g., service performance statistics dropping below allowable threshold). Hence some aspects of the service state should be checked and some management actions may be required. Events are only signals for attention, not necessary for action. Events Monitors can use the publish/subscribe communication paradigm to expresses interest in receiving certain types of event from various internal/external sources. The Events Monitors use the monitoring policies to perform filtering of events. For example they can submit event subscriptions with predicates on the event contents to filter the events they are interested in. In MASC, Event Monitors are used to decouple the adaptation policies from the system and management events. Event Monitors behave like call-back methods that respond to events by triggering process Health State changes if the conditions specified in the monitoring policy are satisfied. The Event Monitors keep a record of the past events in order to be able to process scenarios of events (e.g., a pattern of ordered or unordered events) that may be defined to trigger a health state change. The format the adaptation policy is shown below: POLICY PolicyName ON Health State Specification WHEN Constraints Assertions DO Management Operation(s) CONFIGURATION Adaptation Configuration POST-CONDITION Constraints Assertions The ON clause can specify a process health state. The when-clause specifies the predicates that must be true for actions to occur. These assertions can reference the schema of exchanged messages as well as environment variables such as the number of running process instances. They can also reference the policies of participant services such as the policies describing the transaction properties of the service operations (e.g., whether the operation is retriable, compensatable, cancellable or definite). The do-clause specifies an exception handling action. The latter can be a primitive action or a composite action. The Configuration clause specifies the parameters that govern the behavior of the adaptation policy. This clause leverages WS-Policy [9] to encode the adaptation policy configuration. The configuration also specifies the policy priority, which is used to determine the order of execution if several polices apply per health state. For example, a policy could stipulate that the system should first attempt n retries before failover to a known backup service. Finally the post-condition clause could be used to stipulate the required verification predicates to confirm if the recovery was successful and the bad health state is truly resolved. It could refer to the same verification

4 assertions that were used to confirm the existence of a bad or a deteriorated health state. Currently, the key constructs supported by MASC policy assertions are: retry, substitute, compensate, skip and dynamic update of the service composition instance. (1) Retry policy involves the invocation of the same faulty service hoping that the failure is transient (e.g., a transient database deadlock or transient overloaded server). This recovery construct specifies the number of retries and controls how retries are done, as shown in the listing in Figure 1. However, this strategy is only applicable to idempotent service operations that could receive invocation messages multiple times but only one of them gets processed or when multiple service invocations yield the same results and leave the system in a consistent state (e.g., Book search). In our architecture the retry policy is enforced at the SOAP messaging layer. The retry construct is implemented by placing the messages that fail to be delivered in a retry queue then the queue reader tries redelivery using the pattern specified by the recovery policy. Messages for which processing repeatedly fails are placed in a dead letter queue after exhausting the maximum number of allowed retries and no further delivery will be attempted. Also a fault message can be raised after the last unsuccessful retry. (2) Substitute (hot-swap) the faulty service and dynamically bind to a replacement service that offers equivalent functional and QoS properties. Equivalent services need to have equivalent input messages, preconditions, output messages and effects. This means that they leave the process in an equivalent state after any of the replaceable services is invoked. In some case a faulty service could be replaced by a service composition that has equivalent effects as the faulty service. In most cases, some input/output message transformations are required to address syntactical, structural and business protocol mismatches between services [3]. To implement the substitute policy, our approach introduces the concept of a Virtual Endpoint (VEP), on which multiple equivalent services, acting as a recovery block, can be configured. The Process Orchestration Engine sends requests to the VEP, and MASC infrastructure transparently discovers and selects a replacement service and then redirects the request messages. The selection of services is based on the service selection policies as well as the monitoring data and QoS metrics gathered from prior interactions or from trusted peers. Figure 1. Configuration of a Retry Recovery policy Figure 2. Configuration for Substitute Recovery policy (3) Parallel execution of services, a policy could be attached to the VEP to configure MASC to perform parallel execution of equivalent service and consider the results coming from the first responding service. This

5 strategy is more suitable for data lookup services and freely available services such as Web search. (4) Dynamic update of the service composition instance, the idea here is to augment the Web services composition engine with a Management Endpoint to manipulate running process instances to either (a) perform structure changes such as add/remove a process activity (b) process instance control commands like skip, wait, start, terminate, suspend and resume an activity or a process instance. Adaptation at the process layer is not covered in details in this paper. This is being addressed as part of our proposed AdaptiveBPEL framework [7]. Exception handling policies can be specified by a Process Designer at design time and then they can be attached to a VEP configured in MASC. 3. Implementation and experience This section presents the architecture of MASC middleware with emphasis on the modules that facilitate the enactment of the recovery policies. We are in the process of maturing our prototype implementation of MASC middleware, using the.net framework, Windows Communication Foundation (WCF) and Windows Workflow Foundation (WF) [12]. Although we decided to use.net as our implementation platform, the proposed architecture is not inherently tied to any particular platform. It can be easily extended to Java-based Web services platforms such as Axis 2. MASC uses broker and intercepting filters design patterns [10]. As shown in Figure 3, MASC can be deployed as a configurable add-on to a Web services composition engine to enact adaptation policies. The Web services composition uses MASC as a messaging layer to call external Web services via <invoke> or <reply>, or to wait for response messages via a <receive> activity. MASC acts as a transparent service proxy, clients direct service calls to the virtual endpoints configured in MASC and the later invokes the real services. This deployment allows intercepting, storing, transforming, retrying and routing messages going into and out of the composed services. Basically when a service is provided via MASC, first it needs to be registered with a Virtual Endpoint (VEP) along with its associate runtime policies using the constructs discussed in the previous Section. When an invoke request comes in, MASC takes care of the Find, Select and Bind process using the configured selection and binding policy. During service interactions, various monitoring data is collected MASC middleware architecture Figure 3. Conceptual architecture of MASC MASC groups managed equivalent services into virtual endpoints. Each VEP is bound to one or more mediators. The mediators intercept and manipulate both request and response messages according to the pre-configured policies (e.g., validate an outgoing message against the contract expected by the destination service). Basically the policies define the chain of actions to take on a given message before dispatch. The message is then queued for processing. MASC selects the appropriate service and

6 dispatches the message to it and the response is passed back to the requester via the same path. The diagram in Figure 3 shows MASC architecture Web services composition layer For the process orchestration layer we have selected the upcoming Microsoft Workflow Foundation (WF) [12] because of its simplicity, extensibility and the available documentation. When an incoming message triggers a start activity, the WF runtime engine creates a new process instance and starts it. The engine takes care of persistence, events and alarms handling, as well as Web services invocation. The engine dispatches incoming messages to the right process instance. First if the incoming SOAP message contains correlation data, the engine tries to find the correct instance that matches the correlation data. Otherwise if the request matches a start activity, a new process instance is created. To map abstract Web services defined in the composition to concrete Web services, WF uses a configuration file. In our proposed framework, the abstract WSs should be mapped to a MASC VEP so that the later can perform dynamic Find, Select, Bind and Invoke on behalf of the composition engine. Additionally, the VEP masks and handles any service faults according to the self-adaptive policies attached to the VEP. The adaptation enactment is managed by the adaptation layer Adaptation layer The Adaptation Manager acts as a Policy Enforcement Point (PEP) and it leverages the services of other actuators such as the WS Selection Manager and Messages Adaptation Manager Web services Selection Manger The dynamic mapping of abstract WSs defined in the composition to concrete WSs is outsourced to the WS Selection Manager. This allows shielding the orchestration engine from changes to available services. Hence, adding, modifying and selecting among available services could be done without the need to complicate the process with the routing logic for deciding which concrete services to use. The WS Selection Manager is responsible for the runtime selection among the equivalent services registered with the VEP. The selection of services is based on the QoS metrics gathered from prior interactions or from trusted peers. Various selection policies are supported, a VEP can be configured to choose between registered services in round-robin fashion, or to select the best performing service, or to broadcast the request to all services (registered with the VEP) in parallel and consider the first one that respond, etc.. The following selection policies are supported by the WS Selection Manager: Random Selection, this policy selects the service to be invoked randomly amongst the set of known services. Parallel Invocation, this policy invokes all the equivalent services in parallel using concurrent invocation threads. The first response to be received is returned to the client. All pending invocations are then aborted and their responses are ignored. Best Performing Service, this policy selects the service to be invoked as the one with the best performance considering the last n number of successful invocations Messages Adaptation Manager The Messages Adaptation Manager is responsible for resolving incompatibilities (i.e., structural, value and encoding) and enable interoperation with substituted services. Given that it is not so common to find perfectly substitutable services, MASC provides a set built-in generic handlers (e.g., transform a message with XSLT, split/merge messages) that can be extended and composed in a pipeline by the developers to resolve incompatibilities and mismatching interfaces (i.e., structural, encoding and missing parameter values) and enable interoperation with substituted services. More advanced data flow adaptation operators are being studied such as buffering multiple messages and aggregating them into a single one before sending them to the destination service MASC evaluation and assessment An initial prototype based on the generic architecture presented in the pervious Section was developed as a proof of concept. We conducted a series of benchmarking tests to assess the effectiveness of MASC in enhancing the reliability of Web services composition. The objective of this evaluation is to assess the features of our prototype implementation with regards to its effectiveness and performance under normal and failure conditions. Our goal is to identify areas of the platform that needs further improvement. We used an extended.net-based implementation of WS-I Supply Chain Management (SCM) application [13]. The SCM scenarios are designed as Web services based interactions that simulate the business activity of an online supplier of electronic goods. First a Web client calls the Retailer service's getcatalog operation. When the user submits the order, the Web client calls the Retailer service's submitorder operation. To fulfill orders, the Retailer Web service manages stock levels in three warehouses (WA, WB, and WC). If Warehouse A cannot fulfill an order, the Retailer checks Warehouse B; if Warehouse B cannot, the Retailer checks Warehouse C. When an item in stock falls below a certain threshold, the Warehouse must restock the item from the Manufacturer's inventory (MA, MB, and MC). Each use

7 case includes a logging call to a Logging Service in order to monitor the activities of the services. The customer can track orders by using the getevents operation of the logging facility Web service. During the SCM process enactment, participating Web services can log event by calling the logevent operation of the logging facility Web service. Our goal is to study the impact of introducing MASC middleware on the reliability and the round-trip response time of Web services used in the above SCM application. Additionally, our aim is to discover areas of the platform that needs further improvement. To this end, we simulated multiple concurrent Web service clients, each of which invokes deployed services multiple times. We deployed the SCM backend Web services (The Retailer, the Manufacturer and Warehouses Web services) in a P4 2.8GHz, 1GB RAM server running Windows 2003, Windows Communication Foundation (WCF). The composition engine and MASC were deployed in a Windows XP laptop with 500MB RAM. The machines were connected by 100MB LAN. To be able to compare the response time and throughput metrics, our experimental setup consists of two runtime configurations: one with MASC placed at the client side and other without MASC mediation. Both experiments use the identical application logic implemented in.net. The graph in Figure 4 summarizes some of the performance test results. We simulated multiple concurrent clients, each of which invokes the composite Order Placement service multiple times. We measured the execution time from the perspective of the client application using the Round Trip Time (RTT), which is a measurement in milliseconds from the time the client application sends a request to the point when it successfully receives the full reply from the Service Provider. Execution times were averaged after 500 repetitions. The graph in Figure 8 summarizes some of the performance test results. Avg. Process Execution Time (ms) BPEL 1045 BPEL with MASC Figure 4: Total execution time Direct invocation of participant Web services slightly outperforms channeling Web services interactions through MASC due to the overheads introduces by the added adaptability components (the process execution via MASC is 12 percent slower). Our analysis of the main reasons of the delays introduced by MASC points to the parsing overheads of the XML-encoded adaptation policies. We will enhance the performance of MASC by exploring policy caching mechanisms to offset the degraded performance. To establish the prototype's fault tolerance effectiveness, we periodically injected random exception events at various stages of the process to study the behavior of the system in response to QoS changes and service failures. For service failures, we randomly pick some of the available services and make them unavailable for a random amount of time. For service QoS degradation, we periodically pick some service instances and introduce a random number of seconds delay before sending a reply back to the composition engine. The results of the analysis from 500 repetitions show improved process reliability since 98% of process failures were handled effectively by MASC and the process was able to resume execution past the point of failure. Only a small number of exception scenarios could not be managed by the framework and thus require user intervention, such as the absence of a replacement Web service to substitute a failed service. Under identical conditions, direct invocation of participant services resulted in a process termination when exceptions are injected. These results indicate the effectiveness of MASC, but more analysis is clearly required. A more thorough and complete analysis is currently under way to be able to refine and optimize the system for increased performance. It is worth noting that we assume that Web services interactions channeled through MASC are stateless, meaning that messages are self-contained with enough information (or links to persistent data) to allow the participant service to establish the message context. This assumption is fundamental to adaptability properties of MASC, as this is also an encouraged practice in the Web services literature as in [1]. For services where state management is involved, state migration mechanisms are still to be investigated. 4. Related work and discussion Reliable Web services composition is active area of research and standardization. Several ongoing efforts recognize the need to extend the Web services middleware with mechanisms to ensure dependable service delivery in a Web-centric environment. We adopt a policy-based approach that builds on the emerging selfhealing architectures, such as the one presented in [11]. The key idea is to decouple between a sensor that detects the exceptional events and the effector that corrects them. In [4], Charfi and Mezini propose an aspect-oriented extension to BPEL to enable dynamic weaving of aspects into the service composition. A process runs inside a process container that provides middleware services to

8 BPEL processes. However we argue that some of the QoS aspects that they tried to address, e.g. security and state persistency, can be addressed more naturally via interception at lower-layer messaging middleware rather than augmenting the BPEL engine with the ability to call low-level middleware services. On the other hand our approach is policy-based and operates mostly at the underlying SOAP Layer to shield the Process Orchestration Layer from addressing middleware requirements. Our work is also closely related to the task-based recovery policies advocated by [8] as part of their extended Petri net model, called Self-Adaptive Recovery Net (SARN), for specifying exceptional behavior in Petri net based workflow systems. The proposed recovery policies are generic constructs, such as Skip Recovery Policy, that model the required modifications to adapt the structure of the workflow instance when a task failure event occurs. For each expected failure, these constructs are specified at design time as a sequence of primitive operations, such as delete a transition, to modify the workflow structure. SARN recovery policies are attached with either a single task or a set of tasks called a recovery region. However, SARN recovery constructs are tightly coupled with Petri net concepts such as Places, Transitions and Tokens. Our approach is more generic and does not change the process instance; rather the enforcement of the recovery policies is delegated to the underlying messaging middleware that mediates the Web services interactions. Our ongoing work is also looking at augmenting a process orchestration engine, such as WF [12], with policy-driven aspects weaving mechanisms to transparently enforce reliability policies and dynamically adapt the composition instance [7]. 5. Conclusion Web service compositions are rapidly emerging as critical building blocks for process integration. Hence, their reliable execution must be assured. Our approach cleanly specifies common reliability policies, such as retry and failover, capable of handling common service faults. The identified constructs were integrated into a Web services management middleware, named MASC, to transparently enact the recovery actions. MASC uses the interceptor pattern to add a level of indirection allowing exchanged messages to be introspected, manipulated and routed. This ability also allows fault-handling policies to be transparently applied to service calls by leveraging the emerging Web services policy model. The overarching goal is to ensure that the client of the Web Service is not exposed to, and does not have to deal with, the reliability problems; thus, the client continues to perceive continuous uptime of the Web Service, and it is shielded transparently from the occurrence of faults. The introduced mediation layer dynamically handles potential service faults and transparently hot-swaps to functionally equivalent backup services. This leverages the inherent plurality and the diversity of available services. Experimental results proved the feasibility and effectiveness of our approach and suggest its potential in adding valuable recovery services with limited overhead. In future work we plan to improve and extend our approach in a variety of areas, including policy-driven transactional service compositions as well as adding support for composite events to allow for a recovery action to take place based on multiple related events. Additionally, we are exploring predictive adaptation techniques beside the reactive ones presented in this paper. Moreover, complex recovery schemes are under study to manage checkpointing and recovery of composite services with distributed state. References [1] Alonso, G., Casati, F., Kuno, H. and Machiraju, V. 2003, Web Services: Concepts, Architectures, and Applications, Springer, Berlin. [2] Avizienis, A., Laprie, J.-C., Randell, B. and Landwehr, C. 2004, 'Basic concepts and taxonomy of dependable and secure computing', IEEE Transactions on Dependable and Secure Computing, vol. 1, no. 1, pp [3] Benatallah, B., Casati, F., Grigori, D., Nezhad, H. R. M. and Toumani, F. 2005, 'Developing Adapters for Web Services Integration', in 17th International Conference on Advanced Information Systems Engineering (CAiSE'05), Porto, Portugal. [4] Charfi, A. and Mezini, M. 2005, 'An Aspect based Process Container for BPEL', in First Workshop on Aspect-Oriented Middleware Development (AOMD 2005), Grenoble, France. [5] Erradi, A. and Maheshwari, P. 2005, 'A Broker-based Approach for Improving Web Services Reliability', in IEEE International Conference on Web Services 2005 (ICWS'05), Orlando, Florida, USA. [6] Erradi, A. and Maheshwari, P. 2005, 'wsbus: QoS-aware Middleware for Reliable Web Services Interactions', in IEEE International Conference on e-technology, e-commerce and e- Service (EEE'05), Hong Kong. [7] Erradi, A., Maheshwari, P. 2005, 'AdaptiveBPEL: Policy- Driven Middleware for Flexible Web Services Composition', in EDOC Middleware for Web Services Workshop (MWS'05), Enschede, The Netherlands. [8] Hamadi, R. and Benatallah, B. 2004, 'Recovery Nets: Towards Self-Adaptive Workflow Systems', in 5th International Conference on Web Information Systems Engineering (WISE'04), LNCS 3306, pp , Springer Verlag,, Brisbane, Australia. [9] IBM et al. 2004, Web Services Policy Framework (WS-Policy), September ibm.com/developerworks/library/specification/ws-polfram. [10] Monday, P. B. 2003, Web Services Patterns: Java Edition, Apress. [11] Wile, D. S. and Egyed, A. 2004, 'An Externalized Infrastructure for Self-Healing Systems', in 4th Working IEEE / IFIP Conference on Software Architecture (WICSA'04), Oslo, Norway, pp [12] Microsoft.NET Framework [13] WS-I 2004, SCM Sample Application.