Introduction to Web Services, SOA, and ESBs

Transcription

1 Introduction to Web Services, SOA, and ESBs The Enterprise Integration Perspective Odysseas Pentakalos, Ph.D. Chief Technology Officer SYSNET International, Inc.

2 Agenda Enterprise Application Integration Service Oriented Architecture Characteristics, Advantages, Disadvantages Web Services Characteristics, Advantages, Disadvantages Phase I, Phase II for SOA 2.0 Enterprise Service Bus Characteristics, Advantages, Disadvantages Business Processes in a SOA BPEL, Current state Copyright 2006 SYSNET International, Inc. 2

3 Enterprise Application Integration Definition from Webopedia: EAI is the unrestricted sharing of data and business processes throughout the networked applications or data sources in an organization. Definition from Whatis.com: EAI is a business computing term for the plans, methods, and tools aimed at modernizing, consolidating, and coordinating the computer applications in an enterprise. Copyright 2006 SYSNET International, Inc. 3

4 Demand for Enterprise Integration A number of business drivers are generating growing demand for Enterprise Integration Mergers and acquisitions Constant need to cut costs Constant need to streamline business processes Compliance with government regulations Gartner just released some statistics around the worldwide application integration market, finding a seven percent jump in spending from 2004 to Total revenue totaled $8.5 billion in Gartner says it looks for a similar growth rate this year in the market. Copyright 2006 SYSNET International, Inc. 4

5 Requirements Requirements on an Enterprise Integration solution: The solution should not cost much; remember the need for cost-cutting Should not require extensive changes to existing applications Can handle lots of traffic with high reliability Flexible so that management and reconfiguration is easy to do Copyright 2006 SYSNET International, Inc. 5

6 Enterprise Integration Challenges Typical environment consists of a collection of applications In-house systems: usually developed on the principle of expediency so extensibility and connectivity options are limited Packaged-applications: vendors prefer to keep their system closed and limit integration capabilities Copyright 2006 SYSNET International, Inc. 6

7 Enterprise Integration Challenges The architect needs to reconcile incompatibilities and differences at multiple levels: Business processes are incompatible Data model variations at the syntactic and semantic level Differences in the technologies used for each system Differences in the timing at which systems operate Differences in the interaction patterns: synchronous versus asynchronous From: Understanding SOA with Web Services by Eric Newcomer and Greg Lomow Copyright 2006 SYSNET International, Inc. 7

8 Enterprise Integration Approaches Some of the possible ways of accomplishing integration First taxonomy (from Enterprise Integration Patterns book): File Transfer Shared Database Remote Procedure Invocation Messaging Another taxonomy (EIA JavaWorld article by Abraham Kang): Data-level integration User interface-level integration Application-level integration Copyright 2006 SYSNET International, Inc. 8

9 Enterprise Integration Scenario Loan Broker Scenario; this will be a running scenario throughout the presentation 3 1 Broker 3 Bank Customer 2 3 Bank From: Enterprise Integration Patterns by Hohpe and Woolf. Credit Bureau Bank Copyright 2006 SYSNET International, Inc. 9

10 Service Oriented Architecture

11 Service Oriented Architecture (SOA) The definition of services that encapsulate reusable business function The use of explicit implementation independent interfaces to define services Loosely coupled; services are independent of other services Services are registered in a directory service so that they can be located during design or runtime Service definition is not dependent on transport protocol used to access it Service can be used in a location/platform/technology agnostic approach Copyright 2006 SYSNET International, Inc. 11

12 SOA Interaction Pattern This is the basic interaction pattern in a SOAbased application: Consumer 3 : Bind 2 : Find Provider 1 : Register Registry Copyright 2006 SYSNET International, Inc. 12

13 Service Granularity From: SOA Blueprints Concepts, The Middleware Company Research Team. Copyright 2006 SYSNET International, Inc. 13

14 Service Types Most people think of services as the request-reply type Request/Reply receives information, does something and returns information Worker performs a function or changes state Monitor observes a process and generates events Agent observes a process and takes action based on rules Aggregator combines results from disparate sources Process long running service that implements a business service Copyright 2006 SYSNET International, Inc. 14

15 SOA Vision Multiple applications utilize the services Multiple service providers make services available The lifetimes of the services and applications are fairly decoupled Composite Application Composite Application Composite Application Copyright 2006 SYSNET International, Inc. 15

16 Is SOA a new concept? It is just a new name for a sought-after approach What about DCE, CORBA, RMI, DCOM? Lack of industry wide standardization prevented complete interoperability and platform independence Transport protocols were proprietary or specific to distributed object technology Security concerns and firewalls prevented interaction across boundaries Bad timing with respect to availability of computational power and other industry dynamics Copyright 2006 SYSNET International, Inc. 16

17 Web Services

18 Why Web Services for SOA? Earlier middleware technologies were: Did not support heterogeneous clients and platforms Could not work over the Internet Some of the choices were hard to use Most of the choices were expensive to buy and maintain Web Services address these issues!! Copyright 2006 SYSNET International, Inc. 18

19 XML: The Data Format extensible Markup Language (XML) An XML document is based on a hierarchy of elements and associated attributes Copyright 2006 SYSNET International, Inc. 19

20 XML Schema XML language for defining structure and constraints of an XML document Copyright 2006 SYSNET International, Inc. 20

21 XSLT: Transformation Language extensible Stylesheet Language Transformations Copyright 2006 SYSNET International, Inc. 21

22 Overview of SOAP Definition from SOAP 1.1 Spec SOAP provides a simple and lightweight mechanism for exchanging structured and typed information between peers in a decentralized, distributed environment using XML... Simplicity is a key requirement Extensibility is another key requirement Binding into the network is unspecified Supports various message exchange patterns: request/respond notification callback Copyright 2006 SYSNET International, Inc. 22

23 Structure of a SOAP Message SOAP envelope SOAP header SOAP block SOAP block SOAP body SOAP block SOAP block Copyright 2006 SYSNET International, Inc. 23

24 Example SOAP Message <soap:envelope xmlns:sn=" xmlns:soap=" soap:encodingstyle=" <soap:header> <h:from xmlns:h= </h:from> </soap:header> <soap:body> <sn:getstockquoteprice> <sn:symbol>msft</sn:symbol> </sn:getstockquoteprice> </soap:body> </soap:envelope> Copyright 2006 SYSNET International, Inc. 24

25 Example Web Service <soap:envelope xmlns: > <soap:body> <m:getquote xmlns="urn:xmethods-delayed-quotes > <symbol xsi:type="xsd:string">msft</symbol> </m:getquote> </soap:body> </soap:envelope> HTTP getquote(string symbol) HTTP <soap:envelope xmlns: > <soap:body> <n:getquoteresponse xmlns="urn:xmethods-delayed-quotes"> <Result xsi:type="xsd:float">24.67</result> </n:getquoteresponse> </soap:body> </soap:envelope> Copyright 2006 SYSNET International, Inc. 25

26 WSDL: Introduction Is an XML document itself with a given schema Describes the interface to a Web Service Separates the abstract definition of a service from the implementation Describes the what, the how, and the where of the Web Service Similar to predecessors such as IDL of CORBA and DCOM Copyright 2006 SYSNET International, Inc. 26

27 WSDL Describes Services Web Services Description Language (WSDL) Client Stubs Client Impl. WSDL Platform Tools Server Stubs Requests Server Impl. Copyright 2006 SYSNET International, Inc. 27

28 WSDL: The description language Standard for describing services and their bindings. <definitions>: is the container and defines name and namespaces for WSDL document. <types>: defines any custom data types that are passed in/out of the service. <message>: models the data that is exchanged within method calls <porttype>: operations supported through an endpoint of a web service <binding>: associates a porttype with a protocol <service>: associates a binding with a specific URL Copyright 2006 SYSNET International, Inc. 28

29 WSDL: Sample document Demo of sample from W3C Copyright 2006 SYSNET International, Inc. 29

30 UDDI History Originally defined for the purpose of providing a public business and service registry First few versions came out through UDDI.org UDDI v3.0 is an OASIS standard Currently private registries are more popular in support of SOA-based applications Copyright 2006 SYSNET International, Inc. 30

31 UDDI Basic Concepts Defines a number of elements as part of the data model businessentity, businessservice, bindingtemplate, tmodel Data Model takes the form of an XML document Spec defines APIs for inquiry, publication, and security The UDDI registry itself is exposed as a set of Web Services Copyright 2006 SYSNET International, Inc. 31

32 UDDI Scenarios Design time use Developer searches through the private registry for existing services that can take part in an orchestrated application Interest is in the functionality but not the specific instances Run time use Application searches through the private registry for the service but focus is now on implementations of it Copyright 2006 SYSNET International, Inc. 32

33 EAI using Web Services WSDL Invoke WSDL Query Query Invoke UDDI Server WSDL Copyright 2006 SYSNET International, Inc. 33

34 Web Services: Phase 1 Services were developed to support a specific purpose Registration of interfaces is not necessary Discovery and utilization of interfaces is done at design time Web Services provide platform independence Composite Application Copyright 2006 SYSNET International, Inc. 34

35 Loan Broker with Web Services Copyright 2006 SYSNET International, Inc. 35

36 Web Services-based Integration Advantages (Tactical Approach) Easy to implement Fairly low cost Does not require enterprise-wide agreements Disadvantages Fairly limited in terms of message exchange patterns Lower reusability of the services Consistency across web services implementations is limited Copyright 2006 SYSNET International, Inc. 36

37 SOA Vision Multiple applications utilize the services Multiple service providers make services available The lifetimes of the services and application are fairly decoupled Composite Application Composite Application Composite Application Copyright 2006 SYSNET International, Inc. 37

38 WS-* Standards WS-Addressing Need a standard approach for naming endpoints across protocols WS-ReliableMessaging Need to support reliable messaging on top or unreliable protocols WS-Coordination/WS-Transactions Need to support transactions across services WS-Security Security is crucial for Web Services WS-Policy Need to be able to specify nonfunctional and QoS requirements. Copyright 2006 SYSNET International, Inc. 38

39 WS-Addressing Copyright 2006 SYSNET International, Inc. 39

40 Why do we need Addressing? To find a Web Service you need a URL. Simple right? Address the web service with the URL, set the action in the HTTP headers and you are set. How about intermediaries? How about SOAP over non-http protocol? How about dynamically generated web services? Copyright 2006 SYSNET International, Inc. 40

41 Basic Concepts: Endpoint References Endpoint references: Dynamic generation and customization of service endpoint descriptions. Referencing and description of specific service instances that are created as the result of stateful interactions. Flexible and dynamic exchange of endpoint information in tightly coupled environments where communicating parties share a set of common assumptions about specific policies or protocols that are used during the interaction. Copyright 2006 SYSNET International, Inc. 41

42 Endpoint Reference Definition An Endpoint Reference consists of: Address: is a URI and identifies the address of the endpoint ReferenceParameters: XML elements that are necessary in order to successfully interact with the web service Metadata: provides an extensible container for metadata that describes the endpoint <wsa:endpointreference xmlns:wsa=" <wsa:address> </wsa:endpointreference> Copyright 2006 SYSNET International, Inc. 42

43 Message Information Headers Message Information Headers: headers to allow messages to be addressed to an endpoint all within the SOAP message. To: The URL of the target service <wsa:to> From: The ERP of the message s sender. <wsa:from> <wsa:address> </wsa:from> Copyright 2006 SYSNET International, Inc. 43

44 Message Information Headers ReplyTo: The ERP to which the response should be sent to. FaultTo: The ERP to which the SOAP fault should be sent to. MessageID: Is a URI that uniquely identifies the message Action: Takes the place of the SOAPAction RelatesTo: Element that indicates the MessageID of the caller. Copyright 2006 SYSNET International, Inc. 44

45 WS-Reliable Messaging Copyright 2006 SYSNET International, Inc. 45

46 Why do we need Reliable Messaging? Network failures may prevent messages from being delivered Middleware failures may cause messages from being delivered Intermittent failures may cause duplicates Routing delays may cause out of order delivery Copyright 2006 SYSNET International, Inc. 46

47 WS-Reliable Messaging Concepts Sequence: reliable messaging is enforced within the context of a sequence between two endpoints Acknowledgments: an ack from the server indicates the sequence range of messages received Delivery Assurance Policy: endpoints can negotiate the delivery policy Copyright 2006 SYSNET International, Inc. 47

48 Reliable Messaging Model Initial Sender Ultimate Receiver Application Source Application Source Send Delivery RM Source Transmit Acknowledge RM Destination Receive Copyright 2006 SYSNET International, Inc. 48

49 Reliable Messaging Protocol Endpoint A Create Sequence Create Sequence Response (Identifier) Sequence (Identifier, MessageNumber=1) Sequence (Identifier, MessageNumber=2) Sequence (Identifier, MessageNumber=3, LastMessage) Sequence Ack. (Identifier, Range=1,3) Sequence (Identifier, MessageNumber=2) Sequence Ack. (Identifier, Range=1..3) Terminate Sequence Endpoint B Copyright 2006 SYSNET International, Inc. 49

50 Sequence Message <wsrm:sequence...> <wsu:identifier> [URI] </wsu:identifier> <wsrm:messagenumber> [unsignedlong] </wsrm:messagenumber> <wsrm:lastmessage/>? <wsu:expires> [datetime] </wsu:expires>?... </wsrm:sequence> Copyright 2006 SYSNET International, Inc. 50

51 Sequence Example <wsrm:sequence> <wsu:identifier> </wsu:identifier> <wsrm:messagenumber>10</wsrm:messagenumber> <wsrm:lastmessage/> </wsrm:sequence> Copyright 2006 SYSNET International, Inc. 51

52 Sequence Acknowledgement <wsrm:sequenceacknowledgement...> <wsu:identifier> [URI] </wsu:identifier> [ <wsrm:acknowledgementrange... Upper="[unsignedLong]" Lower="[unsignedLong]"/> + <wsrm:nack>[unsignedlong]</wsrm:nack> + ]... <wsrm:sequenceacknowledgement> Copyright 2006 SYSNET International, Inc. 52

53 Sequence Ack Example <wsrm:sequenceacknowledgement> <wsu:identifier> </wsu:identifier> <wsrm:acknowledgementrange Upper="2" Lower="1"/> <wsrm:acknowledgementrange Upper="6" Lower="4"/> <wsrm:acknowledgementrange Upper="10" Lower="8"/> </wsrm:sequenceacknowledgement> Copyright 2006 SYSNET International, Inc. 53

54 Create Sequence Message Request <wsrm:createsequence...>... </wsrm:createsequence> Response <wsrm:createsequenceresponse...> <wsu:identifier> [URI] </wsu:identifier>... </wsrm:createsequenceresponse> Copyright 2006 SYSNET International, Inc. 54

55 Sequence Termination <wsrm:terminatesequence...> <wsu:identifier> [URI] </wsu:identifier>... </wsrm:terminatesequence> Copyright 2006 SYSNET International, Inc. 55

56 Transactions Copyright 2006 SYSNET International, Inc. 56

57 Why do we need Transactions? In an application based on an orchestration of Web Services all services involved in an operation must come to an undisputed resolution. Typical ACID semantics are useful for some but long-running transactions need different transactional support. Copyright 2006 SYSNET International, Inc. 57

58 Atomic Transactions Atomicity: on completion either all actions or none are completed. Consistency: consistency is the system is preserved after processing multiple concurrent transactions Isolation: intermediate states of the system are not observable outside the transaction Durability: if a transaction commits, the changes are preserved even in after a system failure Copyright 2006 SYSNET International, Inc. 58

59 Business Transactions Long-running transactions: transactions may take hours, days or weeks to complete Need to be able to select a subset of the original participants before a commit Compensating transactions: operations that are used to restore the state in case of a failure Copyright 2006 SYSNET International, Inc. 59

60 Standards and scope WS-Coordination: allows a distributed system to establish a group of participants around an activity participants can register interest in participating in the outcome Coordination protocol can vary but operates upon completion of the activity WS-Atomic: Handles ACID transactions and defines variations of 2PC WS-Business Activity: Allows for the definition of nested scopes of operations Allows for the definition of compensating transactions Copyright 2006 SYSNET International, Inc. 60

61 Coordinator Copyright 2006 SYSNET International, Inc. 61

62 Interactions for Coordination Copyright 2006 SYSNET International, Inc. 62

63 Why do we need Security? Protect messages from someone eavesdropping on the network Prevent tampering with the messages Currently security for web services is done at the transport layer (SSL). This won t work when the same message has multiple links to traverse Need the ability to selectively encrypt parts of the message. Copyright 2006 SYSNET International, Inc. 63

64 WS-Security Specifications Authentication Profiles Username/password Kerberos X.509 Certificates Message Integrity XML Signature XML Key Management Message Confidentiality XML Encryption Copyright 2006 SYSNET International, Inc. 64

65 SOA for Enterprise Integration They all need to talk to one another despite running on top of different protocols They need support for all the WS-* services we talked about The data structures between the elements may not be identical Need to be able to support various message exchange patterns Service Service Service Service Service Service Service Copyright 2006 SYSNET International, Inc. 65

66 Enterprise Service Bus (ESB) Abstract the development of service components from wire protocol and deployment platform from below and the business processes above Facilitates the requirements of a highly-scalable, faulttolerant, message-driven, service-oriented enterprise. Utilizes XML and related-technologies for unifying the message format Provides transformation and routing capabilities to the bus as a service Provides centralized management capabilities of the distributed services components Copyright 2006 SYSNET International, Inc. 66

67 ESB Architecture Provides a standards-based bus and adapters for multiple transports and protocols along with common services Copyright 2006 SYSNET International, Inc. 67

68 Java Business Integration High-level architecture of JBI framework (JSR 208) Copyright 2006 SYSNET International, Inc. 68

69 Business Processes Composition of services into a business process United Airline Reservations Client Travel Reservation Delta Airline Reservations Hertz Car Reservations Hotel Reservations Copyright 2006 SYSNET International, Inc. 69

70 BPEL Business Process Execution Language: Language for describing business processes Convergence of a number of earlier efforts (WSFL, XLANG) Invoke operations in synchronous/asynchronous mode Invoke operations sequentially or in parallel Define fault and transaction handling Compose business processes from business processes Copyright 2006 SYSNET International, Inc. 70

71 Loan Broker with BPEL Broker is defined as a BPEL document 3 1 BPEL Process 3 Bank Customer 2 3 Bank Credit Bureau Bank Copyright 2006 SYSNET International, Inc. 71

72 Enterprise Integration Design Patterns Design patterns for Enterprise Integration Fill the gap between high-level architecture and implementation Pattern Classification Messaging Systems: basic concepts in integration with messaging Messaging Channels: describe channels used for integration Message Construction: describe message types and intent Message Routing: describes provision of routing and brokering Message Transformation: variations of message translator Messaging Endpoints: connecting to and from the messaging system System Management: management of the integration infrastructure Copyright 2006 SYSNET International, Inc. 72

73 Sample EI Patterns Messaging Channels Datatype Channel: all messages on a given channel are of the same type Invalid Message Channel: what to do about invalid messages. Dead Letter Channel: what to do about undeliverable messages. Copyright 2006 SYSNET International, Inc. 73

74 Sample EI Patterns Messaging Routing Content-based Router: routes message based on the message s content Recipient List: routes message to dynamic list of recipients Splitter/Aggregator: one splits message into submessages; other one aggregates back into one Copyright 2006 SYSNET International, Inc. 74

75 Application of EIP Loan Broker using Enterprise Integration Patterns Credit Agency Recipient List Data Enricher Customer Endpoint Aggregator Normalizer Translators Copyright 2006 SYSNET International, Inc. 75

76 Mule: Enterprise Service Bus

77 Mule EIP Server Mule is an open source framework based on the concept of an ESB (mule.codehaus.org) What s with the name? Follows an event-driven architecture System consists of producers, consumers and a message server Configuration of the routing of events between components takes place outside the scope of consumers/producers Container follows the SEDA architecture model Master s thesis work of Matt Welsh at UC Berkeley ( Copyright 2006 SYSNET International, Inc. 77

78 Mule Architecture Copyright 2006 SYSNET International, Inc. 78

79 Conclusion SOA has great potential for finally achieving reusability Web Services is the platform that will make SOA a reality Existing holes to achieve the full vision are currently in development or production An Enterprise Service Bus provides the glue to make SOA a reality today Copyright 2006 SYSNET International, Inc. 79

80 Backup Slides

81 Mule Configuration An XML file defines the integration model: connector: Configuration of the components that attaches to the various providers container-context: Configuration of the container that will provide the components endpoint-identifiers: Configuration of endpoints that provide services interceptors: code to execute before and/or after a component model: Configuration of the components (UMO) that implement the services Copyright 2006 SYSNET International, Inc. 81

82 Mule-based Loan Broker Credit Agency Lender Service Credit Agency Gateway Loan Broker Banking Gateway Bank 1 Bank 2 Bank n Copyright 2006 SYSNET International, Inc. 82

83 Service Definition Loan Broker: (REST Web Service) receives requests for loan quotes Credit Agency: (EJB Service) provides credit information about the customer Lender Service: (Web Service) receives customer loan requests with credit information and forwards requests to appropriate banks Banks: (Web Service) provide quotes on customer loan requests based on request and credit information Copyright 2006 SYSNET International, Inc. 83

84 Endpoint Definition <endpoint-identifiers> <endpoint-identifier name="loanbrokerrequestsrest" value="jetty:rest://localhost:8080/loanbroker"/> <endpoint-identifier name="loanbrokerrequests" value="vm://loan.broker.requests"/> <endpoint-identifier name="loanquotes" value="jms://esb.loan.quotes"/> <endpoint-identifier name="creditagencygateway" value="jms://esb.credit.agency"/> <endpoint-identifier name="creditagency" value="ejb://localhost:1099/local/creditagency?method=getcreditprofile"/> <endpoint-identifier name="lendergateway" value="jms://esb.lender.service"/> <endpoint-identifier name="lenderservice" value="vm://lender.service"/> <endpoint-identifier name="bankinggateway" value="jms://esb.banks"/> <endpoint-identifier name="bank1" value="axis: <endpoint-identifier name="bank2" value="axis: <endpoint-identifier name="bank3" value="axis: <endpoint-identifier name="bank4" value="axis: <endpoint-identifier name="bank5" value="axis: </endpoint-identifiers> Copyright 2006 SYSNET International, Inc. 84

85 Loan Broker Definition <model name="loan-broker"> <!-- The loan broker is used to receive loan requests --> <mule-descriptor name="loanbroker" implementation="org.mule.samples.loanbroker.esb.loanbroker"> <inbound-router> <endpoint address="loanbrokerreqrest transformers= RestReqToCustReq /> <endpoint address="loanbrokerrequests"/> </inbound-router> <outbound-router> <router classname="org.outbound.outboundpassthroughrouter"> <endpoint address="creditagencygateway"/> </router> </outbound-router> <response-router timeout=" "> <endpoint address="loanquotes"/> <router classname="org..routers.bankquotesresponseaggregator"/> </response-router> </mule-descriptor> Copyright 2006 SYSNET International, Inc. 85

86 Credit Agency Definition <!-- The credit agency service will get the credit profile for a customer --> <mule-descriptor name="creditagencygateway" implementation="org.mule.components.builder.reflectionmessagebuilder"> <inbound-router> <endpoint address="creditagencygateway"/> </inbound-router> <outbound-router> <router classname="org.mule.filteringoutboundrouter"> <endpoint remotesync="true transformers="loanquoterequesttocreditprofileargs" responsetransformers="creditprofilexmltocreditprofile address="creditagency"/> <endpoint address="lendergateway"/> </router> </outbound-router> </mule-descriptor> Copyright 2006 SYSNET International, Inc. 86

87 Banking Gateway Definition <mule-descriptor name="bankinggateway" implementation="org.mule.bridgecomponent"> <inbound-router> <endpoint address="bankinggateway"/> </inbound-router>1 <outbound-router> <router classname="org.staticrecipientlist"> <reply-to address="loanquotes"/> </router> </outbound-router> </mule-descriptor> Copyright 2006 SYSNET International, Inc. 87