Enterprise Service Bus

Transcription

1 Enterprise Service Bus Daniel Hagimont IRIT/ENSEEIHT 2 rue Charles Camichel - BP TOULOUSE CEDEX 7 [email protected] 1

2 Intégration - besoins Briques logicielles (applications) A gros grain Distribuées Technologies différentes (protocoles, systèmes, API ) Après le développement Collaboration/Intégration Communication en réparti Adaptation des interfaces et des données Schémas de collaboration complexes (pas que client-serveur) 2

3 Problématique Depuis 20 ans, les DSI se heurtent aux problèmes de Intégrer des applications hétérogènes Construire des architectures logicielles complexes Les maintenir Avec des applications qui n'ont pas été prévues pour le faire 3

4 Intégration vs interopérabilité Définition : L'interopérabilité est la capacité pour un système d'échanger de l'information et des services dans un environnement technologique et organisationnel hétérogène (IEEE, 1990) L'interopérabilité peut être assurée par Le développeur (CORBA, RPC, RMI) L'intégrateur (les applications existent déjà) 4

5 Intégration point à point Technologies adhoc (différentes à travers le temps) The accidental architecture Effet spaghetti 5

6 Les ETL (Extract, Transform, Load) Solution la plus populaire Exportation des données, adaptation et injection dans d'autres applications En mode batch (souvent la nuit) Problème de latence de mise à jour 6

7 Les EAI (Enterprise Application Integration) Comme une multiprise Un connecteur par application L'EAI route les messages entre les applications 7

8 ESB (Enterprise Service Bus) Un EAI décentralisé Utilisation de standards (XML, WS, JMS...) 8

9 Ce qui fait un ESB Un bus (MOM) Des données (XML) Des adaptateurs/connecteurs (WS, ) Un flot de contrôle (routage) 9

10 Les technos associées aux ESB MOM/JMS Messagerie asynchrone (queues, pub-sub) Web Services SOAP, WSDL XML XML, DTD, Schema, XSLT, XPATH 10

11 ESB : les produits Propriétaires BEA Aqualogic (acheté par Oracle) IBM WebSphere Enterprise Service Bus Sonic ESB de Progress Software Cape Clear (spinoff de IONA) OpenSource Mule Apache ServiceMix Jboss ESB OW2 Petals (Toulouse!) 11

12 Mule Mule is a Java-based enterprise service bus (ESB) and integration platform that allows developers to quickly and easily connect applications to exchange data following the service-oriented architecture (SOA) methodology. Mule enables easy integration of existing systems, regardless of the different technologies that the applications use, including JMS, Web Services, JDBC, HTTP, and more. 12

13 What Mule ESB does Decouples business logic Location transparency Transport protocol conversion Message transformation Message routing Message enhancement Reliability (transactions) Security Scalability 13

14 Mule architecture Endpoint Channels for reception Transformer Message transformation/enhancement Router Message flow control (inbound/outbound) Service Component Your integration logic lives here 14

15 Overall view Generally File, WS, mail, JDBC... Generally MOM + XML 15

16 Mule configuration Spring based mule-config.xml Multiple config files using import 16

17 First example <?xml version="1.0" encoding="utf-8"?> <mule xmlns=" xmlns:spring=" xmlns:file=" <model name="fileexample"> <service name="fileservice"> <inbound> <file:inbound-endpoint path="inbox" fileage="500" pollingfrequency="100"/> </inbound> <outbound> <outbound-pass-through-router> <file:outbound-endpoint path="outbox" outputpattern="output.xml"/> </outbound-pass-through-router> </outbound> </service> </model> </mule> 17

18 Endpoints Old style <endpoint address="jms://topic:mytopic"/> <endpoint address="file://work/incoming? pollingfrequency=2000"/> New style (with namespaces) <jms:outbound-endpoint topic="order.topic"/> <file:inbound-endpoint path="inbox" fileage="1000" pollingfrequency="2000" /> 18

19 Endpoints Inbound and outbound Ajax, JDBC, FTP, File, HTTP, JMS, RMI, SSL, TCP, UDP, VM Inbound only IMAP, POP3, Servlet, Twitter Outbound only SMTP New transport/endpoints can be developed 19

20 Transformers Default transformers (associated with endpoint) jmsmessage-to-object-transformer byte-array-to-string-transformer Custom transformers Override default transformers 20

21 Transformers <jms:jmsmessage-to-object-transformer name="jmstostringtransformer"/> <xml:xslt-transformer name="xslt" xsl-file="yourfile.xslt"/> <custom-transformer name="custom" class="esb.yourtransformer"/> <jms:inbound-endpoint queue="query.response"> <transformer ref="jmstostringtransformer"/> <transformer ref="xslt"/> </jms:inbound-endpoint> The default transformer must be reinstalled 21

22 Implementing transformers public class ObjectToXML extends AbstractTransformer { protected Object dotransform(object payload) throws TransformerException { try { StringWriter outwriter = new StringWriter(); IMarshallingContext ctx = BindingDirectory.getFactory( payload.getclass()).createmarshallingcontext(); ctx.marshaldocument(payload, "UTF-8", null, outwriter); return outwriter.tostring(); } catch (JiBXException e) { throw new TransformerException(this, e); } } } 22

23 Implementing transformers public class XMLToObject extends AbstractTransformer { private String targetclassname; protected Object dotransform(object xmldata) throws TransformerException { try { IUnmarshallingContext ctx = BindingDirectory.getFactory(Class.forName(targetClassName)).createUnmarshallingContext(); return ctx.unmarshaldocument(new StringReader(xmldata)); } catch (Exception e) { throw new TransformerException (this, e); } } public String gettargetclassname() { return targetclassname; } public void settargetclassname(string targetclassname) { this.targetclassname = targetclassname; } } 23

24 Routers Route message Inbound : select messages which reach component Outbound : route messages which leave component Are applied after (resp. before) an inbound (resp. outbound) tranformer Most routers are already there New routers can be developed 24

25 Inbound router example <inbound> <forwarding-catch-all-strategy> <jms:outbound-endpoint queue="failure.queue" /> </forwarding-catch-all-strategy> <selective-consumer-router> <jxpath-filter pattern="(//resultcode)='success'"/> </selective-consumer-router> <jms:inbound-endpoint queue="list.in" /> </inbound> default transformer : jms to text 25

26 Common inbound routers (NB : many routers require customization) Router name Description Idempotent receiver Ensures that only messages are received that contain a unique ID. Aggregator Combines two or more messages together and passes them on as a single message. Resequencer Holds back messages and can reorder the messages before they are sent to the component. Selective consumer Allows to specify whether or not you want to receive a certain event. Wiretap router Allows to route certain incoming events to a different endpoint as well as to the component. Forwarding consumer Forwards the message directly to the outbound router without invoking the component. 26

27 Outbound router example <outbound> <list-message-splitter-router> <payload-type-filter expectedtype="java.util.list"/> <jms:outbound-endpoint queue="order.queue"> <payload-type-filter expectedtype="esb.chapter2.order"/> </jms:outbound-endpoint> <jms:outbound-endpoint queue="item.queue"> <payload-type-filter expectedtype="esb.chapter2.item"/> </jms:outbound-endpoint> <jms:outbound-endpoint queue="customer.queue"> <payload-type-filter expectedtype="esb.chapter2.customer"/> </jms:outbound-endpoint> </list-message-splitter-router> </outbound> 27

28 Common outbound routers (NB : many routers require customization) Router name Description Filtering outbound router Routes based on the content of the message Recipient list Multicast router Sends a message to multiple endpoints Chaining router Links various endpoints, the result of one endpoint being used as the input of the next endpoint List message splitter Accepts a list of objects and sends them to different endpoints. Filtering XML message splitter Splits an XML documents and sends parts to different endpoints. 28

29 A component public class ExampleComponent { public void processcustomer(customer customer) { // do something interesting } } <component class="esb.chapter2.examplecomponent"/> 29

30 Integration with Spring Definition of Java object structures in XML <beans xmlns=" <bean id="customer" class="org.demo.customerserviceimpl"> <property name="anotherproperty" value="somestringvalue"/> <property name="customerdao" ref="customerdao"/> </bean> <bean id="customerdao" class="org.demo.customerdaoimpl"> <property name="name" value="manning"/> <property name="address" value="greenwich"/> <property name="clientnumber" value="12345"/> </bean> </beans> Easier configuration 30

31 Integration with Spring <spring:beans> <spring:import resource="components.xml"/> </spring:beans> <service name="comp1service"> <inbound> <file:inbound-endpoint path="work/in" /> </inbound> <component> <spring-object bean="component1"/> </component> <outbound> <outbound-pass-through-router> <jms:outbound-endpoint queue="comp.queue" /> </outbound-pass-through-router> </outbound> </service> 31

32 A full example 32

33 A full example (1/4) <mule> <spring:beans> <spring:import resource="components.xml"/> </spring:beans> <jms:activemq-connector name="jmsconnector" brokerurl="tcp://localhost:61616"/> <custom-transformer class="esb.chapter3.objecttoxmltransformer" name="persontoxml" /> <custom-transformer class="esb.chapter3.xmltoobjecttransformer" name="xmltoperson"> <property name="targetclassname" value="esb.chapter3.person" /> </custom-transformer> <byte-array-to-string-transformer name="bytestostring" /> </mule> 33

34 A full example (2/4) <mule> <service name="fileinboxservice"> <inbound> <file:inbound-endpoint path="chapter3/inbox"> <transformer ref="bytestostring"/> <transformer ref="xmltoperson"/> </file:inbound-endpoint> </inbound> <outbound> <outbound-pass-through-router> <jms:outbound-endpoint queue="log.queue" /> </outbound-pass-through-router> </outbound> </service> </mule> 34

35 A full example (3/4) <mule> <service name="loggerservice"> <inbound> <jms:inbound-endpoint queue="log.queue" /> </inbound> <component> <spring-object bean="loggercomponent"/> </component> <outbound> <outbound-pass-through-router> <jms:outbound-endpoint topic="listener" /> </outbound-pass-through-router> </outbound> </service> </mule> 35

36 A full example (4/4) <mule> <service name="fileoutboxservice1"> <inbound> <jms:inbound-endpoint topic="listener" /> </inbound> <outbound> <outbound-pass-through-router> <file:outbound-endpoint path="chapter3/outbox-1"> <transformer ref="persontoxml"/> </file:outbound-endpoint> XML data </outbound-pass-through-router> </outbound> </service> <service name="fileoutboxservice2"> <inbound> <jms:inbound-endpoint topic="listener" /> </inbound> <outbound> <outbound-pass-through-router> <file:outbound-endpoint path="chapter3/outbox-2" /> </outbound-pass-through-router> </outbound> Person objects </service> 36 </mule>

37 Example with message flow (1/8) JMS to WS wrappers 37

38 Onward flow (2/8) 38

39 Onward flow (3/8) <service name="bookquotelogger"> <inbound> <jms:inbound-endpoint queue="booksearch.input"/> ISBN data (String) </inbound> <component class="esb.chapter4.messageflow.mule.messagelogger" /> <outbound> <multicasting-router> <jms:outbound-endpoint queue="amazon.input"/> ISBN data (String) <jms:outbound-endpoint queue="barnes.input"> <transformer ref="isbntoxml"/> <transformer ref="objecttojms"/> </jms:outbound-endpoint> </multicasting-router> </outbound> </service> 39

40 Backward flow (4/8) 40

41 Backward flow (5/8) Return messages From Amazon On queue : amazon.output Serialized object (BookQuote) From Barnes&Noble : On queue : barnes.output Needs an XML-to-object transformer (with JiBX) to transform into BookQuote object A BookQuote is sent to the aggregatequotes.input queue 41

42 Backward flow (6/8) <service name="cheapestpricecalculator"> <inbound> <jms:inbound-endpoint queue="aggregatequotes.input"/> <custom-inbound-router class="esb.chapter4.messageflow.mule.bookquoteaggregator"/> </inbound> <component class="esb.chapter4.messageflow.mule.cheapestpricecalc"/> <outbound> <outbound-pass-through-router> <jms:outbound-endpoint queue="booksearch.output"/> </outbound-pass-through-router> </outbound> </service> 42

43 Custom inbound router (7/8) (aggregator) public class BookQuoteAggregator extends AbstractEventAggregator { protected EventCorrelatorCallback getcorrelatorcallback() { return new EventCorrelatorCallback() { public MuleMessage aggregateevents(eventgroup events) throws AggregationException { Iterator itevent = events.iterator(); Collection<BookQuote> quotelist = new ArrayList<BookQuote>(); while(itevent.hasnext()) { MuleEvent event = (MuleEvent) itevent.next(); BookQuote quote = (BookQuote) event.getmessage().getpayload(); quotelist.add(quote); create } return new DefaultMuleMessage(quoteList); aggregated } message public EventGroup createeventgroup(muleevent event, Object correlationid) { return new EventGroup(correlationID, 2); } size of the group 43

44 Custom inbound router (8/8) (aggregator) public boolean shouldaggregateevents(eventgroup events) { Iterator itevent = events.iterator(); decide if boolean isamazonpresent = false; aggregation can boolean isbarnespresent = false; be triggered while(itevent.hasnext()) { MuleEvent event = (MuleEvent) itevent.next(); BookQuote quote = (BookQuote)event.getMessage().getPayload(); String companyname = quote.getcompanyname(); if("amazon".equalsignorecase(companyname)) { isamazonpresent = true; } else if("barnesandnoble".equalsignorecase(companyname)) { isbarnespresent = true; } } return isamazonpresent && isbarnespresent; }};} public MessageInfoMapping getmessageinfomapping() { return new MuleMessageInfoMapping() { public String getcorrelationid(mulemessage message) { BookQuote quote = (BookQuote) message.getpayload(); return quote.getisbn(); } get correlation id };}} 44 (ISBN)

45 Content based routing <mule> <xm:xml-to-dom-transformer name="filetodom"/> <model name="routingexample"> <service name="insuranceservice"> <inbound> <file:inbound-endpoint path="insuranceinbox" fileage="500" pollingfrequency="2000" transformer-refs="filetodom"/> </inbound> <outbound> <forwarding-catch-all-strategy> <file:outbound-endpoint path="insuranceexception"/> </forwarding-catch-all-strategy> <filtering-router> <file:outbound-endpoint path="insurancecar" outputpattern="car-${date}.xml"/> different <xm:jxpath-filter pattern="//ins:insurance-type='car'"> <xm:namespace uri=" prefix="ins"/> endpoints </xm:jxpath-filter> </filtering-router> <filtering-router> <file:outbound-endpoint path="insurancetravel" outputpattern="travel-${date}.xml"/> <xm:jxpath-filter pattern="//ins:insurance-type='travel'"> <xm:namespace uri=" prefix="ins"/> </xm:jxpath-filter> </filtering-router> </outbound> </service> </model> </mule> 45

46 Calling a WS and forward <mule> <stdio:connector name="stdioconnector" promptmessage="enter city,country"/> <custom-transformer name="stringtolist" class="org.mule.module.scripting.transformer.scripttransformer"> <spring:property name="scriptenginename" value="groovy"/> <spring:property name="scriptfile" value="tokenizer.groovy"/> </custom-transformer> <custom-transformer name="objecttoarray" class="esb.chapter5.transformation.mule.collectiontoarray"/> WS requires <model name="transformationexample"> an array Java <service name="weatherinvokeservice"> <inbound> object <stdio:inbound-endpoint system="in"> <transformer ref="stringtolist"/> a list of cascaded <transformer ref="objecttoarray"/> </stdio:inbound-endpoint> synchronous </inbound> endpoints <outbound> <chaining-router> <outbound-endpoint address=" wsdl-cxf: WSDL&method=GetWeather"/> <vm:outbound-endpoint path="weather.output"/> </chaining-router> </outbound-router> </service> </model> </mule> 46

47 Connecting with JDBC (read) Table: person id: long primary key name: varchar[255] processed: boolean <mule> <spring:bean name="datasource" class="org.enhydra.jdbc.standard.standarddatasource"> <spring:property name="drivername" value="org.hsqldb.jdbcdriver" /> datasource <spring:property name="url" value="jdbc:hsqldb:hsql://localhost/xdb" /> <spring:property name="user" value="sa" /> </spring:bean> query <jdbc:connector name="hsqldb-connector" datasource-ref="datasource"> <jdbc:query key="get" value="select * FROM person where processed=false" /> <jdbc:query key="get.ack" value="update person SET processed=true WHERE id=${jxpath:id}" /> </jdbc:connector> </mule> 47

48 Connecting with JDBC (read) An inbound-endpoint with the "get" query One message per returned row "get.ack" is executed for each message 48

49 Connecting with JDBC (read) <mule> <model name="jdbc-model"> <service name="jdbc-reader"> <inbound> <jdbc:inbound-endpoint querykey="get" pollingfrequency="3000" /> </inbound> serialized <outbound> object <outbound-pass-through-router> <file:outbound-endpoint path="chapter6/3a-jdbc-read/out" outputpattern="${uuid}-${count}.dat "/> </outbound-pass-through-router> </outbound> </service> </model> </mule> 49

50 Connecting with JDBC (write) <mule> <file:connector name="fileconnector" streaming="false"/> <jdbc:connector name="hsqldb-connector" datasource-ref="datasource"> <jdbc:queries> <entry key="write" value="insert into person (id, name, processed) VALUES(NULL, ${payload}, false" /> </jdbc:queries> </jdbc:connector> </mule> payload of the message = name 50

51 Connecting with JDBC (write) <mule> <model name="jdbc-model"> <service name="jdbc-writer"> <inbound> <file:inbound-endpoint path="chapter6/3b-jdbc-write/in" pollingfrequency="3000"> <file:file-to-string-transformer/> </file:inbound-endpoint> </inbound> byte-array to String <outbound> <outbound-pass-through-router> <jdbc:outbound-endpoint querykey="write" /> </outbound-pass-through-router> </outbound> </service> </model> </mule> 51

52 Connecting with SMTP <mule> <file:connector name="fileconnector" streaming="false" /> <model name="mail-model"> <service name="file-to-mail"> <inbound> <file:inbound-endpoint path="chapter6/4a-mail-smtp/in"> <file:file-to-string-transformer /> </file:inbound-endpoint> </inbound> content of the mail <outbound> <outbound-pass-through-router> <smtp:outbound-endpoint subject="you've got mail from Mule!" host="localhost" port="25" user="mule" password="mule" /> </outbound-pass-through-router> </outbound> </service> </model> 52 </mule>

53 Connecting with POP3 <mule> <pop3:connector name="pop3connector" checkfrequency="5000" deletereadmessages="false"/> <model name="mail-model"> <service name="mail-to-file"> <inbound> <pop3:inbound-endpoint host="localhost" name="mule" password="mule" port="110" /> </inbound> <outbound> <outbound-pass-through-router> <file:outbound-endpoint path="chapter6/4b-mail-pop3/out"/> </outbound-pass-through-router> </outbound> </service> </model> </mule> 53

54 Connecting with FTP (write) <mule> <ftp:connector name="ftpconnector" validateconnections="true" /> <model name="ftp-model"> <service name="file-reader-ftp-writer"> <inbound> <file:inbound-endpoint path="chapter6/5a-ftp-write/in" /> </inbound> <outbound> <outbound-pass-through-router> <ftp:outbound-endpoint user="bob" password="123password" host="localhost" port="2121" outputpattern="${originalname}-${systime}.dat" passive="false" path="/" /> </outbound-pass-through-router> </outbound> </service> </model> </mule> 54

55 Connecting with FTP (read) <mule> <ftp:connector name="ftpconnector" validateconnections="true" /> <model name="ftp-model"> <service name="ftp-reader"> <inbound> <ftp:inbound-endpoint user="bob" password="123password" host="localhost" port="2121" PollingFrequency="10000" passive="false" path="/" /> </inbound> <outbound> <outbound-pass-through-router> <file:outbound-endpoint path="chapter6/5b-ftp-read/out" /> </outbound-pass-through-router> </outbound> </service> </model> </mule> 55

56 Business Process Management Description of an orchestration In terms of message flow Statefull Standard : WS-BPEL (Web Service Business Process Execution Language) Jboss jbpm (Jboss Business Process Management) project jpdl (jbpm Process Definition Language) jpdl integrated in Mule... 56

57 Bibliography General Enterprise Service Bus: Theory in Practice (O'Reilly) Technical Open-Source ESBs in Action (Manning) 57

58 Evolution : Mule AnyPointStudio 58

59 Evolution : Mule AnyPointStudio 59

60 Evolution : Mule Connector market place 60

61 ... 61