Methods and tools for data and software integration Enterprise Service Bus

Transcription

1 Methods and tools for data and software integration Enterprise Service Bus Roman Hauptvogl Cleverlance Enterprise Solutions a.s Czech Republic Abstract Enterprise Service Bus (ESB) is closely related to Service-oriented architecture (SOA) and business integration. It is important to know the functionality and the possibilities of ESB when we are thinking of its usage in the organization. In the following paper there is described the role of ESB in SOA. There is a closer look at the usage of ESB in enterprise integration and an introduction to some common integration patterns which are closely related to ESB. Keywords Service-oriented architecture, SOA, Enterprise Service Bus, Integration, AquaLogic Service Bus 1 Introduction In many companies there are trends to moving to SOA. They invest a lot of financial and also nonfinancial resources to achieve benefits from SOA as soon as possible. Organizations are rethinking their environments and redesigning their architectures. They are creating coarse-grained, loosely coupled s and promoting reuse, that are the basic principles of SOA [2]. In the last time a lot of books and articles appeared, which discuss various aspects of SOA. There can be found a lot of theoretical information, which can be used by organization for SOA implementation. Problem with successful SOA implementation is, that there is a need to learn not a single new technology but a whole series of different technologies. For example XML is technology that is used through the whole SOA, but with XML also come a lot of other technologies like XML Schema, XQuery, Xpath, XSLT and many more. As described in [11] there is a huge potential of SOA in business integration. Because market today demands changes so rapidly, IT organizations need an intelligent infrastructure to gain reuse and delivers reliable integration across the various technologies, protocols, and applications typical in today s business IT environment. In organizations there are many different systems, applications and components which should be integrated or in other words, need to communicate together. There are already some open standards, which help us to integrate various systems communicating through various protocols. Typically, Web Services can be taken into account, which can exposed useful functionality through interfaces and build so reusable components also called s. There are many different definitions of the. In [1] it was defined as A facility providing the with some benefit, such as food or electricity, or some capability, such as communication or SYSTEMS INTEGRATION

2 ROMAN HAUPTVOGL access to information. Services act on behalf of the, shielding the from the implementation details.. There were also defined basic principles of the : Implementation is not part of the. Everything that requires knowledge is part of the. Services are governed by contracts. Contracts define what the does and how and when the will be delivered. As s proliferate in the SOA environment, the traditional approach have been to connect each end point together, creating a sprawling web of point-to-point connections. While this initially seems to be the best way to address connectivity, this web sprawls rapidly and becomes costly. Enterprise Service Bus (ESB) is specifically designed to eliminate such costly point-to-point integrations, which limit IT s ability to efficiently and quickly deploy s that support their SOA initiatives. It is important to know that SOA is not only about redesigning and reimplementing all the systems, applications within the organization, but keep in mind that one of the basic principles of SOA is reuse. This reuse is mainly about creating components, designed in specific manner, that can be used several times. But this reuse can be also conceived as: Reuse as much existing applications and components as possible.. This is probably the hardies to understand, that there is need to decide which systems, applications or components in the company are ready to redesign and re-implement, because it is nearly impossible to accomplish SOA like a big boom. It is iterative process, which needs to mature in the organization. 2 Basic principles When communicating with s, there is on one side which is requesting the information or in another words it can be called consumer. On the other side there is provider which provides desirable information. Both consumer and provider can reside in different environments, use different standards and also different communications protocols like HTTP(S), JMS, FTP, SMTP, RMI. There is need to solve the integration in the unified and standard based manner. And thats the role of the Enterprise Service Bus (ESB). There are a lot of different implementation of ESB. Although we can feel huge influence of the open source technologies in IT nowadays, bigger companies still prefer to choose one of the commercial solutions. This is mainly because with commercial version comes also a support from the organization providing the solution, which is desirable. All the main vendors specializing on the enterprise applications, like BEA, IBM, TIBCO, Microsoft and many more, provide their own integration strategies and also provide platforms for realization of SOA based integration. ESB is one of the main product in SOA stack so it is also provided by them. In the following paragraphs I would like to aim on the BEA Aqualogic Service Bus (ALSB) which is the product of the BEA Systems. BEA AquaLogic Service Bus (ALSB) is a configuration-based, policy-driven ESB. It is targeted for -oriented integration, managing Web Services (WS), and providing traditional message brokering across heterogeneous IT environments. It combines intelligent message brokering with monitoring and administration in a unified software product. From the development perspective ALSB does not follow the standard development life cycle, which among others usually consist of implementation, compile and deployment phase. The development in ALSB is different. The process is configuration driven. It means that there is no need to work with source code, compile it and afterwards deploy your application to the application server, which has a lot of advantages. First of all, the development is faster so IT can react more flexible to business needs. Changes are made within an active session and are propagated to the server as soon as the 366 SYSTEMS INTEGRATION 2008

3 METHODS AND TOOLS FOR DATA AND SOFTWARE INTEGRATION session is confirmed. There is no need to learn a new programming language. There is no need to spent time with development, to learn the source code, to compile it and finally deploy resulting application. It is mainly about the configuration, in other words it can be treated as another clicking tool which can help us to visualize the process flow. This process flow is not the business process, but the composition of the activities like validation, transformation, routing and so on. The responsibilities of these activities will be described later in this paper. It is another step to the effective business integration, the need of mutual communication between IT and business people. Because from the business perspective, SOA-based Business integration starts with the defined business objective and its associated processes. Business analysts use modeling tools to sketch up or visualize the process flow. 3 Architecture The main purpose of the ESB is to provide infrastructure that connects disparate applications and other resources and provides flexible reuse of business components. ALSB consists of several resources which can be configured to create desirable integration. There are two fundamental resources in ALSB. First one is so called proxy, which provides the interface to s requesting (some useful information). The proxy s also define what happens to the message when it enters an ALSB. Second one is business which represent the enterprise that provides some useful information. Both proxy and business s allow decoupling between s and the back-end s. The architecture is shown on the Figure 1. Figure 1: ALSB architecture ALSB proxy proxy business business business business The communications within ALSB is handled in several layers. When the message arrives to the bus inbound transport layer is responsible for handling of this data. This data are raw bytes in form of input/output streams and can be bind to a different transport protocols. The inbound transport layer is simply for getting messages into ALSB and sending responses back. In the inbound binding SYSTEMS INTEGRATION

4 ROMAN HAUPTVOGL layer all the data from the message are taken out and bind to the internal variables of the ALSB. The purpose of these variables is the message processing. They are in the XML form and represent message payload, message headers, attachment and transport specific parameters. Afterwards the processing of the message can be done in the proxy. All the changes are made to the context variables with use of common XML technologies like XQuery, XSLT, XPath. The processing of the message usually finished by routing message to the business, which represent provider. Sending of the message to the provider is done in similar manner but in reverse order. At first the message is composed from context variables, which is done in outbound binding layer. And then in outbound transport layer is message bind to the transport protocol in the form of input/output streams. The communication can be seen on the Figure 2. There is shown request and response path of the message. This way there is no need to take care of low level features, like binding to the transport protocol. Figure 2: Communication within ALSB inbound outbound request transport layer binding layer proxy binding layer transport layer response ESB is backbone of your company, so it should be reliable. All of the messages go through it, so it is good place where to place the monitoring of your systems. ALSB provides the capability to monitor and collect run-time information required for system operations and aggregates run-time statistics. Features that can be monitored are for example: min response time, max response time, overall average execution time, total number of messages, messages with errors, number of validation errors and so on. You can also establish level agreements (SLAs) for the performance of your system, configuring rules that trigger automated alerts. If this alert arise you can be notified for example via mail. SLA guarantee a certain level of performance and also define the system response if those guarantees are not met. 4 Best Practices The main principle of the ESB is to have applications connected to bus in a manner of loosely coupled s that do not have direct knowledge of each other. It is very important, because even if there are changes in the s, consumer do not need to know about them and even do not need to react on them. It is ESB's responsibility to ensure that providers and consumers communicate together in loosely coupled manner. In another words consistent data need to be passed 368 SYSTEMS INTEGRATION 2008

5 METHODS AND TOOLS FOR DATA AND SOFTWARE INTEGRATION within the bus. In previous chapter I have described an architecture which can help us to fulfill this purpose. In this chapter I would like to introduce some best practices. VETO Pattern There is common integration pattern which is delegated to ESB and can ensure that consistent and valid data will be passed through ESB. This pattern is known as VETO pattern. It is acronym made from the first letters of the activities Validate, Enrich, Transform and Operate. As it was mention in previous chapter, all the context variables including message payload are in the form of XML. It means that to perform activities included in VETO we will be mostly working with XML technologies. The validation of the incoming message should be done as soon as the message arrives to the ESB. Its role is to check whether incoming message contains a well-formed XML document and conforms to a particular XML schema or WSDL document that describes the message. If we will perform this task later after some processing, we could find out that we are missing some element, which will lead to rejecting of the message and useless previous processing. The enrichment of the message involves adding of an additionally needed elements, from an external source, to the message to make it more meaningful to a target or application. This action can be performed by calling another s and receives so additional informations. The transform step is responsible for the converting of the message to the target format. It is important to transform message to the valid format in order to prevent target from rejecting it, because it can have its own validations process. The operate step is used to send the message to the target. Split-Join Flow Pattern I have choose the following pattern to demonstrate that there are situations when there is need to use also different products to solve some problems. All the s implemented on the ESB are stateless. There are different integration products and also technologies, like for example Business Process Management (BPM) and Enterprise Java Bean (EJB), which are responsible for implementation of the statefull processes. If there are large messages which require processing for different portions of the message, they can be split into smaller messages. This smaller messages are routed to target s according to the content where they are processed. But for creating the response message back to the there is need to develop an aggregate, which will be responsible for collecting the responses from all the target s and sending message back to the bus. Service bus will process the message and send the response message to the. Development process What I like on ALSB is the different development approach, configuration driven development. All the work in the ALSB is done within an active session. When the session is confirmed, the changes will take an effect. Of course if there are some conflicts or some invalid configurations within the session, session is not allowed to be confirmed. Each confirmed session can have a short description. The changes in the configurations can be seen in the form of the sessions logs, which are these descriptions of the sessions. But the problem is, that it is the only way how to see the changes. That I find not sufficient, especially when you are working in a team. Each confirmed session can be discarded, which will be resulting in the change of the state of the ALSB to the state before the session was started. It is important to choose the description of the session so, that we will recognize the session and in case of problems could execute the rollback. SYSTEMS INTEGRATION

6 ROMAN HAUPTVOGL Also the point of view on version control is different, but you need to keep on mind that implementation is not part of the so there is no need to have common version control system. With version control of our s can helps us some of the SOA governance systems like Enterprise Service Repository or Enterprise Service Registry. They can bring us an effective SOA governance. It keeps SOA on the track to deliver maximum reuse, maximum agility, and maximum Return of Investments (ROI). It addresses common SOA challenges: Lack of visibility into planned and existing s Growing complexities in the enterprise, both business and IT Inability to manage the impact of change Limited, if any, insight into demonstrated savings, ROI and benefits If there is no such SOA governance system it is at least wise to version control the artifacts like WSDL, XSD, policy files and so on. Like in the normal development process there are best practices which also can be applied for development process on ALSB. It is wise to set up the coding conventions, like naming conventions, folder structure and so on. It can help us to get uniform environment. Another thing is to prepare the scripts for moving from development to test environment and from test to production environment. This scripts can help us to set up the proper endpoints of our s. For this purpose can be used so called WebLogic Scripting Tool (WLST). It is BEA tool for scripting the configuration of BEA domains. The secret of its power is its scripting language, Jython, an implementation of the popular Python language in Java. 5 Conclusion In the paper I have tried to describe the role of ESB in SOA. As it was mentioned earlier, the main purpose of the ESB is to provide infrastructure that connects disparate application and other resources and provides flexible reuse of business components. From my own experience I can say that implementation of the processes on ALSB is very fast and effective. I think that ESB is not designed for all the companies, but only for those, which have quite a lot of different s and these s are still evolving. ESB has it special position in the SOA Environments. Where I see the strength of the ALSB is in it's simplicity and rapid response to business needs by quick configuration of routing rules based on changes in business rules. 6 References [1] Jeff Davies: The Definitive Guide to SOA: BEA AquaLogic Service Bus; May 2007; ISBN: [2] Thomas Erl: Service-Oriented Architecture (SOA): Concepts, Technology, and Design; August 2005; ISBN: [3] BEA White Paper: The Integration Journey A Field Guide to Enterprise Integration for SOA [4] BEA White Paper: BEA AquaLogic Service Bus IT's Direct Route to SOA [5] BEA WebLogic Integration, Introducing BEA WebLogic Integration; Version 9.2; November SYSTEMS INTEGRATION 2008

7 METHODS AND TOOLS FOR DATA AND SOFTWARE INTEGRATION [6] BEA AquaLogic Service Bus, Concepts and Architecture; Version 2.6; January 2007 [7] CBDi Forum Report Tim Barrett: Enterprise Integration Middleware Positioning; 2000 [8] SOA World Magazine - Dave Chappell: ESB Myth Busters: 10 Enterprise Service Bus Myths Debunked; May 2005 [9] Buchalcevová A., Hauptvogl R.: Get ready for SOA; System Integration Conference; 2007 [10] Kenny Shin: Practical Enterprise Service Bus Use Cases for SOA; June 2006 [11] BEA White Paper: BEA AquaLogic Service Bus A Technical Review of Architecture and Functionality for Service Integration and Management SYSTEMS INTEGRATION

8 ROMAN HAUPTVOGL 372 SYSTEMS INTEGRATION 2008