Enterprise Service Bus

Transcription

1 We tested: Talend ESB Enterprise Service Bus Dr. Götz Güttich Talend Enterprise Service Bus is an open source, modular solution that allows enterprises to integrate existing or new applications into their IT environment. In our test lab, we installed, configured and worked with ESB to see how it performed in real time. An Enterprise Service Bus (ESB) is used to provision or mediate services in a corporate network. It acts as middleware, handling the flow of data between applications. With an ESB, therefore, there is no need to define dedicated communication interfaces between individual applications. Instead, all applications communicate with the bus, which means they have a common, standardized data exchange hub. An ESB is usually deployed within a modular Service Oriented Architecture (SOA). In this type of environment, it can handle data communication between a variety of systems such as the customer relationship management, accounting and supply chain management modules. It processes queries from a number of different sources, including business applications, mobile devices, partners and Web browsers. For this, it utilizes a wide array of different services, such as Web portals, Web services or file servers. An ESB makes light work of system integration since there is no need to create custom scripts. So more time is spent making decisions instead of integrating systems. ESBs have been around for a number of years now. Solutions like IBM WebSphere, Oracle Fusion and webmethods from Software AG all deliver ESB functionality. These products are highly complex, however, and are best suited to large scale IT infrastructures. Up to now, ESBs have rarely been deployed in small to medium sized 1 networks. This is where Talend s ESB offering comes in. As mentioned above, the product is based on open source solutions, making it both highly scalable and relatively uncomplicated. As such, it is a viable option for environments that would previously have been considered too small for an ESB.

2 Closer look at Talend ESB Talend ESB combines an Eclipse based tooling environment with messaging, Web services, intelligent routing and data transformation. Talend enables developers to easily build Patterns (EIPs). EIPs are standard templates used to address common integration needs. Talend s development environment provides graphical palettes of component and Talend Administration Center. It speeds deployments and upgrades, allowing administrators to rapidly respond to integration requests. Application and data service events are easily monitored with drill down capabilities into individual service statistics for in depth analysis of inter process service activity. Talend ESB is available in three different versions. The Standard Edition is an open source version, the capabilities of which are limited to creating and using services. Talend Open Studio for ESB includes the Eclipse based development environment. Talend Enterprise ESB the solution we used in our test also has various management functions like a development environment for multiple users. A comparison of the different versions can be found at The Talend Administration Center is a centralized deployment console to manage the ESB solution Web and data services to quickly integrate heterogeneous IT environments, both on premises, across companies and in the cloud. The services can also be re used across multiple applications for maximum efficiency. On top of this, Talend ESB comes with failover and load balancing functions. The Eclipse based tooling environment allows developers to build, compile, test and deploy Java Web services, REST applications, data services and messaging routes from a single interface. In addition, developers can design integration solutions based on Enterprise Integration connector icons. To create new software, all developers have to do is drag the palette icons they require to their workspace and then define the associated parameters. In all, the Talend library has over 450 connectors supporting practically all types of sources and targets for data integration, data migration and data synchronization. These include files, databases, big data solutions and more. It is even possible to insert custom code, if required. We will describe our experience of creating software components later on. Talend ESB has a centralized deployment console called the 2 Technical background information Certain Talend ESB components are built on Apache. Web services, for instance, are based on the Apache CXF framework, which can be used to create SOAP and REST Web services. For service integration, however, Apache Camel is used. This rule based routing and conversion engine uses the EIPs mentioned above and is programmed with Java. The runtime container is based on Apache Karaf. The test For the test, we installed Talend ESB on a Windows 8 test PC with 8 GByte RAM, 750 GByte hard disk and an Intel Core Quad processor with 2.4

3 GHz clock speed. The Java environment used was Version 7 Update 17 the most recent one available at the time of the test. Following installation, we created a simple Hello program with Talend s Eclipse based development tool. When users variable so that the installer can find the Java environment installed on the system, and start the setup routine. You will then see a wizard displaying the license information. It will first ask for the license file, and then you have to specify the Our Hello program and the icons we used send their name to this program via the ESB, it returns the string Hello {Name of user}!. We then made the program available via the bus. After that, we submitted a database query via the ESB again using the development environment. Finally, we integrated an existing application with the ESB. For this, we used Talend s free demo car rental application supporting reservation and customer management functionality. It can be used for all test scenarios once the ESB has been installed. Installation Talend ESB is supplied in the form of an installation file and a depot file, which contains all the necessary components. To install the software, you simply save both files to the same directory, set the JAVA_HOME components you wish to install and their target location. For client configurations, for example, it often makes sense to install just the development environment. For our test, however, we decided to install all the components. Administrators also have the option of setting up the server components (Administration Center, Command Line, Job Server and Runtime) as system services or as products to be started manually. To complete the setup wizard, you are asked to configure the database for the Talend Administration Center and define the SVN server and similar settings. Getting started Once installation was complete, we started the Web server manually using Tomcat and the Administration Center plus Karaf as the container. We did this 3 because we had not set these up as system services in our test environment. Then, with our browser, we went to the page dministrator/. We logged in here using the default login data admin@company.com/admin. For security reasons, the default account only gives you authorization to manage user accounts. This is how Talend ensures that no standard login data can be used during day to day use of the system, and we found this a very sensible precaution. We completed the first step of creating an administrator account, logged out of the system and then logged in again as administrator. The administration tool The Talend Administration Center uses a transparent tree structure to give users fast access to the configuration items they are looking for. The administration tool allows you to define settings for projects and users, view the Dashboard with connections and job analyses, monitor services and manage the servers. As we first needed a server for our test, we created one at this point. To do this, we defined the communication ports, gave the server a name, specified the host and the timeout, etc. As an option, communication with the server can also be secured via SSL. Once we had defined the server, it appeared on screen, immediately displaying its status and the current configuration details. Administrators have the option of switching to the Apache Karaf Web Console by clicking the Admin Server button if they need to. Talend ESB uses Apache Archiva as its repository management software. When we

4 tried it out, we noticed that the installation routine read the path to the Java environment during setup and then entered it in the Archiva file wrapper.conf. If you update your Java installation and the path changes (for example, if you update from Java 7 Update 15 to Java 7 Update 17), it is not enough to reset the JAVA_HOME variable because the wrapper.conf entry will still be there. If you then try to start Archiva, the system will respond with an error message. The only way to get the system up and running again is to manually change the path in the wrapper.conf file. A workaround solution for this problem would be to always use the same fixed path to the Java environment. Talend intends to fix this problem in one of its upcoming software releases. The Hello application For the next part of our test, we created the Hello application. We started by opening the Talend development environment and created a new project. We used the icons mentioned earlier to set up the Hello program. Specific functions are represented by icons in the development studio. They are grouped in a palette on the right hand side under categories such as Big Data, Business Intelligence, Cloud, File, Databases, ESB, Custom Code, and so on. When the developer drags an icon from one of these groups and drops it on their workspace, the Talend Development Studio generates the code automatically in the background, so all the user has to do is define the parameters for the icons. Let us assume that a user wants to define an Access database as the data source. All they have to do is go to the folder Databases/Access in the palette and drag the AccessInput component to the workspace. Then it s just a matter of entering details like the Access version, the path to the database, the user nearly every task can be accomplished using icons. This reduces manual programming to zero in most cases. As such, even users with little or outdated programming expertise will find the solution easy to use. For our Hello program, we used a similar approach to that described Once a service has been compiled and made available, the development studio displays the URL of the service in the workspace (here: name, etc. in the component s configuration dialog boxes. This completes the configuration process for the data source and the database is now ready to be used. In the next step, the user can for example select or re arrange particular database entries using a mapping function. To do this, they just have to drag a mapping icon to the workspace, define the mapping function and create a link between the data source and the mapping icon (by drawing a line between the two icons with the right mouse button). To finish off, the user just needs to generate a data output (again using an icon) to enable data transformation. Since many of the functions are pre defined, 4 for the data transformation demo program. Our objective was not just to transform data, but to communicate with the ESB. The program setup was similar, however. First of all, we created a new service called Hello and imported the WDSL schema. This ensured that the service metadata was available in our repository so we could reuse it in other components. We then added a new job to the service. This opened the workspace, which already contained two icons. The first was an ESBProviderRequest icon. This receives service requests from the bus and forwards them as appropriate. The second icon was an ESBProviderResponse component, which sends the response to the bus. Next, we

5 dragged an XMLMap icon from the XML folder in the palette and added it to our project. We added a schema for the transferred data (using the WDSL schema data we had previously saved in our repository) and modified the data flow via the mapping function so that every user who sends their name to the Hello service via the ESB receives the response Hello {Name of user}!. All we had to do was establish a link between the request and response components with the mouse and edit the response text so that Hello is inserted before the input and a! string after the input. By now we had completed our program, and so were able to compile it and make it available using the Execute command in the configuration toolbar beneath the workspace. To use the Hello service, we had to create software that would send the relevant names to the ESB. We did this by creating a new job and adding a FixedFlowInput icon from the palette. We again added an XMLMap component, plus one ESBConsumer and two LogRow components. The FixedFlowInput routine generated the input. We could instead have taken a database or similar, but to keep the demo program simple, we decided to define a static table with three names which we created directly during the input icon configuration process. The XMLMap component had the job of forwarding the input of the input routine to the service. We imported the service information within the mapping function from the repository it had been saved there when the service was defined and linked it with the input from the names table using drag and drop. The ESBConsumer function then called the service and delivered the response. We had to enter the URL of the service address of our previously created Hello program in the configuration options. The studio provides this as output after compiling the software and making it available in the workspace. So that we could see something on the screen, we used the two LogRow components mentioned earlier. One of them accepted the response and the Service for checking book titles second was used as the destination for any error messages. Once we had started the job, the three names we had entered in the table and the associated Hello messages appeared in the output space of the development environment. The program therefore worked as expected. Based on our previous experiences, we concluded that the software development with Talend ESB is a smooth and easy process for IT professionals in enterprise networks. 5 Deploying Hello program in a Talend runtime container Once we had verified that our software worked correctly, we set about exporting the program to the Talend runtime container so we could use it independently of the development studio. We right clicked our Hello service and selected the command Export Service. A Save as dialog then appeared. We saved the software directly to the Deploy subdirectory of the Talend runtime container (directory \esb\container\deploy) to make it available for immediate deployment. When the export was completed, our Hello service was available in the container. To test its functionality again, we used the same demo program as before. All we had to do was change the port in the service URL from the test port of the studio (8090) to the default port of the Talend container (8040) in the configuration dialog of the ESBConsumer function. We did this to make sure that the system communicated with the right service. We started the program

6 again and everything worked just like before, except now we could also view the software output in the output window of the container. Routing As mentioned earlier, Talend ESB includes a powerful routing function. This can be used to source versions of the ESB software. Data queries After completing the Hello software test, we turned our attention to the data query job. Here the user is free to choose any data source from a database to a small file. For our test program, we used a CSV file containing a number of book titles and the names of the authors. For communication with the bus, we again used a job with the ESBProviderRequest and ESBProviderResponse icons. Mapping was again handled by an XMLMap component. For this job, we also added our data source to the program, and The graphical user interface of the connected it to the software using Rent a Car demo a File/Output FileInputDelimited distribute inputs to various icon. We then configured the instances of a service, generate a mapping function so that the variety of outputs or filter out program compared a book title duplicate data. query received from the bus with the title column in the CSV file It also supports different software and returned the title to the bus if versions used by various partners a hit was found. This completed to send data specifically adapted the program so it was ready for to the version in question. network access. Incidentally, the However, a detailed description program we used to send the of this functionality would go titles to the bus in the test was a beyond the scope of this test. We specially adapted version of the will just mention that routes can program we had previously used be created using the mediation to send the names to our Hello capability of the development service. studio and they can be configured using icons like in the Rent a Car demo software development process. Talend provides a car rental demo application for the ESB. Routes can also be exported and This software is programmed to deployed in exactly the same manage customers and vehicles way as services. Routing is and make reservations for described in great detail in the specific dates. The code for the comprehensive and helpful application is available in the documentation found on Talend s development studio, but there is website. This can be downloaded also the option of integrating the for free, as can the two open software as a complete 6 application in the runtime environment of the ESB. In this test, we chose the latter option since we had already tested service provisioning via the studio. The data for the demo application is located in the Talend installation folder under esb\examples\talend\tesb\rent a car. Apache Maven must be installed on the computer used for the software build process. If this is the case, the program can be installed by executing the mvn clean install command in the Rent a Car directory. When we had completed this step, we set about deploying the demo services in the Talend runtime container. This step is executed via the OSGi command line and is completed after entering two commands. The same applies to deployment of the demo client application. We encountered no hitches whatsoever when testing the integration of an application in Talend ESB. Testing the Rent a Car demo Once we had installed the Rent a Car demo, we launched the graphical user interface and used it to simulate a customer completing a rental transaction. To do this, we selected a customer account from the customer database when the tool booted and entered the rental date and return date. When we clicked Find, the application called CRMService via the ESB to find the customer data and customer status information. The software then displayed a list of available vehicles to choose from. After that, we clicked Reserve. The application then invoked

7 ReservationService to forward the query. Simultaneously, the The list of available rental cars program showed us a confirmation message with the reservation details. In addition to a graphical application, the demo also includes a command line application, allowing the user to search the databases and make reservations. What s more, the demo can be extended with a Service Locator component and a Service Activity Monitoring (SAM) function. different containers and simulate server failover. In addition, the demo shows how the Web services security feature (WS Security) protects Web services and how repository management software works with the demo. All of these functions go beyond the scope of this test. They can, however, be tried out with the free versions of the software, which are available to download from the Talend website. They are also comprehensively described in the ESB Getting Started Guide. We would therefore recommend that you install ESB and explore its performance and functionality for yourself. Summary Talend ESB is relatively easy to get up and running and to manage. We installed the software in a matter of minutes, The system can therefore be configured and managed with ease. But the main highlight of ESB has to be its development environment and icons, allowing users with little or even no programming expertise to easily create data services and routes. The Talend ESB development environment does more than just facilitate software development, however. It also provides users with solution integration, testing and deployment support. The other components of the ESB solution, like the Web services stack, load balancing and high availability features, containers and a repository, are equally impressive. All of this makes Talend ESB an excellent choice for enterprises of all sizes thinking about introducing a service bus, but reluctant to go with a product from one of the big players such as IBM or The Service Locator publicly registers the service endpoints. This means that service consumers are able to locate endpoints during runtime, without having to know where exactly the endpoint is running. In addition, the Service Locator component allows service providers to register or deregister their endpoints automatically and thus make them available to users on demand. The function of the SAM, however, is to monitor service calls. The data collected can be used to compile usage statistics and pinpoint faults. Users can also run the demo in two The repository with Archiva and were instantly impressed with both the clear layout and range of capabilities of the graphical administration console. 7 Oracle. This mid sized segment opens up a completely new market for Enterprise Service Buses.