WHITE PAPER Talend Enterprise ESB Technical Overview
Table of Contents INTRODUCTION... 3 INTEGRATION WITHOUT CODING... 3 Productive Tools... 3 Best-in-Class Data Services... 4 COLLABORATE ON PROJECTS... 4 Promote Build Process Best Practices... 4 DEPLOY AND MANAGE CENTRALLY... 5 Security In a Snap... 5 UNDERLYING DISTRIBUTED ARCHITECTURE... 5 Versatile and Flexible Web Services... 6 Dynamic Mediation and Intelligent Routing... 7 Enterprise Messaging... 7 POWERED BY APACHE INTEGRATION PROJECTS... 7 ENTERPRISE CLASS... 8 Establishing Trust and Authentication... 8 Identity Management and Authorization... 9 Service Registry... 9 Service Activity Monitoring... 9 Supported by Key Apache Committers... 9 Easy To Extend... 10 OPEN SOURCE LICENSE... 10 START TODAY... 11 APPENDIX A... 12 ABOUT TALEND... 15 2
Introduction Talend Enterprise ESB is the first to combine application integration with data management to allow businesses to cope with the volume of data in an increasingly connected world. Talend Enterprise ESB enables developers to easily build reliable, scalable and secure REST, Web and data services to quickly integrate heterogeneous IT environments, both on-premises and in the cloud. Built on an Apache licensed microkernel architecture, Talend Enterprise ESB is a reliable and scalable enterprise service bus (ESB) that allows development teams to holistically address integration projects combining application integration and data management for complex, heterogeneous IT environments. Talend Enterprise ESB frees data management processes that are traditionally bound to specific applications by abstracting them as standards-based services that can be reused by other applications. The reuse of data services across multiple applications maximizes the value of initial investments in standalone data integration solutions. Integration without Coding Only Talend allows developers to develop, build, test and publish Java Web services, REST applications, data services and messaging routes all from a single user interface. Talend Enterprise ESB combines the productive tooling features of Talend Open Studio for ESB with advanced deployment, collaboration, security and management functions all which are described in more detail in the following sections. Productive Tools Talend Enterprise ESB (Talend ESB) includes an innovative, Eclipse-based tooling environment for modeling, configuring, and deploying integration solutions. Projects are deployed faster and developers are more productive as they respond to integration requirements. Talend ESB eliminates the lengthy learning curve typical of most integration tools and allows developers to leverage existing skillsets through an easy-to-use, drag-and-drop Eclipse environment as they rapidly integrate systems, applications and data sources. Users can simply drag-and-drop basic integration components and connectors onto a graphical workspace, draw connections and relationships between them, and set specific properties. 3
Best-in-Class Data Services Talend ESB allows users to effortlessly combine data integration with a Web service using a wide assortment of components and connectors. Users can leverage a WSDL-first approach to integration to efficiently deploy multiple data services behind a single interface on a common ESB runtime. Talend ESB leverages the industry s leading open source data integration capabilities to provide access to a library of over 450 connectors, supporting all types of sources and targets for data integration, migration or synchronization. Talend ESB supports data services creation to free data management processes, whether they reside as structured or unstructured data, EDIFACT, SAP connectors, or event-based ETL. Data services take such processes that are traditionally bound to specific applications and often batch in nature, and abstracts them as standards-based services that can be reused by other applications. Collaborate on Projects Talend ESB facilitates collaboration by allowing users to store and share information in a common repository while preserving version integrity throughout the lifecycle of the integration project. Developers work in the innovative, Eclipse-based tooling environment for modeling, configuring, and deploying integration solutions. The graphical workspace supports the drawing of connections and relationships between components and allows users to set specific properties. It includes basic integration components and connectors to create Web services, data services, REST applications and mediation routes. Promote Build Process Best Practices The Talend Repository further facilitates best practices and collaboration in integration projects by storing ESB development and operations artifacts such as projects, integration patterns, metadata and other items and allowing them to be shared among a team of developers or administrators. Talend ESB encapsulates Software Development Lifecycle (SDLC) best practices, by allowing multiple repositories to be deployed. Software artifacts are versioned and can be promoted between repositories, depending on the stage within the development lifecycle process. The Repository supports Sonatype Nexus OSS as the preferred default archive repository. 4
Deploy and Manage Centrally Talend ESB allows organizations to more rapidly respond to integration requirements by deploying and managing upgrades through a centralized deployment console. Developers and development administrators gain complete control over the deployment and upgrade process for Web services, REST applications, data services and messaging routes deployed to Talend ESB. Talend ESB also includes a graphical console that facilitates administration of production integration environments by providing service activity monitoring (SAM) and service locator capabilities. Talend s SAM functionality tracks messages across multiple service invocations, allowing users to drilldown into inter-process application and data service events through this event-based monitoring capability. Users can perform in-depth analysis to address service level agreements (SLAs). The Talend Administration Center (TAC) supports role-based access control (RBAC) such that only administrators with the appropriate privileges can setup and deploy ESB instances using the secured repository. The TAC also includes extended event logging with Kibana-based visualizations for monitoring services and message flows. Any type of event or message can be tracked, stored, and used to trigger an actionable process, making it easier to manage security events, auditing, user activity, and service and message lifecycles. You can store all of your logging information into a centralized location for easier processing and analysis. Security In a Snap Talend ESB provides many wizards and graphical tools to easily create and manage policies, security services, and identity and access management requirements. The XML Key Management Specification (XKMS), which Talend contributed to Apache CXF, uses the web services framework to make it easier for developers to secure inter-application communication using public key infrastructure (PKI). Underlying Distributed Architecture Talend ESB employs a fully distributed architecture, which means it does not depend on a central server, or hub, to support applications and users. A distributed approach to SOA allows the creation and deployment of services through a variety of options. This includes deploying service enablement on the same machine as the application, on the client side, or as an intermediary. 5
The most efficient of these deployment patterns involves service enablement as part of the application itself. The key point of the distributed approach to SOA is that it does not impose any particular deployment architecture, such as hub and spoke or mid-tier server and enables services using a minimum of effort and expense. It is easily and infinitely scalable to fit customer environments and to accommodate growth. The smallest unit of the ESB is a lightweight container into which legacy system adaptors can be integrated or application services run. Talend ESB is not dependent on a specific transport technology. Rather, it can use the most appropriate transport mechanism available, thus ensuring interoperability with customers systems that do not use Talend ESB. In cases in which asynchronous transport is called for, Talend ESB uses a messaging system while keeping both systems independent. Talend ESB also supports the ability to add quality of service policies including security, reliability, transactionality and configure them as needed. Versatile and Flexible Web Services Talend ESB allows Java developers to quickly create RESTful and SOAP Web services as part of a SOA. Its modular, lightweight architecture allows users to build composite applications for reuse across multiple projects. Talend ESB comes pre-configured to run on any Java container used in the vast majority of IT environments today. Streamlined deployment container setup and configuration-not-code increase developer productivity, allowing programmers to focus on business logic instead of IT complexity. 6
Talend ESB offers a comprehensive set of features for building secure Web service applications using established industry standards for XML encryption, authentication and authorization. Developers can add these features to allow consumers to be configured for automatic access. Dynamic Mediation and Intelligent Routing Integrated business operations involve many complex relationships among various systems and applications and what to process when and where can change rapidly. Talend ESB provides dynamic mediation and routing capabilities based on enterprise integration patterns (EIPs) to meet the demands of these ever-changing systems (see Appendix A for a list of supported EIPs). You can base mediation and routing decisions on architecture, business rules and deployed enterprise integration patterns. Talend ESB enables such routing decisions at the endpoint. This eliminates bottlenecks and results in a scalable, high performance solution. Talend ESB includes features for developers who need to integrate ESB services with BPM processes, such as Bonitasoft. ESB and BPM services can be connected using SOAP services, REST services and Camel routes. Enterprise Messaging Powered by Apache ActiveMQ, the most popular open source messaging provider, Talend ESB minimizes hardware and software costs by making efficient use of computing resources. Each message broker can handle myriad connections while processing thousands of persistent messages per second with minimal latency. ActiveMQ supports native clients in Java, C#, and C++ for connectivity to all of your enterprise systems. It can be deployed as a stand-alone broker or embedded with applications for peer- to-peer architectures. Powered by Apache Integration Projects Talend invests, promotes and nurtures the underlying technology of Talend ESB through its participation in the Apache Software Foundation (ASF), a leading open source community recognized for its transparency and for delivering enterprise-grade, freely available software products that are designed and built from a diverse community of users. Talend ESB is powered by the leading Apache open source integration projects, including: Apache CXF an open source services framework, CXF helps you build and develop services using front-end programming APIs like JAX-WS and JAX-RS. 7
Apache Camel a powerful open source integration framework, Camel lets you create Enterprise Integration Patterns to implement routing and mediation rules. Apache ActiveMQ the most popular and powerful open source messaging provider, ActiveMQ is fast and fully compliant with the JMS standard. It supports many Cross Language Clients and Protocols, supports EIPs and many advanced features. Apache Karaf a small footprint, OSGi-based runtime, Karaf provides a lightweight container onto which various components and applications can be deployed. Apache Cellar a clustering solution for Apache Karaf powered by Hazelcast. Apache Syncope for managing digital identities in enterprise environments. These integration projects are deployed widely in many major global enterprises and provide application integration capabilities for multiple industry-leading middleware solutions today. Enterprise Class Talend ESB is built, tested, QA d and certified using Talend s real world experience deploying large-scale enterprise IT environments. Talend ESB is available as a single package preconfigured and ready to run right out-of-the-box and Talend provides documented examples to reduce the learning curve, so developers can get started quickly. Establishing Trust and Authentication Trust can be difficult to establish within and across IT domains. Talend ESB provides a Security Token Service (STS) Framework, which allows trust to be established between parties using the latest security standards for Web Services. The STS Framework issues interoperable security tokens to negotiate claims between consumers, services providers and 3rd parties, establishing trust and authentication. By abstracting authentication through brokers, you can establish secure connections through a common trusted entity in situations where parties cannot establish trust directly. Through the STS Framework, clients and services can securely and transparently authenticate during connections without custom coding. The STS Framework also supports Security Assertion Markup Language 2.0 (SAML 2.0) to federate security credentials and can be extended to create custom STS, tailored to meet the needs of existing security environments. 8
Identity Management and Authorization Talend provides identity services (who are you) and authorization services (what are you allowed to do). The Talend Identity Service is based on Apache Syncope and provides centralized management of runtime users and roles. The Talend Authorization Service is based on the extensible Access Control Markup Language (XACML) standard and provides access control at the service / service operation level for SOAP services and access control at the URL / verb level for RESTful services. Service Registry Talend's Service Registry supports the many real-world situations where multiple providers for the same logical service differ significantly in the quality of service they provide and/or the assumptions they make about the caller (such as requiring authentication with a specific security token etc.). Policies are a common and standardized way of describing such assumptions in a machine-understandable fashion. With Talend Service Registry support is provided for the authoring and creation of these policies and WS-Policies. It uses a wizardbased user interface (UI) to help the developer complete these tasks, and then allows you to store and retrieve policy documents with the Talend Service Registry service. The Talend Service Registry also provides a simple interface that allows administrators to browse the contents of the registry, deploy and un-deploy artifact collections and perform queries. Service Activity Monitoring All hardware and software no matter how reliable eventually fails. Through the Service Locator, Talend ESB provides automatic and transparent failover and load balancing via dynamic endpoint registration and lookup through Apache Zookeeper. Service Locator maintains the availability of the service to help meet demands and service level agreements (SLAs). Talend ESB also supports the management of services through its Service Activity Monitoring functionality. Service Activity Monitoring captures events and stores this information to facilitate in-depth analysis of service activity and track-and-trace of messages throughout a business transaction. This can be used to analyze service response times, identify traffic patterns, perform root cause analysis and more. Supported by Key Apache Committers Talend ESB provides a fully certified and tested solution with support from the same key Apache developers that build the software. Talend employs Project Management Committee (PMC) chairs and members for Apache CXF, Apache Camel, Apache Karaf, and Apache ActiveMQ. By registering with the Talend Community, developers can collaborate with peers, 9
submit feedback or issues through JIRA and interact with Talend engineers to gain valuable advice. Easy To Extend As your requirements expand, you can easily upgrade Talend Open Studio for ESB to Talend Enterprise ESB to Talend Platform for Enterprise Integration. Talend Platform for Enterprise Integration adds business process management, advanced data mapping, and data quality functions into a highly scalable and available architecture. More information on the differences between the products is online here: http://www.talend.com/products/matrix/platform-forenterprise-integration. Open Source License No longer restricted to large enterprise IT, Talend continues to democratize the integration market by making the ESB more affordable, accessible and easier to use. Open source enables enterprises to leverage a more open and affordable integration platform to reduce complexity. Talend provides users of open source with documentation, training and other services to help build robust enterprise software that can be trusted to run mission-critical IT systems all backed by professional support when needed. Talend Open Studio for ESB and its source code is provided under the LGPL v3 open source license agreement terms. Talend ESB Standard Edition (SE) provides the core ESB runtime which is licensed under Apache License v2.0. This grants users the openness and flexibility to get started quickly without the large upfront costs of enterprise software. Talend ESB SE is a pure runtime-based ESB, that is based on the most popular Apache integration projects Apache CXF, Camel, Karaf and ActiveMQ. Talend Enterprise ESB and Talend Platform for Enterprise Integration come as subscription licensed products. 10
Start Today Integration projects come in all shapes and sizes. The integration solution that you choose needs to be flexible to accommodate a variety of use cases. If the business processes and applications you are looking to integrate are mission-critical to your business, your integration solution must also be reliable, secure and deliver high performance. The future of a business is hard to predict. Choose an integration solution that is extensible to easily adapt to these changes. Integration solutions should be open, modular and easy-to-use to provide the greatest return on investment to the organization. And finally, integration projects can be complex. Make sure that your solution doesn t require an army of expensive consultants or the purchase of a stack of technology that you do not intend to use. Talend provides professional training, documentation and support around Talend ESB to help build robust enterprise software. As part of its support offerings, Talend includes legal guarantees of intellectual property protection (IP indemnification) to safeguard mission-critical IT systems. Both Talend Open Studio and Talend ESB SE are free to download and use, for as long as you want. No budget battles or endless delays just faster, easier application integration starting today. Talend Enterprise ESB is the easy-to-upgrade solution when you need the features and support required for advanced security, management and deployment. To download Talend Open Studio for ESB or Talend ESB SE, please visit www.talend.com. 11
Appendix A Talend ESB supports the key Enterprise Integration Patterns (EIPs) shown below, and many more. Messaging Systems Message Channel How does one application communicate with another using messaging? Message How can two applications connected by a message channel exchange a piece of information? Pipes and Filters How can we perform complex processing on a message while maintaining independence and flexibility? Message Router How can you decouple individual processing steps so that messages can be passed to different filters depending on a set of conditions? Message Translator How can systems using different data formats communicate with each other using messaging? Message Endpoint How does an application connect to a messaging channel to send and receive messages? Message Routing Content-Based Router How do we handle a situation where the implementation of a single logical function is spread across multiple physical systems? Message Filter How can a component avoid receiving uninteresting messages? Dynamic Router How can you avoid the dependency of the router on all possible destinations while maintaining its efficiency? Routing Slip How do we route a message consecutively through a series of processing steps when the sequence of steps is not known at design-time and may vary for each message? Throttler How can I throttle messages to ensure that a specific end point does not get overloaded, or we don t exceed an agreed SLA with some external service? Loop How can I repeat processing a message in a loop? Recipient List How do we route a message to a list of dynamically specified recipients? Splitter How can we process a message if it contains multiple elements, each of which may have to be processed in a different way? Aggregator How do we combine the results of individual, but related messages so that they can be processed as a whole? Sampling How can I sample one message out of many in a given period to avoid downstream route does not get overloaded? Delayer How can I delay the sending of a message? Resequencer How can we get a stream of related but out-of-sequence messages back into the correct order? Composed Message Processor How can you maintain the overall message flow when processing a message consisting of multiple elements, each of which may require different processing? Scatter-Gather How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply? Load Balancer How can I balance load across a number of endpoints? Multicast How can I route a message to a number of endpoints at the same time? 12
Messaging Channels Point-to-Point Channel How can the caller be sure that exactly one receiver will receive the document or perform the call? Publish-Subscribe Channel How can the sender broadcast an event to all interested receivers? Dead Letter Channel What will the messaging system do with a message it cannot deliver? Guaranteed Delivery How can the sender make sure that a message will be delivered, even if the messaging system fails? Message Bus What is an architecture that enables separate applications to work together, but in a decoupled fashion such that applications can be easily added or removed without affecting the others? Message Construction Event Message How can messaging be used to transmit events from one application to another? Request Reply When an application sends a message, how can it get a response from the receiver? Correlation Identifier How does a requestor that has received a reply know which request this is the reply for? Return Address How does a replier know where to send the reply? Message Transformation Content Enricher How do we communicate with another system if the message originator does not have all the required data items available? Content Filter How do you simplify dealing with a large message, when you are interested only in a few data items? Claim Check How can we reduce the data volume of message sent across the system without sacrificing information con tent? Normalizer How do you process messages that are semantically equivalent, but arrive in a different format? Sort How can I sort the body of a message? Validate How can I validate a message? Messaging Endpoints Messaging Mapper How do you move data between domain objects and the messaging infrastructure while keeping the two independent of each other? Event Driven Consumer How can an application automatically consume messages as they become available? Polling Consumer How can an application consume a message when the application is ready? Competing Consumers How can a messaging client process multiple messages concurrently? Message Dispatcher How can multiple consumers on a single channel coordinate their message processing? Selective Consumer How can a message con- sumer select which messages it wishes to receive? Durable Subscriber How can a subscriber avoid missing messages while it s not listening for them? Idempotent Consumer How can a message receiver deal with duplicate messages? Transactional Client How can a client control its transactions with the messaging system? Messaging Gateway How do you encapsulate access to the messaging system from the rest of the application? Service Activator How can an application design a service to be invoked both via various messaging technologies and via nonmessaging techniques? 13
System Management Detour How can you route a message through intermediate steps to perform validation, testing or debugging functions? Wire Tap How do you inspect messages that travel on a point-to-point channel? Log How can I log processing a message? 14
About Talend Talend provides integration that truly scales. From small projects to enterprise-wide implementations, Talend s highly scalable data, application and business process integration platform maximizes the value of an organization s information assets and optimizes return on investment through a usage-based subscription model. Ready for big data environments, Talend s flexible architecture easily adapts to future IT platforms. And a common set of easyto-use tools implemented across all Talend products enable teams to scale developer skillsets, too. More than 1,800 active subscribers worldwide leverage Talend s solutions and services. The company has major offices in North America, Europe and Asia, and a global network of technical and services partners. For more information, please visit www.talend.com. Contact Us www.talend.com/contact info@talend.com partners@talend.com sales@talend.com Talend 2013 WP157.2-EN 15