Cape Clear s Enterprise Service Bus (ESB)

Cape Clear s Enterprise Service Bus (ESB) How Cape Clear Software applies SOA and Web service principles to deliver a proven ESB solution WWW.CAPECLEAR.COM

Cape Clear s Enterprise Service Bus (ESB) (May 2005) Copyright 2005 Cape Clear Software, including this documentation, all demonstrations, and all software. All rights reserved. The document is not intended for production and is furnished as is without warranty of any kind. All warranties on this document are hereby disclaimed including the warranties of merchantability and fitness for a particular purpose. Trademarks Cape Clear is a registered trademark of Cape Clear Software. Cape Clear Software, Cape Clear Orchestrator, Cape Clear Data Transformer, Cape Clear Studio, Cape Clear Server, and Cape Clear Manager are trademarks of Cape Clear Software in the United States and other countries. All other company, product, and service names mentioned in this product may be trademarks or service marks of others. CPV-DOC-3062 Copyright 2005, Cape Clear 2 Software Inc. Page 2 of 17

Contents Introduction... 4 SOA and ESB... 6 ESB: A New Tier in the Enterprise Architecture...7 Not All Data Is XML Data and Not All Applications Are XML-Aware...12 The Key Capabilities of an ESB...12 Cape Clear s ESB Product Approach...15 Summary...17 Copyright 2005, Cape Clear 3 Software Inc. Page 3 of 17

Introduction Organizations in all sectors of business and government are pursuing Service-Oriented Architecture (SOA) initiatives in response to their need for increased business agility. SOA approaches enable the creation of powerful, flexible networks of software services that closely mirror the actual business processes of an organization. These services, which expose and leverage organizations existing data and applications, can flexibly evolve to meet changing requirements and can be readily integrated with processes spanning customers, partners, and suppliers. Against a backdrop of constrained IT budgets and increasingly demanding line-of-business customers, delivery on the promise of SOA must happen rapidly, and it must happen in a cost-effective way. By leveraging the power of XML and Web services as an enabling technology, the Enterprise Service Bus (ESB) product category is making the rapid attainment of SOA a reality. The Enterprise Service Bus is a new integration architecture that draws on the disciplines of Service-Oriented Architecture and the power of Web services standards to radically change both the technology and the economics of application integration and the development of new, orchestrated business services. This paper provides an overview of SOA and ESB, explains how the ESB differs from the traditional integration solutions that have come before it, and describes in detail the functionality and capabilities of Cape Clear Software s standards-based, infrastructureagnostic ESB product suite. The generally accepted attributes of an Enterprise Service Bus include: Deep, native Web services capabilities. Support for the development and delivery of new orchestrated services. Sophisticated mapping and transformation capabilities. Extensive multi-protocol messaging support. Content-, metadata-, and address-based routing capabilities. Flexible deployment architectures, both inside and outside the firewall, with deep Quality of Service for enterprise-class systems. Integration support for a wide range of existing applications and data sources. In addition to the above attributes, expect the ESB to provide the following capabilities: Improved ease-of-use. Configure, don t program! The ESB is proving to be radically faster and simpler to implement than other types of application integration approaches. Copyright 2005, Cape Clear 4 Software Inc. Page 4 of 17

Increased ROI. The ESB is delivering an order-of-magnitude better economics than EAI or customized integration approaches. Infrastructure-agnostic. The ESB does not lay down additional infrastructure its sole purpose is to connect and leverage the existing infrastructure already in place, regardless of how proprietary, heterogeneous, and disconnected it may be. Gartner predicts that more than half of all large enterprises will have an ESB running by year-end 2006. 1 This impressive adoption rate is no doubt driven by the massive move to SOA, and the proven ease of use and ROI gained from using ESBs over traditional approaches. 1 Predicts 2005: Application Integration, ESBs and B2B Evolve, Roy W. Schulte et al, 1 November 2004. Copyright 2005, Cape Clear 5 Software Inc. Page 5 of 17

SOA and ESB The discipline of Service-Oriented Architecture is (and has long been) well understood. Gartner Group first referred to the concept in 1996, and the idea arguably goes back to the early 1970s. More recently, Gartner has made bullish predictions about the likely prevalence of SOA approaches: By 2008, SOA will be a prevailing software-engineering practice, ending the 40-year domination of monolithic software architecture. 2 SOA is a set of principles: an approach to developing reusable services. The Web servicesbased standards suite is a way to implement a SOA in practice. Good Web services design fosters SOA discipline, and good Web services tools and platforms inherently mandate SOA approaches. ESB is a term initially coined by Gartner for a category of products that implement SOA disciplines using Web services approaches. Figure 1: The path from business agility to the ESB 2 Service-Oriented Architecture Scenario, Yefim Natis, 16 April 2003. Copyright 2005, Cape Clear 6 Software Inc. Page 6 of 17

SOA principles, as applied to good business service and Web service design, are discussed in more detail in the whitepaper Principles of SOA Design, which is available from http://www.capeclear.com. Forrester Group, in a prescient 2002 report 3, succinctly summarized the distinction between the then-current state of the majority of Web services implementations and the state to which they were rapidly evolving. Figure 2: The evolution of Web services ESB: A New Tier in the Enterprise Architecture Application servers are well suited for hosting complex back-end business logic. Integration servers are good at integrating packaged applications (although not via XML). It is Cape Clear s view that the new era of service-oriented, enterprise-software architectures requires that a new tier emerge in the enterprise architecture where business services are rapidly, opportunistically, and cost-effectively defined and implemented. The associated architecture is shown in Figure 3 below: 3 Web Services Platform Shootout, Forrester Research, October 2002. Copyright 2005, Cape Clear 7 Software Inc. Page 7 of 17

Figure 3: Layers in the Enterprise Service Bus This figure also describes the steps in creating flexible business. First, existing enterprise assets (including packaged and custom applications, databases, and other information sources) are Web service-enabled. Then they are connected to the ESB and associated with a more coarse-grained service interface, which is closer to the business definition of the service than to its technical implementation. The document-oriented nature of SOA promotes interface descriptions that are familiar to business users, such as issue a purchase order, for example, versus the low-level API interfaces that are common to applications. Connection to the ESB also associates a service policy with the service, addressing matters such as security, availability, error handling, messaging, and so on. Finally, at the business process tier, individual business services (either internal or external Web services standards mean that firewalls can be readily spanned) can be rapidly orchestrated together into new composite business services. Copyright 2005, Cape Clear 8 Software Inc. Page 8 of 17

Gartner defines an ESB in the following way 4 : Enterprise service buses (ESBs) are a new kind of middleware that combines features from several previous types of middleware into one package. ESBs support Web services by implementing Simple Object Access Protocol (SOAP) and leveraging Web Services Description Language (WSDL) and Universal Description, Discovery and Integration (UDDI). Many ESBs also support other communication styles that involve guaranteed delivery and publish-and-subscribe; those that don't soon will. All ESBs provide some value-added services beyond those found in basic communication middleware, such as message validation, transformation, content-based routing, security, service discovery for a serviceoriented architecture (SOA), load balancing, failover and logging. Some services are built into the ESB core, while others run in "plug in" modules. ESBs have a distributed architecture wherein some services are executed near the application programs, rather than in a central hub. ESBs support Extensible Markup Language (XML) and often also support other message formats. Figure 4: How the ESB fits in (Source: Gartner) 4 Predicts 2004: Enterprise Service Buses Are Taking Off, Roy W. Schulte, 19 December 2003. Copyright 2005, Cape Clear 9 Software Inc. Page 9 of 17

The Enterprise Service Bus provides a way to define and deliver highly customized services and views of services, closely aligned with the client s perspective, and without intruding on deep back-end application logic. The Enterprise Service Bus is a simple way to do integration within a Service Oriented Architecture. -- Forrester, Dec 2004 An enterprise service bus enables a service-oriented architecture by acting as an intermediary layer of middleware through which a set of reusable business services are made widely available Roy Schulte, Gartner, 2004 An ESB helps enterprises obtain the value of SOA by increasing connectivity, adding flexibility that speeds change, and providing greater control over use of the important resources it binds. The industry transition to messaging and ESB will mark an inflection point triggering a new, massive wave of innovation around businesses use of their information resources. Roy Schulte, Gartner, 2004 An ESB helps enterprises obtain the value of SOA by increasing connectivity, adding flexibility that speeds change, and providing greater control over use of the important resources it binds. Mike Gilpin, Forrester Research, 2004 the use of the Web services stack as a cheaper, simpler EAI alternative is a nobrainer. Rachel Chalmers, The451, 2004. So how do the capabilities of the ESB differ from application servers and integration brokers? Although both of these product categories have deep capabilities within their respective spheres of competence, they are not well suited for implementing a Service- Oriented Architecture in environments with an installed infrastructure already in place, for example: Application servers, principally J2EE-based, are widely used to develop and deploy back-end server logic. However, application servers are more suited to new application development than to integration tasks. The application-server approach to SOA is to auto-enable all Java interfaces as WSDL, which leads to complex and Copyright 2005, Cape Clear 10 Software Inc. Page 10 of 17

non-interoperable service interfaces a very bad thing for a number of good reasons 5. The J2EE JBI initiative is in many ways similar to many of the ESB concepts outlined here, except that it is tightly coupled to the enterprise Java architecture. Integration brokers, as realized in EAI suites, have historically been used for the integration of packaged applications via specific and often heavily customized adapters. However, although EAI was a significant improvement over custom integration coding, it suffers from some very significant drawbacks: o o o o o The messaging layers of EAI vendors are proprietary. The adapters offered by EAI vendors are typically proprietary, not packaged, and are being rendered obsolete by the advancing embrace of Web service-enabled APIs in enterprise application suites. Most EAI products have one central control point, dramatically impacting performance and failure modes. The APIs that drive the EAI suites are complex and difficult to learn. The skills to operate them are scarce and expensive. The use of EAI suites typically results in inflexible integrations that cannot be rapidly re-purposed and re-cast in support of agile business initiatives. These shortcomings, coupled with the pressure on costs, the drive towards standards, and the emergence of SOA as the preferred architectural paradigm, means that EAI is largely unsuited to the challenge of modern-day enterprise integration. Figure 5: A historical view of enterprise integration 5 See http://www.capescience.com/articles/wsdlfirst/index.shtml. Copyright 2005, Cape Clear 11 Software Inc. Page 11 of 17

Not All Data Is XML Data and Not All Applications Are XML-Aware Many integration solutions and ESBs assume that all the data and applications that need to be incorporated in an integration effort are already accessible as Web services or XML. Unfortunately, this is an unrealistic assumption. A wide variety of content stored in structured, semi-structured, and unstructured formats (such as spreadsheets, databases, comma-separated files, and other flat files) often comprises a significant portion of an integration project. Perl scripts, custom Java code, manual processes, and extensive data re-entry are often used to incorporate this data into an integration project. An ESB enables users to rapidly build XML-based on-ramps and off-ramps to and from the Web services world for any data format. This enables organizations to define, capture, and manipulate the semantics of their business data as it flows through communications pipes, and not just at the programming endpoints. Many structured and standard data formats in a variety of industries (for example, EDI, SWIFT, ACORD, and PARLAY) either do not have standardized XML representations or, where they do, are not yet widely used in practice. The need to process these data types gives rise to the need to process and manipulate non-xml data in the context of ESB-based integration efforts. Another common source of non-standard data occurs when company boundaries are crossed in the course of the execution of a business process. These data transformation capabilities are especially useful for incorporating business partners data in the scope of integration, especially when their systems interfaces are not exposed. The Key Capabilities of an ESB An ESB enables more efficient value-added integration of the variety of different application and data components behind a service-oriented façade and by applying Web services technology to the problem. The physical product architecture is described in more detail below. The detailed functional requirements for an ESB product suite are also outlined below. Deep, Native XML and Web Services Capabilities and Support for Standards WSDL is the starting point for the activities and processes of an ESB, so an ESB must offer deep support for WSDL design, editing, manipulation, and processing, as well as deep support for XML Schema and related standards, including XPath and XSLT. This should not be confused with simply providing a thin veneer of Web services support over a proprietary core. In addition, an ESB must offer support for the latest Web services standards, including the WS-I Basic Profile and Basic Security Profile. Copyright 2005, Cape Clear 12 Software Inc. Page 12 of 17

Workflow/Orchestration The ability to flexibly compose a variety of service atoms into a range of different composite workflows (which are then themselves recursively addressable as services!) enables organizations to seamlessly align their IT infrastructure with their business needs. Easily orchestrating services into new, value-added business services creates business agility. BPEL has become the widely adopted standard for implementing workflow. An ESB should provide the tooling, execution, and monitoring environments for building business processes using the BPEL standard. Mapping and Transformation Support for graphical mapping and transformation is fundamental to the ESB, since it is frequently necessary to semantically match service interfaces, as well incompatible data sources. Transformations can be between XML schemas, or into and out of XML for non- XML data types. Extensive, Multi-Protocol Messaging Support An ESB must support multiple synchronous and asynchronous messaging transports. To depend on any one or any small set dramatically impacts its applicability in real-world installations. Supported transports include HTTP(S), WS-ReliableMessaging (WS-RM), JMS, FTP/file drop, and SMTP/e-mail. Content-, Metadata-, and Address-Based Routing Routing decisions should be capable of being executed in a highly dynamic fashion, based on either header or WS-Addressing information, metadata, or any value inside a given document (content-based routing). Flexible Deployment Architectures with Deep Quality of Service for Enterprise-Class Systems A major benefit of the ESB architecture is that it is distributed: it has no central point of control or failure. A loosely coupled ESB deployment can support sophisticated serviceclustering arrangements to ensure high levels of system throughput, responsiveness, and fault tolerance. Adapter Services The magic bullet of EAI was the application adapter. EAI vendors succeeded in the 1990s on the back of a popular misconception (which they helped create): that it was possible or desirable to package into an off-the-shelf adapter any integration requirement for a given enterprise application. This approach was flawed on a number of levels: Copyright 2005, Cape Clear 13 Software Inc. Page 13 of 17

Every user of a major application uses it differently, resulting in different data models, different deployed modules, different process templates, different security models, and so on. A one-size-fits-all approach to integrating with the application was never feasible. The vast majority of enterprise application vendors have been supporting standard, often Java-based APIs into their products for a number of generations. Many also offer XML-based document access. This shifts the integration problem from one of syntax (the ability to reach applications via formerly proprietary transports and APIs) into one of semantics (given that it s now straightforward to reach into this application, how can its usage semantics be rationalized with the new service being developed?). Cape Clear provides powerful tooling that automatically generates Web service interfaces for Java, CORBA, and database-accessible enterprise applications that do not themselves expose Web service APIs. Cape Clear s Data Transformer functionality provides powerful, graphical tools to enable semantic compatibility. The full capabilities of the Cape Clear ESB can then be brought to bear on these service-enabled interfaces, incorporating them into the SOA world. Infrastructure-Agnostic The raison d'etre for an ESB is to bring integration and new value-added service delivery to existing environments, regardless of the type or complexity of the infrastructure already in place. An ESB must be able to work in highly heterogeneous environments of application servers, message transports, applications, and data. An ESB is infrastructure-agnostic, is not tightly linked to any one vendor s products, and does not add another layer of infrastructure. Be very skeptical of an ESB if it requires its own proprietary JMS or other proprietary middleware. Copyright 2005, Cape Clear 14 Software Inc. Page 14 of 17

Cape Clear s ESB Product Approach Cape Clear s approach to the ESB is straightforward: Radical simplicity and ease of use. In order to achieve the productivity benefits promised by an ESB strategy, products must enable less-skilled developers to be more productive at solving complex integration problems in a new way. This imposes requirements throughout the product set in terms of ease of installation, learning curve, complexity, and the user-interaction model. The Cape Clear ESB offers a highly graphical, next-generation set of products its primary design goal is to drive complexity out of the integration process. Figure 6: The Cape Clear Orchestrator graphical user interface SOA and Web services standards from the ground up. Cape Clear fosters a WSDL first / schema first approach to SOA. Cape Clear provides powerful tooling for WSDL and schema design. This fosters inherently good SOA discipline something that a code first approach can never claim. Copyright 2005, Cape Clear 15 Software Inc. Page 15 of 17

Figure 7: How Cape Clear Server works Enterprise-class products. Cape Clear has over 200 customers fully deployed in mission-critical and performance-intensive environments, requiring 24x7 uptime and support. Cape Clear Software offers a comprehensive and proven ESB, comprised of five tightly integrated products: Figure 8: The components of the Cape Clear ESB Cape Clear Studio provides an Eclipse-based graphical environment for designing and developing Web services. Cape Clear Server is a powerful, high-performance Web services-based platform for integrating applications. Cape Clear Data Transformer radically simplifies the integration of diverse data sources. Cape Clear Manager provides all the tools required to manage, monitor, and secure deployed services and integrations. Cape Clear Orchestrator enables the creation, execution, and monitoring of BPELcompliant business processes. Copyright 2005, Cape Clear 16 Software Inc. Page 16 of 17

Summary ESB is a meaningful and rapidly developing product category, which enables the rapid introduction of SOA discipline to integration projects and new service delivery. Adopting SOA offers the potential for significant technical and economic benefits, but only if a proper, fully functional ESB is used. Cape Clear has long led the way in applying Web services to the task of integration and new service delivery, and has the experience, technology, product, and solution skills to help you successfully implement your ESB and SOA initiatives. For more information, visit http://www.capeclear.com/esb. Copyright 2005, Cape Clear 17 Software Inc. Page 17 of 17