1 CHAPTER 1 INTRODUCTION Internet has revolutionized the world. There seems to be no limit to the imagination of how computers can be used to help mankind. Enterprises are typically comprised of hundreds if not thousands of applications that are custombuilt, acquired from a third-party, part of a legacy system, or a combination thereof, operating in multiple tiers of different operating system platforms. It is not uncommon to find an enterprise that has 30 different Websites, three instances of SAP and countless departmental solutions. A customer placing a new order may require the coordination of many systems. The business needs to validate the customer Identity, verify the customer s good standing, check inventory, fulfill the order, get a shipping quote, compute sales tax, send a bill, etc. This process can easily span across five or six different systems. From the customer s perspective, it is a single business transaction. In order to support common business processes and data sharing across applications, these applications need to be integrated. Application integration needs to provide efficient, reliable and secure data exchange between multiple enterprises applications. Enterprise Application Integration (EAI) is the integration of applications at the enterprise level. EAI is the heartbeat of integration. It can be described as, "The process of integrating multiple applications that were independently developed, may use incompatible technology and remain independently managed." From this perspective, EAI is not a technology but a collection of
2 tools, techniques and technology which enable applications to effectively interoperate. This is not to say that integration and middleware apply only at the enterprise level. Indeed, middleware can be used at many levels: within a single system (component integration) across the enterprise (application-to-application (A2A) integration) Across the supply chain (business-to-business (B2B) integration) Integration brokers, for example, may be used on all levels, but their specific features and functions vary. CORBA and COM are commonly used at the component level, while message brokers, file brokers or ETL brokers are often used on the A2A level. Additionally, portals and specialized B2B brokers are used for managing interactions between companies. EAI connects existing and new systems to enable collaborative operation within our entire organization. This also assists in making real-time operating decisions, bringing all critical information to our fingertips and lowering operating costs (both Information Technology (IT) and business). Organizations must provide customers, employees, suppliers, distributors and numerous other parties with the information they need, when they need it. Business systems and processes can no longer remain isolated and disparate. The success of a business now demands better performance from easier application integration. Enterprise computing has progressed enormously in just the last few years. As more and more corporations become Web-enabled and find them relying on a myriad of applications, the ability to evolve and integrate existing applications becomes significant. Virtually all enterprise organizations at some time face the problem of integrating different applications and database systems. It is not a simple matter for an enterprise to discard its existing applications, or even
3 overhaul its established business processes, to effect a change in its business model. These kinds of changes are financially expensive to undertake and daunting in terms of human resources. Many enterprises cannot afford to make such changes or discard existing systems. Thus, it is critical for enterprises to be able to leverage their investments in their existing enterprise infrastructure and applications. In these situations, enterprise application integration assumes a great importance. EAI enables an enterprise to integrate its existing applications and systems and to add new technologies and applications to the mix. It also helps an enterprise to model and automate its business processes. It has always focused on a company s IT department integrating new software modules or applications with its existing systems. EAI is the approach for integrating information systems. As more and more corporations become web-enabled and find them relying on a myriad of applications, the ability to evolve and integrate existing applications becomes significant. Virtually, all enterprise organizations face the problem of integrating different applications and database systems. It is not a simple matter for an enterprise to discard its existing applications or even overhaul its established business processes as it is financially expensive and daunting in terms of human resources. Thus it is critical for enterprises to be able to leverage their investments in their existing enterprise infrastructure and applications. EAI enables an enterprise to integrate its existing applications and systems and to add new technologies and applications to the mix. Initially Service Oriented Architecture (SOA) and Event Driven Architecture (EDA) emerged as complementary concepts in distributed computing. A detailed study was undertaken to choose the right approach for
4 EAI. In the course of the research it was found that these two concepts are actually not complementary to each other but can happily coexist. This research work highlighted that combining these two concepts can provide advantages by getting the best of both worlds. An architecture comprising these two concepts has been proposed to realize the benefits of both EDA and SOA. This proposed architecture has been named as Event Driven Service-Oriented Architecture (EDSA) and has been applied to EAI. The existing system, implemented using EAI approach is facing issues in terms of reliability, scalability and throughput. This system had all the services deployed on a single integration server and was using the request-reply paradigm. The system was completely synchronous which led to poor response times. It is re-engineered and implemented using a hub-and-spoke architecture. Most of the services were redesigned to be asynchronous. An event-driven SOA-based architecture is designed for EAI to overcome the limitations of the existing system. The request- reply paradigm has been replaced by the publish-subscribe paradigm. This has considerably reduced the response time to about 70%. The SOA concept of registry is implemented using an integration broker. Since there are no direct calls to services there is location transparency. Each module is independent of the other and can be distributed and hence it is easier to add new services. The concept of clustering is implemented which leads to better reliability and fault tolerance. The proposed architecture has been implemented to an insurance system and a software organization and the resulting system is found to be more scalable, accurate and reliable than the existing system. The thesis is organized in six chapters.
5 Chapter 1 gives the introduction part of the thesis explaining the need for EAI and outlining the contents of the thesis. The biggest demand of enterprises today is to share data and processes across heterogeneous computer systems without having to make sweeping changes to the applications or data structures. The current trends in system architecture for integration revolve around SOA and EDA. An event-based system has components, which are completely decoupled from each other since they communicate only through the publication and subscription of events and never directly with each other. When multiple systems need to communicate with each other, EAI comes to the rescue. In day-to-day life, interoperability is a key requirement. Within an organization there are myriad systems each functioning excellently as individuals but when it comes to reusability or information sharing, they fail miserably. A new architecture is proposed combining the features of SOA and EDA to support the current business requirements. When information propagates from one system to the other there is a data inconsistency because the systems are heterogeneous. By introducing the integration layer in the proposed architecture, consistency is achieved. The proposed work can be applied to any organization where heterogeneous systems exist. Chapter 2 explains the architectures for System integration. EAI defines semantics for application and data integration. EAI defines a standard methodology for applications and data sources to communicate. The application integration layers viz., business process layer, integration layer and application layer are explained. A few EAI patterns are outlined. The EAI types viz., information-oriented integration, application services-oriented integration and business process oriented integration are explained. SOA is an IT strategy that organizes the discrete functions contained in enterprise applications into
6 interoperable standards-based services that can be combined and reused quickly to meet business needs. The SOA component architecture is explained. The characteristics of SOA are explained. The concept of a lightweight SOA where a local registry is used without UDDI is described. The EDA categories viz., simple event-driven integration, EDA with integration brokers, EDA that are directed by business process management and complex-event processing applications are discussed. EDA makes use of the publish-subscribe paradigm. Chapter 3 explains the proposed Event-Driven Service-Oriented Architecture (EDSA). An EDA provides the ability to send, receive and respond to unpredictable business information and events asynchronously. This combined with SOA, creates a real-time enterprise architecture that enables real-time business. The functional elements of the proposed real-time enterprise architecture are discussed. The design phase involved designing an architecture using publish-subscribe model. The advantages of the event-driven publishsubscribe paradigm and its uses are explained. An architecture stack is introduced which consists of 4 layers viz., System Layer, Integration Layer, Business Layer and Application Layer. The proposed architecture has been tested for typical applications and these illustrations are discussed in Chapter 4. An existing Insurance application has been taken as a case study. The existing architecture faces issues in terms of scalability, throughput and reliability. Also all the transactions are synchronous causing serious performance bottlenecks. The new architecture uses the EDA with integration broker architecture and addresses these issues. It is implemented on the webmethods Integration Platform.
7 Chapter 5 discusses the results and inferences. EAI has been found to be extremely useful for an organization. The concepts of SOA and EDA have been applied to EAI resulting in the proposed EDSA architecture. The proposed system has been found to be more reliable as there is no manual intervention. Since information exchange is through structured Extensible Markup Language (XML) / Simple Object Access Protocol (SOAP) formats rather than conventional ad-hoc formats, it results in reliable transmission of data. The addition of new applications doesn t affect the performance. Since the subscribers are decoupled from the publisher, changes in one system do not impact the others. It is easier to add new systems without impacting the other existing systems. This results in quicker delivery of systems. As the number of services increases, the response time increases drastically in the existing system whereas the response time remains almost a constant in the proposed system. Chapter 6 concludes the research work and lists the future enhancements. A new architecture has been proposed in this research work by combining the concepts of SOA and EDA. The proposed architecture has been designed and successfully implemented for two industrial applications. The proposed system is observed to be accurate, reliable and scalable. The proposed architecture provides interoperability and reusability with increased throughput. However, the cost of EAI tools is very high. The proposed architecture can be extended for grid computing, which specifies a standard architecture, infrastructure, protocols and application programming interface (API) for building an open enterprise system.