Generic ESB Monitoring Architecture Compliant With JBI

Transcription

1 Generic ESB Monitoring Architecture Compliant With JBI Marek Psiuk Tomasz Bujok Supervisor: Prof. Krzysztof Zieliński Department of Electrical Engineering, Automatics, Computer Science and Electronics AGH University of Science and Technology, Kraków, Poland Thesis submitted for the Degree of Master of Computer Science in AGH University of Science and Technology 2008

2 Uniwersalna architektura monitoringu ESB zgodna ze specyfikacj a JBI Marek Psiuk Tomasz Bujok Promotor: prof. dr hab. inż. Krzysztof Zieliński Wydział Elektrotechniki, Automatyki, Informatyki i Elektroniki Akademia Górniczo Hutnicza, Kraków, Polska Praca Magisterska Akademia Górniczno Hutnicza 2008

3 SOA will be used in more than 50 percent of new mission-critical operational applications and business processes designed in 2007 and in more than 80 percent by 2010 Gartner Raport, 2007 Abstract Currently, the significance of the Enterprise Application Integration (EAI) in the field of Service Oriented Architecture (SOA) is gradually increasing. The quality of the integration patterns is constantly ameliorating. Progressive integration solutions are designed, published and offered to the community by most of the prominent IT vendors. Today, Enterprise Service Bus (ESB) is one of the most meaningful solutions and top-selling products on the market of the integration services. There is, however, no agreed-upon definition of what an ESB actually is. Java Business Integration (JBI) presents an approach to standardizing the ESB pattern by specifying a concrete, well-defined and Java-centric model of ESB. The first version of the specification does not mention any kind of monitoring solution for JBI, which is the vital drawback of the concept. Increasing complexity of the SOA-oriented IT systems implies higher importance of the selective monitoring, which allows to mitigate the problem of the management of software infrastructure. The main goal of the thesis is to design and implement a system which adds a monitoring and management logic to a Java Business Integration container in such a way that the logic is fully transparent and compliant with the JBI specification and allows to gather, process and expose sophisticated information regarding the qualitative parameters and operational reliability of the JBI container. The design of the system was performed in accordance with the Experimental Driven Development (EDD) methodology. At the beginning a statement of functional and non-functional requirements was created. Afterwards, an analysis and prototype construction were performed. At the end, the prototype was meticulously examined during a set of experiments which were conducted, in order to provide best-fitting architectural solutions which are compliant with the aforementioned requirements. Finally, it was possible to design and implement functional and innovative realization of the JBI monitoring solution called GlassBus. Performance test have proofed that the average JBI performance deterioration, while being monitored, is not higher than 20 percent. Not only did the results of the work described in this thesis fully satisfy the authors, but they also amazed the academic supervisor. Progressive architectural solutions introduced in GlassBus ensure its forthcoming success.

4 Contents Abstract Declaration Acknowledgements i ix x 1 Introduction Enterprise Application Integration Enterprise Service Bus Java Business Integration Monitoring Thesis statement Research contribution Organization of thesis Enterprise Integration Theory Preamble Definition of the Enterprise Application Integration Reason for integration Challenge Integration taxonomy Integration factors Enterprise Integration Patterns Topologies Fundamentals Integration Fundamentals Properties of integration solutions Point to point Integration Broker Integration Message Bus Integration Comparison Enterprise Service Bus Service Oriented Architecture as ESB environment ESB Definition ii

5 Contents ESB implementations Summary Java Business Integration Background Architecture JBI Implementations Summary Conclusion JBI Monitoring Preamble Monitoring abstraction JBI monitoring status quo Monitoring from JSR ServiceMix monitoring OpenESB monitoring Clustering Status quo implications Application monitoring theory Application Response Measurement Business Activity Monitoring JBI Monitoring Challenge JBI context influence Application theory influence Conclusion Executive summary of the monitoring architecture Preamble Problem definition Non-functional requirements Functional requirements Introducing GlassBus Monitoring Abstraction Conclusion GlassBus Research Preamble Experiments placement Information acquiring mechanism Possible options Realisation JBI monitoring mechanisms Monitoring information Correlation heuristics Problem definition Realization iii

6 Contents 5.6 Persistence Database choice Persistence abstraction Communication middleware Communication theory MOM Implementation provider JMS Sender JMS Receiver Communication security Management middleware Possible options Realisation Hierarchical Data Management Possible solutions Realization Installation and runtime environments Deployment assumptions Deployment scenarios Realization Monitoring interface Possible solutions Realization Logging Possible options Realization Conclusion GlassBus Implementation Preamble Architecture Overview Logical Structure Domain model GlassBus Agent GlassBus Engine External interface model Management and selectivity model Invocation flows Physical Structure Configuration Installation Implementation status Performance Tests Test environment ServiceMix OpenESB iv

7 Contents Summary Conclusion Conclusions Overview The status of Java Business Integration Future of JSR JBI GlassBus achievement Appendices 171 A Source code 172 Bibliography 186 v

8 List of Figures 2.1 Integration scopes Point to point integration Broker integration Classes of brokers Direct Broker Sequence Diagram Indirect Broker Sequence Diagram Bus integration Topology Summary Location of ESB in SOA environment Architecture of JBI JBI Messaging Model Message Exchange Flow ServiceMix clustering OpenESB clustering Application invokes methods on ArmTransaction Connection between ARM Interface and Agent Sample distributed system Transaction and parent-child correlators Message Oriented Middleware ARM patterns Business process management life-cycle CEP-oriented BAM architecture Possible Architecture Positioning GlassBus architecture overview ME FLOW PATTERN example ME SUB FLOW PATTERN example ME TRANS example nr ME TRANS example nr BUSS TRANS PATTERN example GlassBus architecture overview - placement of the experiments vi

9 List of Figures 5.2 Message correlation algorithm overview ActiveMQ Broker Topology Out-of-the-box scenario Distributed scenario Engine broker clustering Message Driven Pojos architecture GlassBus deployment to Single monolithic bus GlassBus single out-of-the-box deployment GlassBus complete architecture General logical structure JBI Topology JBI Message Exchange Patterns GlassBus Dao GlassBus Agent GlassBus Engine External monitoring interface The Agent in the Engine registration sequence diagram Messaging propagation sequence diagram Topology propagation sequence diagram Physical structure of GlassBus implementation ServiceMix topology for performance tests OpenESB topology for performance tests Comparison of GlassBus performance overhead vii

10 List of Tables 2.1 Topologies comparison JBI implementations Clustering comparison JBI implementations comparison Abstraction contextual evaluation AOP Requirement compliance JBI parameters requirement compliance JBI parameters requirement compliance Persistence Requirement compliance Tight vs. Loose coupling discussion Communication middleware comparison Basic comparison of the middleware performance ActiveMQ performance comparison - embedded brokers ActiveMQ performance comparison - embedded and remote brokers JMS sender performance comparison JMS transmission reliability Communication middleware - requirement compliance Management requirement compliance Hierarchical data - requirement compliance Deployment Requirement compliance Monitoring interface - requirement compliance Logging Requirement compliance Timing path - MEP relations Performance of glazed ServiceMix GlassBus overhead in ServiceMix Performance of glazed OpenESB GlassBus overhead in OpenESB GlassBus effort perspective Proof-of-Concept projects effort perspective viii

11 Declaration The work in this thesis is based on the research carried out at the Department of Electrical Engineering, Automatics, Computer Science and Electronics of the AGH University of Science and Technology in Kraków, Poland. No part of this thesis has been submitted elsewhere for any other degree or qualification and it is all our own work unless referenced to the contrary in the text. It has to be admitted that all work enclosed in the thesis is the result of a joint and collective research conducted and approved by both of the authors. Organization of the thesis as well as the outline and content of the chapters were also agreed upon. It has to be mentioned, however, which part of the text describing the aforementioned research, is the individual contribution of each author: Tomasz Bujok wrote Chapters 1, 2, 4 and the following Sections of Chapter 5 : 5.1, 5.2, 5.5 (without 5.5.2), 5.7, 5.9, Marek Psiuk wrote Chapters 3, 6, 7 and the following Sections of Chapter 5 : 5.3, 5.4, 5.5.2, 5.6, 5.8, 5.10, 5.12, Copyright c 2008 by Tomasz Bujok and Marek Psiuk. The copyright of this thesis rests with the authors. No quotations from it should be published without the authors prior written consent and information derived from it should be acknowledged. ix

12 Acknowledgments First of all, we would like to express our sincere gratitude and appreciation to our thesis advisor, Prof. Krzysztof Zieliński, for his continuous inspiration, encouragement, patience and careful guidance during our study years. He gave us the opportunity to minutely consider the problems of IT and he taught us how to meticulously analyze them from different perspectives. He has also shown a great and consistent interest in our project during its design and development which greatly refined its quality. We would also like to thank our families for the guidance, support and love they provided us with in particular through the work on the thesis, without whom we would not have finished. In conclusion, we would like to express our gratitude to Magdalena Gawłowska-Bujok and Katarzyna Hussar for an indispensable and thorough proofreading. Tomasz Bujok, Marek Psiuk Kraków, Poland September 9, 2008 x

13 Chapter 1 Introduction We are what we repeatedly do. Excellence, then, is not an act, but a habit. Aristotle, Ancient Greek Philosopher, Scientist and Physician. This chapter briefly introduces the basic terms and concepts of the Enterprise Application Integration, Enterprise Service Bus as well as Java Business Integration. It also explains the importance of monitoring, which is a key aspect of every modern enterprise system. This chapter also presents the goals and main assumptions of the thesis. 1.1 Enterprise Application Integration Today, enterprises are facing numerous challenging problems induced by development of the global village and the Internet itself. On the one hand, company customers constantly demand real-time service, better performance, higher quality and lower prices. On the other hand, the market is constantly undergoing dynamic changes to which companies have to react, which creates competitive growth. The strategy of sustainable development makes the companies become more agile in the sense of ability to react to the requirements that rapidly alter and demands of the environment. One of the most important relations which have to be shaped by an efficiently operated company, whose quality cannot deteriorate during the organization s evolution, are: B2B - Business to Business - a set of relations between specified business and cooperating businesses. B2C - Business to Consumer - a set of relations between specified business and a consumer. Business agility is mostly dependent on business infrastructure. In addition, the infrastructure determines the agility of the business as this very infrastructure has to provide the company with the possibility of implementing new business processes through the integration and cooperation between existing ones. All these modifications have to be implemented as fast and efficiently as possible, which reduces business costs. The explained process of Enterprise Integration has to be performed at the level of computer applications which is a core of every modern organization. 1.2 Enterprise Service Bus In computing, an enterprise service bus (ESB) refers to a software architecture construct. This construct is typically implemented by technologies found in a category of middleware infrastructure products, usually based on recognized standards which provide fundamental services for more complex architectures via an event-driven and standards-based messaging engine (the

14 1.3. Java Business Integration 2 bus) [1]. What is very interesting, this definition of ESB by Wikipedia is only one of the wide variety of explanations. There is no agreement on what ESB actually is in the enterprise community. The majority says that it is an Enterprise Application Integration instrument. All of these matters mentioned here are thoroughly investigated in what follows. 1.3 Java Business Integration Java Business Integration (JBI) [2] is a formal specification developed under the Java Community Proces (JCP) [3]. The first version (1.0) was released on the 17th of August 2005 and is referenced by the Java Specification Request numbered 208 (JSR-208) [2]. The second version (2.0), referenced by JSR-312 [4], was supposed to be released in the second quarter of 2008 but is already late. The main goal of JBI is to address the problems of Enterprise Application Integration and businesstwo-business solutions (B2B). JBI is an implementation of the ESB integration mechanism. 1.4 Monitoring Increasing complexity of IT systems, in particular enterprise solutions, implies higher importance of monitoring and management, which allows to mitigate the problem. Authors of Java Business Integration were aware of these facts since the specification introduces advanced management mechanisms in the sixth chapter of the JSR-208. However, it has to be admitted that the monitoring mechanisms introduced in the specification are rather basic and do not provide sophisticated functions. Therefore, great prospects are opened for the enhancements of the monitoring capabilities of JBI. 1.5 Thesis statement The main goal of the thesis is to state that the design and implementation of a generic and reusable monitoring architecture which is complaint with JBI is possible and beneficial for the JBI communities. Thesis statement that formalizes aforementioned matter, reads as follows: There exists a processing model which enables the construction of a system which adds a monitoring and management logic to a Java Business Integration container in such a way that the logic is fully transparent and compliant with the JBI specification and allows to gather, process and expose sophisticated information regarding the qualitative parameters and operational reliability of the JBI container. This dissertation presents the mentioned model as well as its implementation and experimental verification. Target implementations of the JBI-compliant runtime environment used during the verification are: OpenESB 1 and ServiceMix Research contribution The essential contributions of the presented thesis read as follows:

15 1.7. Organization of thesis 3 Theoretical model - general assumptions on the transparent JBI monitoring model based on the Aspect Oriented Programming 3 and Message Oriented Middleware 3. Theoretical approach is also based on Application Response Measurement 3 as well as Business Application Monitoring 3. Concrete architecture - specification of a concrete JBI monitoring architecture based on the aforementioned theoretical model. Proof of concept - implementation of the JBI monitoring mechanism based on the concrete architecture. Evaluation - performance and usability tests run in distributed JBI environment. 1.7 Organization of thesis This thesis is organized in the following way: Chapter 2 overviews the existing application integration technologies presenting complete background of ESB and JBI technologies. At the end, precise definition of these concepts is presented in order to provide a clear view of the domain of the thesis. Chapter 3 examines the taxonomy of monitoring systems. It also contains the status quo of the JBI monitoring on the basis of JSR-208 specification as well as OpenESB and SerivceMix proprietary solutions. At the end, an overview of some helpful monitoring solutions is provided. Chapter 4 presents the executive summary of the monitoring architecture in form of the functional and non-functional requirements. It also thoroughly explains the notions and concepts introduced by the final solution - called GlassBus. Chapter 5 concisely formulates the results of experimental driven development carried out in order to choose the best fitting architectural solutions for GlassBus. Chapter 6 describes the architecture and implementation of GlassBus. It forms a complete development documentation of the project. Results of the performance tests are enclosed at the end of the chapter. Chapter 7 summarizes the conclusions and reflections concerning final solution and future works in the field of JBI. 3 The concept is precisely explained in the following chapters.

16 Chapter 2 Enterprise Integration Theory Always design a thing by considering it in its next larger context a chair in a room, a room in a house, a house in an environment, an environment in a city plan. Eliel Saarinen, a Finnish-American architect and city planner. 2.1 Preamble The main goal of this chapter is to meticulously outline the domain of the thesis. At the beginning, the general idea of Enterprise Application Integration is presented. The section contains an explanation of integration reasons, challenges as well as integration taxonomy which helps to summarize all factors regarding application integration. The main emphasis is put on the importance and complexity of the discussed issue. Next section contains the integration fundamentals in form of the integration patterns which state the building bricks of every integration technology. Two core patterns of mediation and federation are explained. At the end, common properties and characteristics of every integration solution are identified and described. Discussed problem is explicated in the next section as three main integration patterns are described. These are: Point-to-point integration, Broker integration and Bus integration. Every pattern is carefully investigated. Explanations are supplemented with diagrams which present topology of each solution. At the end, advantages and disadvantages are thoroughly discussed, pointing the crucial differences between patterns. The core part of this chapter is dedicated to explaining the concepts of Enterprise Service Bus as well as Java Business Integration. These are the most interesting technologies, since they compose the exact environment of the desired monitoring solution. Both definitions are of crucial importance as there is much misunderstanding of these concepts - no only in the Internet but also in the IT community. Therefore, proper explanation is the key to accurate formulation of borders between the system and environment. 2.2 Definition of the Enterprise Application Integration The description of the EAI presented in the first chapter (1.1) can be brightly summarized by this single sentence: EAI is the soluble glue needed for modular relationships that allow organizations to be flexible and responsive to market demands. [84] The commonly used term refers to the modernization, consolidation and coordination of the applications in an enterprise [5]. According to one of the Gartner reports, [84] the EAI is the unrestricted sharing of data and business processes among any connected application or data sources in the enterprise. EAI can be also introduced as the uses of software and computer systems architectural principles to integrate a set of enterprise computer applications. [6]

17 2.2. Definition of the Enterprise Application Integration Reason for integration Huge enterprises typically consist of hundreds of applications that are proprietary-built, obtained from a third-party, part of a legacy system, operating in multiple tiers, operating systems, platforms and localizations. An enterprise that comprises thirty different web sites, three instances of SAP and countless departmental solutions is commonly seen. There are many reasons for the fine granularity of contemporary enterprise applications: Design complexity - writing business applications poses great difficulties. Creating a single, monolithic, extendible and easily manageable application to run a complete business enterprise is almost impossible. Software age - it is not very common that all applications used in an enterprise are written at the same point of time. They are being increasingly added during the growth of an enterprise. Technologies - some of the enterprise applications are designed using wide variety of specified, different technologies, as it was the general architectural recommendation at the current stage. Legacy solutions - are present in many enterprises. These are the applications with unavailable source code, no support, no documentation or which are simply too expensive to be redesigned. Local requirements - distributing business functions and business operations across multiple applications provides an enterprise with flexibility to select the best fitting option at the specified situation. Control - sometimes it is easier to have all administrative tasks done locally in every single application as it improves local accountability. Better performance - fine granularity gives the possibility to tune the underlying hardware specially for the needs of the installed pieces of software. Bad design - application are written ad-hoc without performing adequate research. Unknown scope - current integration goals were unknown at the specified application design stage. On the one hand, users such as customers or business partners which interact with the frontend application do not generally think about the system boundaries and limitations when they inter-operate with a business. They simply want to execute business functions and get the expected results regardless of the count and complexity of internal systems that the business consists of. The view of an enterprise has to be unified, simple and functional to the customer. On the other hand, every single application requires specified amount of autonomy which provides developers with space for innovation, improvisation and improvements. Without these elements, when application resides in rigidly integrated, rigorous environment, it could quickly become unprofitable and, in the end, dead. Therefore, there is a strong and urgent need for enterprise application integration patterns which cope with all aforementioned aspects which specify successful and well-considered integration solutions.

18 2.2. Definition of the Enterprise Application Integration Challenge During the previous years some of the traditional ways of performing the Enterprise Integration have failed, some have shown that the process has to be investigated and improved further on. The most important aspect of the EAI is the way how it is performed. It should be straightforward, standardized, manageable and cheap. According to [82] there is much work to be done in this field: Integration of information systems is expensive and time consuming. Between 20% and 40% of labor costs can be traced to the storage and reconciliation of data. In addition, 70% of code in corporate software systems is dedicated to moving data from system to system. One of the reports [7] claims that: A well-designed integration infrastructure will enable businesses to become more adaptive in their business processes, while reducing information latency and business cycle times. New, standards-based integration technology is driving down the cost of application integration by approximately 50%. All facts and numbers show the importance of the properly performed EAI Integration taxonomy Enterprise Integration can be considered and performed on different scopes. Please see Figure 2.1 and explanation below which present the crucial technical aspects of enterprise integration [82]. Figure 2.1: Integration scopes 1 Requirements and principles - determine the functional and non-functional requirements of the integrated enterprise. Use-case modeling is one of the ways of achieving this step. Business integration - determines and integrates business processes which span across multiple applications. Business process re-engineering (BPR) is often performed at this stage. 1 Figure downloaded from [82].

19 2.2. Definition of the Enterprise Application Integration 7 Presentation integration - deals with the creation of the consistent and unified view of the enterprise providing single front-end application. Main users are: employees, customers, partners, and suppliers. Data integration - ensures that all applications understand the data format used in cooperating applications. Data modeling consists of: normalization, integration and consistency of data. Control integration - deals with different types of communication between applications. Determines the messaging models and protocols. Connectivity - investigates communication protocols which are used in the applications. Ensures the inter-connectivity between applications. Quality attributes - Put emphasis on the quality attributes of the integrated application and ensures that the business requirements are translated into quality attributes. Application integration - is the main goal, spans all the layers of the integration model. Ensures that all different applications finally work together Integration factors Companies that want to avoid drawbacks and pitfalls during and after the integration, face many challenges while choosing the proper integration solution. What is obvious, two main options are: build or buy. There are many factors on the basis of which the decision should be taken. The list presented below outlines some of the most important ones. Quality - What is the quality of the solution? Does it fulfill all the functional and nonfunctional requirements? Cost - What is the total cost of the solution? What is the cost to quality factor? Effectiveness - What is the effectiveness and efficiency of the solution? Speed - How long does it take to integrate? How many people have to be involved? Complexity - How complex is the integration solution? Is it based on open standards or proprietary solutions? Is it well documented? Flexibility - What is the nature of the integration? Is the integrated system difficult to modify? What is the time to react to the specified problems known a-priori? Are the elements tightly coupled? The list is not complete, but it briefly shows the problems that every company has to face during consideration of integration. As stated here [7], two main factors are cost and speed. Companies have to think strategically and tactically, putting more emphasis on the integration architecture and usage of open-standards which are easier to learn and faster to use. Companies should make an assumption that a product will be replaced in two years, which greatly influences the price negotiations.

20 2.3. Enterprise Integration Patterns Enterprise Integration Patterns Enterprise Integration Patterns are simply widely known design patterns which tell how and using which building bricks the integration should be performed. They are the set of reusable rules and solutions to commonly known problems occurring during the integration. A great set of enterprise integration patterns is presented in the book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf [88]. Right now, the most important integration patterns will be discussed Topologies Fundamentals Topology defines an arrangement of elements. When discussing the topology of an enterprise system we can easily notice two kinds of elements: hardware artifacts - physical components of the computer system. software artifacts - a set of computer programs and applications. We can also distinguish two levels of abstractions: physical level - describes the way that each hardware artifact physically connects to the network. It also describes the means, the sequence, and the protocol that hardware artifact uses to communicate. logical level - describes the arrangement of channels, collaborations, and mechanisms that different software artifacts residing on hardware artifacts use to interoperate. The subject-matter of this thesis focuses only on the logical level, discussing the paradigms and design patterns that different software artifacts implement to make the whole topology interoperable Integration Fundamentals According to this definition [6] there are two main integration foundations which all other integration patterns derive from. These are: Mediation; Federation; Mediation Mediation pattern or mediator pattern is one of the behavioral design patterns described in the book by Gang of Four [83]. It is a software design pattern that specifies an unified mediator element with a commonly known interface which acts as a intermediary. Elements of the system do not interact with each other. They communicate through the mediator which encapsulates the mediation logic instead.

21 2.3. Enterprise Integration Patterns 9 Federation There are numerous unclear opinions about the popular federation pattern. Some people call it a facade, the others - a federation. There is, however, a clear difference between these two patterns. Facade is a software design pattern that provides a unified interface to a system, for example, to reduce the dependencies or wrap a poorly written APIs with a single well-designed API. A federation pattern is a more abstract integration pattern which defines interoperation between systems or applications. If a set of systems provides the same functions and can interoperate forming one system that provides the the same functions as the constituent systems, then architecture is based on the federation pattern. Federations are main means which provide systems with scalability. There are two types of federations [8]: homogeneous federations - composed of like subsystems with at least the same interfaces and usually the same implementations and exposing the same interface for the composed system ; heterogeneous federations - where some amount of mediation is required to coerce some subsystem ; The confusing misunderstanding is probably caused by the Data Federation Pattern [9] that addresses the data integration problem. It has a similar approach to a facade pattern in object programming while having the word federation in its name Properties of integration solutions Whenever application integration is considered there are many common architectural elements (later called integration services), regardless of specified integration pattern. Data transformation and connectivity are the aforementioned pieces of critical importance. The list presented below, based on article by Microsoft Press [10], tries to summarize all of these common aspects. Messaging (Routing) - Core of the integration system. It supports data reconciliation, transformation and routing among integrated applications. Adapters and Connectors (Transformation) - Elements which create an abstraction layer for the transfer and data protocols. It is an extendible framework which supports connectivity to all of the specified applications. Supporting Platform - Services to support cooperation of application regardless of hardware and software platform. Management and Administration (Registration) - Services to support deployment of artifacts, monitoring and runtime management of the integrated system. Security - Provides authentication and authorization mechanisms, like single sign-on 2. Monitoring for B2B Processes - Services whose main goal is to monitor the business transactions that are executed in the integrated system. Monitoring consists of status activity monitoring and failure monitoring. 2 sign-on

22 2.3. Enterprise Integration Patterns 10 Business Process Management - If business-to-business transactions are to be automated through the exchange of messages and documents, it is necessary for an EI platform to provide a set of support functions for these document exchanges, such as receipts and acknowledgments, activity status monitoring, failure or out-of-process alerts, and compensatory events. This feature set is one of the most important to consider when differentiating truly comprehensive EI platforms. [10] Point to point Integration This is the first and most intuitive pattern or anti-pattern which began to gain a bad reputation in the last decade. It mainly represents the build approach which was mentioned earlier. The general rule is that during implementation companies write custom, point-to-point interfaces and facades in order to perform the integration of specified applications. P2P topology is presented in Figure 2.2. Figure 2.2: Point to point integration 3 It derives from mesh networks where every node can connect to each other. Mesh networks can be classified for example into the group of ad-hoc networks, like for example MANET. It was introduced on the battle-fields in the army, when a data transmission among mobile nodes without any other infrastructure was urgently required. The fact that there was no need for any kind of mediation between the nodes was most important. They used exactly the same transmission protocol and data format, thus all the nodes could connect to each other without additional activities. This assumption is not correct with the reference to the application integration. Since the general purpose is completely different, it is quite obvious that it is hard to expect that all features will be beneficial. In an enterprise application usually uses different data formats and transfer protocols. It results in the fact that whenever an application requires a connection with another applications, all aspects of integration, like data reconciliation and transfer protocol unification, have to be hard-coded in the integration code. The repeated process is the major drawback of the point-to-point integration pattern. It leads very soon to the commonly known n squared problem, which means that there 3 Figure downloaded from [11].

23 2.3. Enterprise Integration Patterns 11 are at most n squared connections among applications. Any kind of addition of new application results in at most n new connections and interfaces, which actually causes additional costs. This kind of topology, known as Big ball of mud [12], appears to have no distinguishable architecture, it is hard to manage and code reusability is poor. Summarized list of the point-to-point features is presented below. Advantages: Simplest approach, quickest option for small enterprises; Minimizes the data transfer time as the connection is direct; Provides redundancy of connection paths (multi-hop connections), in case of direct connection failure; Graph theory can be applied to the path management; Disadvantages: Integration anti-pattern; Addition of new system results in at most n new connections and integration interfaces; Unmanageable in big systems with n squared connections; Proprietary integration code is duplicated and it is difficult to maintain, to extend, to test, and to manage; Does not scale at all ( Big ball of mud architecture), good only for small systems; Implements neither federation nor mediation pattern. Mediation interfaces are hard coded in every application; It is very hard to say that the point to point integration pattern is bad, and all the other patterns are good. However, when rating the solution according to the criteria introduced in Chapter the result is very poor. Every aforementioned integration service has to be implemented as the proprietary solution, which is expensive, hard to manage and, in most cases, unreusable. P2P integration is successfully applied in small enterprises with not many applications and connections but the integrator has to be aware of the main pros and cons during the integration in order to choose the best fitting solution Broker Integration Broker integration pattern was invented to fill the gap created by the P2P integration. P2P solutions were beneficial for small enterprises where the number of application was minimal and all connections were nonvolatile and commonly known. However, when the companies faced growth problems, P2P became inefficient, expensive and unmanageable. Therefore, broker integration pattern was introduced.

24 2.3. Enterprise Integration Patterns 12 Figure 2.3: Broker integration 4 Pattern presented in Figure 2.3 introduces the definition of centralized integration broker which links the applications. The main goal of a broker is to decouple source systems from target systems by assuming responsibility for interoperability and coordination of communication. All integration services are incorporated into the central broker which acts as a intermediary among applications. It is located in the center of the system, between the source and target applications. Usually the whole process of communication goes through the broker. Thanks to the centralized configuration, there is a reduced number of connections which come to n where n is the number of systems. It is a great improvement in comparison to P2P topology with n squared connections. When an application is added to the enterprise system, integration with only one element - central broker has to be performed. During this process the integrator has a wide variety of integration services residing in the broker, thus there is no need to write the whole integration code. However, there are also drawbacks. The broker is a single point of failure, which means that when it is down, the whole system is down. The problem can be resolved by clustering the central machine on which broker resides, but such scalability is limited to the scalability of this machine. What is obvious, the advantages of the centralized topology automatically entails disadvantages which come out of the general rule of this topology. There are three main classes of brokers[13] (Figure 2.4): Direct brokers; Indirect brokers; Message brokers; 4 Figure downloaded from [11].

25 2.3. Enterprise Integration Patterns 13 Figure 2.4: Classes of brokers 5 Direct Broker Its main distinctive characteristic is the way in which it coordinates communication. After establishing the initial communication, the two parties communicate directly (without the mediation of the broker) using specified proxies. Figure 2.5 which is presented below depicts sequence of invocations: 5 Figure downloaded from [13].

26 2.3. Enterprise Integration Patterns 14 Figure 2.5: Direct Broker Sequence Diagram 6 Indirect Broker Indirect broker coordinates the communication differently. It is especially useful when a careful control over the communication is required. Figure 2.6 which is presented below depicts sequence of invocations: 6 Figure downloaded from [13]. Figure 2.6: Indirect Broker Sequence Diagram 6

27 2.3. Enterprise Integration Patterns 15 Message Broker aka. Hub and Spoke The hub and spoke distribution paradigm has been known for many years. It is a implementation of a star-network topology. It also derives from American airline industry. In 1955 [14] Delta Airlines pioneered the spoke-hub model at its hub in Atlanta, reducing the usage of point to point model. At the beginning only direct flights, operated by the middle size airplanes, were scheduled. Later, model started to alter and direct flights were replaced by hub-to-hub flights, operated by huge airplanes and local spoke-to-hub flights. In 1978 this model was annexed by several airlines and it has been widely used until nowadays. The main idea is the reduction of connections, which leads to more efficient use of transportation resources. While this example is connected with the airline industry, a few advantages are simply observable in reference with the enterprise integration. Coming back to the IT field, a message broker, also referred to as a hub and spoke, is the specialized version of the broker pattern. It uses message oriented middleware (MOM) as the communication platform where messages are the information carrier. The most important fact here is that the message broker can expose different interfaces. It does not enforce the usage of the agreed-upon unified data format. To receive the messages, application have to register themselves in the broker using the specified mechanism. Main responsibilities of the message broker [15]: Receiving messages; Determining recipients and performing the routing; Handling transformation; Sending a message to the recipients; Broker summary Summarized list of the broker features is presented below. Advantages: Clear structure - implements the mediation pattern. Simplicity - reduces the number of connections among applications. New application has to be integrated only with the hub which deals with the data unification, communication and translation. Centralization - provides flexibility and centralizes the configuration. Single point of configuration modifications. Services - provides a set of implemented integration services which are used during the integration. No proprietary spaghetti code has to be written. Convenience - topology is more intuitive. Management and monitoring is convenient.

28 2.3. Enterprise Integration Patterns 16 Disadvantages: Single point of failure - when the broker is down, the enterprise is down. Congestion point - performance of the system is mainly dependent on the performance of the broker. It can be a bottleneck. Complex proprietary solution - brokers are usually heavy, all-in-one proprietary solutions based on closed, corporate standards, which makes them very complex to understand and use. Using such solutions indicates in many cases hiring a third-party, consulting company which performs integration and trains developers and stuff how to use the corporate broker. Undoubtedly, it generates high costs. Broker pattern solved the n squared problem caused by P2P unstructured integration. Pattern is successfully implemented and brought into the real business. Main implementations are licensed, proprietary, vendor-dependent solutions, which are very expensive. The utilization of the technology cannot be done ad-hoc as the level of complexity is very high, which also increases the integration costs. Work force costs are also worth mentioning as it takes a lot of time to get acquainted with all vendor-specific solutions Message Bus Integration The term bus derives from computer networking as well as from computer architectures. In a bus network a set of nodes is connected using a shared communication line - the bus. In computer architecture a bus is a subsystem which is responsible for transferring the data between computer components. Unlike the point-to-point connection, a bus can logically connect a set of nodes using the same shared communication line. Normally, each bus specifies a set of connectors to plug the elements. The idea, which is very intuitive and straightforward, is depicted in Figure 2.7. Figure 2.7: Bus integration 7 Let s take a closer look at the bus architecture, being more precise at the message bus, since it is the implementation of the bus paradigm in the application integration field. The general purpose 7 Figure downloaded from [11].

29 2.3. Enterprise Integration Patterns 17 is to enable applications to work together in a decoupled manner, so that the applications can be plugged in and out without affecting the behavior of others. Message bus behaves as the message broker between plugged applications. It automatically reduces the number of point-topoint connections as the only connection which an application has to initiate, is the link with the bus. Message Bus is a combination of: common description language; common command set; messaging infrastructure; which allows different systems to interoperate using a specified set of well-known interfaces. The main idea of the bus integration is based on the agreed-upon common description language (CDL) which could be understood by all parts of the system. It is a layer of abstraction. Whenever an application has to be plugged into the bus, there are certain steps to follow: describe the data in the CDL format; describe the services in the CDL format; When an application wants to use another application, it simply sends messages to the bus using the common command set. Prepared messages have to comply with the specified format which the bus expects (CDL). Invocation arguments as well as the invocation results are also described using the CDL, so the interoperability is provided. Messaging infrastructure provides the application with the possibility to communicate in a reliable and robust way. Messages are the carries of information. Usually, the bus does not preserve the message order as this additional logic is not required. The main motto of the Message Bus can be described in such a way: logically centralized, physically decentralized [11]. The bus, which is the main building brick of the integrated application, supports distribution in a federated manner, which means that constituent buses cooperatively implement the greater system with the same functions as the constituent buses. This is the exact example of the federation pattern described in Section Distribution and federation are the key aspects provided by the bus. The messaging mechanism of the bus can be implemented using one of these: Message router; Publish/subscribe mechanism; Message Router The main goal of message router is to consume messages from one channel and republish them to different target channels depending on a set of conditions. What is also important, message router does not change the content of the routed messages. The taxonomy of message routers is presented from the Enterprise Integration Book [88]. Content-Based Router - examines the message and makes routing decision based on the content of the message;

30 2.3. Enterprise Integration Patterns 18 Message Filter - has only one output channel; when message matches specified criteria, it is routed, when not it is dropped; Recipient List - computes recipient list, then sends copy of message to all recipients; Splitter - publishes new message for each element of the original message Aggregator - gathers and correlates messages, then sends correlated messages as a one message to the output channel; Resequencer - contains an internal buffer; it stores out-of-sequence messages until a complete sequence is obtained, then the messages are routed; Composed Message Processor - splits the message up, routes the sub-messages to the appropriate destinations and re-aggregates the responses back into a single message; Scater-Gather - routes a request message to the a number of recipients; collect the responses and distill them into a single response message; Routing Slip - attaches the routing list to the message and sends the message to the first recipient from the list; the recipient sends the message to the next recipient from the list; the process is repeated until the end of the list is reached; Process Manager - uses a process manager to generate processing sequence and to determine next step on the basis of intermediate results; Publish/Subscribe Mechanism The main goal of publish/subscribe mechanism is to send a message, whenever it is published, to all subscribed recipients. There are three main types of Publish/Subscribe pattern [16]: List-Based Publish/Subscribe - keeps a list of published topics and subscribers and notifies each one on event; Broadcast-Based Publish/Subscribe - does not determine the subscribers; broadcasts the message to all recipients in the bus; Content-Based Publish/Subscribe - matches message against specified criteria and then sends message to interested subscribers; Summary Currently, message Bus pattern goes one step further and takes the lead. Not only does it contain all advantages of the broker architecture, but it also solves some inconvenient problems. It simplifies and standardizes the approach of integration as well as introduces a federated manner of solving the broker clustering problem. Message buses are one of the most sophisticated integration solutions and are nowadays widely used in business.

31 2.3. Enterprise Integration Patterns Comparison Figure 2.8 summarizes the main integration topologies showing their hierarchy. topologies were described in the chapter above. All of these Figure 2.8: Topology Summary 8 At the end a short comparison of the Message Broker and Message Bus will be presented as these two patterns are frequently confused. 8 Figure downloaded from [13].

32 2.4. Enterprise Service Bus 20 Table 2.1: Topologies comparison Topologies comparison Message Broker Message Bus Implements the pattern of mediation; Implements the pattern of federation; Physically centralized broker; Logically centralized, physically decentralized bus; Distribution of integration logic only by clustering the central machine on which the bro- of buses, local clustering of each bus is also Distribution of integration logic by federation ker resides; available; Broker is a monolithic software; All integration services are embedded into the hub; Usually, broker can be used in all or nothing mode ; Exposes many different, proprietary interfaces, each for a distinctive application type or data model; Usually a proprietary solution; 2.4 Enterprise Service Bus Different components cooperate to provide integration services; Pluggable architecture provides reusability; Selective usage model of bus functions is possible; Based on one common description language which attempts to reconciliate data model and application specific approaches; Standardized solutions exist; It has to be admitted that there is no agreed-upon definition of what an Enterprise Service Bus (ESB) actually is. The term was initially invented by Sonic Software to refer to their solution named SonicXQ later renamed to SonicESB. There is much ambiguity, confusion and buzz around ESBs caused by companies which seem to offer ESB like solutions. To prove that concept, some quotations presenting contrasting ESB definitions will be presented [79]: A Web-services-capable infrastructure that supports intelligently directed communication and mediated relationships among loosely coupled and decoupled biz components. Gartner Group; The ESB label simply implies that a product is some type of integration middleware product that supports both MOM and Web services protocols. Burton Group; A standards-based integration backbone, combining messaging, Web services, transformation, and intelligent routing. Sonic Software; An enterprise platform that implements standardized interfaces for communication, connectivity, transformation, and security. Fiorano Software; To put it bluntly: If you have WebSphere MQ and other WebSphere brokers and integration servers, you have an ESB. Bob Sutor, IBM; The Enterprise Service Bus is a uniform service integration architecture of infrastructure services that provides consistent support to business services across a defined ecosystem. The ESB is implemented as a service oriented architecture using Web Service interfaces. CBDI;

33 2.4. Enterprise Service Bus 21 Smoke screen created by this definitions completely covers the main goal and general purpose of the ESB. Straightforward architectural definition of this pattern will be presented later on, but firstly the environment in which the ESB resides, will be briefly introduced as these two elements are tightly connected Service Oriented Architecture as ESB environment. Service Oriented Architecture is a technology-independent software architecture which specifies a certain set of rules how to build and maintain distributed applications. It presents an approach to designing computer systems which consist of autonomous services. The discussion on what a service actually is, looks somehow similar to the discussion about the definition of an object in early nineties. OASIS defines service as [68]: a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description. Simplier, the service represents an autonomous business logic or technical functionality defined by an agreed-upon service description. Figure 2.9 presents the overview of SOA. Figure 2.9: Location of ESB in SOA environment 9 In the SOA environment services consist of: contract - specifies the pre and post conditions, functional and non-functional requirements as well as the type of the service; implementation - the core business logic which operates on the provided data and returns the result; interface - short description of the service; 9 Figure downloaded from [70].

34 2.4. Enterprise Service Bus 22 The main guiding principles of SOA accroding to [70] are: loose coupling, reuse, granularity, modularity, composability, componentization, and interoperability. Not going into details, it means that services are reusable and granular building bricks or the low level entities which application should consist of. Whenever a system is built, a developer chooses required services, designs the business process and composes the applications using, for example, a graphical tool. Great emphasis is put on the reusability as it was a main concern of the software industry. What is very important, SOA is technology independent. It is a common mistake to claim that SOA is tightly coupled with the Web Services. SOA architecture can be implemented using a wide variety of technologies, like [70]: SOAP, REST, RPC, DCOM, CORBA, WCF or last but not least Web-Services. Summing up, SOA is a modern, thoroughly different approach to software design from the previous ones, as earlier computer systems were formed as monolithic, coarse-grained, unreusable entities. The primary goal of SOA is [56] to align the business world with the IT world in a way that makes both more effective ESB Definition SOA architecture defines the set of rules how to design and implement a distributed application. These rules apply whenever new applications are created, as well as when old or legacy systems are redesigned. ESB pattern greatly fits into the SOA since it is fully compliant with its paradigms. It encompasses all amenities of SOA as well as provides a set of sophisticated features, described later on. Definition of an ESB presented in this work goes as follows: ESB is a pattern which is the SOAoriented extension of the bus integration pattern. It is an integration pattern as it fulfills all the required conditions which are described in Chapter However, ESB goes beyond as it is SOA oriented. It is not supposed to implement SOA architecture but it was invented to facilitate the engineering and utilization of modern SOA. Alike SOA, ESB is not bound to any specific technology, it only defines a set of features which a ESB solution should consist of. Generally, on the highest level of abstraction ESB provides: Bus integration pattern; SOA runtime environment; ESB as integration pattern In the area of SOA, integration can be described as service virtualization [57], what describes the feature of virtualizing the following factors during the service interactions: Protocol and pattern - the participants of interaction need not use the same transfer protocol (TCP, UDP, HTTP, SOA, etc.) or interaction pattern (synchronous/asynchronous, requestresponse, only-in, etc.). ESB should provide the whole required mediation. Interface - requester and responder need not use the same interface of the same service. ESB should provide the ability to separate Business Service (contract 2.4.1) from Service Implementation (implementation 2.4.1) and interface;

35 2.4. Enterprise Service Bus 23 Identity (transparency of the service location) - the requester of interaction need not know the identity (for example address) of the responder to request a service. Service could be provided by any of providers from different physical locations. The list of providers and their identities is known only to the ESB and, what is important, may change without impacting the requester side. ESB as a runtime environment One of the most important principles of ESB as the runtime environment is the aspect oriented connectivity [57] which allows to inject cross-cutting aspects between service invocations in order to add the support of: security, management, logging, auditing and transaction management. This feature is similar to the Request Interceptor specified in the Enterprise Service Beans standard. ESB should also embed the advantages of an application container and provide the services with the runtime facilities such as the support of deployment in a high availability configuration. The environment should also support scalability, management and reliability. ESB capabilities According to Mark Wright, [108] ESB should be defined by specifying the core facilities which form the main characteristics of an ESB. They can be grouped under the parent, logical component which they belong to. However, in this work the definitions as well as the grouping are extended, redefined and brought to the different level of abstraction. The list of ESB logical components and their functions is presented below. Rules Engine (can be implemented by the Message Router, described in detail here: 2.3.6) Routing; Message Transformation; Message Enhancement; Message Processing; Service Registry: Service Mapping - ability to map service contract into the corresponding service implementation; Service Mediator: Protocol Transformation - ability to expose service using different types of protocols; possible thanks to the usage of CDL; Business Processor (ability to coordinate complex business processes): Service Orchestration - orchestrates service in order to form a business process; services are coordinated in a centralized way, coordination requires a central point - mediator; processes can be specified using, for example, BPEL;

36 2.4. Enterprise Service Bus 24 Service Choreography - coordination realized in a decentralized manner; can be implemented using the routing slip function of the message router; Service Container: Security (Authentication and Authorization); Transaction Support; Context Propagation; Management; Logging; Deployment environment; Service Manager (Monitoring features will be discussed later on): Business Monitoring - ability to track, monitor, analyze and profile logical business operations, executed in the bus; System Monitoring - ability to monitor main system characteristics and quality informations; Service Level Agreement - business auditing; Federation of ESBs Federation (2.3.2) is one of the main concepts which the message bus pattern is based on. However, in reference with Enterprise Service Bus, this paradigm was specified mainly theoretically; some of the implementations do not support it at all. Currently, some trends of change are emphatically visible since the newest Forrester report [85], dated April 2008, reads as follows: The enterprise service bus (ESB) is not a single entity across the enterprise but a federation of ESBs and other SOA infrastructure that work together to ease and control service usage across the enterprise. According to this report: ESB federation enables multiple ESBs to work together, seamlessly and transparently, to enable enterprisewide access to and management of services across multiple domains, each of which has potentially different requirements for security, visibility, mediation, quality-of-service, and management. This definition, completly compliant with the federation pattern described earlier, seems to head of the right direction as the main drawback of the up-to-date ESB solutions is noticed and there are attempts to resolve it. According to the report, three models of federation are possible: Direct federation - ESBs which host services are directly connected; Brokered federation - many ESBs, one acts as a mediator and hosts no services; all the other ESBs host services and connect with each other using the mediator; Full federation - many ESBs, one acts as a mediator but also can host services; direct or indirect communication possible;

37 2.4. Enterprise Service Bus ESB implementations There are many software solutions which are claimed to be ESB compliant, however some of them have the ESB only in theirs name. The list presented below outlines the main ESB implementations presenting the name of the product as well as the producer. Aqua Logic Service Bus, BEA; WebSphere ESB, IBM; JBoss ESB, JBoss; OpenESB, SUN Microsystems; ServiceMix, Apache; Enterprise ServiceMix (FUSE), Iona; SonicESB, Sonic Software; ESB, IWay Software; WSO2 Enterprise Service Bus, open project; ChainBuilder ESB, Bostech; Mule, open project; PEtALS, OW2; BizTalk Server products family, Microsoft; Summary ESB is currently one of the most up-to-date solutions in the area of enterprise integration and service oriented architecture. It is considered to have significant influence on design of distributed software solutions across the world. It proved to be very successful and now it is a top-selling product offered by most of the major IT market players. However, in a meaningful article [56] Bobby Woolf, respectable software architect and co-author of the Enterprise Integration Patterns book, questions the legitimacy of an ESB. The author claims that it is not beneficial to adopt an ESB without an attempt to identify, design and implement the autonomous services, which are the underlying, building bricks of every SOA. It is stated that clients want to build only ESBs not paying attention to business requirements, as they seek rapid profit growth. They want to create an infrastructure and then wait and hope that some SOA will use it. Author puts the emphasis on the fact that ESB without services does not create a business value as this is a useless infrastructure without customers (services) which benefit from it. The main motto reads as follows: Don t build an ESB by itself; build it as part of a SOA.

38 2.5. Java Business Integration Java Business Integration In contrast to the ESB, definition of the JBI is clearly specified in JSR 208. Althought the word ESB can be found only once in the whole document, these two terms have a lot in common. According to the specification, [2] Java Business Integration (JBI) seeks to address EAI problem by creating a standards-based architecture for integration solutions. This infrastructure allows thirdparty components to be plugged in to a standard infrastructure, and allows those components to interoperate in a predictable, reliable fashion despite being produced by separate vendors. It is anticipated that this ability to interoperate will create a multivendor ecosystem which will give rise to large pool of integration-related technologies that can be sourced by end users. In addition, this ecosystem will foster new innovations in integration technologies since it will permit innovators to concentrate on a particular technology or problem area, without having to worry about providing all the other pieces needed to build a complete integration solution. (...) JBI defines what is sometimes termed a service container in Enterprise Service Bus (ESB) systems. To sum up, JBI standardizes the ESB pattern by specifying a concrete, Java-centric model of ESB Background Looking back, JBI is the last but not least integration technology presented in this work. All the previous definitions were introduced to present the taxonomy of enterprise integration patterns, describe the most popular solutions and preciously outline the background and origin of JBI. At the current stage, when it is stated that JBI derives from ESB which derives from bus integration, all underlying concepts should be well-known. JBI fulfills all conditions to be characterized as enterprise service bus (2.4.2). The list below accurately explains and summarizes all the JBI features with the reference to ESB. JBI is an integration pattern (2.4.2) as it virtualizes: communication protocol; communication pattern; interface and identity; JBI is a runtime environment (2.4.2). It is often called a container of containers : container for plug-in components; container for service artifacts which resides in plug-in components; JBI acts as intermediary. Components do not interact with each other directly, they communicate by means of mediated message exchange governed by the normalized message router (NMR) instead (2.3.6). JBI is SOA-oriented as it embraces a service-oriented approach. It is based on Web Services model. JBI encompasses all the capabilities each ESB should consist of (2.4.2). JBI is standardized and based on open standards:

39 2.5. Java Business Integration 27 JBI is dependent on either J2SE 1.4 or J2EE 1.4; JBI uses WSDL(1.1 and 2.0) as the common description language (CDL); Interacion makes use of XML based messages (XML 1.0); Processing uses the JAX-RPC 2.0 model; JBI makes use of Java Management extenstion (JMX); JBI defines service container defined by ESB pattern; Architecture Figure 2.10: Architecture of JBI 10 JBI consists of many elements which operate together to form runtime environment for the utilization of SOA. The solution is complex (Figure 2.10), thus its description is grouped by the logical component or function. It is impossible to briefly describe all the features JBI contains, prior JBI knowledge is required. Therefore, some general ideas of high-level architecture will be presented. Unified Service Description JBI specifies a declarative model of services using Web Services Description Language (WSDL versions 1.1 and 2.0). Agreed-upon and unified service description is crucial as every party understands it - mediation among service invocations is not required. Message-based services are declared on two levels (Figure 2.11): abstract and concrete. This kind of design derives from 10 Figure downloaded from [51].

40 2.5. Java Business Integration 28 Figure 2.11: JBI Messaging Model 10 object-oriented design, where interface and implementation are decoupled. Specification of abstract and concrete service model keeps the configuration tidy and avoids redundant pieces of information. Abstract service model describes a service in a general way. It describes messages which are sent during invocations of operations and groups these operations into interfaces. Concrete model specifies services as the implementation of the abstract interfaces. A service consist of endpoints which offer access to the service implementation through different bindings protocols. Aforementioned mechanisms present the ability of JBI to separate the business service (contract) from service implementation and interface, which is explained in detail in Section Normalized Message A concept of message normalization is used in JBI. It is the carrier of data between consumers and providers. It encompasses invocation arguments as well as operation results. Normalized message consists of three main parts: Message Content (payload) - normalized XML document, conforming to specified message scheme; content carrier; Message Properties - metadata of the message; holds additional data added to the message during the processing of the message; metadata can be added by plug-in components as

41 2.5. Java Business Integration 29 well as system components; Message Attachment - additional part of the content, which is,for example, non-xml, can be attached as an attachment; data handler for such content has to be specified; The main goal of the normalized message is to provide interoperability between components. Normalized Message Exchange Message exchange is the logical entity which describes communication between consumer and provider in JBI. Not only does it serve as the container for normalized messages that take part in the exchange, but it also represents the JBI-local portion of service invocation. Message exchange introduces the communication pattern which specifies steps of communication. There are four communication patterns: In-Only - standard one-way messaging pattern; consumer initiates with a message; provider replies only with status response that indicates that the exchange is over; Robust In-Only - reliable one-way messaging pattern; consumer initiates with a message; provider replies with status or fault response which indicates the correctness of the exchange; In-Out - standard two-way messaging pattern; consumer initiates with a message; provider replies with status or fault response which indicates the correctness of the exchange; consumer responds with a status; In Optional-Out - standard two-way messaging pattern; provider response is optional; Normalized Message Router Delivery of the messages which take part in message exchanges is provided by the normalized message router (NMR) which is the core of JBI. It performs operations with different qualities of service depending on multiple factors, like application state or nature of the message. Three qualities of service are supported by the NMR: Best effort - no guarantee of message delivery; message can be dropped, delivered once or more than once; At least once - message will be delivered once or more than once; Once and only once - message will be delivered exactly once; Components JBI introduces two types of components which can realize two different roles: consumer and provider. Components are installed during the process called Component Installation. Binding Component (BC) - BC provides the virtualization of communication protocol and pattern by performing normalization and denormalization from and to the protocol-specific

42 2.5. Java Business Integration 30 format. It provides the NMR with the possibility to deal only with normalized communication format. Each BC understands one communication protocol and behaves like connector which allow different kinds of communications with the bus. BC can perform operations playing two roles (separately or simultaneously) : consumer role - it enables the services deployed in the bus to communicate with external services. provider role - it enables external customers to communicate with the services deployed in the bus. Service Engine (SE) - SE provides business logic services and transformation services in the JBI environment as well as consume such services. SE enables to construct low-grained complex processes by aggregating and orchestrating other services as well as creates simpler, fine-grained services, for example data transformation and enhancement. SEs act as containers as they form the runtime environment for services. Whenever a new service is created it is deployed to specified SE which supports the technology using which the service was implemented. SE can serve as service provider, consumer or both. Component Framework JBI provides the component framework which enables the components (BCs and SEs) to interact with the JBI environment. All the interfaces are agreed-upon and specified. There are two types of interfaces: API (application program interface) - exposed by the JBI framework to BCs and SEs; defines the contract between JBI environment and pluggable components in order to achieve functional goals; SPI (service provider interface) - interfaces implemented by the components; Artifacts JBI components behave as a type of container which new artifacts are deployed to in order to add new service consumer or provider logic. For example, it is possible to deploy new style sheets to XSLT-transformation service engine to provide it with new transformation logic. The process of adding such component-specific artifacts is called Artifact Deployment. There are two types of artifacts: Service Unit (SU) - single deployment package, destined for a single component; content of service units is not specified as SUs are opaque to JBI; each SU must contain a JBI-defined descriptor file in which static services (endpoints) produced and consumed by this SU are defined; Service Assembly (SA) - equivalent of consumer application; SU artifacts are grouped under a higher level entity, called service assembly (SA), which is a set of SUs. Each SA contains service deployment descriptor which specifies to which component each SU is to be deployed. SA concept is sometimes called composite service description (CSD);

43 2.5. Java Business Integration 31 CSD plays a significant role as it provides the JBI environment with the information about service connections between SUs and providers. These data are grouped under two categories: Description of static services provided and consumed as a result of deployment of a service unit [2]. Description of static service interconnections between the services units of the service assembly, as well as dependencies on services offered outside of the service assembly (i.e, services provided elsewhere in the JBI environment [2]. Management JBI bus is fully administrable through the JMX. Specification outlines several JMX beans in order to provide full control over the JBI environment. Most significant functions of the management beans are the life cycle management of the components, installation of engines and bindings as well as deployment of component artifacts to executive environment. Message Exchange Flow One of the most important JBI environment s functions is to route message exchanges during the interaction between service consumer and service provider. It illustrates all features that are under the hood of JBI. Figure 2.12 depicts the standard In-Out Message Exchange. Figure 2.12: Message Exchange Flow 11 The interaction consist of many phases. 11 Figure downloaded from [73].

44 2.5. Java Business Integration An external service consumer sends a service request to a binding component (acting as a consumer component in this case) using particular transport protocol. The consumer component normalizes the service request using specified format described in WSDL model. BC creates message exchange, creates in message with the normalized service request and sends it to JBI Environment. 2. The JBI environment selects service provider and routes message directly to it. The provider pulls the message exchange using accept method and obtains invocation parameters from the in message. 3. The Provider component invokes business logic requested by the consumer. The results of the invocation are stored in the normalized form in the out message of the message exchange. 4. Message exchange with the out message is sent to the JBI environment. 5. JBI environment recognizes the message exchange and routes it directly to the consumer which pulls it using accept method. 6. The result of the requested service is denormalized and returned to the consumer using the particular protocol and transport, which the consumer previously used. 7. The consumer component sends status to the provider component. 8. The providing component pulls the status message from NMR using accept method. 9. Message exchange terminates. Federation The main drawback of JBI is that it specifies neither distribution nor federation. This fact significantly diminishes the wide variety of the ESB usage scenarios, which JBI implements, causing that this technology places itself between hub and spoke and ESB pattern JBI Implementations The following table presents the state of the art of JBI implementations at two different points in time: the beginning of the research concerning this thesis and the current moment. Table 2.2: JBI implementations Q Q JBI based JBI certified JBI based JBI certified ServiceMix OpenESB ServiceMix OpenESB Mule Mule OW2 PEtALS JBossESB ChainBuilder Fuse ESB

45 2.6. Conclusion 33 At the beginning, it was decided that OpenESB and SericeMix would be evaluated as these were the only two solutions that were worth mentioning with reference to JBI compliance. OpenESB was certified to be fully JBI compliant, while ServiceMix was considered to be almost JBI compliant. ServiceMix developers claimed that only some slight incompatibilities exist. In the first quarter of 2008, it turned out that PEtALS is also an absorbing solution which is probably ahead of ServiceMix. However, it was too late to include it in the subject of the performed research Summary The future of JBI is currently unknown. The approach to standardizing enterprise service buses seems to head off the right direction. The first version of the specification was approved by most of the major ESB vendors, which set great stores on the tight standardization of ESB. However, later on the general assumptions of the JSR group were partially fulfilled as not many steps were taken by major ESB vendors to unify existing implementations in compliance with JBI. Only two JBI certified solutions were designed from scratch after the publication of the JSR, which indicates that there were not many movements in the ESB community. It is also very hard to evaluate the market share of these solutions which does not seem to be meaningful. The uncertain future of JBI is confirmed by the revision summary of the second version of specification which was rejected by the two main ESB providers that remain unconvinced of the value of JBI solution for SOA. At the current moment, it is hard to predict the significance of JBI. According to many resources, the future of ESB is beyond any doubt, only time can clarify the role played by JBI. 2.6 Conclusion The presented chapter outlined a thorough examination of the Enterprise Application Integration. Theoretical research was very important at this stage of work as it broadened the view of the core concepts which are discussed across this paper. Precise definition introduced in this chapter helped to demystify the real differences between integration patterns and laid a grate foundation of clear and accurate theoretical background. It occurred that point-to-point integration is the most intuitive solution. However, its applicability is very limited. It is beneficial only for small systems consisting of restricted number of elements. If that border is outrun, problems rapidly accumulate. Broker integration seemed to eliminate main problems met in the previous pattern. It is scalable and expansible. The main problem concerning broker integration is that its implementations are usually proprietary and based on closed standards. It results in an expensive and unmanageable solution which is hard to maintain. Broker is also a single point of failure, which is unacceptable in some cases. Bus integration pattern is supposed to solve all problems which are mentioned above. Logically centralized, physically decentralized integration topology which is based on commonly known standards and technologies. Shortcomings are almost imperceptible. ESB tends to develop this trend helping to facilitate the utilization of SOA. At the end Java Business Integration (JBI) is introduced. It is defined as a standardized, java-oriented implementation of ESB. All its architectural characteristics are described in order to provide a close examination of the specification. Most of this information is required by following chapters whose main goal is to investigate JBI in order to propose its generic monitoring architecture. Chapter is finished with discussion

46 2.6. Conclusion 34 concerning the future of JBI. It is very hard to predict, whether the specification will be successful since many signs precede the gradual deterioration of its significance. All of the facts mentioned earlier formed a complete description of the target technologies. They have also deepened the knowledge in the specific domain of the thesis.

47 Chapter 3 JBI Monitoring It is not enough to have great qualities; We should also have the management of them. Francois de la Rochefoucauld, a French classical author, leading exponent of the Maxime. 3.1 Preamble The previous chapter introduced the main concepts: Enterprise Application Integration, Enterprise Service Bus and Java Business Integration, which are the subject of the monitoring system presented in this thesis. This deep analysis of ESB and JBI essentials gives an appropriate background for discussing the aspect of monitoring enterprise applications, which is the content of this chapter. The content is structured in a top-down manner. At the beginning, an enterprise monitoring abstraction is provided which is extended in the following sections relating to JBI specification and some already defined monitoring theories. Finally, the incremental approach is summarized by identification of JBI monitoring as a non-trivial challenge. The monitoring characteristics introduced in the monitoring abstraction are referred to throughout the rest of the chapter and allow to evaluate the monitoring capabilities defined in JBI as well as those provided by OpenESB and ServiceMix implementations. The evaluation is performed in JBI status quo section, which additionally discusses the important feature of clustering. It is discussed in the context of ESB federation introduced in the previous chapter. Clustering support increases difficulty of JBI monitoring systems and implies the need of a theoretical background provided by Application Response Measurement and Business Application Monitoring. The last section discusses the influence of all aspects mentioned in this chapter and tries to grasp the architecture implications. It has to be stressed that the entire content of this chapter is intentionally kept at an abstraction level, which does not refer directly to GlassBus solution that has been introduced later. Focusing on enterprise monitoring aspects without solution, the context gives a better justification for decisions made in further chapters. 3.2 Monitoring abstraction This section tries to identify some general well-known monitoring abstraction and, having failed to do so, it proposes a simple substitute in the form of a monitoring system classification. According to a research performed for the purpose of this thesis, a general monitoring theory applying to Enterprise Service Bus environment does not exist. There are specifications aiming to standardize application server monitoring, e.g. JSR-77: J2EE Management [17], but such an approach does not comply with ESB. A theoretical background would allow for a later contextual

48 3.2. Monitoring abstraction 36 definition of our solution s capabilities. The shortcomings and advantages of the system would be clearly visible in the context of a more general monitoring abstraction. The lack of a monitoring abstraction was the motivation of the subject analysis, which resulted in putting forward a monitoring system classification. Monitoring systems can be classified on the basis of their various characteristics. After evaluation of the key characteristics related to monitoring, the following system division has been created: 1. Mechanism of information acquiring How does a system measure the resources that are the subject of monitoring? log analysis the information regarding the resources is extracted from the logs produced by the resources. It requires constant log parsing and identifying the data that is important from the monitoring point of view. code instrumentation the resources are equipped with new functionalities that allow to intercept monitoring events as soon as they appear and on this basis to update the monitoring parameters. 2. Model of monitoring parameters evaluation What patterns are used for the querying of resources? constant pulling the resources are constantly being queried. When the response changes, the monitoring engine updates the parameters. The information is pulled from the resources to the system. event-driven tracing the resources trigger events at the status change that are received by the system and processed to some monitoring parameters update. The information is pushed by the resources to the system. 3. Mechanism of information exposition How can the external system access the system s monitoring parameters? log file change of the monitoring parameters or periodical status is reported to a log file. well-defined interface each monitoring parameter is exposed through an interface which defines syntactical and semantic details. 4. Model of monitoring parameters exposition What patterns are used for the exchange of the data with external systems? request-response interface data exchange is realized with a plain old request-response in a synchronous way (information is pulled by an external actor). Method signatures (or some generic data format) is known a priori. notification oriented interface the data is pushed to an external actor by means of notifications. Generic data format is known a prori. 5. Monitoring subject When analyzing a computer machine (regardless of whether it is one computer or a whole cluster), the monitoring parameters can obviously be divided to system specific and application specific. A list of sample parameters specific to each monitoring type can easily be provided:

49 3.3. JBI monitoring status quo 37 application free RAM memory, free hard disk space, CPU usage, number of system processes, number of running applications, each application RAM and CPU usage, network bandwidth usage [107]; system number of related network connections, response time from the user s point of view, persistence level errors, business level errors, modules usage statistics, business operation performance; Depending on the particular monitoring system s scope and subject, the division into application and system can be related to different aspects. 6. Time context How is the available monitoring information related to the present state? historical information there is a record of monitoring parameters history. Queries about both the past and present status can be made. current information only the present status is maintained. Past information is overwritten with newer information. 7. Monitoring update delay How far from the present time is the newest available monitoring state positioned? monitoring of real-time there is no delay between the event and monitoring parameter update. monitoring of relative near past there is significant delay between the event and update. The proposed classification takes the meaningful characteristics of a monitoring system into account, but does not perform any detailed analysis. Its purpose is to provide some reference point to a wide range of specifications, technologies, patterns and implementations that are within the scope of this thesis. 3.3 JBI monitoring status quo In order to propose a solution for a monitoring JBI container, a state-of-the-art analysis has to be performed. As it has been defined in the introduction (1), the objective of the thesis is to propose a system complying with JSR-208 and successfully operating on the open source JBI implementations: ServiceMix and OpenESB. This section goes through JBI, ServiceMix and OpenESB analysis and it extracts information relevant to monitoring, ipso facto creating the environment for establishing the JBI monitoring solution Monitoring from JSR Java Business Integration introduces basic management and monitoring mechanisms based on the Java Management Extensions (JMX) technology. Thanks to the design of the JMX architecture, API, patterns and appropriate services supporting both networks and applications, it allows to control and query various resources: network elements, system compounds, application objects, enterprise components. JMX architecture s flexibility is achieved by a division into the following layers: Instrumentation level;

50 3.3. JBI monitoring status quo 38 Agent level; Distributed services level; The resources managed with JMX are instrumented with Managed Beans (MBeans) and are the primary architecture entities residing on the instrumentation level. Further JMX details are not described in this thesis authors assume that JSR-003 and JSR-160 [18, 19] are known to the reader. JBI specification uses MBeans from the instrumentation level to define the management skeleton interface, which provides the following functionalities: Installation of engines and bindings (components); Life cycle management of components (start/stop controls); Deployment of component artifacts to engines and bindings; Monitoring and control; Unfortunately, most of the methods of JBI s MBean interfaces are used for management and there are only a few that can provide functionality of monitoring. The following methods have been identified as valuable for monitoring (description of the methods has been omitted due to their self-explanatory names): AdminServiceMBean.getBindingComponents(); AdminServiceMBean.getEngineComponents(); DeploymentServiceMBean.getDeployedServiceUnitList(componentName); DeploymentServiceMBean.getDeployedServiceAssemblies(); DeploymentServiceMBean.getServiceAssemblyDescriptor(saName); DeploymentServiceMBean.getDeployedServiceAssembliesForComponent(component); LifeCycleMBean.getCurrentState(); InstallerMBean.isInstalled(); These methods can provide the basic information about the structure of the bus: what BC, SE, SU and SA are installed/deployed and which of them are currently running. Such information is valuable, however there are other important statistics that have to be exposed in a complete monitoring solution. The conclusion is that JSR-208 does not define a sufficient monitoring infrastructure. The specification basis has to be enriched in order to comply with the assumptions from the theoretical discourse included in the previous chapter (3.5.2).

51 3.3. JBI monitoring status quo ServiceMix monitoring The monitoring capabilities of ServiceMix exposed by JMX do not really go beyond those defined in JBI. In ServiceMix versions used for the purpose of this thesis, i.e and 3.2.1, only the following monitoring parameters were available 1 : getinboundexchangecount; getinboundexchangerate; getoutboundexchangecount; getoutboundexchangerate; These parameters can be obtained in the context of a particular endpoint or component giving the information about MessageExchanges counts and rates. ServiceMix extension to JSR MBeans does not introduce any significant functionalities. However, the Bus is equipped with other interesting mechanisms meaningful for monitoring. CorrelationId The first of ServiceMix s monitoring mechanisms is CorrelationId. In order to avoid confusion it has to be stressed that it is used in two meanings: as the name of the mechanism which processes - correlates - MessageExchanges that flow through the Bus and as the name of MessageExchange s property, which preserves the actual id of a particular correlation. Each JBI MessageExchange is uniquely identified through its own id. ME can appear in the bus as a reaction to some external event (e.g. invocation of some web service exposed by a http binding component) or as a reaction to processing some other MessageExchange (internal event). MessageExchanges created while processing internal events can be correlated with a parent MessageExchange. The process of correlation can be performed recursively up to the point when MessageExchange has been created for an external event response. It allows to correlate each MessageExchange with a root MessageExchange triggered by an external event. ServiceMix s CorrelationId mechanism uses the above-mentioned correlation process and marks each Message- Exchange with a property of CorrelationId. The value of CorrelationId property is the id of the related root MessageExchange. The mechanism of correlating MessageExchange into what is known as the JBI transaction is presented in great detail in Section 4.3. Please refer to it for a complete understanding. Event interception ServiceMix introduces a very flexible mechanism for intercepting various events occurring in the bus. The mechanism relies on a definition of event listeners provided in the package org.apache.servicemix.jbi.event. The implementation of each listener is notified about specific events. The following listeners interfaces are available: 2 : 1 Information about parameters available in earlier versions of ServiceMix is available here: 2

52 3.3. JBI monitoring status quo 40 ComponentListener notifies about component installation, deinstallation, starting, stopping, shutting down. DeploymentListener notifies about file modifications (ServiceMix specific). EndpointListener notifies about endpoint registration and unregistration. ExchangeListener notifies about sent and accepted MessageExchanges. ServiceAssemblyListener notifies about ServiceAssembly deployment, undeployment, starting, stopping, shutting down. ServiceUnitListener notifies about ServiceUnit deployment, undeployment, starting, stopping, shutting down. In order to be able to use the described mechanism, an implementation of one of the above interfaces has to be provided and registered in ServiceMix using the following method: JbiContainer.addListener(EventListener listener). Unfortunately, such an event interception is not compliant with JBI specification. Additionally, there is no similar mechanism in OpenESB listeners are specific to ServiceMix. There are few useful components which use the mechanism of event listeners. They are presented in the following paragraphs. Component and Endpoint visualization ServiceMix has a built-in functionality of bus topology visualization. Notifications of ComponentListener, DeploymentListener and EndpointListener allow to track changes in the topology. Thanks to that, maintaining of the installed, started components and the endpoints deployed to them is possible. The knowledge about topology is a good basis for further analysis of MessageExchange flows occurring between the endpoints in the topology. The topology is provided in a file with the components and endpoints graph described in the DOT language [20]. DOT is a simple plain-text description language which is understandable to both humans and computers. DOT files created by ServiceMix have to be further visualized by one of the many tools capable of DOT handling 3. This step is performed outside of ServiceMix by the users themselves. MessageExchange flow visualization Another ServiceMix s visualization tool which is complementary with the topology provides information about MessageExchange flows. Implementation of ExchangeListener is used, which allows to intercept events of sending and accepting Message- Exchanges in DeliveryChannels of all components running in the bus. MessageExchange interception is enhanced with CorrelationId feature allowing for creating trees of message exchange flows. As in the case of the topology description, the flow is also provided in DOT language files and the final step of visualization is the user s responsibility. Tracing the paths of MessageExchanges is a meaningful capability that may be used for debugging business logic deployed to the bus. 3 Graphviz - ; AT&T Grappa - john/grappa/ ; C++ BGL /libs/graph/doc/index.html

53 3.3. JBI monitoring status quo 41 ServiceMix Audit Audit is one of the ServiceMix s modules. It focuses on MessageExchanges, which to some extent is analogical to the flow visualization. However, Audit is not concerned with correlation, maintaining instead a historical record of MessageExchanges. Recorded MessageExchanges are also intercepted in ExchangeListener (see: flow visualization). The module provides an abstraction of the persistence layer allowing for flexible change of the persistence realization. At the moment, two realizations are available: FileAuditor storing exchanges in regular files; JdbcAuditor storing exchanges in any JDBC compatible resources. For the purpose of performance only one table with a message id and serialized content is used; The persistence layer is additionally enhanced with a full-text search engine - Lucene 4. It appropriately indexes recorded message exchanges allowing for further content-based search. Just as in the case of the flow visualization, it can be used for debugging, but also as a means of statistics and some business oriented analysis (cf. Business Intelligence 3.4.2). Spagic Spagic describes itself as a SOA Enterprise Integration Platform. On the homepage, the following description is provided: Spagic is a solution composed by a set of visual tools and back-end applications oriented towards design, realisation, deploy and monitoring of ESB infrastructures adherent to the SOA paradigm. The solution s monitoring aspect is interesting for the purpose of this thesis, but the remaining details are out of its scope [21]. Spagic monitoring refers to an abstraction of a higher level process (analogical to business process) which is realized by means of ESB, BPM and also JBI. Spagic uses ServiceMix for the purpose of Java Business Integration. The higher level process part, realised by JBI (ServiceMix), can be actively monitored. It comes to following the business transaction through its endpoints and reporting success or possible failures. Monitoring uses two ServiceMix s mechanisms: ExchangeListener used to intercept MessageExchanges; CorrelationID for business transaction constructing; Spagic s broad functionalities and its monitoring aspect prove the usefulness of the monitoring mechanisms of ServiceMix OpenESB monitoring OpenESB monitoring realization approach differs from the ServiceMix approach. OpenESB lacks valuable mechanism introduced by ServiceMix: CorrelationId and Event Listeners, but it provides a well-designed Sierra framework dedicated to runtime monitoring. The framework is described on OpenESB Wiki pages [22], which, along with our experience and the results of experiments, provides the main reference for this section. Sierra monitoring focuses mainly on performance characteristics which span across system and application monitoring types defined in Section 3.2. This means that every JBI role will benefit from the monitoring parameters provided by Sierra (cf. 3.2). The performance evaluation is applied very systematically. Firstly, the framework finds the primary JBI entities, in which the context 4 Apache Lucene -

54 3.3. JBI monitoring status quo 42 performance can be evaluated: Normalised Message Router, Delivery Channel, Endpoint (their detailed description can be found in JSR-208 [2]); then it identifies the operations and classifies them as common or entity-specific: Common Active exchanges, Completed exchanges, Faulted exchanges, Error exchanges, Exchange duration, Throughput, Historical distribution of MEP by type; NMR Active Components, Active Endpoints; Delivery Channel Timing path information; Each operation is analyzed and the monitoring parameters are derived from it. It allows to provide a set of statistics at each level of the JBI abstraction. The Sierra framework monitors the following statistics for default: Framework StartTime, StartupTime, UpTime, Component count, Endpoint count, ServiceAssembly count; NMR Time spent in each component, Time spent in DeliveryChannel, Time spent in NMR Per Component/Endpoint Queried by Component/Endpoint; Component StartTime, StartupTime, UpTime, Total # of activated endpoints, Total sent/received requests (i.e. in msgs), Total sent/received replies (i.e. out msgs), Total sent/received faults, Total completed exchanges (consumed/provided), Total active exchanges (consumed/provided), Total error exchanges (consumed/provided) Response Time, Status Time, Active MessageId s, Waiting MessageId s; Service Assembly Uptime, Last startup timestamp, Startup time, Stop time, Shutdown time (Startup, Stop and Shutdown are also accessible per constituent Service Unit); Endpoints ActivationTime, UpTime, Total sent/received requests (i.e. in msgs), Total sent/received replies (i.e. out msgs), Total sent/received faults, Total completed exchanges (consumed/provided), Total active exchanges (consumed/provided), Total error exchanges (consumed/provided), Response Time, Status Time, Active MessageId s, Waiting MessageId s; Additional Service Assembly and Framework levels were introduced. The Framework is equivalent to OpenESB cluster abstraction. At the Framework level, the parameters can be queried in the context of the whole cluster or specific OpenESB instance. It results in the monitoring solution being in line with OpenESB clustering described in Section Apart from the listed monitoring parameters, the Sierra monitoring has the functionality of recording message exchanges going through the Bus. This mechanism is not yet completely implemented, but final functionalities will allow to extract interesting message exchange data, including the timing path that will be described later. The message exchange history can be evaluated in two ways: saving to a log file for later processing; runtime results aggregation on the basis of dynamic or static conditions;

55 3.3. JBI monitoring status quo 43 Timing path analysis and capability of dynamic aggregation will make historical records a great tool for solving performance issues. The Sierra monitoring has powerful functionalities which provide a complete view of the whole JBI bus. Presentation layer Presentation layer is not provided in the discussed monitoring mechanism. Instead, proper JMX interfaces are exposed. They are more flexible than the integrated presentation capability because JMX MBeans can be easily connected to and used for realization of any kind (stand-alone or webbased) user interface. Timing Path Sierra provides great details of performance analysis. MessageExchange communication has well-defined internals which are important compounds of general performance. To find the reason of poor performance of some MessageExchange, Sierra analyzes timing from both the consumer s and provider s point of view. Both approaches allow to find various points in JBI send() accept() path, where important measurements can be done. Four main areas from consumer s point of view have been identified. They are depicted in the following diagram 5 : Consumer Provider send() ---> > accept() 3 +< send() 4 accept() <--+ 1) Time spent in sender (first time is costly, others very cheap) 2) Waiting in queue for accept() (provider specific delay) 3) Processing time (provider) 4) Waiting in queue for accept() (consumer specific delay) Sum of above is total time (from consumer pov) These are the core measurement areas of Message Exchange Patterns (MEP) defined in JSR- 208, however identified more of them can be identified when performing analysis of each MEP 5 The diagram is borrowed from OpenESB wiki:

56 3.3. JBI monitoring status quo 44 from the provider s and consumer s point of view. These identified timings are available for every endpoint registered in the bus. Notifications On the Sierra wiki page, the mechanism is described in the following way [22]: The events that are the target of this mechanism are any events which change the effective state of the JBI runtime. Notification clients should be able to register with the JBI Runtime and monitor events which occur in the JBI runtime and affect the states of Components (Binding Components and Service Engines), Shared Libraries, Service Units, Service Assemblies, and the JBI runtime itself. Notifications will be emitted whenever one of these entities undergoes an operation which changes its state or adds or removes it from the system. The following notifications are available: Service Assembly Deployed, Started, Stopped, Shut down, Undeployed; Service Unit Deployed, Started, Stopped, Shut down, Undeployed; JBI Runtime Ready, Started, Stopped; Component Installed, Started, Stopped, Shut down, Uninstalled; Shared Library Installed, Uninstalled; This mechanism is realized by JMX Notifications. The specification provides the following motivation for introducing notifications: The management interface of an MBean allows its agent to perform control and configuration operations on the managed resources. However, such interfaces provide only part of the functionality necessary to manage complex, distributed systems. Most often, management applications need to react to a state change or to a specific condition when it occurs in an underlying resource.. Notification model is very generic thanks to it being created on Java Event Model [76] basis. It provides few abstractions: Notification signals any type of management event. Realized by the interface of the same name. Broadcaster allows Liteners to register their interest for specific Notifications. It is realized by NotificationBroadcaster interface, which must be implemented by MBeans emitting notifications. Listener entity receiving Notifications from Broadcaster. It is realized by NotificationListener, which is implemented by object requesting Notification reception. JMX standardization makes Notifications a compatible way of exposing inter-jbi events, which allow to actively inform about events occurring in the bus. The described events focus on the bus topology and therefore the mechanism can be compared with the ServiceMix s mechanism Component and Endpoint Visualization. Sierra solution has the advantage of real-time updating, but it lacks the ServiceMix s visualization compatible format (the DOT Language).

57 3.3. JBI monitoring status quo 45 Implementation JBI Conformance Sierra monitoring is realized by inserting a monitoring code into existing OpenESB implementation. The functionalities of monitoring parameters extraction and exposition have been added to the following classes: com.sun.jbi.nmr.messageservice (MBeans and some statistics); com.sun.jbi.nmr.deliverychannel (Some statistics); com.sun.jbi.nmr.messageexchangeproxy (Some statistics); com.sun.jbi.nmr.messageexchangeimpl (Some statistics); com.sun.jbi.framework.componentstatistics (MBeans and some statistics); com.sun.jbi.nmr.registeredendpoint (Add: Endpoint statistics); com.sun.jbi.nmr.stats.nmrstats (New: the history information); The implementation is tightly coupled with the bus code and it is OpenESB-specific. It cannot be reused in other JBI-compliant bus implementation Clustering In today s enterprise, system distribution is becoming more and more crucial. Business cannot operate without constant inter-department communication and organizations are starting to perform aggressive expansions that often span over few countries. Enterprise system of such an expanding organization induces requirements which can be perfectly addressed by ESB and by Java Business Integration in particular. JBI specification describes a wide set of meaningful functionalities of this container of containers, however the subject of distribution is not among them. In Section 14 of JSR-208, the following information is provided: JBI, as currently defined, is not distributed. Distributed implementations may require support in the JBI specification to address issues that arise only in distributed systems. Clustering and distribution architecture is not defined by specification, probably due to pioneering in enterprise service bus standardization. However, it is suggested that the JBI implementation should address clustering support on their own. Probably experience gathered during such clustering evaluation will be the basis of JSR-208 refinement and complementing. The buses described before, ServiceMix and OpenESB, address the issue of clustering. The following part of this section is devoted to the analysis of their distribution mechanisms. ServiceMix ServiceMix provides the following cluster definition two or more ACTIVE ServiceMix container instances that are networked together. It is cited from the ServiceMix clustering description available on the homepage [23]. Along with the experience gathered during the experiments, the clustering description is the main reference for this section. The container supports the following distribution patterns:

58 3.3. JBI monitoring status quo 46 high availability(ha); containers clustering; The ServiceMix cluster is realized by means of an inter-container connectivity which allows to form a network of containers. Containers can be dynamically added and removed from the network. Connectivity has its internal intelligence providing remote component awareness each container is aware of its peer container s components. It is realized by listening for remote component registration/deregistration events. Maintaining awareness is a requirement for distributed message routing realized in the cluster scope (depicted in Figure 3.1). Figure 3.1: ServiceMix clustering 6 Distributed message routing is equipped with the capability of load balancing processed messages. It is the reason for significant benefits. The mechanism can balance requests in two ways, which enhances the distribution patterns listed before: high availability Installation of the same component on multiple containers to provide increased capacity and additional HA capabilities. If one container fails the same component in another container can still service the request. [23] containers clustering Partition workload among container instances whereby different containers can handle different tasks and flows can span multiple containers. [23] 6 Figure has been downloaded from ServiceMix s clustering description [23].

59 3.3. JBI monitoring status quo 47 Of course, in the case of load balancing it is hard to avoid a single point of failure, unless using hardware load balancer, but it is out of the scope of the bus capabilities. The ServiceMix Architecture is based on a SEDA paradigm - Stage Event-Driven Architecture [100]. Instead of introducing a complex, in terms of concurrency, inter-threading communication, the functionalities are divided into parts, assembled in separate modules and inter-connected by means of some Message Oriented Middleware. It removes the concurrency complexity and assures embedding and separation of functionalities. In order to realize the SEDA paradigm, ServiceMix needed highly functional messaging oriented middleware. ServiceMix has chosen the Apache s ActiveMQ, which names itself most popular and powerful open source Message Broker and Enterprise Integration Patterns provider. ActiveMQ has a very wide set of capabilities and it has finally been incorporated into solution of the JBI monitoring - please refer to Communication middleware section in Chapter 5. OpenESB OpenESB does not directly provide clustering capabilities. It uses the functionality of GlassFish Application Server [24], which implies that OpenESB cannot be clustered when running in the stand-alone mode. However, the bus is tightly bundled with the application server and there is no rationale (apart from lower computing and memory consumption) that would favor using the stand-alone mode. Glassfish clustering is delineated in Figure 3.2. GlassFish cluster encompasses one main Domain Administration Server and any number of agent nodes which are managed by DAS through the JMX API. Instances of agent nodes can span across multiple hosts (with the possibility of more than one instance per host), therefore allowing for any kind of distribution. The following GlassFish cluster definition can be found on OpenESB Wiki page [25]: A cluster is a logical entity encompassing zero or more server instances. Simply speaking, a cluster is a collection of application server instances that can distribute a workload throughout the clustered application instances for optimal performance. These server instances share the same set of applications, resources, and configuration information. A clustered server instance belongs to exactly one cluster, and inherits everything from that parent cluster. Instances in a cluster can extend over any number of computers. The most important information is that inter-cluster instances share the same set of resources and applications. It is the implication of specific clustering type that is described on the same Wiki page: Sun Java System Application Server supports clustering of homogeneous application server instances (containing the same set of JBI components, applications, and configuration information) installed on a single host or on multiple hosts. Applications that run on each application server instance are independent, but are also manageable by an administration infrastructure, either through web browser based (DAS) or command line clients.

60 3.3. JBI monitoring status quo 48 Figure 3.2: OpenESB clustering The conclusion is that the so called OpenESB clustering is mainly restricted to central cluster management, performed by DAS. Components, service assembles and bus configuration can be easily controlled from one point but it implies that each clustered instance contains the same set of those elements - it is depicted in Figure 3.2. Additionally, the application logic deployed - by means of DAS - to agent nodes runs in an entirely independent way. It is a meaningful shortcoming of the clustering capabilities: load balancing GlassFish provides a built-in load balancer, but in each case it has to be configured manually by JBI developers. high availability Mirroring of instances fits for the purpose of HA, but restricted OpenESB- GlassFish clustering integration forces to implement HA at the JBI component level. Such architecture has already been designed for the BPEL component [26]. workload partition the situation is similar to the one from HA - restricted clustering implies implementation at the component level. The above-mentioned BPEL component clustering design covers workload partition as well. [26]. ESB distribution context Along with the definition of ESB in Section the Federation of ESBs idea was introduced. As it has been stated, the ESB trends incline towards federation bus pattern, which is built of interconnected Enterprise Service Buses. Compliance with this pattern is determined by clustering capabilities of ESBs, which are federated. ServiceMix and OpenESB, which challenge the clustering functionalities, can be questioned about the compliance with the federation idea. Question evaluation is performed on the basis of buses clustering comparison depicted in Table 3.1.

61 3.3. JBI monitoring status quo 49 Table 3.1: Clustering comparison Functionality OpenESB ServiceMix Central Management Yes No Load balancing Manual Yes High availability Not on bus level Yes Workload partition Not on bus level Yes The federation is interpreted as a seamless and transparent ESB inter-operation. In order to achieve this, a workload partition is inevitable. Additionally, large-scale federations cannot evade the problems of scalability and reliability implying the need of load balancing and high availability. Therefore, it can be concluded that only ServiceMix is capable of assembling ESB Federation. Additional ServiceMix supplement of central management would allow for a complete enterprisewide federation enhancement Status quo implications Even though the described JBI containers are implementations of the same specifications, monitoring mechanisms of those two buses vary. Comparison of ServiceMix and OpenESB monitoring essentials is outlined in Table 3.2. Table 3.2: JBI implementations comparison Functionality OpenESB ServiceMix JMX interface JBI compliant MBeans; Broad JBI compliant MBeans; A few management and monitoring simple counters per endpoint; functionality of Sierra; JBI flow tracing N/A CorrelationId Event interception Functionalities inlined in the Framework of flexible listeners source code Visualization N/A Topology description; Message flow description; Historical MessageExchange Among Sierra functionalities ServiceMix Audit module stats Performance analysis In-depth timing path Basic MessageExchange flow rates Event-driven external interface JMX Notifications N/A Clustering Limited bus mirroring pattern Fully distributed bus, compliant with ESB paradigms, capable of both high availability and functionalities distribution Monitoring JBI Conformance Not Compliant Not Compliant The JBI specification is not complete in terms of monitoring capabilities definition. The lack

62 3.4. Application monitoring theory 50 of completeness implied introducing a proprietary mechanism in each bus implementing JBI. It resulted in solutions with a very wide and in-depth monitoring scope, but not compatible with each other. Therefore, a general JBI monitoring solution is an abstraction of which pursuit is not an easy task. 3.4 Application monitoring theory Every IT system, design and implementation has to be preceded by a deep theoretical research which gives a foundation for later Software Engineering. This section introduces a theoretical basis in the form of the Application Response Measurement specification and the Business Activity Monitoring technology. The theory is discussed, evaluated and confronted with practical experience Application Response Measurement ARM stands for Application Response Measurement. It is a standard proposed by OpenGroup 7 consortium [27] which allows to monitor the availability and performance of complex, looselycoupled, service oriented applications. The standard defines bindings for both C and Java language. Thanks to that, any application written in these languages can be instrumented and further monitored. Hence, the JBI implementations that are in the scope of this thesis are written in Java, the ARM analysis has been restricted to the newest ARM Java Bindings - version 4.1 [78]. The clue of ARM is the performance evaluation from the point of view of the application itself. Thanks to that, the ARM user has a precise knowledge about timing of each business transaction that takes places in the application. Additionally, the user knows the exact time breakdown of the transaction, i.e. the time spent in each transaction part. It allows to detect bottlenecks very easily. Performance is not the only aspect of ARM. It allows to answer many important questions [78]: Are transactions succeeding? In case of transaction failure, what is the cause? What is the response time experienced by the end-user? Which sub-transactions of the user transaction takes too long to execute? Where are the bottlenecks? How many of which transactions are being used? How can the application and environment be tuned to be more robust and to perform better? The ability to answer the above questions is valuable for every kind of application. Further analysis in this section shows which of the mentioned aspects are usable for monitoring JBI. 7 According to Wikipedia [101] Version 1 of ARM was developed jointly by Tivoli Software and Hewlett Packard in Version 2 was developed by an industry partnership (the ARM Working Group) and became available in December 1997 as an open standard approved by the Open Group

63 3.4. Application monitoring theory 51 ARM Architecture In order to monitor a given application using ARM, a few steps have to be performed. The most important is to instrument the application that is going to be monitored. Instrumentation is a process of source code modification. Invocations of ARM bindings (well-defined interfaces) are added to parts of the code that needs to be monitored. For example, before invocation of some business method X, the notification of ArmTransation.start(X) is inserted. Analogically, after the end of such invocation, ArmTransation.stop(X) is inserted. Additionally, during the transaction, update() notifications can be sent which inform about status change. Instrumentation operations are delineated in Figure 3.3. Figure 3.3: Application invokes methods on ArmTransaction 8 An application instrumented with ARM (informally ARMed application) has to be equipped with Management Agent, which performs analysis and reporting. It is depicted in Figure 3.4. Figure 3.4: Connection between ARM Interface and Agent 8 Every ARM implementation provides a library for the purpose of linking to the application being monitored. The library implements the ARM Interface the invocations of which were put into the code during the instrumentation. The function of the library is to forward monitoring information from the application to the Monitoring Agent. ARM functionalities This subsection discusses the details of ARM functionalities that allow to answer the important questions stated in the previous subsection. As it has been mentioned before 3.4.1, the ARM key 8 Figure was extracted from the evaluated ARM specification [78]

64 3.4. Application monitoring theory 52 functionality is to monitor all details of business transaction. In real scenarios, business transactions are often distributed across many systems. ARM is able to monitor such type of transactions by means of an inter-transaction correlation mechanism. For the purpose of business transaction monitoring analysis, a sample distributed system is depicted in Figure 3.5. Business transaction in a distributed system is divided into smaller-transactions which are performed individually in components of the system. ARM allows to correlate transactions in different components, which belong to the same business transaction. At the level of ARMed application, the relation between parent and child small-transactions is remembered and reported to the Monitoring Agent. The process is delineated in Figure 3.6. Figure 3.5: Sample distributed system 9 Figure 3.6: Transaction and parent-child correlators 9

65 3.4. Application monitoring theory 53 When all information is reported to the Monitoring Agent, parent-child relations can be extended and the entire flow of business transaction together with exact timing statistics can be calculated. The described analysis is targeted at synchronous transactions. Version 4.1 of the ARM specification introduces new functionalities of asynchronous transactions monitoring. It provides capabilities of monitoring transactions based on some asynchronous mechanism. It is very common that Message Oriented Middleware (MOM) [28] is used for this purpose. ARM supports few patterns of transaction that are based on some abstraction of queueing framework (MOM). They are presented in Figure 3.7. Figure 3.7: Message Oriented Middleware ARM patterns 10 JSR-208 introduces message patterns that are asynchronous. Therefore, the functionality of monitoring them is important for the purpose of this thesis. 9 Figure was extracted from the evaluated ARM specification [78] 10 Figure was extracted from the evaluated ARM specification [78]

66 3.4. Application monitoring theory 54 OpenARM This subsection evaluates available ARM implementations. Stable mature implementation is crucial for successful usage in JBI Monitoring. Few commercial ARM implementation have been found: Exinda Networks [29]; Oracle Siebel [30]; IBM Tivoli s WebSphere [31]; The evaluation of these implementations was impossible because using commercial solutions was out of the scope of the thesis assumptions. Instead, focus was put on Open Source [32] implementations. Unfortunately, only one such implementation was available: OpenARM [33]. OpenARM implements the Java part of ARM bindings and complies with version 4.0 of the specification [77]. OpenARM s implementation was initiated by Mark Masterson, who discovered an absolute lack of reference and free implementations of ARM specification. He describes his motivations in the following way: ARM itself is a pure specification. The Open Group does not provide a reference implementation they don t provide any kind of implementation at all. Traditionally, they have left that for the various commercial vendors who are members of the working group. And there are a number of commercial, proprietary implementations of ARM, notably from Tivoli and HP. These implementations are fairly expensive, however. [34]. According to Masterson, OpenGroup makes the so called open specification but the lack of reference implementation makes ARM less valuable. Masterson was forced to implement OpenARM relying only on the specification (and on his own previous ARM experience, of course). Unfortunately, the quality of OpenARM is not comparable with commercial solutions. In the evaluation, a sample code from ARM specification was used, but in order to compile it against OpenARM a modification has to be performed 11. Another shortcoming of OpenARM is the lack of compliance with version 4.1 of specification. OpenARM is relatively obsolete (last version was released in 2005) and does not have functionalities for asynchronous transactions. It is a real shortcoming for the purpose of using it in the JBI monitoring. ARM usefulness The following facts gathered about ARM have impact on possible ARM usefulness: ARM provides the specification of Java interfaces but does not provide any reference implementation. The only open source implementation - OpenARM lacks functionalities essential for JBI. The objective of full ARM compliance can be achieved only by first implementing it (it is, of course, beyond the scope of this thesis). Successful implementation of ARM does not assure flexibility of exchanging it to some other implementation: 11 Evaluation code is presented in Listing A.1

67 3.4. Application monitoring theory 55 The lack of open source implementations of ARM 4.1. JBI s messaging mechanism is a very specific case of asynchronous communications and there are aspects which have not been taken into account in the ARM design. Therefore, there is a risk of incompatibility between a self-implemented ARM and other third-party implementations. The facts imply a high complexity of ARM usage. The main obstacle is the lack of mature implementation of ARM 4.1. In case of OpenARM being not sufficient, the most reasonable solution seems to be: not implementing the ARM directly; not using the complex ARM interface definitions; deriving ARM ideas which introduce total new quality to application measurement; taking into account questions mentioned in Section 3.4.1; Business Activity Monitoring Business Activity Monitoring is another effort of standardizing or rather defining monitoring in context of business and enterprise systems. On 1 April 2002, Gartner, Inc. [35] published an article entitled Business Activity Monitoring: Calm Before the Storm [93], which introduced the idea of BAM. The abbreviation stands for Business Activity Monitoring but it is also often referred to as Business Activity Management. Gartner defines BAM as an extension to regular real-time monitoring, which provides additional business oriented performance indicators. BAM is targeted at large, heterogeneous enterprise systems where information has to be gathered at many levels, both internally and externally. Only such approach gives a complete view on the business operations and processes, which BAM allows to monitor and manage. Even in 2002 Gartner forecasted a rapid development of BAM: By 2004, BAM will grow to become the major force driving application integration s deployment and benefits. Vendors will depend on BAM for survival, and enterprises will tout BAM for its competitive advantage. BAM is the next big thing that application integration vendors want and need. Forecasts turned out to be true today quick google search returns many commercial solutions which allow to integrate BAM into existing enterprise architecture. Some of them are listed below: Tibco Business Activity Monitoring Software 12 ; Progress BAM 13 ; Systar BusinessBridge ServiceVision 14 ; WebMethods for Process, Infrastructure, B2B and SAP 15 ; Adobe LiveCycle Business Activity Monitoring ES 16 ; 12 activity monitoring/default.jsp

68 3.4. Application monitoring theory 56 Business Intelligence BAM is based on an older approach Business Intelligence, which first appeared in October 1958 in an IBM Journal article [92]. An often cited BI definition was proposed in article A Brief History of Decision Support Systems [96]: BI describes a set of concepts and methods to improve business decision making by using fact-based support systems. BI is sometimes used interchangeably with briefing books, report and query tools and executive information systems. Business Intelligence systems are data-driven DSS. Business Intelligence allows to keep under control the former, current and future state of a business system. The state is constructed mainly on the basis of warehouse data collected in the past, but also occasionally using operational data. BI allows to notice business trends, customer habits and market changes, thanks to which upcoming occasions for business enhancement are never missed. Business Process Management At the beginning, the BAM realization was identified as closely related to Business Process Management. Business Process is a set of correlated activities and tasks conducted both by people and IT infrastructure (Deep business process analysis is out of the scope of this thesis. It can be found in literature [81, 97]). It turns out that in today s aggressive business world an organization s business processes and their effective management is the key to success. BPM is a process of changing business approach of an organization to adapt it to the expectations of customers and partners. BPM provides constant optimization of an organization s processes (a process optimization process [104]) which allows to most effectively strive for its goals. The importance of BPM triggered the creation of Business Process Management Initiative [36], which tries to standardize most common business processes identified in organizations. Such standardization and good practices of management lead to enhancing B2B, ebusiness and enterprise in general. For the purpose of BPMI such standards as: Business Process Modelling Language (BPML), Business Process Modelling Notation (BPMN), Business Motivation Model (BMM), Semantics of Business Vocabulary and Business Rules (SBVR) have been introduced. However, they are out of the scope of this thesis. The BPM s optimization is realized by means of business process management life-cycle depicted in Figure 3.8. Figure 3.8: Business process management life-cycle Figure was borrowed from the Wikipedia:

69 3.4. Application monitoring theory 57 Monitoring is a very important phase of BPM cycle which triggers the real optimization process realized as re-orchestration of business processes. BAM used the BPM approach to orchestrate enterprise with event gathering mechanism, which resulted in the shortcoming of forcing the enterprise to invest in BPM, before integrating BAM in the organization s infrastructure. Fortunately, BAM evolved to use Complex Event Processing (CEP), which bring event detection and correlation to a higher quality level. Thanks to that, BAM dependency on BPM has been reduced. Complex Event Processing Complex Event Processing (CEP) is a relatively new solution capable of enhancing such technologies as: BAM, BPM, EAI. According to CEP home site [37], the first bigger symposium on CEP subject took place in March 2006, but it has greatly evolved until now. CEP is targeted at enterprise applications and allows them to comprehend all of the information flowing through their IT infrastructure. CEP focuses on events that appear on various tiers of enterprise applications and allows to: intercept all events; filter out events that are out of concern and keep those which are valuable; detect complex multi-event patterns; enrich events with information of potential impact on higher management objectives; provide events correlation and abstraction; provide event hierarchies; provide event relationships such as causality, membership, timing and event-driven processes; report (act on) events in real time; CEP goes deep into the enterprise infrastructure and allows to handle events produced by such technologies as RFID [37, 105]. An example of using the CEP technology taken from the Wikipedia [105]: Car has several sensors - one that measures tire pressure, one that measures speed, and one that detects if someone sits on a seat or leaves a seat. The car is moving and the pressure of one of the tires moves from 45 PSI to 41 PSI over 15 minutes. As the pressure in the tire is reducing, a series of events containing the tire pressure is generated. In addition, a series of events containing the speed of the car is generated. The car s Event Processor may detect a situation whereby a loss of tire pressure over a relatively long period of time results in the creation of the lossoftirepresure event. This new event may trigger a reaction process to note the pressure loss into the car s maintenance log, and alert the driver via the car s portal that the tire pressure has reduced. This example delineates the basic functionalities realized by CEP: event interception, filtering, contextual interpretation, correlation and, finally, reporting.

70 3.4. Application monitoring theory 58 BAM The fact that BI heavily depends on static data warehouses makes it inflexible to dynamic changes. BAM changes the approach and turns to real-time, event based data acquiring which can be further aggregated, analysed and presented also in real-time [91]. BAM focuses on monitoring organisation s business activities both internal and external related to customers and partners. Such activity could be a business process (orchestrated by BPM) or more general organisation s process which spans across multiple tiers and different applications. It also allows to monitor transactional resources such as Web services and message queues and thanks to Complex Event Processing, correlate heterogeneous events that take place in context of those resources. BAM targets to be useful for operation managers and upper management giving real-time status of all business activities. The basic functional steps of Business Activity Monitoring can be outlined in the following way: gather the data from intercepted events; correlate events according to general patterns; aggregate the data according to general assumptions and user preferences; analyze the data; present the data in real time; BAM has far more potential than the above-listed functionalities. It is a natural effect of enhancing more static Business Intelligence to real-time monitoring. BI forces human interaction for purpose of manual analysis of the collected and further processed data. It is required to conclude and perform steps towards business activity modifications. BAM goes one step further and allows to define patterns and conditions, which are applied during the data processing. When the condition is met, an alert is generated triggering notification about specific situation in the system. Additionally, an action can be bound to a condition which allows for a straight-forward reaction to some diagnosed problems. An exemplary scenario of BAM integration taken from the Wikipedia [103]: Mobile telecommunications company is interested in detecting a situation whereby new customers are not set up promptly and correctly on their network and within the various CRM and Billing solutions. Low level technical events such as messages passing from one application to another over a middleware system, or transactions detected within a database logfile, are processed by a CEP engine. All event relating to an individual customer are correlated in order to detect an anomalous situation whereby an individual customer has not been promptly or correctly provisioned, or set up. An alert can be generated to notify technical operations or to notify business operations, and the failed provisioning step may be restarted automatically. The above scenario gives an idea of the power brought by Business Activity Monitoring. The described sample and strictly tailored functionality of detecting proper customers set up in CRMs and Billing solutions is not a feature that can be easily deployed. BAM extensions to BI imply higher deployment difficulty. The advantage of strictly tailoring forces every organization willing to integrate BAM to resolve a specific case study which differs from one organization to another. In order to overcome this deployment issue, BAM experts identified some common templates

71 3.5. JBI Monitoring Challenge 59 that can be applied to well known scenarios: Banking, Manufacturing, Stoke Brokering [103]. Nevertheless, the deployment complexity causes BAM adoption cost to increase. 3.5 JBI Monitoring Challenge JBI Monitoring section (3.1) provided background for the thesis subject discussing various monitoring aspects. The background going through the abstraction level, status quo and monitoring theory imposes quite strict positioning of possible monitoring solutions that can be applied to the Java Business Integration. This chapter discusses possible solution positioning and provides conclusions for a later architectural design JBI context influence In order to discuss how deeply the background of JSR-208 and its evaluated implementations influence the architectural possibilities, a JBI contextual analysis of monitoring abstraction is performed. As identified in Section 3.2, there are seven characteristics of monitoring system classification. All of them can be applied straightforwardly to JBI apart from monitoring subject. In order to apply this simple characteristic to JBI, the two types of monitoring approaches have to be evaluated in the context of JBI bus being the monitoring subject. Monitoring subject characteristic The JBI bus is a container for application logic deployed in the service units (introduced in Section (2.5)). Such relation can be interpreted as JBI bus being a system and service unit logic being applications. With such interpretations, the characteristic can be extrapolated to JBI, however monitoring parameters will be totally different. Examples of monitoring parameters in the system vs. application characteristic: system == JBI container container uptime, total sent/received requests, total sent/received replies, total completed exchanges (consumed/provided), total error exchanges (consumed/provided), total number of activated endpoints, NMR performance; application == JBI business logic endpoint uptime, first endpoint invocation TIMES- TAMP, last endpoint invocation TIMESTAMP, endpoint response time, business logic execution performance, business faults statistics; The proposed characteristic is not very formal, but has the advantage of flexible extrapolations to a wide range of subjects, including JBI. In the context of computer machine, the information gathered at the application level gives the idea how the computer is usable to the final user. Application providers can have feedback about defects and shortcomings of their applications. In case of a serious error or performance degradation at the application level, the administrator often has to consult system monitoring parameters to check for possible system failures. So for computer machines two roles can be identified: system administrator and application provider. In Section 3.5 of JBI specification we can find definitions of roles specific to JBI: Engine Developer providers of Service Engines;

72 3.5. JBI Monitoring Challenge 60 Binding Developer providers of Binding Components; JBI Environment Provider creator of JBI bus implementation; J2EE Platform Provider administrator of a J2EE platform which hosts JBI in application server environment; JBI Application Developer Service Assembly providers which create business logic deployed to SE and external endpoint descriptors deployed to BC; In the proposed characteristic, the system parameters would be in the first place valuable to the JBI Environment Provider. Their can be also important for the J2EE Provider, who provides environment for JBI and probably wants to monitor and manage it. Developers of JBI Components - SE and BC - will benefit from both monitoring types. The system will provide the status of their components with bus interactions and application will inform about the results of service units logic execution. The last role of the Application Developer will obviously benefit from the application monitoring type only. The fact that JBI roles are in line with the defined monitoring characteristic proves that it fits in JBI. Monitoring abstraction evaluation The JBI extrapolation of the monitoring subject characteristic allows to use a defined abstraction for means of comparing: JBI specification, ServiceMix and OpenESB implementations. The following characteristics of monitoring abstraction have been identified: 1. mechanism of information acquiring; 2. model of monitoring parameters evaluation; 3. mechanism of information exposition; 4. model of monitoring parameters exposition; 5. monitoring subject; 6. time context; 7. monitoring update delay; The following table evaluates each of the above characteristics:

73 3.5. JBI Monitoring Challenge 61 Table 3.3: Abstraction contextual evaluation No. JBI OpenESB ServiceMix 1. not defined code instrumentation code instrumentation 2. constant pulling events from inserted monitoring event listeners code 3. management (not well-defined JMX JMX (limited), log files, DOT monitoring) interface files 4. request-response request-response and notifications request-response 5. application and system application (endpoint, SU, SA context) and system (component, nmr, framework context) application (message flow visualization) and system (listeners related to components) 6. current information historical and current historical and current 7. no real-time interface real time of notifications relative near past by constant events pulling The comparison which resulted in the abstraction evaluation confirmed the conclusions from the JBI monitoring status quo section (3.3). Specification is not complete in terms of all mentioned characteristics which are supplemented by each implementation. This fact along with the incompatibility of ServiceMix and OpenESB (3.3.5) implies that JBI monitoring is not a trivial challenge for: defining, designing and implementing. Evaluation of themonitoring subject characteristic proves that, in order to have a fully functional monitoring system, parameters at both the system and application level have to be provided. Some of the JBI parameters are encountered at both monitoring levels or their semantics simply elude positioning in the taxonomy. Therefore, strict monitoring parameter to a monitoring type assignment has been omitted in this thesis. However, during the later design of the architecture solution, focus was put on covering both types of monitoring approaches. It allowed to assure that the system will be usable to every role defined in the JBI specification. Another evaluation conclusion relates to the mechanism of information acquiring selectivity. According to Proactive Application Monitoring article by Alexandre Polozoff [95], log parsing has many drawbacks in comparison to continual data collection (realisable by code instrumentation). Log analysis encompasses going through everything that happens in the system, which makes it far less efficient than focusing on events coming from an instrumented code. Such approach makes selectivity very hard to introduce. Our experience tells us that a system with monitoring information available only in log files requires building another system of which the only purpose is to analyze produced log files and extract important information. Too much information is in fact a disinformation Application theory influence The previous section concluded that the JBI context implies monitoring subject to be a significant challenge. The background of the application theory 3.4 complements with that impression and identifies more challenges of the JBI monitoring solution. BAM and ARM are not at the same level: ARM is quite a detailed specification, while BAM is rather a paradigm of functionalities that enterprise monitoring should be equipped with. Although both technologies were not designed

74 3.5. JBI Monitoring Challenge 62 with special JBI-related aspects in mind, some of their elements have been identified as being useful for JBI. As described in Section 2.5, JBI is an abstraction of a container which is capable of hosting other containers. The JBI specification does not define exact functionalities that the container should provide. Everything depends on the capabilities of Service Engines and Binding Components that will be installed. Without knowing the detailed capabilities, it is hard to design a monitoring system which will be deeply integrated with the business logic executed in JBI. Business Activity Monitoring is a technology which integrates very tightly with the monitoring subject (resulting in the above-mentioned high deployment complexity). The reason for this tight integration is the functionalities that go beyond the basic ones listed in Business Activity Monitoring (3.4.2) section, i.e.: conditions and patterns for business parameters being evaluated, analyzed, alerted and reacted to in real time. BAM s high deployment complexity results in challenging a requirement which does not fit to JBI the capability of patterns and the conditions for evaluating the business logic cannot be provided without earlier, at least general, definition of business that will be conducted. Therefore, implementing pure BAM capabilities is questionable. For the purpose of BAM elements usable for JBI, further CEP-oriented architecture of Business Activity Monitoring analysis was performed. The result of the analysis is simple division of functionalities between CEP and BAM which is depicted in Figure 3.9. In a situation when BAM is built on top of CEP, all basic functionalities enriched with CEPrelated feature are delivered by the latter technology. Only the most advanced, business-oriented capabilities such as Service Level Agreement are realized by BAM layer. Architectural context leads to the conclusion that the JBI monitoring solution fits rather as an abstraction of Complex Event Processor built into the JBI bus. The approach focusing on event analysis is in line with the assumptions from the previous section, where it was pointed out as being superior to massive log parsing and constant parameters pulling. The JBI monitoring is unable to provide advanced BAM features which are out of the scope of JSR-208. In order to define more precisely the application theory influence, general characteristics of the monitored events have to be provided. For this purpose, the ideas from the ARM specification have been derived. ARM focuses on business transaction monitoring which is not a trivial task in a heterogeneous and distributed environment. In order to reconstruct the transaction s complete report, the system has to analyze, correlate and build hierarchies of inter-transaction events that are intercepted (as delineated in Section 3.4.1). Therefore, the functionality of the JBI transaction monitoring solution could be viewed as Complex Event Processing tool focusing on business transaction constituent events. The real-time transaction monitoring feature turns out to be a very useful feature. However, the capabilities will be more complete if also some historical information is stored for the purpose of a later analysis and exposition as context information for real-time status. In order to accomplish this, the idea of data warehouses from Business Intelligence has been leveraged. It could be interpreted that leveraging BI resulted in addition of persistence functionality to the CEP technology. Having it mind the loosely-coupling and the compatibility with SOA derived from ARM, a possible JBI monitoring architecture has been depicted in Figure In order to provide a complete theoretical foundation of architecture, a list of functionalities derived from CEP and interpreted in the context of ARM is defined: gathering the data through event interception;

75 3.5. JBI Monitoring Challenge 63 Figure 3.9: CEP-oriented BAM architecture aggregating the data at various levels; data filtering; providing the event context; correlating the events belonging to the same transaction; constructing event hierarchies to provide a complete transaction report; constructing event relationships with detailed timing information; reporting information in a flexible and not only real-time-oriented way; The Section concludes that the best choice is not to strictly comply with the ARM specification, but to derive from it. This thesis derives some of the ARM innovative monitoring approaches it has already been depicted in Figure 3.10 and in the above-mentioned functionalities. However, there are a few more ARM-related capabilities that are provided:

76 3.6. Conclusion 64 Figure 3.10: Possible Architecture Positioning Monitoring transaction success and, in case of failure, reporting the reason; Transaction timing along with a complete timing breakdown across component; The ease of finding bottlenecks; Support for both synchronous and asynchronous distributed transactions; Minimization of influence of (slowing down) the application being monitored; Preserving the loosely-coupled architecture between the ARM application nodes and the ARM monitoring center(it is called ARM Agent in the ARM specification); 3.6 Conclusion This chapter provided a deep insight into the subject of monitoring enterprise applications and JBI in particular and it allowed to grasp significant conclusions. First of all, the monitoring abstraction section identified that there is no general enterprise monitoring theory that could serve as a foundation for GlassBus. It forced the authors to introduce a substitution in form of general monitoring characteristics which allow to evaluate the JBI monitoring status quo. It turned out that JSR-208 proposes only restricted management

77 3.6. Conclusion 65 interface and lacks a significant standardization of monitoring facilities. Additionally, the specification finds the distribution and clustering as being out of the scope of JBI version 1.0. It implies that OpenESB and ServiceMix implementations differ in clustering mechanisms and monitoring facilities. Further analysis of the application monitoring theory resulted in the conclusion that neither BAM nor ARM can be directly used in the monitoring of JBI. The lack of mature open-source implementation of Application Response Measurement was the reason for restriction to extracting most valuable ARM ideas and incorporating them into the JBI monitoring solution in a proprietary way. Business Activity monitoring turned out to be too demanding in terms of the monitoring capabilities oriented at business analysis - advanced triggers, alarms and instant reactions. It has been concluded that the presented monitoring solution should rather be based on the concept of Complex Event Processor (CEP), which is in fact a mechanism for conducting the BAM functionalities. The presented monitoring application theory allowed to grasp main ideas and directions of enterprise monitoring. It enables to create awell-designed, functional and flexible solution of which architecture principles cannot be rejected without denying fundamentals of Application Response Measurement and Business Activity Monitoring. The last section of this chapter introduced the assumptions of a system compliant with the ideas of ARM and providing CEP functionalities. It identified the JBI monitoring as a significant challenge - the constraints resulting from the JBI status quo and the application monitoring theory were found as being not trivial.

78 Chapter 4 Executive summary of the monitoring architecture I think it is an immutable law in business that words are words, explanations are explanations, promises are promises - but only performance is reality. Harold S.Geneen, an American Accountant and Industrialist. 4.1 Preamble Previous chapters outlined a complete background of the problem discussed in this paper. Second chapter (2) presented the fundamentals of an enterprise integration theory as well as introduced the standardized ESB in form of JBI. Not only did the third chapter (3) explain the purpose of JBI monitoring, but it also inspected its current status quo. Finally, some basic concepts of the JBI monitoring architecture were formed. The main goal of the current chapter is to elucidate and develop these concepts in order to present executive summary of the JBI monitoring architecture. Concepts are discussed with reference to previous chapters forming a consistent view of the designed solution. Functional and non-functional requirements are means of expression, which makes the message explicit and well-defined. The main emphasis is put on the performance and the compliance with JBI. Next section introduces the final solution called GlassBus whose main goal is fulfillment of requirements mentioned above. Its brief description is followed by the architectural diagram which depicts the monitoring topology. At the end short description of each GlassBus element is provided. Finally, a monitoring abstraction is presented. It contains a thorough explanation of notions and concepts introduced by GlassBus in order to provide a full understanding of JBI monitoring fundamentals. The most important aspect is the definition and demarcation of the business transaction - top-level entity which is monitored. 4.2 Problem definition General purpose of the presented thesis is to design and implement monitoring solution compliant with JBI. However, this sentence does not clarify the architectural details as well as particular functionalities which are required in the final solution. At the beginning of the work, it was impossible to estimate what monitoring features are practicable. The process of requirements engineering was performed in order to design the characteristics of the system. It consisted of five stages [64]: environment evaluation - analyzing the JBI specification in order to gain thorough knowledge; requirements elicitation - gathering the requirements from JBI community;

79 4.2. Problem definition 67 analysis - checking for consistency and completeness; specification - documenting the requirements; verification - making sure the specified requirements are correct; As a result, main concepts of the system are presented in form of the functional and non-functional requirements. It is a statement that identifies a necessary attribute, capability, characteristic, or quality of a system in order for it to have value and utility to a user. [75]. It is worth mentioning that presented requirements were used as input during the design stages of development which are described in the fifth chapter (5). The description of the requirements is brief as it is only a shortened summary of the features which were already introduced or are described in detail later on Non-functional requirements Non-functional requirements are the constraints or quality requirements. Other terms for nonfunctional requirements are quality attributes, quality goals and quality of service requirements. They specify criteria that can be used to judge the operation of a system. Summing up, they define how a system is supposed to be [66]. JBI Compliance The implementation of the monitoring mechanisms should be based only on APIs defined in JBI. No proprietary features should be enclosed by the target solution. The general assumption is that the system should be able to monitor any kind of JBI-compliant ESBs. Performance-oriented implementation Any kind of significant deterioration of JBI performance should NOT be noticeable during the operation of the monitoring system. High performance is one of the key aspects since it has a great impact on the business efficiency. Implementation should consist of as little computation logic as possible, every construct has to be well-considered and performance-oriented. Loosely coupled architecture Elements of the monitoring system should be loosely-coupled due to the following reasons: It is the best way of monitoring of loosely-coupled applications. It is the requirement of ARM, described in Section Runtime environment support It should be possible to run the final solution on the application server as well as in the stand-alone mode. It ensures that the system is reusable under all environmental conditions.

80 4.2. Problem definition 68 Non-invasiveness for the code of JBI Designed architecture should on no account invade the JBI code. It is not possible to apply code changes, recompile the code and produce substitute binaries. Operation of the monitoring system should be transparent from the point of view of JBI container. Simplicity The usage of monitoring system should be simple and straightforward. Initial installation should consist of default settings in order to provide the working configuration as quick as possible. It does not mean that the system should lack sophisticated and advanced functionalities. They should be, however, configurable according to the principle of abstraction. Usefulness for JBI community Monitoring system should be innovative and useful for the community of JBI users. There is no goal in implementing a product which is similar to all the others. Due to deficiency, any kind of monitoring solutions for JBI are highly anticipated. However, this fact should not deteriorate its quality. The ease of finding bottlenecks One of the main goals of the monitoring solution is to enable the JBI administrator to tune the performance of the business transactions executed in JBI environment. APIs as well as mechanisms should be implemented in compliance with this requirement to form advantageous platform for optimization of business processing. High manageability Accomplishing of an administrative task should be easy to perform. The system should be easy to maintain and manage. Avoid Not Invented Here syndrom Not Invented Here (NIH) [67] also called as reinventing the wheel is a term which describes a onerous software culture that avoids reusing existing products which come from different vendor or products family. Efforts which are split up usually result in a group of identical software distributions differing only in a slight manner. During implementation every requirement for a software feature has to be carefully evaluated in order to find a robust, tested and supported software library. Own proprietary code has to be omitted unless a fundamental need exists. Architecture accordant with best-practices of software design Best-practice [53] is a technique which introduces a general, reusable and effective solutions to a commonly known tasks. It assumes that with the usage of proper, most efficient methodology a competitive outcome can be brought with as few drawbacks as possible. Best practices of software design should be applied during the development of the monitoring solution. That induces:

81 4.2. Problem definition 69 usage of commonly-known, recommended solutions; usage of specified tools, templates and libraries; usage of design patterns; writing of a qualitative code; Open-source dependencies Straightforward requirement which oblige the authors to use only software artifacts which are released as Open Source 1. Use of monitoring capabilities defined in JSR JSR-208 specification defines a set of JMX beans which provide a basic knowledge about conditions and qualities of operating JBI. In compliance with the NIH requirement it makes no sense to implement it once more. Designed solution should benefit from all contributive techniques defined in JBI, no matter how technically sophisticated they are Functional requirements Functional requirements describe the functionalities of the system as concrete functions which the system is able to execute. They can consist of diagrams, technical details, data manipulation and processing charts in order to specify particular behaviors of the solution. Summing up, they describe what the system is supposed to do [58]. Completeness of monitoring capabilities Monitoring capabilities of the system should not be limited by anything different from the JBI specification itself. The selection of underlying technologies and architectural decisions is supposed to be careful and well-considered in order not to constrain, but to provide the system with a wide variety of possibilities. Parameters for application and system monitoring types Requirement characterizes two classes of monitoring parameters which the system should provide: application parameters; system parameters; They are described meticulously in Section Open Source - source movement

82 4.2. Problem definition 70 Real-time and historical data Requirement specifies two kinds of monitoring data which should be made accessible by the solution: real-time data; historical data; The system should continuously evaluate the conditions and qualities of JBI providing the user with real-time metrics as well as aggregated historical analysis from the specified time interval. Topology and flow information Requirement defines two types of events that the system has to track. Not only should it analyze the information about message exchanges which flow in the bus, but it should also follow all topology changes of the runtime environment. Notifications The system should encompass the mechanism of notifying the subscribed users about specified conditions or alert-situations which are agreed-upon. It is similar to the mechanism introduced by OpenESB (Section: 3.3.3). This functionality allows to conduct business activity monitoring (BAM) described in Section Support for synchronous and asynchronous operations JBI supports synchronous and asynchronous operations in the MessageExchange processing (send and sendsync methods). The system should be able to monitor and profile both of these interaction patterns. Transaction monitoring All transactions should be precisely monitored in order to provide the full spectrum of metrics concerning business processing. Stored pieces of information should concern: parties which took part in the transaction; information flow; exact timing path: business processing time, time spent in queues, etc.; result: success or failure; It is compliant with ARM transaction monitoring described in Chapter Clustering requirements On the one hand, JBI does not standardize the clustering of buses. On the other hand, many proprietary mechanisms exist as OpenESB supports clustering/mirroring, while ServiceMix introduces federation of JBI elements. All these facts should be taken under consideration during the design stage in order to support both clustering scenarios.

83 4.3. Introducing GlassBus 71 Monitoring parameters extending those proposed in OpenESB OpenESB introduces a long description of monitoring mechanism enclosed in the Sierra release. Not only should the final solution derive from it but it should also extend its capabilities. Event listeners based on the proprietary solution introduced in ServiceMix ServiceMix provides a functionality of event interception, which is described in Section However, this mechanism is not compliant with JBI, since it relies on definition of events listeners provided in package org.apache.servicemix.jbi.event. The system should reimplement this feature in compliance with JBI. CorrelationId mechanism from ServiceMix SeriveMix is the only implementation of JBI specification 5, which defines the mechanism of message correlation. This mechanism combines the features of ARM (Section 3.4.1) as well as CEP (Section 3.4.2). It is one of the most anticipated features in the final solution. 4.3 Introducing GlassBus Section outlined some basic concepts of the generic monitoring architecture compliant with JBI. In order to propose an initial point of reference, a first draft of the monitoring solution is presented in Figure 4.1. Its name is GlassBus. The name is not coincidental since it fits well to the group of Glass-like solutions (GlassBox 6, GlassFish 7, etc.). The main goal of GlassBus is the fulfillment of requirements described above. GlassBus defines three types of elements which form the monitoring topology: Monitoring Agent - sensor of the system - implementation of the Event Listener which encompasses some functionalities of the Complex Event Processor (both terms introduced in Chapter 3.5.2); Monitoring Engine - brain of the system - implementation of the ARM monitoring center (3.10); its main goals are: reception of the data sent by Monitoring Agents; storage of the data in the data-store; exposition of the processed monitoring data to external clients; Communication Backbone - enables Monitoring Agents and Engines to communicate in a loosely-coupled manner; The description of GlassBus architecture is brief. It is only an overview of the concepts which are extended in the subsequent chapters ServiceMix is based on JBI, but it is not certified to be fully JBI compliant. 6 GlassBox - Troubleshooting agent for Java applications GlassFish - Open Source Application Server -

84 4.4. Monitoring Abstraction 72 Figure 4.1: GlassBus architecture overview 4.4 Monitoring Abstraction GlassBus defines a wide variety of notions and concepts which are referenced in the following chapters. This section accurately explains these concepts in order to provide a full understanding of GlassBus fundamentals. Brief reminder of JBI notions which are explained in detail in Section 2.5: Operation - low-level business function; Endpoint - entity encompassing one or more operations; Service Unit - entity encompassing one or more endpoints; Component - JBI component, container of SUs; Service Assembly - entity encompassing one or more SUs; JBI instance - operational JBI container; Cluster - logical entity encompassing zero or more JBI instances; Notions introduced by GlassBus: ME FLOW - Instance of a ME FLOW PATTERN. Equivalent of single Message Exchange performed in JBI container. ME FLOW PATTERN - ME FLOW represents the JBI-local portion of a service invocation. It is distinguishable by its two ends:

85 4.4. Monitoring Abstraction 73 Consumer location: cluster, JBI instance, SA, Component, SU, endpoint; Provider location: cluster, JBI instance, SA, Component, SU, endpoint; ME FLOW PATTERN is an abstract notion (template) which describes location attributes (consumer and provider location) of a concrete ME FLOW. ME FLOW is a concrete instance of a specified ME FLOW PATTERN performend in the JBI environment. The relation between ME FLOW and ME FLOW PATTERN is identical to the relation between object and a class in the programming languages. ME SUB FLOW - Instance of a ME SUB FLOW PATTERN. ME SUB FLOW PATTERN - Sub-group of ME FLOW PATTERNs. Description of its two ends is extended since operation is added: Consumer location: cluster, JBI instance, SA, Component, SU, endpoint, operation; Provider location: cluster, JBI instance, SA, Component, SU, endpoint, operation; Notion similar to ME FLOW PATTERN, however, it relates to ME SUB FLOWs. ME TRANS - Instance of ME TRANS PATTERN. ME TRANS PATTERN - Definition of a single transaction 8 which is described as a sequence of correlated ME SUB FLOW PATTERNs. BUSS TRANS PATTERN - A group of ME TRANS PATTERNs whose initial ME FLOW PATTERN is identical. BUSS EP - Consuming endpoint in which BUSS TRANS PATTERN begins. BUSS OP - Consuming operation in which BUSS TRANS PATTERN begins. The main goal of the section presented above is to name and standardize the concepts of JBI monitoring which are used in this paper. The terminology is quite complex and may be unclear at the beginning, however, it is caused by the complexity of the field of JBI monitoring. Description located below tries to clarify all the notions. ME FLOW (ME SUB FLOW) and ME FLOW PATTERN (ME SUB FLOW PATTERN) Abstract definition (class): ME FLOW PATTERN as well as ME SUB FLOW PATTERN are the representations of the MessageExchange which is defined in JBI. The introduction of these notions was necessary as they extend the amount of information encompassed by a simple MessageExchange. The advantage of ME FLOW PATTERN and ME SUB FLOW PATTERN is that they contain accurate consumer information which lacks in the MessageExchange. It was also essential to decouple ME FLOW PATTERN and ME SUB FLOW PATTERN. At the beginning they seem to be similar. However, ME SUB FLOW PATTERN relates to operations, not endpoints. The information about the consumer operation which is added to the ME SUB FLOW PATTERN is hard to gather in a JBI compliant way. It is sometimes omitted by the JBI implementations, since it is not mentioned in the JBI specification. Thus, in order to provide a generic concepts of 8 In this context, transaction means a group of logically connected message exchanges.

86 4.4. Monitoring Abstraction 74 JBI monitoring, these two terms were decoupled. Figures 4.2 and 4.3 depict presented concepts. Lifecycle: ME FLOW PATTERN, as well as ME SUB FLOW PATTERN, begins when the corresponding MessageExchange is initially sent (passed to NMR using one of the send methods by a component). ME FLOW PATTERN, as well as ME SUB FLOW PATTERN, terminates when the corresponding MessageExchange terminate. Concrete definition (object): ME FLOW as well as ME SUB FLOW are the concrete instances of the patterns mentioned above. For example: ME FLOW NR1 was performed at 01.54PM and lasted 13,2seconds. Figure 4.2: ME FLOW PATTERN example Figure 4.3: ME SUB FLOW PATTERN example

87 4.4. Monitoring Abstraction 75 ME TRANS and ME TRANS PATTERN Abstract definition (class): ME TRANS PATTERN defines a group of ME FLOW PATTERNs or ME SUB FLOW PATTERNs which are correlated. It represents a set of logically connected message exchanges which are performed in a sequential or parallel manner. Figures 4.4 and 4.5 depict the presented concept. Lifecycle: ME TRANS PATTERN begins when its initial ME FLOW PATTERN (or a ME SUB FLOW PATTERN) begins. Initial means the first one in the sequence of invocations. ME TRANS PATTERN ends when its last ME FLOW PATTERN (or a ME SUB FLOW PATTERN) ends. Last means the last on in the sequence of invocations. Concrete definition (object): ME TRANS is a concrete instance of the pattern mentioned above. For example, Figure 4.4 presents ME TRANS which consisted of four ME FLOWS: ME FLOW 1 sequentially initiated ME FLOW 2, ME FLOW 3 and ME FLOW 4. It was performed at 2.01PM and lasted 57,2seconds. Figure 4.5 presents different ME TRANS which consisted of five ME FLOWS since ME FLOW 4 is added. This two figures presents instances of two different ME TRANS PATTERNs. Figure 4.4: ME TRANS example nr.1

88 4.4. Monitoring Abstraction 76 Figure 4.5: ME TRANS example nr.2 BUSS TRANS PATTERN Abstract definition (class): BUSS TRANS PATTERN defines a group of ME TRANS PATTERNs whose initial ME FLOW PATTERN (or a ME SUB FLOW PATTERN) is identical. After performing the initial ME FLOW PATTERN, ME TRANS PATTERN can consist of various ME FLOW PATTERNs depending on the parameters and conditions of the transaction. All these combinations create, however, different routes of the same logical business transaction. BUSS TRANS PATTERN classifies all this routes, which are actually different ME TRANS PATTERNs, into one business transaction. Figures 4.4 and 4.5 greatly depict two routes of one business transaction. BUSS TRANS PATTERN, presented in Figure 4.6 is one of the most important concepts of GlassBus. BUSS TRANS PATTERN lifecycle is analogous to the lifecycle of ME TRANS PATTERN. Concrete definition (object): Concrete instance of BUSS TRANS PATTERN is obviously an instance of one of the ME TRANS PATTERNS which forms the BUSS TRANS PATTERN since it is a single route of a business transaction. Thus, there is no sense in defining BUSS TRANS notion.

89 4.5. Conclusion 77 Figure 4.6: BUSS TRANS PATTERN example 4.5 Conclusion Presented chapter outlined in detail the requirements imposed upon designed solution. Not only did it present the most demanded functionalities but it also explained the relationships between them and the high-level concepts described in the third chapter (3). The process of requirement s engineering resulted in 24 functional and non-functional requirements, which are presented above. Not all of the requirements are of the same importance. Next chapter thoroughly describes the extent to which they were fulfilled. The chapter also presented an overview of the GlassBus architecture. GlassBus is the realization of the Complex Event Processor introduced in Chapter 3.5.2). Topology consists of: Monitoring Agent - sensor of the system - gathers and sends monitoring data to the Monitoring Engine; Monitoring Engine - brain of the system - receives, processes and exposes the monitoring data to external clients; Communication Backbone - enables Monitoring Agents and Engines to communicate in a looselycoupled manner; Details of the GlassBus architecture are presented in the fifth (5) and sixth (6) Chapter. The last section of the chapter presented notions and concepts regarding JBI monitoring. Terminology is complex due to the complexity of the discussed field. Introduced notions form a foundation which is referenced later on in the following chapters, thus its full-understanding is required before proceeding. The main goal of these definitions is to propose and standardize different types of operations and communications which are performed in the JBI bus. It is done in order to introduce JBI monitoring theory. The most important aspect is the definition and demarcation of the logical business transaction proposed by the authors of the thesis. All the facts form a great preface to the following chapter which describes experiments and architectural decisions made during the design stage. Functional and non-functional requirements form a great shortlist of demanded features and an inevitable point of reference. GlassBus terminology presented at the end

90 4.5. Conclusion 78 enables the authors to precisely communicate the message using an agreed-upon, fully-understandable notions.

91 Chapter 5 GlassBus Research A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it. Albert Einstein, German born American Physicist. 5.1 Preamble The previous chapter (4) provided a glimpse of the GlassBus architecture. Main functionalities and expectations were described in form of the functional and non-functional requirements. In order to provide best-fitting architectural solutions which are compliant with these requirements, a wide set of experiments was performed. Experiments were made at every design stage, whenever a nontrivial problem was faced. These kind of development methodology is sometimes referred to in the literature as Experimental Driven Development. Every experiment was thoroughly documented and it consisted of: a general description, research results and the discussion regarding requirements compliance of the chosen realization. The list presented below outlines the description and goal of each experiment conducted during the process of design and development. Information acquiring mechanism - Clarifies the way in which GlassBus will instrument the JBI bus in order to gather monitoring information from running instances. JBI monitoring mechanisms - Defines the set of monitoring capabilities of GlassBus as well as explains the underlying mechanisms. Correlation heuristics - Defines the methods of business transaction identification and monitoring thanks to the mechanism of ME FLOW s correlation. Persistence - Investigates the newest persistence solutions. Experiment consists of three parts: database engine choice, persistence layer choice, persistence domain design. Communication middleware - Determines the best communication method for GlassBus in order to provide the interconnect among GlassBus elements on agreed-upon conditions. Management middleware - Designs the specification, interface and implementation of the management mechanism for GlassBus which is accordant with best-practices, for example, composite 1 design pattern. Hierarchical Data Management - The problem of hierarchical data management is not trivial in Java. There is no standardized library which copes with all the possible problems and requirements. Due to that fact, a thoughtful examination is performed in order to choose the best solution of hierarchical data management in GlassBus. Installation and runtime environment - Provides the simplest and most robust solution of GlassBus installation and configuration which supports both: stand-alone and in-container execution. 1 Composite design pattern - pattern

92 5.2. Experiments placement 80 Monitoring interface - Investigates the ways of the remote interface exposition in reference with the characteristics of the exposed data. Logging - Provides robust debug and logging solution for GlassBus. 5.2 Experiments placement Figure 5.1: GlassBus architecture overview - placement of the experiments Figure 5.1 depicts an overview of the GlassBus architecture. Aforementioned experiments are related to different parts of the system. The Information acquiring mechanism and the JBI monitoring mechanisms are related to the sensors. Persistence is related to the Engine s datastore. Middleware regards various aspects of Agent-Engine communication. Finally, Monitoring interface is responsible for exposing the data processed by engine to external actors. The rest of the experiments relate in general to GlassBus and cannot be explicitly placed in the architecture diagram.

93 5.3. Information acquiring mechanism Information acquiring mechanism This experiment is related to one of the monitoring classification characteristics introduced in Section 3.2 mechanism for acquiring monitoring information. It is a fundamental building block of every monitoring system and GlassBus is no exception here. Flexibility of this mechanism gives the foundation to a wide range of the JBI monitoring capabilities: MessageExchange interception, correlation and topology tracing, which allows to supersede the monitoring features of ServiceMix and OpenESB. The crucial assumption is that the mechanism must not modify the source of the chosen buses: ServiceMix and OpenESB. Any modification would result in the solution being not useful for the JBI community because every user would be forced to patch the source code and recompile the whole bus Possible options The Section 3.2 defines the following kinds of mechanisms for acquiring information: log analysis; code instrumentation; Only the log analysis is non-invasive for the code of bus, but it does not comply with other requirements. Information does not have to be logged in real time (there can be some undefined delay). Therefore, the real-time requirement may not be met. Processing logs is not efficient - a performance problem. But the most significant problem results from the lack of a logging format standardization. Each bus would need a proprietary mechanism for parsing and interpreting logs (the verbosity of which has to be accordingly tuned and the information of which could be far from completeness) that would be unacceptable. Therefore, the only reasonable choice was code instrumentation, which complied with every requirement apart from being non-invasive for the source code and thus being not useful for JBI buses communities. The problem is solved by the code instrumentation applied through Aspect Oriented Programming, which was introduced for the first time in the article Aspect-Oriented Programming published by Kiczales et al. in 1997 [90]. The article provides the following AOP goal definition: The goal of Aspect-Oriented Programming (AOP) is to make it possible to deal with crosscutting aspects of a system s behaviour as separately as possible. The JBI aspect that GlassBus needs to crosscut is of course monitoring - AOP fits into GlassBus s demands. The crosscutting can be performed on join points, being an abstraction of points in a program s control flow. AOP introduces pointcuts for the purpose of the join points set specification. The aspect framework provides various criteria that can be used to specify what join points are within the scope of a particular pointcut. Finally, advice are introduced allowing to execute some code (aspect logic) in the join points defined by the pointcuts (i.e. in the context of pointcuts). In the AOP terminology: the advice of the aspect logic is applied to the code in the pointcut scope, which can execute in a before, after or around manner. The combination of the pointcuts, advice and some additional logic forms an aspect which is the primary AOP modular entity used for crosscutting various system concerns. These are only the very basics of Aspect Oriented Programming (inspired by AspectJ Programming Guide [80]), the details of which are assumed to be known to the reader of this thesis.

94 5.3. Information acquiring mechanism 82 The code gathered in the aspect has to be applied to the original program code and such an operation is called weaving. According to The AspectJ Development Environment Guide [109], weaving can be applied in three different ways: Compile-time (source) must be used when the aspect modifies the original program and other code parts rely on this modification. Weaving other than compile-time will result in a compile error. Post-compile (binary) class files (or whole JARs) are modified during weaving process. Load-time binary weaving deferred until the point that a class loader loads a class file and defines the class to the JVM [109]. The only weaving type complying with all requirements is load-time, which allows to apply the code instrumentation in runtime just before the bus starts. Therefore, a modification of the bus source or binary is not needed Realisation AspectJ is the most mature and powerful AOP framework for Java and therefore it has been chosen for the purpose of the GlassBus s monitoring aspect. In order to enable load-time weaving, AspectJ provides an appropriate agent in the form of a jar file that has to be given as an argument to the JVM invocation - -javaagent:pathto/aspectjweaver.jar. Weaving is achieved by overriding JVM s class loader and appropriately changing the bytecode of the loaded classes, which are in weaving scope. The agent waits for jars in classpath that contain the META-INF/aop.xml file, which provides configuration for the aspects enclosed in a particular jar. Sample aop.xml is available in Listing A.8. It defines packages that are included in the weaving scope, aspect names, i.e. fully qualified class names of the aspects as well as concrete aspects which extend the abstract aspects provided in the jar. Defining concrete aspects at the XML level provides a very flexible mechanism of adjusting the aspect s details which can be environment-dependant it is suited to the needs of adapting aspects to two different JBI implementations. The described monitoring aspect does not make any assumptions about the crosscut implementation, apart from the JBI compliance. The pointcuts on the well-defined JBI interfaces are perfectly enough for intercepting all needed information appearing in any bus that is JBI compliant. Further pointcuts details are provided in Section 5.4, while the rest of this section focuses on description of the aspect weaving configuration in ServiceMix and OpenESB. ServiceMix ServiceMix has recently been adopted to several Application Servers: JBoss, Geronimo and Tomcat, but the immaturity level of those adoptions did not allow to maintain all functionalities of the stand-alone mode, which is currently the most common ServiceMix deployment scenario. Therefore, the GlassBus solution focuses on supporting the ServiceMix stand-alone scenario. The weaver file - aspectjweaver.jar - is placed in ServiceMix s lib directory along with jars containing the aspects. The final step requires a modification of the servicemix.bat, which is the bus s startup script. The JVM invocation has to be modified accordingly: "%JAVA%" "-javaagent:%servicemix_home%\lib\aspectjweaver.jar"...

95 5.4. JBI monitoring mechanisms 83 OpenESB Contrary to ServiceMix, OpenESB is most commonly deployed to the GlassFish application Server. GlassBus supports this deployment scenario. GlassFish allows to maintain separate domains which contain their own configuration of application server instance that can be further extended to cluster of a particular domain s application servers. GlassBus monitoring aspect is tailored for the domain of the GlassFish server it does not modify the higher configuration. The aspectjweaver.jar, the jars with aspects, as well as their dependencies are placed in the domains lib directory. The configuration of the domain is enclosed in domain.xml, which has to be modified accordingly for the purpose of weaver integration: <jvm-options> -javaagent:${com.sun.aas.instanceroot}/../../lib/aspectjweaver.jar </jvm-options> Requirement compliance of the chosen realization is presented in Table 5.1. Table 5.1: AOP Requirement compliance Non-Func NoCodeChange Thanks to load time weaving, the source code is not modified. JBI Only JBI interfaces, but weaving appliance is bus-specific - JBI compliance is slightly broken. Env Stand-alone and Application Server is supported. Performance AOP instrumentation allows to process events in an efficient way without influencing the bus s performance. Usefulness Maintained thanks to the possibility of runtime usage. Func Notifications Can be the result of events intercepted by AOP. Real-time Proper event processing allows for both kinds of data exposition (later discussion in 5.4). Sys & App Flexibility of code instrumentation allows to cover both types of monitoring parameters - see: JBI monitoring mechanisms The previous sections introduced the general idea of JBI monitoring as well as the domain of business transactions in the JBI environment. The main goal of this section is to present the underlying mechanisms, which allow to gather the specified monitoring data Monitoring information As described in the JSR-208 Monitoring Status Quo 3.3.1, the capabilities of the JBI management/- monitoring interface are far from being sufficient. It implied introducing the information gathering

96 5.4. JBI monitoring mechanisms 84 mechanism depicted in Section 5.3. In order to provide a complete gathering mechanism, a definition of JBI points that will be crosscut by the aspects has to be supplied. This section provides a definition of those points which are referred to as pointcuts in the AOP terminology. The pointcuts and the accompanying advice are grouped into aspects. GlassBus-Agent s aspects are divided into two groups: messaging interception and topology interception. Messaging interception Careful analysis of the JBI specification allowed to identify the elements related to messaging. As it has been depicted in Section 2.5, the crucial communication element of a bus is NMR - Normalized Message Router, which provides functionalities of normalization, denormalization and, of course, routing. The JBI components have to directly interact with NMR and for this purpose they are equipped with the Delivery Channel, which is specified by regular Java interface. It allows to define pointcuts which rely only on the provided interface of the Delivery Channel. Such pure JBI pointcuts are the GlassBus s essentials, which provide the elegance of not binding to specific OpenESB and ServiceMix implementation and restrict to relying only on JBI, thus providing innovative monitoring solution fully compliant with JSR-208. Each method of the Delivery Channel has been crosscut resulting in the following pointcuts (ME - MessageExchange, DC - DeliveryChannel): pointcut accept(dc): execution(me accept()); pointcut accepttimeout(timeout, DC): execution(me accept(long)); pointcut send(me, DC): execution(void send(me)); pointcut sendsync(me, DC): execution(boolean sendsync(me)); pointcut sendsynctimeout(me,long,dc): execution(boolean sendsync(me, long)); Unfortunately, in the process of the development it turned out that the ServiceMix s DeliveryChannel implementation is not fully compliant with JSR. It introduces one additional method: processinbound, which allows to inject MEs into the channel in a push manner. It is equivalent to the accept method, which receives the message in a pull manner. In order to overcome this noncompliance, GlassBus leverages AspectJ s flexible mechanism of extending abstract aspects. The above-mentioned pointcuts are enriched with an abstract pointcut called pushaccept and enclosed in the JBIMessagingInterceptor aspect (the code is provided in Listing A.4). GlassBus provides separate concrete versions of the JBIMessagingInterceptor dedicated to each JBI implementation. Concrete aspect is provided by means of the aop.xml file mentioned in the previous section 5.3. The aop.xml configuration for ServiceMix is provided in Listing A.8, while OpenESB s is presented in Listing (A.9). A detailed AOP configuration of GlassBus artifacts will be described in Chapter 6. Such solution allows to support both OpenESB and ServiceMix without denying full JBI compliance. Messaging pointcuts are advised in an around manner. It allows to trace the parameter state before and after a particular method invocation. Additionally, every JBIException occurring in DeliveryChannel can be intercepted and appropriate notification can be sent to GlassBus-Engine. It obviously supersedes the capabilities of ServiceMix s messaging listeners which do not cover the exceptions interception. The processing of each ME interception is aligned with the knowledge about topology - delivery channel being the invocation subject is checked every time against channels registered during

97 5.4. JBI monitoring mechanisms 85 the initialization and starting(accordingly init and start method of ComponentLifeCycle interface) of the JBI components. MessageExchanges acquired from the pointcuts are subjected to correlation heuristics described in the next section (5.5), which allow to join separate MEs into flows of business transactions. The described basic JBI pointcuts are sufficient for the needs of the monitoring parameters presented earlier. Topology interception The JBI s topology state changes occasionally. Therefore, its dynamics cannot be compared with the dynamics of the messaging events. JBI JMX interfaces expose information about topology but their usage forces to constant interfaces pulling and listening for changes. As it has been mentioned in Chapter 3, such approach is undesirable. It implies motivation for emerging a smart topology interception by means of AOP. Topology interception has been enclosed in two aspects: JBITopologyComponentInterceptor(Listing A.5) and JBITopologySUInterceptor (Listing A.6). The first aspect focuses on tracing the bus s components. It is realized by crosscutting all methods of thecomponentlifecycle interface, thus allowing for intercepting the following component state changes: init, start, stop and shutdown. Thanks to that, GlassBus acquires information about every change in the bus s Binding Components and Service Engines. The JBITopologySUInterceptor focuses on artifacts deployed to the JBI components, which encompass: ServiceAssemblys (SA), ServiceUnits (SU) and Endpoints. Every JBI component should implement a Component interface (cf. SPI in Section 2.5) which provides setters and getters for the ServiceUnitManager - an entity that every JBI component is equipped with. ServiceUnitManager crosscutting allows to intercept all events related to service units including: deploy, init, shutdown, start, stop and undeploy. After intercepting an SU event, it has to be correlated with parent ServiceAssembly. Unfortunately, it was impossible to achieve it through an SA event interception. In order to discover the parent SA, the JBI s DeploymentServiceMBean has to be used: 1. List of ServiceAssemblies deployed to the bus has to be retrieved using the getdeployed- ServiceAssemblies(). 2. For each SA the descriptor is acquired by means of the getserviceassemblydescriptor method. The descriptor is the jbi.xml file described in specification. It has to be parsed in order to extract compounding Service Units. 3. Name of the considered SU is matched to extracted Service Units, which allows to discover parent Service Assembly. The JBITopologySUInterceptor aspect has its internal registry which is used to remember and update relations between SA and SU. It reduces the need of redundant Service Assemblies descriptors parsing. Of course, finally the relations are propagated to Engine and stored in a hierarchical data model along with the rest of the topology information. The described logic covers SA, SU and their correlation. In order to intercept a complete topology view, information about internal and external JBI endpoints is needed. Each endpoint is

98 5.4. JBI monitoring mechanisms 86 registered by particular SU. GlassBus is capable of acquiring such ServiceUnit-Endpoint correlation. Endpoints-related information is gathered through JBI s ComponentContext crosscutting. The pointcuts are maintained for the following methods: ServiceEndpoint activateendpoint(qname servicename, String endpointname); void deactivateendpoint(serviceendpoint endpoint); void registerexternalendpoint(serviceendpoint externalendpoint); void deregisterexternalendpoint(serviceendpoint externalendpoint); JSR-208 states that in order to start using internal or external endpoint, it has to be accordingly activated or registered in NMR, therefore above pointcuts are sufficient for tracing all changes in the bus s endpoints. Correlation between SU and endpoints is maintained through the ServiceUnit execution flow analysis. It is assumed that the endpoints operations are always executed in a particular Service- Unit s execution flow. It is not a requirement of the JBI specification but it is a natural consequence of the bus s design. Analysis is realized by means of Java s ThreadLocal 2 functionality. In order to allow for the flow analysis, pointcuts of ServiceUnits are advised in an around manner, opposite to pointcuts of endpoints, where simple before advising is sufficient. The algorithm of the SU-endpoint correlation is depicted below: 1. Pointcut of ServiceUnit is intercepted. Before the invocation of the proceed method, the aspect marks the ThreadLocal variable with a SU name. After that, the proceed method is invoked. 2. Pointcut of Endpoint is intercepted. Aspect checks the value of the ThreadLocal variable. Thanks to the variable s per thread value preservation, the Endpoint is able to discover the parent SU. 3. After returning from the proceed method in the pointcut of the ServiceUnit, the ThreadLocal variable is reset. For additional details please refer to the aspect s code (Listing A.6). JBI routing analysis In the course of the research in has been revealed that intercepting ServiceUnits events does not allow to acquire all information related to the SA and SU deployment. ServiceAssembly descriptor can contain additional connection metadata. Specification describes its usage in the following way: The connections element is used to declare interconnection metadata for the assembly, as a set of connection elements. These data declare mappings from consumer-provided message exchange addresses to provider service endpoints. Exact mappings evaluation algorithm depends on provider linking, which is specified in the SU s descriptor. The SU descriptor can be extracted during the ServiceUnit s deploy or init, where the serviceunitrootpath is supplied. JBI defines the following types of provider linking: 2

99 5.5. Correlation heuristics 87 Standard links The NMR performs its normal routing, including the application of service connections. The evaluation algorithm is simple: 1. If the destination is set to existing endpoint, connections are ignored and ME is sent to that endpoint. 2. If the destination endpoint does not exist, connections are looked up in order to map the endpoint to the real destination. Hard links The consumer always specifies the endpoint of a provider directly, and the routing is not affected by service connections. Soft links The consumer always specifies the endpoint of a provider using an indirect address defined in the connections. GlassBus is capable of acquiring ServiceAssembly connections metadata and taking it into account in the JBI transactions tracking described in the next section. Requirement compliance of the chosen realization is presented in Table 5.3. Non-Func JBI Usefulness JSR-based Simplicity Func Parameters Table 5.2: JBI parameters requirement compliance All monitoring parameters (apart from small exceptions) are acquired by the JBIcompliant crosscutting and analysis. Monitoring parameters completeness and capabilities supremacy implies that Glass- Bus solution is desirable and usable for JBI communities. Some MBeans defined in JSR-208 are used to acquire the complete monitoring information Monitoring parameters list provides a simple JBI knowledge summary GlassBus provides all acquirable information about JBI s messaging and topology, which are more complete than combined ServiceMix and OpenESB monitoring parameters and cover both thesystem and application monitoring classification characteristics. Topology&Flow Both completely covered thanks to the JBI pointcuts Send/ASync Both types of MessageExchange patterns are intercepted by messaging pointcuts. ME Result Thanks to exceptions interception, both success and failure of the MEs and the entire transactions can be monitored. Cluster JBI scope of Cluster is covered by monitoring parameters list SMix Events GlassBus interception supersedes ServiceMix s - exceptions monitoring 5.5 Correlation heuristics As described in Section 4.4, a group of logically correlated Message Exchanges (ME FLOWs) forms a business transaction. The main goal of this experiment was to design, test and implement mechanism compliant with JBI which would provide GlassBus with the functionality of the very ME FLOW correlation.

100 5.5. Correlation heuristics Problem definition Message Correlation is the concept of tying messages together. Usually it occurs when one service calls another services during a longer business processing. The calls are logically coupled but physically decoupled. The usage of correlation ids to track the flow of logically related messages is a commonly used mechanism in systems based on messaging middleware. Generally correlation is a specification of a group of markers encompassed by messages and a set of messaging actions which specify how the processing should carry and preserve the unique values of these markers that were initialized by the first message [65]. On the one hand, JBI does not introduce any kind of hierarchical relations for Message Exchanges that depend on the invocation tree. On the other hand, message correlation was one of the most important functional requirements of GlassBus as it combines the key features of business transaction monitoring introduced by ARM (Section 3.4.1) as well as capabilities of Complex Event Processor enclosed in BAM (Section 3.4.2). Numerous experiments and researches were conducted in order to provide this highly anticipated mechanism. First off, JBI implementations were evaluated. It quickly turned out the ServiceMix provides a proprietary implementation of the message correlation functionality which is introduced in the section regarding ServiceMix (3.3.2). After inspecting the code and contacting the ServiceMix developers the general idea of the implementation was known. The mechanism is based on particular implementation of ServiceMix s components which delegate single and the same thread to message transmission during each business transaction performed by the component. Trace of messages sent and received by each thread enables to correlate messages. The general assumption of this mechanism is broken by the BPEL service engine which is based on the JACOB 3 framework. JACOB deals with two key issues in implementing BPEL constructs: Persistence of execution state; Concurrency; The aforementioned features enable to break and flatten the business process execution stack and rely on explicit communication channels to handle control flow [52]. Not going into details, it has to be said that message correlation mechanism implemented in ServiceMix does not support it, which causes that the correlation is rather based on an optimistic heuristic than on a deterministic algorithm. OpenESB, however, lacks even such mechanism. Message correlation was not directly addressed by the JBI specification, thus any kind of this mechanism was not implemented in OpenESB. That fact was confirmed by Ronald Ten-Hove, a software architect who sits in the JBI experts group. In his Ron described that this mechanism was seriously discussed, but finally rejected in the first version of the specification. In addition, he did not possess sufficient knowledge, whether it would be enclosed in the awaited second version. All these facts caused that an own JBI-compliant mechanism had to be implemented. 3

101 5.5. Correlation heuristics Realization Another research was performed in order to find the best fitting solution. It was spotted that required functionality is in a measure similar to the features of a transaction provider. JBI is fully transactional since transaction can span the flow of a Message Exchange which contains required properties: field: String JTA TRANSACTION PROPERTY NAME; method: boolean istransacted(); Transaction provider can somehow correlate the Message Exchanges as it is equipped with the functionality of spanning multiple messages in single transaction. According to the specification, NMR is able to share transaction context between JBI engines/bindings. The first solution was the instrumentation of the NMR logic in order to extend its functionalities. NMR was supposed to be provided with the capability of correlation id propagation. However, all of the PoC 4 implementations failed. The listing presented below outlines fragments of the JBI interfaces which, according to specification, concern transactions. 1 public interface javax. jbi. component. ComponentContext { 2 Object gettransactionmanager ( ) ; 3 } 4 5 public interface javax. jbi. messaging. MessageExchange { 6 Object getproperty ( String name ) ; 7 void setproperty ( String name, Object obj ) ; 8 String JTA_TRANSACTION_PROPERTY_NAME ; 9 boolean istransacted ( ) ; 10 } public interface javax. jbi. messaging. NormalizedMessage { 13 Object getproperty ( String name ) ; 14 void setproperty ( String name, Object value ) ; 15 } Transactional approach - reasons for failure: interfaces regarding transactionality (presented above) are to weakly defined in JBI; all work is performed backstage by the transaction provider and NMR whose interfaces are unknown; This solution had to be rejected in order to fulfill the requirements of JBI compliance and noninvasiveness of the JBI code. Finally, it was decided that a heuristic similar to that applied in ServiceMix will be designed. It was obvious that its efficiency of action will probably not be deterministic since messaging flows can become very complex. It was also known, that the scope of correlation in some cases could be limited and span only a part of the transaction. However, heuristic was probably the only solution that was possible to implement in compliance with the GlassBus requirements. 4 Proof of concept

102 5.5. Correlation heuristics 90 Message correlation It was decided that the heuristics will derive from the aspect oriented programming and the crosscutting of the JBI code. It is based on threads and Normalized Messages analysis. The general idea is that: There is a thread which uses references of Normalized Messages which are enclosed in parent and child Message Exchanges. The references are used one after the other which enables the correlation. It is worth mentioning that this concept may prove to be false in certain situations as it makes assumptions on the component thread management, which is not standardized by JBI. The concept of the algorithm is presented in Figure 5.2 Figure 5.2: Message correlation algorithm overview The algorithm consist of two elements: Data Analyzer; MessageExchange correlator; Data analyzer gathers data required for message correlation: 1. NormalizedMessage to MessageExchange correlation; 2. NormalizedMessage to Thread correlation;

103 5.5. Correlation heuristics MessageExchange state tracing; The first operation correlates Normalized Messages with Message Exchanges to which the messages belong. In JBI the relation between these objects has different direction since the Message Exchanges composite the Normalized Messages. It is done by the pointcut which traces the createmessage() operation of the Message Exchange object: 1 pointcut createnormalizedmessage ( MessageExchange me ) : call ( NormalizedMessage createmessage ( ) ) && target ( me ) ; Result: It is possible to get a reference of Message Exchange having only a reference to one of its Normalized Messages. The second operation traces the usage of Normalized Message objects and marks the threads which touches particular Normalized Messages. It is done by the wildcard-pointcut which traces all touches of Normalized Messages: 1 pointcut threadmessagemarker ( NormalizedMessage nm ) : target ( nm ) && call( (.. ) ) Result: It is possible to get the references of all NormalizesMessages which were touched by specified thread. The third operation trace the status of the ongoing Message Exchanges. It is done by a set of pointcuts which trace the group of send and accept operations defined in DeliveryChannel. They are described in Section Result: It is possible to check the state of Message Exchange (AC- TIVE, DONE, ERROR) having only its reference. The message correlation is performed always before the first send, sendsync or sendsynctimeout operation of the new Message Exchange. It enables to correlate new Message Exchanges before they are sent by NMR. Tracing of send operations is done by three pointcuts: 1 pointcut send ( ME, DC ) : execution ( void send ( ME ) ) ; 2 pointcut sendsync ( ME, DC ) : execution ( boolean sendsync ( ME ) ) ; 3 pointcut sendsynctimeout ( ME, long, DC ) : execution ( boolean sendsync ( ME, long ) ) ; The description of the correlation logic reads as follows (overview of the algorithm is presented in Figure 5.2): 1. Before sending the new Message Exchange MX by thread T get the list of all Message Exchanges touched by thread T. 2. Check the state of all these Message Exchanges and classify them into two groups: A:active and F:finished. 3. Correlation of MX with other Message Exchange is performed if one of the correlation conditions is fulfilled: if (A.size 5 == 1) - it deals with the IN-OUT Message Exchange when parent exchange waits for the results of child exchanges; 5 A.size = size of group A;

104 5.5. Correlation heuristics 92 if ((A.size == 0) && (F.size() 6 > 1) && isinscope(f.pop()) 7 ) - it deals with the IN-ONLY Message Exchange when parent exchange spawns child exchanges but finishes before their results are computed; As said before, presented heuristic is imperfect and provides a workaround solution. Assumptions made by the algorithm may prove to be false in some cases. The simplest failure scenario is when three different thread pools are used to: send, process and receive exchanges. However, the heuristic was thoroughly tested on OpenESB and SerivceMix in many usage scenarios and the results were outstanding. The only component which caused troubles was the BPEL component from ServiceMix. It introduces concurrency abstraction layer which is similar to that presented in the failure scenario. XML marking The fact that the thread-based heuristic was not capable of correlating messages in ServiceMix s BPEL component - Apache ODE, was the reason for further extension of GlassBus s correlation mechanism. It has been decided that, instead of one heuristic, a whole set of heuristics should be available. Intercepted ME FLOW should be analyzed by all available heuristics, so in the end the algorithm can choose appropriate correlation result. Proper design of such solution has been performed. It is depicted in the UML diagram in Section The correlation described earlier and based on threads was named ThreadHeuristic. The second supplementary heuristic was named XMLHeuristic. It has been decided that these two heuristics are sufficient for the purpose of the GlassBus dissertation, but correlation mechanism design allows to easily add heuristics in the future. The purpose of the mentioned XMLHeuristic was to correlate messages in Apache ODE BPEL component. In a situation when it was impossible to make any assumptions about the processing of MessageExchanges, the analysis went into the direction of content marking. It can be assumed that in a situation where two MessageExchanges are correlated, some information is transferred from the first exchange to the second one should take place. In other words, it can be assumed that some part of the first ME s content should appear in the second ME. It is obvious that there can be many scenarios where this assumption is broken. Fortunately, in the course of the evaluation of the most common business process constructions it turned out that this content assumption is true. The problem of the described idea is that content alteration can result in a MessageExchange s processing error. MessageExchange s content is enclosed in NormalizedMessage in the form of an XML payload. XML payloads that fly through the JBI bus should often conform to schemas of various interfaces (WSDLs most of the time). It is also true in Apache ODE - messages are standardized by schemas deployed along with the BPEL process code. In such case any XML modifications break the schema compliance what is unacceptable. Fortunately, an XML modification was discovered that does not break the desired schema compliance. It is based on a mechanism of namespaces introduced by XML. Such namespace can be defined in any place of an XML file and its occurrence is not restricted by the schema. Therefore, there is no obstacle in enriching XML tags with the following attribute: 6 F.size = size of group F; 7 Checks whether the exchange is in scope of the correlation by examining the time-stamp of the exchange termination;

105 5.5. Correlation heuristics 93 1 xmlns:cor= correlationid Unfortunately, it is hard to predict which part of the XML content should be marked. For the purpose of simplicity, XMLHeuristic takes a brute force approach and marks every tag found in the content of the traced MessageExchange. Please examine exemplary XML content provided below: 1 <tns: getloanquoterequest x m l n s: t n s= urn:logicblaze:soa:loanbroker > 2 <t n s : s s n> </ t n s : s s n> 3 <tns:amount>100000</tns:amount> 4 <t n s : d u r a t i o n>48</ t n s : d u r a t i o n> 5 <t n s : s c o r e>48</ t n s : s c o r e> 6 <t n s : l e n g t h>48</ t n s : l e n g t h> 7 </ tns: getloanquoterequest> Due to the marking, the exemplary XML content is modified in the following way: 1 <tns: getloanquoterequest x m l n s: t n s= urn:logicblaze:soa:loanbroker xmlns: cor= correlationid > 2 <t n s : s s n xmlns:cor= correlationid > </ t n s : s s n> 3 <tns:amount xmlns: cor= correlationid >100000</tns:amount> 4 <t n s : d u r a t i o n xmlns:cor= correlationid >48</ t n s : d u r a t i o n> 5 <t n s : s c o r e xmlns:cor= correlationid >48</ t n s : s c o r e> 6 <t n s : l e n g t h xmlns:cor= correlationid >48</ t n s : l e n g t h> 7 </ tns: getloanquoterequest> The correlationid is the value of the correlation id assigned to the traced MessageExchange. The proposed modification not only complies with the XML schema, but also does not change the semantics of the XML. It fits perfectly for the needs of the discussed content-based heuristic. The problem of the XML marking was a significant increase of the content size. In case of a large XML message with many small tags, adding the correlation id attribute to every tag can even lead to doubling the size of XML. Such overhead is unacceptable in the scenario of the distributed JBI, where MessageExchanges are sent over the network. In order to mitigate it, XML marking takes place after the accept method of DeliveryChannel. It is a point where ME is received from the network and the processing is going to begin. After the processing, the send-out of the child ME is intercepted and correlation attributes are removed from the XML content, which is about to be sent on the wire. Thanks to such an approach, communication overhead is avoided. Unfortunately, there is some parsing overhead, but it does not influence the time and memory complexity of parsing in a significant way. As described, XMLHeuristic marks every XML content before the DeliveryChannel s accept and before its send methods. When CorrelationId attributes are found before particular ME sendout, it is a proof that this ME is correlated accordingly to the value of the CorrelationId. Requirement compliance of the chosen realization is presented in Table 5.3.

106 5.6. Persistence 94 Non-Func JBI Compliance Non-invasive Usefulness for JBI community Performanceoriented Ease of finding bottlenecks Best-practices Func CorrelationId Topology and Flow information Parameters extending Sierra Clustering requirements Table 5.3: JBI parameters requirement compliance Mechanism is fully JBI compliant. Thanks to AOP programming, solution is non-invasive for the JBI code. Message correlation is a long awaited feature in the JBI community. Designed heuristics fill this gap and make business transaction tracking possible. It was obvious from the beginning that the message correlation mechanism will probably deteriorate the efficiency of the bus. However, this feature was so important, that the performance requirements were brought to the background. Message correlation enables to track long-running business transactions which span across many Message Exchanges. It is one of the most important features of the performance profiling as a developer has the possibility to visualize the performance of each constituent flow and to find the slowest parts of the processing. Heuristics are not complicated solutions, but the code encompassed by these pieces of software was written according to the best practices of software design. The experiment provided the implementation of the message correlation mechanism which formed one of the most important requirements of GlassBus. Mechanism enriches the flow information and provides the business transaction monitoring. OpenESB does not introduce message correlation - designed mechanism extends monitoring features described in the Sierra list. Mechanism is compliant with the clustering requirements. 5.6 Persistence The section Application theory influence (3.5.2) concludes that real-time monitoring capability will be more complete if also some historical information is stored for the purpose of later exposition and analysis. The GlassBus architecture takes that into account and provides a flexible persistence layer which is composed of the following elements: database engine exact technology providing DBMS (Database Management System); persistence interface database engine abstraction allowing to interchange DBMSes; persistence domain the design of information that will be persisted; Persistence discussed in this section is related only to the GlassBus Engine, which accumulates all information intercepted in the system. The GlassBus Agent Persistence is relevant only in the context of communication middleware (5.7) and is handled by the MOM framework [28] used in GlassBus, assuring reliability of the agent-engine communication.

107 5.6. Persistence Database choice The main assumption about DBMS is not to use heavy engines that need separate deployment procedure and are accessible externally through TCP - i.e. regular client-server architecture. Examples of such engines: MySQL, DB2, Informix Dynamic Server, Firebird, Microsoft SQL Server, PostgreSQL. The priority was to use lightweight database solution which would allow to minimize management and assure straightforward procedure of GlassBus Engine deployment (it is described in Section 5.10). On the other hand, a complex and distributed enterprise bus can generate great amount of monitoring data which has to be persisted. In such a situation a mature, scalable clientserver DBMS would be more appropriate. In order to make such scenario possible, internals of DBMS have been hidden behind persistence abstraction interface (discussed in the next section), of which the only requirement is a JDBC-compliant database. Database engine layer can be therefore easily interchanged with any database capable of JDBC. Additionally, DBMS was not the main focus of the GlassBus research, therefore robustness and simplicity of embedded databases were more important than wide functionalities of client-server solutions. The above-mentioned embedded databases, chosen for GlassBus lightweight persistence solution, are libraries coupled with the application and accessed through a well-defined interface. In the case of Java technology, the interface is JDBC and the library coupling equip the JVM with all functionalities of DBMS. The following embedded persistence engines have been evaluated: Apache Derby 8 mature implementation gaining an increasing popularity (among others it is used in ServiceMix and ApacheMQ). It has been adopted by SUN as JavaDB and is shipped with newest JDK 6.0. It can be also used in a regular, not-embedded client-server mode. The database library size is about 2 MB. H2 9 It is the first branch of Hypersonic SQL project 10 created by Thomas Mueller, who has discontinued it in 2001 and started to implement H2 (Hypersonic 2) from scratch. The database library size is about 1 MB but it contains also the a Console web application, web server and other tools. HSQLDB 11 It is the second branch of Hypersonic SQL, which, contrary to H2, shares its codebase. The database library size is about 700 kb. Derby DB with its 2MB footprint seems to be the heaviest choice. In spite of it being favored by Apache, many users report performance problems and point to H2 and HSQLDB as less popular but interesting alternatives. Thomas Mueller - the creator of H2 - publishes DB tests [94] that allow for comparison with various DBMS, not only embedded ones. Part of the test is performed by H2 creator himself and the other part is the result of PolePosition benchmark [38], which is an independent benchmark test suite to compare database engines and object-relational mapping technology. Both benchmarks show that in most case H2 is a lot more efficient that Derby and tends towards being slightly faster than HSQLDB. What is also important, H2 outperformed regular DBMSes like PostgreSQL and MySQL

108 5.6. Persistence 96 A similar PolePosition benchmark has been published by the HSQLDB team. Unfortunately, H2 was not among the compared databases, but other (embedded or not) engines have been clearly outperformed. It is obvious that such tests can lack impartiality, but the introduction of a third-party benchmarking solution - PolePosition - lowers such risk. In addition, Thomas Mueller defends his performance results in a public discussion with Francois Orsini [39]. Mueller agrees that some tests can be impartial, but he was not able to find a test in which Derby would clearly outperform H2. Performance test results and feature comparison presented on H2 homepage 12 resulted in choosing H2 as the GlassBus s persistence engine. H2 provides a wide variety of persistence strategies which can be simplified to: in-memory persistence; file persistence; In-memory alternative is obviously faster but the lack of data continuity between engine failures discredited it, leaving the option of regular file persistence Persistence abstraction A layer of persistence abstraction was introduced in order to comply with software engineering good-practices, which favor decoupling implementation from the interface. The layer provides a regular Java interface. The signatures of the interface methods are defined through domain classes described later on. The purpose of this layer is to translate this object oriented interfaces to relational internals of underlying DBMS. As it has been mentioned before, the assumption is that DBMS faces with JDBC, which assures flexibility in changing the engine. It is an analogy to interchanging the implementation in terms of IT good practices. The goal could be realized in three different ways: 1. JDBC SQL by-hand the data in persistence domain is manually stored, retrieved and searched by means of regular SQL statements executed through a JDBC driver. 2. JDBC BLOBs the persistence domain is defined by classes of which instances are serialized and written to DB as BLOBs [102]. 3. ORM framework a third-party framework is used to map objects into relations in a standardized way. The second solution has a significant shortcoming - the data cannot be queried on the basis of the content. In case of a query, every record has to be read and deserialized. The specificity of the GlassBus domain makes it unacceptable. The first solution s implementation complexity depends on the complexity of the domain, which is not trivial. Additionally, any change in domain would trigger a modification in the code executing SQL. The SQL dialect of each database slightly differs, which makes the compliance with any JDBC-capable database more difficult. It could be mitigated by means of the Spring s Framework 13 JDBC support, which facilitates transaction handling and exception processing, but still it comes to reinventing the wheel building

109 5.6. Persistence 97 Object-Relational Mapping and that conflicts with the requirement of avoiding NIH. Therefore, it was decided to use an existing ORM solution. Object-Relational Mapping There are many ORM frameworks available today, which makes the choice difficult. Some of the most popular: Hibernate; Oracle s Toplink; JDO - Java Data Objects; All frameworks started with keeping the object-relational mapping configuration definition in an XML file. Obviously, mapping s XML schema was different from one framework to another, but it was not a problem because, along with the framework interchange, the mapping could be superseded. At the same time, the evaluation of EJB-2.1 in production environments revealed an enormous complexity of XML configuration in EJB-2.1 systems maintenance, described as XML hell. Java 5.0 introduction leveraged the potential of annotations allowing to bring ORM framework to a higher quality level. Each framework provided its own definition of annotations set used to annotate simple POJOs 14 and by means of that, configuring the object-relational mapping. It was the main reason for the ORM inter-framework non-compliance - POJOs annotated for one framework did not compile for others. At this tame JSR-220 Expert Group [40] worked on the EJB-3 specification, which was expected to simplify EJB-2.1, also in terms of persistence. Sun JPA FAQ comments it in the following way: (...) simplification of EJB CMP was not enough, and that what was needed was a POJO persistence framework in line with other O/R mapping technologies available in the industry. This POJO persistence framework was called Java Persistence API (JPA) and introduced a new standard of ORM frameworks. Its flexibility made it usable not only in EJB scope, but also for the purpose of regular J2SE applications. Java Persistence API JPA Advantages: The entire Java community can use a single, standard persistence API. It draws upon the best ideas from persistence technologies (Toplink, Hibernate, JDO). It was decided that combining several persistence technologies would be better than making one of the ORM frameworks (e.g. Hibernate) a standard. The end of incompatible non-standard persistence models for object/relational mappings. It supports a rich, SQL-like query language. It supports the use of pluggable persistence providers. 14 Plain old java objects

110 5.6. Persistence 98 Java Persistence effort triggered adoption of existing ORM solutions to the standardized JPA. It was not difficult because JPA was founded on those ORM frameworks. Right now, there are many commercial and open source JPA implementations available: CocoBase, DataNucleus, GlassFish, TopLink, EclipseLink, Hibernate, OpenJPA, Cayenne. It has been decided that JPA fits perfectly into the GlassBus persistence needs providing a simple and well-standardized solution. Most popular and mature JPA implementations are Hibernate and Toplink. Their quality is similar, but Toplink s Essentials (open-source community edition of Oracle s TopLink product) was classified as reference implementation the use of which will force full specification compliance allowing to avoid problems during future interchange of persistence providers. Spring enhancement The motivation for Spring Framework 15 was high J2EE complexity and belief that its development should be less difficult. Spring encompasses many functionally-varied modules which aid development of enterprise application by providing a framework for convenient realization of well-known enterprise good practices. The aim of spring is not to supersede J2EE (however some mechanisms can be used interchangeably with J2EE), but to enhance it. The support for Java Persistence API was introduced in Spring 2.0, allowing to leverage its ORM enhancements [41]: Ease of testing - ability to swap the implementations and the locations of the configuration files. Common data access exceptions - conversion from a proprietary to common DataAccessException hierarchy. General resource management - spring application contexts. Integrated transaction management - wrap mapping code with interceptor managing transactions. Spring-JPA causes dramatic increase of unit testing ease. JUnit extends AbstractJpaTests class from Spring, which allows to perform each test in a separate transaction that is roll-backed at the end. It provides the advantage of not changing the state of the database, which could be troublesome. Spring JPA enhancement has been evaluated in two ways: 1. with JpaTemplate - it integrates more tightly with Spring and therefore allows to instantly benefit from its mentioned ORM enhancements; exceptions are: translation and automatic transactions handling by means of the transactional EntityManager. 2. without JpaTemplate - leaving the developer with more freedom of choosing particular features. Transactions and exceptions support can be enabled (by means of XML configuration or annotations) or they can be handled in another specific way. 15

111 5.7. Communication middleware 99 It has been decided that the first option of Spring s JPA enhancement will be most beneficial for GlassBus. Requirement compliance of chosen realization is presented in Table 5.4. Non-Func NIH Env Performance Usefulness Open-source Bestpractices Func Historical data Table 5.4: Persistence Requirement compliance Avoiding implementation of own ORM solution. Solutions can be used in both J2SE and J2EE (stand-alone, app. server). H2 was one of fastest DBMS. JPA is currently mature and does not cause bottlenecks. The user does not have to install and configure external DBMS. All dependencies are open-source. Abstraction layer provides a well-defined demarcation between the system and DBMS, which allows changing persistence providers. Available through persistence. 5.7 Communication middleware Communication middleware provides an environment whose main goal is to enable manifold software artifacts to set up a conversation and transfer the data. As stated earlier in Section 4.3, GlassBus consists of two kinds of software artifacts: monitoring agents and monitoring engines, which form a monitoring topology. The general purpose of this experiment was to determine the best communication middleware for GlassBus in order to provide the interconnect on specified conditions Communication theory Currently, enterprise applications are characterized by a wide variety of numerous concepts for communication. This heterogeneity of communication mechanisms is caused by the manifold distribution requirements of enterprises. As stated in the second chapter (2) modern organizations consist of many distributed software elements which have to be integrated. Communication middleware which form the way of the system communication, is one of the most important elements. According to this article, [71] there are three levels of heterogeneity of distribution techniques: Level of communication modes - Many characteristics of a communication mode exist. The main one is the interaction pattern: synchronous or asynchronous interaction. However, multiple variations of these basic modes exist. Level of runtime features - Describes the additional runtime features that are provided by the communication framework. Level of products - Describes numerous products which incorporate capabilities described above.

112 5.7. Communication middleware 100 Characteristics of communication middleware Wide variety of characteristics of the communication middleware exists. These are important as they describe the main capabilities and emphasize the vital differences between solutions. The next subsection presents the main attributes that should be taken under consideration during the application design. Capabilities can be classified into the aforementioned levels: communication modes or runtime features. Synchronous versus Asynchronous Interaction - Synchronous communication is characterized by the strict request/reply communication pattern which it implements. Request is followed by the immediate response which is awaited by the requester during a blocking call. It is required that both sides of communication (requester and provider) are available and fully functional, at the same point of time, to perform the successive conversation. Asynchronous communication is characterized by the fact that the requester and provider are decoupled. Typically, requester sends a request but no immediate response is awaited. The sender can retrieve the result at any point of time, after the provider returns the result. It is possible thanks to the intermediary that administers the communication. It is NOT required that both sides of the communication (requester and provider) are available and fully functional, at the same point of time, to perform the successive conversation. Interface versus Payload Semantics - In general, communication is based on a bidirectional conversation in which both sides send and retrieve pieces of information. Typically, the requester invokes a transaction or operation the results of which are sent back to the client by the provider. The encapsulation of the invocation logic is normally performed in one of two ways. The first way is based on interface semantics. The requested transaction set is defined as a type of semantically rich interface which consists of procedures that have meaningful names which describe their purpose. Interface semantics provide the programmer with an intuitive interface which is easy to understand and use. The second way is based on payload semantics. In this case the invocation logic is encompassed in a message payload and sent to the provider which is determined only by its address. The provider has to recognize the payload and invoke associated business function. Payload semantics is usually used in a message oriented middleware (described later) which provides a set of get/put or send/receive functions. Tight versus Loose Coupling - tight and loose coupling are widely discussed in the IT world. There is no certain definition what it actually means. Coupling, which referes to joining things together can be discussed on different levels. The table, created by [71], entirely covers this issue.

113 5.7. Communication middleware 101 Table 5.5: Tight vs. Loose coupling discussion Level Tight Coupling Loose Coupling Physical coupling Direct physical link required Physical intermediary Communication style Synchronous Asynchronous Type system Interface semantics Payload semantics Interaction pattern OO-style navigation of complex Data-centric, self-contained object trees messages Control of process logic Central control of process Distributed logic components logic Service discovery and binding Statically bound services Dynamically bound services Platform dependencies Strong OS and programming language dependencies OS- and programming language independent Runtime Features - There is a group of additional runtime features which characterize the capabilities of communication middleware solutions. These are: security support, fault tolerance, load balancing, transaction handling, logging, usage metering, auditing, etc. Commonly known solutions There is a set of existing and commonly known communication solutions. They were invented on the trot and each of them was described as the next main, evolutionary step in the field of communication infrastructures. The list below briefly describes the main ones. Physical - The oldest and one of the most primitive methods. Programmers create a communication framework by direct implementation of data transmission. At the beginning programmers had to create network packets, send and receive them as well as handle the errors. Later on, higher-level protocols like TCP/IP and their software stacks reduced the amount of labour, however the abstraction level was still to low. Remote Procedure Call (RPC) - Provides the semantic of a local procedure call for distributed applications. The mechanism is similar to local procedure call, but the invocation is routed through the network to another host, which is the provider of the procedures. Distributed Objects - As the object-oriented programming emerged as the successor of the modular programming based on procedures and functions, the mechanism of RCP was implemented in the object-oriented manner. Its main goal is to make a local object accessible to another applications residing on various host across the network. Usually, distributed objects are administered by an Object Request Broker (ORB) which facilities the remote creation, location, invocation and deletion of the objects. Message Oriented Middleware (MOM) - Mechanism based on the idea of messaging queue. Message is the carrier of the data while the queue is the container which can hold and distribute the data. The queue, which acts as an intermediary, decouples the sender and the receiver. MOM usually provides a number of service levels, additional runtime features and transactional capabilities enabling the programmer to build dynamic, reliable, flexible and high-performance systems.

114 5.7. Communication middleware 102 High level solution - There are many other communication infrastructures which mediate between web servers, backend systems, databases, computing resources, resource pools and concurrent clients. These are, for example, transaction monitors or application servers. However, they are out of scope of the current experiment, as their applicability is not suitable for the demanded solution. The table presented below outlines the features of the aforementioned solutions with reference to the characteristics of communication middleware. It compares mechanisms, not their implementations, thus the classification has only demonstrative purpose. It is possible to implement a RPC solution which is loosely coupled or has a payload semantics, however it is out of scope to present untypical solutions. Table 5.6: Communication middleware comparison Feature RPC Distributed Object MOM Synchronous Blocking, synchronous, Blocking, synchronous, Asynchronous; Publishsubscribe vs. Asynchronous request-reply; Callback request-reply; Callback available available available Interface vs. Modular interface Object-oriented interface Payload semantics Payload Coupling Tight Tight Loose Runtime features Location, security Security, fault tolerance, load balancing, transaction handling, logging, usage metering, and auditing Abstraction over transport prot. Remote creation, location and deletion of the objects; security Not necessarily Yes Yes Realization As presented above, there are many communication frameworks which offer a considerable set of functionalities. However, there are many factors which influence the choice of middleware, since GlassBus has very specific requirements. The main nonfunctional demand is the high performance as the general assumption presumes that the efficiency deterioration of the monitored bus has to be minimal. To fully understand the value of performance, GlassBus architecture has to be reminded. Monitoring agents are the sensors of the system. They gather and process the monitoring information and then send it to the monitoring engine, which is the heart of topology. The number of conversations between agents and engine is huge as every Message Exchange causes one monitoring event. The most important fact is that the agent code is injected to the JBI code, which means that it directly enlarges the amount of work executed during the operation of JBI. The main place which could cause bottlenecks is the code of the information sender, thus the performance of communication middleware is so crucial.

115 5.7. Communication middleware 103 Short experiment which compares three communication mechanisms was performed to determine the performance of the solutions. The tested code was not tuned to include general performance optimizations, such as, for example, message batching; only native optimizations were performed. The experiment was supposed to show the performance of a pure, single sending operation. During the experiment communicates (10KB each) were send between sender and receiver. Only the sending time was measured, as the receiving time does not influence the performance of the monitored bus. The experiment had been repeated ten times, after that, average time was computed. The general purpose was only to have a quick glance on the performance issue. The results are presented in Table 5.7. Table 5.7: Basic comparison of the middleware performance Mechanism Implementation Mean Std. dev. Physical Java TCP sockets 2,0 sec. 0,28 sec. MOM ActiveMQ ,5 sec. 0,41 sec. Distributed Object RMI over TCP 10,3 sec. 1,13 sec. After precise analysis of GlassBus requirements as well as characteristics, features and performance of the existing middleware solutions it was decided that MOM would be used. It turned out that physical solution is the fastest, which was quite obvious at the beginning. However, after tuning the native properties of MOM, it was only two times slower than the test winner, which sets great stores on the performance and capabilities of GlassBus. Not only are high-end implementations of JMS nearly as fast as raw java sockets, but they also contain an incomparable amount of runtime features which provide GlassBus with a wide variety of capabilities that are described later on MOM Implementation provider During the performance test presented above, ActiveMQ was used as the implementation of MOM. It was chosen as it is a communication provider in ServiceMix. After getting promising results of the performance test, another experiment was performed in order to determine which JMS implementation is the best fitting solution for GlassBus. A considerably big set of products was evaluated: ActiveMQ 16 Open MQ 17 JBoss Messaging 18 JORAM 19 OpenJMS

116 5.7. Communication middleware 104 It quickly turned out that ActiveMQ is the most performance-oriented and mature implementation. High efficiency is its design milestone what greatly fulfills the most significant requirement of GlassBus. The next subsection presents the architecture and main features of ActiveMQ. Apache ActiveMQ is one of the most popular and powerful open source Message Brokers and Enterprise Integration Patterns providers. It is the implementation of the Java Messaging Service standard described in JSR-914 [60]. The abundance of features which it contains, causes that it is impossible to briefly describe all of them, thus the next section presents only the main capabilities which had an impact on the GlassBus architecture. Figure 5.3 presents the topology of ActiveMQ brokers. Figure 5.3: ActiveMQ Broker Topology 21 ActiveMQ Topologies ActiveMQ supports two modes of broker execution: Stand-alone - Message broker resides in a separate virtual machine which is located on the same or different host than some (or all) consumers or providers. Embedded - Message broker resides in the same virtual machine as at least one consumer or provider. ActiveMQ also supports two scenarios of communication governance: 21 Figure downloaded from [46].

117 5.7. Communication middleware 105 One broker, many clients - The typical scenario of JMS communication. Only one broker (stand-alone or embedded) exists and acts as a intermediary among clients which produce and consume messages. This is implementation of the hub and spoke architecture based on client/server mechanism. Many brokers, many clients (Store and forward network federation) - High-end, sophisticated JMS communication scenario which provides massive scalability of a large messaging fabric. It extends the hub and spoke architecture in which sole broker is the single point of failure. In this scenario brokers are connected together and form a network (or cluster) of brokers so that the system could survive failure of any particular broker, machine or subnet. Clients are allowed to connect to any broker in the network and fail over to another broker in case of failure. Network of Brokers provides store and forward mechanism to distribute messages from brokers with producers to brokers with consumers, which allows to support distributed queues and topics across a network of brokers. Normally, the connection between brokers is one-way. In order to create bidirectional conversation two connection have to be opened. As it generated synchronization troubles in some usage scenarios, from version 5.x of ActiveMQ, a network connection can be optionally enabled to be duplex. Auto discovery of brokers is supported, so clients can automatically detect and connect to a broker out of a logical group of brokers as well as brokers can discover and connect to other brokers to form large networks. Auto discovery is based on a hard-coded list of available brokers or on a multicast discovery protocol. ActiveMQ Clustering Clustering is a large concept and can span across many aspects of resource redundancy. Network of brokers mechanism, which is described above, can be discussed with reference to clustering, as it provides redundancy of the next-hop broker for producers and consumers. In case of a failure clients can fail over to another broker on the network to preserve the ongoing conversation. However, mechanism whose main goal is to distribute messages to clients connected to distributed brokers, has its limitations. It is not hard to imagine a situation in which operating broker gathers messages in queue for which, at the specified point of time, no consumers exist. The messages are stored, but not forwarded, since no consumer exists. If the very broker fails, the messages are unavailable. In addition, if the broker does not persist messages, they are lost forever. Therefore, ActiveMQ introduces two mechanisms which cope with this problem and form a disaster recovery (DR) solution. Pure Master Slave - This mechanism provides messages replication to a slave broker, so even in case of hardware failure of the master s machine, no messages are lost, as the slave broker takes over, making stored messages available with imperceptible delay. This is a shared-nothing, fully replicated topology with independent brokers. Replicated Message Stores - An alternative mechanism to master slave which contains slight differences. Two brokers (master and slave) are not fully independent, as they share a common message store, for example SAN. In case of a master s broker failure slave broker connects to the shared store of messages and takes over ongoing conversations. Recommended for brokers which have access to highly-available stores, such as RAID drives or SANs.

118 5.7. Communication middleware 106 ActiveMQ Transports ActiveMQ provides a set of transport protocols which enables the clients to connect to brokers or the brokers to connect to each other. The VM Transport - One of the most interesting transport ways. Allows clients to connect to the embedded broker which exists in the same virtual machine. The connection uses direct method invocation to omit the overhead of the network communication. The TCP Transport - Allows clients to connect to the remote broker using a TCP socket. The UDP Transport - Allows clients to connect to the remote broker using a UDP socket. The NIO Transport - Similar to TCP transport, however new version of Java IO (NIO) is used. The SSL Transport - Allows clients to connect to the remote broker using a secure, SSLencrypted, TCP communication. The Peer Transport - Provides a peer to peer network. Every client contains embedded, local broker and uses the VM transport to communicate with it. Broker establishes network connections to other embedded brokers residing on different clients, forming a peer to peer, JMS network. The Multicast Transport - Allows clients to connect to the remote broker using multicast. The HTTP and HTTPS Transport - Allows clients to connect to the remote broker using HTTP/HTTPS tunneling which helps to avoid problems with connectivity due to corporate firewalls. ActiveMQ Persistence ActiveMQ supports numerous persistence configurations. No persistence; JDBC Persistence without journaling; JDBC persistence with high performance journal; AMQ Message Store (with Kaha persistence); The most important option is the AMQ Message Store which is the default storage for ActiveMQ version five and above. Accorging to the authors [45], this embeddable and transactional message storage is extremly fast, reliable and easily recoverable. Description, taken from the ActiveMQ site [45] describes the store in details: Messages themselves are persisted in the data logs of the journal - with references to their location being held by a reference store (by default Kaha) for fast retrieval. References to messages are held in memory, and periodically inserted into the reference store to improve performance. The messages are stored in data logs, which are individual files, typically 32mb in size (though this is configurable, they can be larger if the size of a message is large than the file size). When all the messages in a data log have been successfully consumed, the data log file is marked as being ready to be deleted - or archived - which will happen at the next clean up period.

119 5.7. Communication middleware 107 ActiveMQ Performance The performance of ActiveMQ was meticulously evaluated in order to find the best configuration. Results of the performance experiments are presented below in Table 5.8 and 5.9. Table 5.8: ActiveMQ performance comparison - embedded brokers Sender Receiver Embedded broker; VM transport; Persistence Embedded broker; VM transport; Persistence disabled; disabled; Receiver was started after the sender had finished. Mean: 1,73 sec; Mean: 13,04 sec; Std.dev.: 0,34 sec; Std.dev.: 1,17 sec; Embedded broker; VM transport; Embedded Embedded broker; VM transport; Embedded Kaha Persistence; Kaha Persistence; Receiver was started after the sender had finished. Mean:5,48 sec; Mean:22,29 sec; Std.dev.: 0,63 sec; Std.dev.: 2,37 sec; Embedded broker; VM transport; Persistence Embedded broker; VM transport; Persistence disabled; disabled; Receiver was running simultaneously with sender. Mean: 5,02 sec; Mean: 10,12 sec; Std.dev.: 0,78 sec; Std.dev.: 1,37 sec; Embedded broker; VM transport; Embedded Embedded broker; VM transport; Embedded Kaha Persistence; Kaha Persistence; Receiver was running simultaneously with sender. Mean:16,9 sec; Mean:23,5 sec; Std.dev.: 1,81 sec; Std.dev.: 3,54 sec; During the experiment, which was similar to the one presented in Section 5.6, communicates, whose size was ten kilobytes, were sent between sender and receiver. Transmission time was measured on the sender and receiver side. Transmission time was averaged to the execution time of sender and receiver since the overhead of other operations performed by these parties does not have a negligible effect on the results. The experiment had been repeated ten times, after that, average time was computed. Table 5.8 shows the impact of persistence and communication parallelism on the performance of the sender and receiver. The fastest option is the solution with disabled persistence when operations of sending and receiving are performed one after another. However, it is hard to assume that in the JBI environment the communication between parties will not be simultaneous. The table 5.9 shows the performance comparison of the persistence mechanisms. Kaha persistence is the fastest option. But, what is worth mentioning, it deteriorates the efficiency of the sender almost two-fold. While comparing the results of tests presented in Table 5.8 and 5.9, it is easily observable that sender is ten-fold slower when it uses remote broker than when local broker is used at the server side (1,73 sec. versus 13,13 sec.).

120 5.7. Communication middleware 108 Table 5.9: ActiveMQ performance comparison - embedded and remote brokers Sender Receiver Remote broker; TCP transport; Embedded broker; VM transport; Persistence disabled; Receiver was running simultaneously with sender. Mean: 13,31 sec; Mean:13,78 sec; Std.dev.: 2,31 sec; Std.dev.: 1,97 sec; Remote broker; TCP transport; Embedded broker; VM transport; Embedded Kaha Persistence; Receiver was running simultaneously with sender. Mean:23,43 sec; Mean:22,96 sec; Std.dev.: 3,65 sec; Std.dev.: 3,93 sec; Remote broker; TCP transport; Embedded broker; VM transport; JDBC Persistence over Derby data engine; Receiver was running simultaneously with sender. Mean: over 1000 sec. Mean:over 1000 sec. Impact on solution architecture Experiments described above had a great impact on the architecture of the final solution. ActiveMQ provides Glassbus with an excellent flexibility and a lot of features which makes the architecture tidy and robust. Backstage work of the configuration is done completely by ActiveMQ. The most important aspect of the solution is that no changes in the code are required in order to switch the configuration scenario. Whole system is configurable through a set of XML files containing all settings and properties. As described in the installation Section 6.5 GlassBus supports two deployment scenarios: out-of-the-box distributed Figure 5.4 presents the middleware configuration in the out-of-the-box scenario. Both parties (sender and receiver) resides in the same virtual machine, thus they use single embedded broker and VM transport. This configuration ensures that the performance is not deteriorated since direct method invocation is used in order to omit the overhead of the network communication. Figure 5.4: Out-of-the-box scenario

121 5.7. Communication middleware 109 Figure 5.5 presents the middleware configuration in the distributed scenario. Each party uses embedded broker which all together create a peer to peer network of brokers. Embedded brokers are used, as, according to the performance tests (5.8 and 5.9), they provide the best sending efficiency. Sender code uses VM transport to communicate with the broker, what is extremely fast. The heaviest processing of the sending operations is deferred to the broker code, which makes an insignificant impact on the performance of JBI. Figure 5.5: Distributed scenario Figure 5.6 presents clustering of the engine broker. Monitoring engine is connected to the master using any kind of transport. In case of failure of the master broker, engine connects to the slave broker and continues the ongoing conversations what makes the system reliable and highly-available.

122 5.7. Communication middleware 110 Figure 5.6: Engine broker clustering Presented configurations are only the proposed options. Thanks to ActiveMQ, brokers can be configured in various ways, enabling the GlassBus administrator to choose the best fitting option to fulfill the specified requirements. Performance tests, which are presented earlier, showed that even sophisticated persistence mechanism deteriorates the efficiency of the solution. Thus, in GlassBus message persistence is disabled by default. It can be enabled at any time by a single change in the XML configuration file of each broker (Appendix: A.3), however communication reliability and disaster recovery is provided in a different way (Section: 5.7.4). Engine side broker is clustered what ensures that no messages are lost. Brokers on the agent side are not clustered since they reside in the JBI environment. The tight coupling with the environment causes that in case of a hardware or software failure of the JBI-host the whole system is down, which makes monitoring impossible - considering that clustering is useless JMS Sender JMS sender is a software artifact which is used to send information from agents to engines in the GlassBus environment. The list presented below outlines the tasks performed by the JMS sender during the execution of the send operation: 1. Get transmission-resource which consists of: JMS connection; JMS session; JMS message producer; 2. Perform message transmission; 3. Release transmission-resource;

123 5.7. Communication middleware 111 Two problems concerning the usage of the JMS sender had to be solved in order to design a robust and efficient solution. The first one regards thread safety. GlassBus agent consist of aspects which are injected to the code of JBI, which causes that the JMS sender is used simultaneously in many threads aggregated by the thread pools of the bus. The second one regards resource acquisition since the process of creating and releasing the transmission-resource from scratch (initiating the connection, negotiating the parameters, creating session, creating message producer, closing all resources, etc.) is very complex, heavy and inefficient. According to the best practices of software design there are at least two solutions which cover the presented issue: 1. Send and receive operations of the JMS sender are synchronized; JMS sender acquires single transmission resource at the creation time and uses it during the whole life-cycle. 2. Send and receive operations of the JMS sender are NOT synchronized; JMS sender uses transmission resource pool which copes with the thread-safe resource management; In order to choose the most efficient solution some performance tests were conducted. The results are presented in Table Tests consisted of two experiments which investigated the performance of proposed solutions on a multi-core machine. They were repeated ten times - then the average time was computed. Experiment number one - One and only one sending thread. Thread performs send operations. Message contains ten kilobytes of data. Duration of the whole sending process measured. Experiment number two simultaneous sending threads and 10 pooled transmission resources; Thread performs send operations (all together). Message contains ten kilobytes of data. Duration of the whole sending process measured (sum of the duration of the sending process in each thread). Table 5.10: JMS sender performance comparison Experiment Solution number one Solution number two One Mean: 3,77 sec. Std.dev.: 0,42 sec. Mean: 4,1 sec. Std.dev.: 0,31 sec. Two Mean: 3,95 sec. Std.dev.: 0,35 sec. Mean: 1,1 sec. Std.dev.: 0,12 sec. Experiment one, which tested the sequential (one-threaded) JMS sender, shows that the second solution is slightly slower. This is quite obvious since minimal overhead due to resource pool management exists. Experiment number two shows what is good under the hood of the second solution. It greatly improves the performance in the multi-threaded environment as the operations are executed simultaneously and do not have to share single transmission-resource, which greatly improves the throughput of the communication backbone. The efficiency is almost four-fold better. After evaluation of the performance tests solution number two was chosen. At the beginning JMSTemplate 22 provided the implementation of the JMS sender. However, later on due to performance bottlenecks, which were caused by redundant computation logic encompassed by this solution, proprietary implementation 23 was proposed. It is characterized by 22 org.springframework.jms.core.jmstemplate - Spring Framework pl.edu.agh.glassbus.agent.sender.jmssender A.2

124 5.7. Communication middleware 112 the fact that it is the simplest possible way of implementing the JMS sender. Auxiliary performance test which are off the record showed that the performance is between two and three times better. Usage of transmission resource pool is transparent for the JMS sender as all the backstage work is performed by the connection factory. JcaPooledConnectionFactory 24 from Java Connector Architecture (JCA) 25 container provides the high-performance implementation of thread pooling, resource pooling, transaction handling and consumption of resource adapters [63]. Summary of all features contained by the JMS sender implemented in glassbus: Sending mechanism - proprietary implementation of JMS sending logic. Send and receive operations of the sender are NOT synchronized; JMS uses transmission resource pool which copes with the thread-safe resource management; Transmission reliability - local JMS transactions (described in Section 5.7.4). Resource management - JMS transmission resource pools provided by JCA container. Broker failure protection - not provided, in case of host failure the whole monitoring system is down JMS Receiver JMS receiver is a software artifact which is used to receive information sent by agents in the GlassBus environment. The list presented below outlines the tasks performed by the JMS receiver during the execution of the receive operation: 1. Get transmission-resource which consist of: JMS connection; JMS session; JMS message producer; 2. Perform receipt of message; 3. Store message in a data-store; 4. Release transmission-resource; In order to provide implementation of the receipt mechanism which is fully compliant with the best practices of the software design, a short research was performed. It quickly occurred that Message Driven Beans (MDB) from Enterprise Java Beans (EJB) specification 26 fulfill all the requirements. Let us consider a brief overview of MDB capabilities [61]: Executes upon receipt of a single client message. Is asynchronously invoked. 24 org.jencks.amqpool.jcapooledconnectionfactory - Jencks Container MDBs were introduced in EJB-2.0 spec - JSR-19; EJB JSR-220 is currently the final version of the specification;

125 5.7. Communication middleware 113 Can be transaction-aware. May update shared data in an underlying database. Does not represent directly shared data in the database, although it may access and update such data. The problem is that MDBs are deployed in an EJB container within the application server and that is the only runtime environment which supports execution of MDBs. One of the non-functional requirements of GlassBus says that the whole system has to be operable in a stand-alone mode (without application server), thus this solution was rejected first off and a research concerning lighter mechanisms was undertaken. Finally, Message Driven Pojos solution provided by the Spring-based Java Connector Architecture container was chosen (Figure 5.7). Authors characterize the solution in such a way [47]: We have a lightweight, easily embeddable Spring based JCA container which allows us to provide MDB like functionality inside any Java application without requiring a full EJB container. This allows us to support message driven pojos using dependency injection for efficient JMS consumption together with pooling of the pojos using lightweight containers rather than relying on EJB. The JCA container also makes it easy to programatically at runtime create new message driven pojos rather than relying on the fixed deployment-time only option with EJB. The greatest advantage of this solution was the fact that it is natively supported by ActiveMQ, what makes the integration effortless. Previously JCA container was a part of ActiveMQ, it was decoupled later on. Figure 5.7: Message Driven Pojos architecture 27 Craig Walls [55] explains what are the main differences between MDBs and MDPs: MDBs must be run in EJB container - MDPs can be run anywhere, even in simple main method; 27 Figure downloaded from [59] and redesigned.

126 5.7. Communication middleware 114 MDBs require the lifecycle methods of javax.ejb.messagedrivenbean to be implemented; - MDPs have to implement only core javax.jms.messagelistener interface. MDPs support all features of Spring Framework: dependency injection and AOP support (including Spring s support for declarative transactions and Acegi s support for declarative security). MDPs XML configuration is more verbose than annotations used by MDBs. Transmission resources are pooled using JcaPooledConnectionFactory, the same mechanism as in the JMS sender what ensures high-performance of the message-listener to broker communication. The last aspect which had to be considered was the reliability of message transfer and storage. According to ActiveMQ resources [49] there are four possible solutions: 1. JMS auto-acknowledgement; 2. JMS explicit acknowledgement via Message.acknowledge(); 3. JMS local transactions; 4. XA distributed transactions; JMS is a reliable communication middleware. All messages are acknowledged, there is no JMS unacknowledged message transfer. No great functional difference between mode 1 and 2 exists. The only thing is that in explicit-acknowledgement mode the programmer has the possibility to acknowledge the message by explicit method invocation. Acknowledging a consumed message automatically acknowledges the receipt of all messages that have been consumed by its session, what provides a batching mechanism. The main difference between modes 1 & 2 and 3 & 4 is that mode 1 & 2 does not support rollback and redelivery in case of a failure during the message procession - what characterizes mode 3 & 4. It is also a common misconception that JMS local transactions are slow - there are no efficiency differences between modes 1 & 2 and 3, thus it is recommended to use JMS local transactions in order to provide the system with the rollback functionality. JMS local transactions also support message batching since one commit can acknowledge a group of consumed messages. The Table 5.11 summarizes all the features. Table 5.11: JMS transmission reliability Auto-ack Explicit-ack Local-transactions Reliable transfer YES YES YES Message batching NO YES YES Delivery rollback NO NO YES The forth solution (XA transactions) is completely different from the previously introduced. It is a distributed transactions mechanism which provides atomic transactions for multiple transactional resources. For example, receiver uses two data sources: JMS queue and database. Its main function is to store the content of the message in the database upon receipt. However, there is a small time window between the completion of the insert operation and the message acknowledgment. In case of a network/hardware/process failure inside that window at least two possible problems exist depending on the sequence of the performed operations:

127 5.7. Communication middleware database insert; 2.message acknowledgment - the message will be redelivered and the system will process duplicates. 1.message acknowledgment; 2.database insert - the message will not be stored, the content will be lost forever; XA transactions introduce the two-phase commit protocol which overcomes the mentioned difficulties, but, as usual, at the cost of performance, latency and complexity. The protocol requires multiple syncs to disk to ensure the proper recovery under all failure scenarios. What is worse, slow disk syncs have to be performed per transaction - batching is impossible. According to ActiveMQ developers [50], the best solution in that case is to use: Local JMS transactions (mode 3); Specified order of processing: 1.database insert; 2.message acknowledgment; Idempotent consumer which performs duplicate message detection in order to omit processing of duplicates; The pseudo-code solution reads as follows: 1 onmessage ( Message msg ) { 2 try { 3 if ( msg. notprocessed ( ) ) { 4 jdbc. commit ( ) 5 } 6 jms. commit ( ) 7 } 8 catch ( Exception e ) { 9 jms. rollback ( ) 10 } 11 } Not only does it solve the problems which are mentioned above but it is also very efficient and encompasses message batching, which is one of the most important features of the JMS receiver used in GlassBus. Summary of all features contained by the JMS receiver implemented in glassbus: Receipt mechanism - Message Driven Pojos provided by the Spring-based JCA container which is a lightweight equivalent of EJB container. Processing reliability - Equivalent of the XA distributed transaction which is performanceoriented and supports message batching (local JMS transaction, specified order of processing, idempotent consumer). Resource management - JMS resources are aggregated and managed by JCA container which provides, for example, JcaPooledConnectionFactory. It ensures the most proficient way of resource pooling and usage. Broker failure protection - Message persistence disabled due to performance overhead - failure scenario solved by redundant, clustered ActiveMQ broker - Figure 5.6.

128 5.7. Communication middleware Communication security Considering the aspect of security in all its bearings, it could be divided into three separate features: authentication, authorization and communication encryption. It is out of scope of this work to describe details of the mechanisms which are mentioned above, however it is worth mentioning that ActiveMQ provides all of these features on three different levels: broker, queue and message through various providers. Authentication - mechanism provided by JAAS 28 ; Authorization - provided by AcitveMQ proprietary authorization plugin; Communication encryption - provided by SSL transport - (Section: 5.7.2); More information could be found on ActiveMQ website [48]. Requirement compliance of chosen realization is presented in Table Table 5.12: Communication middleware - requirement compliance Non-functional Loosely coupled Provided by the usage of MOM. Flexibility of the underlying implementation architecture (ActiveMQ) provides GlassBus with a wide variety of con- figuration scenarios. Environment All software artifacts can be run in stand-alone mode as well as on the application server. Message Driven Pojos mechanism is an equivalent of required but rejected EJB mechanisms which can be run only in the EJB container. Performance Performance was the key aspect of the implemented solutions. A big set of performance tests (documented and off the record) was performed in order to ensure that the solutions are efficient and fast. NIH Implementation of features which are already provided by numerous software libraries, was omitted. The only exception is the own implementation of the JMS sender. However, the tested implementations were too slow. Best-practices Numerous researches and experiments were conducted to provide best solutions for problems which were met during the system design. Various design patterns are used in GlassBus. Open-source All dependencies are open-source. Simplicity Configuration of the communication middleware is straightforward. No changes in the code are required to support numerous deployment scenarios. All properties are configurable through XML files. Functional Clustering requirements Communication backbone do not impose any kind of limitations that could hinder the support of JBI clustering. In addition, ActiveMQ is used in ServiceMix, what makes the integration effortless in that case. 28

129 5.8. Management middleware Management middleware The first section of JBI Status Quo (3.3.1) delineates that the specification covers rather the management than monitoring concerns. The result of that is some management infrastructure that must be provided by every JBI implementation. The same section cites the JBI specification and proves that the bus s clustering and distribution are not standardized. It is therefore not surprising that the management framework defined in JBI is not suitable for scenarios of clustering in OpenESB and ServiceMix. OpenESB and ServiceMix analysis against this issue reveled non-trivial impediments. Both buses provide JMX interfaces by means of widely used RMI connectors, the requirement of which is binding to specific IP number and port. Additionally, the connector is protected with a user name and password that is configured by the particular container s administrator. It is easy to imagine a cluster encompassing dozens of buses, each with its own ip, port, user name and password of RMI connector. Such infrastructure is highly unmanageable, because central administration tool needs knowledge about RMI connector s details, which can be the subject of more or less often changes. Another shortcoming comes from the direct localization binding, which implies violation of the loosely-coupling paradigm. The depicted implications are not compliant with GlassBus requirements. Therefore, they are overcome by proper management middleware Possible options The previous section discusses several options of conducting communication between GlassBus components. The shortcomings of solutions based on sockets or RMI are presented. The proposed communication middleware subjects to a sole monitoring issue where information flow direction is mostly from the Agent to the Engine. The management slightly differs because its specificity requires both communication directions - querying and passing of management parameters from the Engine to the Agent is necessary. However, the difference does not prevent to incorporate conclusions about the MOM supremacy, which resulted in using MOM for the management realization. It allows to maintain the GlassBus loosely-coupling and removes the need of combining and integrating different middleware mechanisms. The result is a coherent monitoring/management middleware. The most convenient incorporation of MOM into management is simply using a communication middleware defined earlier - network of ActiveMQ brokers - and building management on top of it. After the final choice of the mechanism for conducting management, a detailed realization of passing management data has to be proposed. The following options have been taken into account: XML over JMS The data is enclosed in XML files, which are sent by means of communication middleware. It forces to define some protocol which would provide XML schema based syntactical definition and semantical interpretation enclosed in a parser logic. Separate parsers would have to be implemented on the agent s and engine s sides. The specificity of the management requires employing the request-response pattern, which is needed while engine gets or sets values of particular attributes. It implies introduction of mechanism realizing request-response through MOM, which comes to implementation of correlation between messages sent as requests and those sent in response. Additionally, features

130 5.8. Management middleware 118 of reliability: resending and response timeouts are needed for the mechanism s completeness. Such solution can be conceived as a capability of the method invocation built on top of message-oriented middleware. JMX over JMS This approach extends the first idea by adding the abstraction of JMX. This implies all the above-mentioned elements: communication protocol (it does not have to be XML), message parser, message interpreter and a message reliable correlator used to acquire request-response which is used to method invocation establishment. However, this solution is more management-oriented. It allows for natural extension of the mechanism built into ServiceMix and OpenESB. Existing MBeans are simply exposed by a MBean Server using JMS transfer protocol. Therefore, no additional logic mapping parsed XML to JBI MBeans is needed. Both proposed solutions are not trivial to implement, but additional extensions provided by the JMX over JMS results in its higher implementation complexity. However, it is undeniable that the latter solution provides a mechanism superior to the first one. Finally, it has been decided that GlassBus will be equipped with the JMX over JMS management middleware. The decision was motivated by discovering a framework that allows to conduct it Realisation The Lingo framework [42] has been identified as being ideal (and probably the only one) for realization of JMX over JMS. The framework describes itself as: lightweight POJO based remoting and messaging library, which extends it to support JMS. Among the wide range of capabilities, Lingo provides a synchronous request-response (like RMI) message pattern which has been identified as being crucial for both XML and JMX over JMS. Furthermore, Lingo provides a readyto-use mechanism of JMX over JMS 29, which allows to expose any MBean and to access it through a proxy on the client s side using bare JMX interfaces. The additional Lingo s advantage is ActiveMQ dependency. Employing ActiveMQ in communication middleware implies that using Lingo (management middleware) on top of ActiveMQ (communication middleware) is straightforward and does not cause any integration problems. Unfortunately, recently Lingo has not been under active development - last version was released in October It was probably the reason for some Lingo s defects that had to be solved in order to use it for the GlassBus s purposes: Lingo provides the capability of binding MBeanServer to specific Topic name by means of the environment properties: destinationgroupname and destinationservername. Unfortunately, the mechanism extracting these properties and applying them was broken and demanded patching the Lingo s code. Bug in waiting for response timeout in Topic-based communication resulting in hang-up. It was fixed by patching the Lingo code. Lingo was using ActiveMQ Topics for means of communication, which provides functionality of aggregating results from operations or queries on MBeans exposed by MBeanServer that Lingo connects to. Such functionality was not needed by GlassBus and topics have 29

131 5.8. Management middleware 119 been identified as being less reliable than queues. The Lingo code has been patched, allowing GlassBus management to be enhanced by the reliability of JMS queues. Some problems with ActiveMQ s temporary queues, which are used to conduct requestresponse pattern, have been identified. It has been discovered that ActiveMQ versions earlier then 5.x do not provide usable duplex communication feature, therefore connecting into network of brokers needed a significant amount of time. It has taken into account the design of GlassBus s Agent and Engine. After overcoming problems encountered in Lingo GlassBus management, the middleware was ready for establishing further design details. Architecture of management and monitoring When introducing the management requirements in this section only the GlassBus-Agents managing was taken into account. However, the JMX over JMS capabilities with method invocation among them, allow to realize broader functionalities. When designing Agent and Engine interactions it was concluded that Agents cannot simply start to send monitoring information prior to some previous preparation. The need of agent into engine registration procedure has been identified and JMX over JMS allows to handle this task. In order to achieve this, not only Agents, but also Engine has to expose its MBeanServer by means of communication middleware. Therefore, the final architecture assumes that every GlassBus component is equipped with the capability of both exposing and accessing MBeans in a loosely-coupled, message-oriented way. Additionally, each component is uniquely named by a simple string, which provides an addressing mechanism. It allowed to elaborate an algorithm sketch of the agent s and engine s behavior. The algorithm is presented in details on Sequence diagrams in Section Requirement compliance of chosen realization is presented in Table Non-Func Usefulness JSR-based Simplicity Loosely- Coupling Highly manageable NIH Open-source dependencies Func Cluster Table 5.13: Management requirement compliance The GlassBus user can avoid accessing MBeans of clustered JBIs by inconvenient RMI connectors. GlassBus retains all JBI MBeans thanks to forwarding them to the Engine. No middleware implementation was needed (apart from Lingo patching). Completely retained by using communication middleware. Achieved by Preserving JMX advantages. Not implementing functionalities already provided by Lingo. Lingo is open-sourced - uses Apache 2.0 license. JMX over JMS can manage federations and clusters easily.

132 5.9. Hierarchical Data Management Hierarchical Data Management JBI topology consists of many elements which remain in hierarchical relationships. For example: JBI instance is a part of a cluster, SA is a part of a JBI instance, SU resides in a SA, endpoint is a part of SU, set of operations form an endpoint, etc. GlassBus is supposed to trace, store and provide the external client with the topology information; therefore during the design some conceptual problem were faced and solved, in order to encompass all aforementioned capabilities. Three main problems which occurred are: What data structure should be used to store the JBI topology information during runtime? How to persist the topology information? How to provide the external client with the topology information? Possible solutions The introduced problem, which could actually be defined as the quandary of creating, managing and persisting hierarchical data, was never a trivial one. Java programming language does not provide the generic, hierarchical data collections such as graphs and trees, thus a surrogate, which is proprietary implemented, has to be introduced every time. The format and the nature of the data are crucial aspects while choosing the best fitting option. Let us consider the functional requirements of the needed solution on the lower abstraction level. It is possible to divide the problem into two separate parts: Data management: Data kind - topology information remaining in hierarchical relationships; Data type - Java primitive data type, mainly String; Data operations - addition of new data to the specified place (parent, child), searching (eg. all disabled SUs), modifying (eg. changing the state of SU), deleting (eg. removing undeployed artifacts); Data exposition - topology information is exposed to external clients using primitive data format; Data persistence - topology information is gathered by monitoring agents; then information is sent to monitoring engine which stores it in a datastore; One of the first-thought solutions is to code the own, proprietary implementation. There would be two steps of the code design: implementation of the generic data structure and its functionalities as well as implementation of the domain classes and the persistence code. The solution would be very flexible as it can be tuned to match all the distinctive requirements. Data structure would contain all demanded methods, providing sophisticated management functionalities and the persistence would be also incorporated, forming a monolithic solution. However, presented idea does not comply with the best practices of software design. Usually, implementation of a proprietary solution of a commonly known problem is expensive, mundane, redundant and error-prone. It could take hours to fix all the bugs and tune the performance to the long-expected

133 5.9. Hierarchical Data Management 121 level. Proprietary implementation is also often tightly coupled to a specified set of technologies such as persistence vendor etc. To sum up, it is very hard and time-consuming to design a generic all-in-one solution and whenever it is not urgently required own implementation should be avoided. Thus, on that basis, it was rejected first off. Another option was to use a polished, production software library which would be tested, reliable and functional. It complies with the paradigm of writing as little code as possible, providing the programmer with the working, tested and supported solution. The best possible option in the presented case would be a combination of a hierarchical data model [62] with a hierarchical data store [74]. After performing a technical reconnaissance it was stated that there is no commonly used product which fits all. In addition, it was hard to find any popular, tested and mature implementation. Many tutorials on the Internet ([72], [54]) instruct how to build hierarchical data frameworks but they provide no implementation at all. There are some approaches to use the javax.swing.tree packages as the tree provider in Java. However, this data structure is oriented to Swing programming combined with JTree, which in our case implies a lack of useful capabilities and a lot of redundant and unusable features which impede the proper usage. Finally, a best fitting option was found. It occurred that many developers recommend the usage of Document Object Model (DOM) library as the data structure. It is mainly known as a XML parser but could also perfectly serve as a document creator. After evaluating some implementations no disadvantages of this approach were found. List below presents all the features of the DOM approach: DOM libraries contain a wide variety of functions for hierarchical data management. Structure of the tree can be modeled as a XML scheme which forces the proper format of the document. DOM provides XPath query language which is similar to SQL query language. It enables to perform parameterized searches across the elements of the document in order to find specified data. DOM provides lightweight approach of persistence since all the data can be persisted into a physical XML file. It works out-of-the-box and no further configuration is required. This persistence model is also independent of data model which is a great advantage. In the presented case data is gathered as primitive data types and exposed to the external clients as primitive data types (mainly Strings). The best approach would be to omit the creation of value-objects hierarchy, often called as value-object anti-pattern, to operate on the data. DOM gives that possibility: XML scheme forces the proper semantic format of the structure, thus primitive data types can be stored and retrieved from data store - troublesome data mapping is avoided Realization JDOM implementation as the hierarchical data provider; XPath as the functional query language for hierarchical data;

134 5.10. Installation and runtime environments 122 XML file as a hierarchical data store; Requirement compliance of chosen realization is presented in Table Non-functional Best-practices Open-source Functional Topology and Flow information Table 5.14: Hierarchical data - requirement compliance Design is compliant with best-practices of software engineering. All dependencies are open-source. This experiment is a part of topology functionality realization Installation and runtime environments It has been stressed many times that the essential GlasBus s goal is JBI compliance. This experiment, however, does not relate directly to it. The aim of this experiment is to provide a solution for deploying GlassBus on enterprise infrastructure built from either ServiceMix or OpenESB buses. The requirement for deployment is to create one user-friendly, coherent tool that will overcome difficulty of the two different buses environments: application server and stand-alone. Additionally, the tool has to support both deployment scenarios: single monolithic bus; distributed federation of buses; Deployment assumptions As it has been introduced earlier in Chapter 4, GlassBus encompasses two main components: GlassBus-Agent and GlassBus-Engine. The Agent has to be deployed to each bus, the monitoring of which is desirable, while the Engine s assumption is the deployment to some external resource dedicated to gathering, processing and exposing monitoring data. Agent integration is combined of the following steps: 1. installing all needed dependencies; 2. installing appropriate aspects for each bus; 3. configuration of information acquiring mechanism - AspectJ weaving described in Section 5.3; 4. configuration of logging framework; Additional configuration of Agent s initialization procedure is not needed because the JBITopologyComponentInterceptor aspect (Listing A.5) described earlier (Section 5.4) listens for topology changes and launches GlassBus-Agent as soon as one of bus s JBI components initializes. Integration details of GlassBus-Engine will depend on the particular environment.

135 5.10. Installation and runtime environments Deployment scenarios The GlassBus s strategy of installation and integration evolved during the analysis of all possible buses deployment scenarios. This section depicts this evolution, discussing how ServiceMix and OpenESB can be deployed. Single monolithic bus This is the primary scenario, where only one bus instance is employed. The OpenESB s application server oriented approach was the motivation for the initial idea of enclosing GlassBus- Engine in the WAR package. Unfortunately, it imposed a non-trivial deployment to ServiceMix. OpenESB deployment of both Agent and Engine is straightforward, but ServiceMix s stand-alone container implies installation and configuration of another application server just for the purpose of the GlassBus-Engine WAR package. Figure 5.8 depicts the described scenario. Figure 5.8: GlassBus deployment to Single monolithic bus As it is grasped in Figure 5.8, in the case of ServiceMix there is a disadvantage of external agent-engine communication overhead, but more important is the inconvenience of additional application server installation. The GlassBus usefulness requirement priority made such inconvenience unacceptable. In order to overcome it, GlassBus was equipped with the Out-of-the-box deployment scenario, which does not require installation of any additional elements. GlassBus-Engine WAR package encloses JAR-embedded logic which is initialized by WEB application. The Out-of-the-box GlassBus removes WAR dependency by installing the Engine s logic directly into either ServiceMix or OpenESB (simple placing in a lib directory) and by adding an aspect which initializes the Engine. The procedure of Engine initialization is analogical to the Agent s - the GlassbusEngineIgnitor aspect (Listing A.7) pointcuts the JBI s ComponentLifeCycle.init(ComponentContext cc) method and starts GlassBus-Engine along with the first JBI component. Glassbus components disposition available through the described approach is depicted in Figure 5.9. As it is shown, there is no need for redundant application server in the case of ServiceMix. In the case of OpenESB, the Application Server is still available, but its environment is not used directly by GlassBus-Engine, which is embedded into the JBI container.

136 5.10. Installation and runtime environments 124 Figure 5.9: GlassBus single out-of-the-box deployment The Out-of-the-box GlassBus s feature allows to maintain installation simplicity and removes the need of external communication in the single monolithic bus scenario. The feature makes possible, but does not impose, the in-bus Engine deployment. The user can choose the most convenient scenario. In ServiceMix in-bus option is obviously more convenient, but in the case of OpenESB it is difficult to judge what would be more preferable by the user. Distributed cluster of buses Complying with the requirement of cluster monitoring support, the deployment mechanism allows to integrate GlassBus into existing OpenESB or ServiceMix clusters (cf. clustering in Section 3.3.4). Mechanisms described in the single monolithic bus scenario are perfectly enough to support the clustering case, which can be covered in several possible ways: OpenESB 1. GlassBus-Agent is deployed to each instance of OpenESB in the cluster. GlassBus- Engine WAR package is deployed to an external application server. 2. GlassBus-Agent is deployed to each instance of OpenESB in the cluster. GlassBus- Engine is deployed in the Out-of-the-box mode to one of the clustered application servers, which becomes the main monitoring node. ServiceMix 1. GlassBus-Agent is deployed to each instance of federated ServiceMixs. Separate not federated ServiceMix instance is used just for the purpose of GlassBus-Engine Out-ofthe-box deployment. 2. GlassBus-Agent is deployed to each instance of federated ServiceMixs. GlassBus-Engine is deployed to a separate external Application Server.

137 5.10. Installation and runtime environments GlassBus-Agent is deployed to each instance of federated ServiceMixs. GlassBus-Engine is deployed to one of the instances from federation, which becomes the main monitoring node. The first option in OpenESB and the first two in ServiceMix are probably more adequate to larger buses clusters where scale of monitoring information requires separate resources for gathering and processing. The latter options are suitable for small clusters where they are applied by promoting one of the nodes to monitoring center. The user can freely choose the most convenient scenario Realization It has been decided that the described functionalities will be covered by a separate component: GlassBus-Installer. It is enclosed in an executable jar file allowing therefore for a straightforward downloading and installation. GlassBus-Installer features the following functionalities: While executed in the root bus directory (either ServiceMix or OpenESB) or executed with path argument pointing to such a root, it can automatically detect the bus type and GlassBus components already deployed to it. Installation procedure is performed interactively - the user is questioned about components he/she wishes to install and to initially configure. the following components can be installed: GlassBus-Agent into ServiceMix, OpenESB; GlassBus-Engine into ServiceMix as out-of-the-box, OpenESB as out-of-the-box; The deployment of GlassBus-Engine WAR package is left to the user. It can be achieved by any means available in the particular application server. GlassBus-Installer allows for instant out-of-the-box deployment. The user downloads the GlassBusinstaller jar and executes it in a bus s root directory. While questioned he replies that both agent and engine should be embedded into the bus. In two minutes time everything is ready. In distributed scenario GlassBus-Agent has to be manually installed to each node that is going to be monitored. Fortunately the GlassBus-Installer simplicity allows to easily script installation task and execute it in automatic manner (for details please refer to Section 6.6). Requirement compliance of chosen realization is presented in Table 5.15.

138 5.11. Monitoring interface 126 Non-Func Env Performance Usefulness Best practices Func Clustering Table 5.15: Deployment Requirement compliance Out-of-the-box and WAR GlassBus-Engine options cover both environments. In one instance the scenario of ServiceMix external communication overhead can be avoided thanks to the out-of-the-box mode. Single, interactive GlassBus-Installer component for all functionalities (apart from the WAR deployment, which is covered by application server). Possibility of automatic scripted installations. The user can choose the most suitable option of GlassBus integration. Avoiding employing not necessary resources - removing demand of Application Server for ServiceMix clustering. Supported through various deployment possibilities - most suitable can be chosen Monitoring interface GlassBus monitoring system gathers a wide variety of information and characteristics about ESB environments compliant with JBI. In order to enable access to these information a formal, stable, and agreed-upon interface (API) has to be formed and exposed to external clients. Applications can use this API to directly access the application layer of GlassBus and use the business logic of the system. The problem of choosing the proper technology of the GlassBus API exposition is discussed across this section Possible solutions Exposition of the business interface can be discussed on different levels. During the design of computer systems there is always a problem of defining and determining the borders of the system - the point of contact between system and the environment. In GlassBus, it was decided that the system would expose business application interface (BAPI) as well as graphical user interface (GUI). Text or command-line interface will not be provided. Business API There are many common ways of business API exposition. They can be classified into two main groups: local exposition - plain old java interfaces which expose functionalities of a library/system in the same Java virtual machine which the system resides in; remote exposition - exposes functionalities remotely, which means that they can be accessed from any other host across the network; eg. RMI, CORBA, Web-Services, etc.; There were two communications models which support what was required in GlassBus: Request - Response; Publish - Subscribe (Notifications);

139 5.11. Monitoring interface 127 After a short evaluation, it was decided that Java Management Extensions technology would be used. Not only does it fulfill all the requirements, as all the communication models are provided, but it also provides the abstraction over communication protocol. There is a set of supported, most popular communication protocols, which can be extended in an unlimited fashion using Generic Connection Architecture. Furthermore, the chosen technology is also oriented to monitoring and management, which greatly fits in. Coming to an end, simple, standardized tools of generating statistics and charts from JMX beans exist, which gives the possibility to avoid redundant code creation. It is also worth mentioning that GlassBus architecture does not limit the exposition of business logic to any kind of technology. The API is designed according to the best practices of objectoriented software design, which enables a developer to expose the API in a straightforward way using any kind of proper technologies. Graphical User Interface The realization of graphical interface is an additional feature of GlassBus. It is an end-to-end interface which enables an external user to interact with a system using graphical icons and indicators. GUI can be built from scratch or built on the top of a Rich Client Platform (RCP). According to many sources, there are many advantages of RCP [69]: well tested, solid foundation; provides: windows and panels management, plugin support, messaging infrastructure, service discovery, etc.; incorporates: help system, log system, etc.; It was decided that Netbeans Rich Client Platform would be used [69] since it is well documented and many tutorials exist. This approach would be also compliant with OpenESB as its all graphical tools are provided as Netbeans plugins Realization BAPI - exposed by Java Management Extensions beans which are oriented to monitoring information exposition. It is referred in the rest of thesis as External monitoring interface. GUI - realized on the top of Netbeasn RCP as a netbeans plugin. Requirement compliance of chosen realization is presented in Table 5.16.

140 5.12. Logging 128 Non-functional JBI Compliance Environment Best-practices Open-source Functional Notifications Table 5.16: Monitoring interface - requirement compliance It is not adequate to speak about JBI compliance in that case. However, this solution is compliant with the architectural idea of JBI paradigms which define JMX management and monitoring beans. JMX beans do not rely on JEE. They can be exposed in standalone or application server environment. Design is compliant with best-practices of software engineering. One of the JMX main function is to expose monitoring and management information. All dependencies are open-source. JMX notification mechanism Logging Every computer system has to be equipped with a well-designed logging facility, which is used through its whole lifetime. Increasing complexity of IT systems forces improvements in the logging mechanism, the role of which becomes more and more important. Enterprise solutions complexity is definitely higher than average, therefore poor logging design or its susceptibility to defects are not affordable. Logging facility exploitation in an IT system can be demarcated by the point of releasing the system to the production environment. It results in the following exploitation phases: Development In the development phase logging is expected to provide verbose information related to the code flow and data, which is processed, stored or passed between particular system entities. The verbosity level has to be adjustable and scope-sensitive, because sometimes only limited system scope needs analysis or verification. Production When the development finishes, the system is launched to production and logging verbosity is not needed anymore. There should be an easy way of disabling it. The code performance cannot be affected by parts related to logging of the one needed earlier, debugging information. In the production only the significant information should be logged, which encompasses critical failures, major errors, warnings and notifications related to the important system state. The amount of the logged data has to be limited to mitigate information noise. GlassBus is a monitoring solution for systems conducting enterprise integration. Therefore, the logging standards have to be maintained at least in the enterprise quality. Additionally, tight integration with ServiceMix and OpenESB implies the requirement of conforming with their logging policy. It is inevitable for effective logging in both development and production phases of the bus instrumented with GlassBus.

141 5.12. Logging Possible options The described logging facilities requirements are well-known and there are many logging frameworks for Java language which comply with them. The following options have been considered for the purpose of employing in GlassBus: 1. Log4j 30 the most popular Java logging framework. It can be treated as a logging standard which introduced terms like: Logger, Formatter, Appender, Filter, Logging Level. 2. Java Logging API 31 functionality of JRE introduced in Java 1.4. It was build on the basis of Log4j, but unfortunately it lacks many Log4j s functionalities - e.g., there is a very limited number of appenders. 3. Own proprietary solution implementation of simple logging framework complying with GlassBus s requirements. Implementation of own logging solution does not comply with Best practices and does not fulfill the NIH requirements. There is no need of reinventing the wheel and implementing logging frameworks, which are already available. Log4j seemed to be the best choice, but the alternative of logging embedded into JRE could not be easily disregarded. Before choosing the final logging framework, an analysis of logging in ServiceMix and OpenESB has been performed: ServiceMix The bus uses the Log4j logging framework, but in an indirect way. The basic logging interface is provided by Apache s Java Commons Logging (JCL) 32, which is actually not an implementation of a logging facility, but rather a thin abstraction layer which allows to employ underneath any logging framework. OpenESB The Sun s solution employs the Log4j s less powerful alternative - JDK Loggers, which is used in both GlassFish and OpenESB. The differences between buses logging frameworks imply significant difficulty of equipping GlassBus with a solution that could be easily integrated with them. An interesting JCL feature is the possibility of deferring logging framework choice, therefore any of the aforementioned frameworks could be finally chosen and optionally further replaced according to the needs. Such approach most completely comply with the GlassBus s requirements - for example, the framework s abstraction layer is a common design pattern, which is recommended by software engineering best practices. GlassFish Documentation 33 mentions the JCL usage, but does not recommend it: Application components may use the Apache Commons Logging Library to log messages. The platform standard JSR-047 API, however, is recommended for better log configuration. Despite this fact, it has been concluded that using commons logging allows to overcome the problem of the buses logging frameworks differences. Therefore, it has been decided to employ JCL in GlassBus. It gives the possibility of integration with Log4j in ServiceMix and Java Logging API in OpenESB

142 5.12. Logging Realization Logging experimenting with JCL was restricted to an integration with Log4j. It was perfectly enough for the purpose of this research and simply left the possibility of later integration with loggers from Java API in the scope of the solution s improvements and future work. It implied integration of Log4j into GlassFish, which turned out to be supportable by the container. Following the steps on GlassFish s wiki 34 Log4j has been integrated with ease. As described in sections: Information acquiring mechanism 5.3 and Installation and runtime environment 5.10, GlassBus integration is performed by placing its jar-embedded logic into the bus s lib directory. It does not allow to cleanly separate GlassBus logging from the configuration of the bus. It is not a problem in OpenESB, where Log4j is used only for GlassBus. Therefore, there is nothing to interfere with. Unfortunately, ServiceMix s every component heavily depends on Log4j, which is why some integration strategy is indispensable. It was concluded that the cleanest solution will be to simply modify the ServiceMix s logging configuration by adding appenders and loggers dedicated to GlassBus. A sample final configuration can be looked up in Listing A.10. Such an approach removes the requirement of embedding logging configuration into each GlassBus artifact, which would be totally unmanageable because of the sealing into jars. This solution does not relate to GlassBus-Engine Web Application. Application servers provide flexible way of separating logging internals of web application from that of the server. Logging of GlassBus-Engine web app. was solved by embedding Log4j dependencies into WAR package therefore achieving total separation of logging infrastructure. The ease of WAR logging embedding in the case of GlassBus-Engine did not prevent the full understanding of the complexity of the application servers logging environments, the essentials of which are described in Ceki Gulcu s Supporting the Log4j RepositorySelector in Servlet Containers article [86]. The author explains that the separation of WEB application logging is not trivial because simple embedding of Log4j libraries in WAR can be overridden by libraries installed in the server scope (actually, it depends on particular server implementation). The author proposes to override Log4j s RepositorySelector, which would allow to maintain separate loggers hierarchies for each web application. It seemed to be the most complete solution. Currently, however, simple Log4j embedding complies with all GlassBus requirements, which is why it has finally been chosen. The solution can be further improved in the future according to suggestions of Ceki Gulcu. Integration of JCL was straightforward thanks to the automatic detecting Log4j availability and defaulting to it. No additional configuration was needed. JCL usage was restricted to two classes: LogFactory and Log. Equipping CLASS with logging was achieved by a schematic Log attribute: private static final Log log = LogFactory.getLog(CLASS.class); Java commons logging suggests logging best practices, which have been incorporated into Glass- Bus: Code Guards each logging of INFO and lower level is preceded by checking if such logging should be performed. It allows to avoid log arguments evaluation when logging should not be performed. 34

143 5.13. Conclusion 131 Messages Priorities ensuring that the log message severity is appropriate to its content. Default Message Priority by default, messages of DEBUG and lower priorities should not be logged. Info or Debug maintaining a reasonable balance between the amount of info and debug messages, so that information noise and lack of information in production environment can be avoided. Logging Exceptions assuring that exceptions are logged when necessary and that sometimes it does not eliminate the requirement of rethrowing them. Requirement compliance of chosen realization is presented in Table Non-Func Env Usefulness Best practices Open-source dependencies Simplicity Table 5.17: Logging Requirement compliance JCL and Log4j allows to cover both environments. JCL does not need any configuration and Log4j can be easily adjusted through the main container s configuration modification. GlassBus-Engine WAR logging configuration is separated and also easily adjustable. JCL abstraction layer is in line with IT best practices. Additional logging specific practices have been incorporated. JCL and Log4j are both licensed on Apache License Version 2.0 from January Introduction of complex RepositorySelector has been avoided thanks to a simple Log4j in-war embedding Conclusion Albert Einstein s maxim introduced in the chapter s beginning was inspiration of so called Experimental Driven Development approach presented throughout the GlassBus Research. Subject s complexity implied that only well documented experiments would be convincing proof for choosing best realizations of discussed problems. The most important outcome of this chapter is related to particular realizations justification, which was provided by means of requirements compliance extracts. Elements compliance can be summarized in following way: Information acquiring mechanism - Acquiring information by means of AOP crosscut allows to avoid the need of the JBI container s code modification. Simple installation of jars with aspects and running container with AspectJ weaver is sufficient for proper instrumentation. JBI monitoring mechanisms - The flexibility of AOP crosscut allowed for a straightforward design of JBI monitoring. It is based solely on methods from JBI interfaces documented in JSR-208, which results in the full compliance with JBI. Correlation heuristics - The previous two mechanisms allowed for realizing the heuristics that are fully JBI-compliant and non-invasive for the JBI code. Priority of JBI compliance pursuit resulted in mechanisms that do not always function properly - it was an inevitable trade-off.

144 5.13. Conclusion 132 Persistence - This element had not the highest priority. The main goal was to maintain the level of abstraction which will assure the ease of future replacement. The performance of chosen lightweight H2 DBMS was sufficient for the purpose of the research. Communication middleware - Fortunately, it was possible to propose a fully loosely-coupled messaging backbone. It was very important for the support of OpenESB and ServiceMix clustering. Additional advantage of the achieved loosely-coupling is an unequivocal demarcation between the backbone and the code built upon it, which is not aware of the backbone distribution details. Even more important was the achievement of high sender performance, which allowed to avoid significant performance deterioration of the running JBI container. Management middleware - The fact that the management is built on top of the communication middleware implies the inheritance of its all advantages. The most important functionality is the abstraction of the method invocation, which is realized by Lingo s JMX over JMS. Hierarchical Data Management - Software engineering best practices was the main requirement, which have accompanied the realization of this element. It resulted in adapting well known XML solutions fitting perfectly for the management of hierarchical data. Installation and runtime environments - The main goal was a straightforward installation and configuration. It was the reason for introducing the out-of-the-box installation type, which not only provides installation ease, but also boosts the single bus scenario performance in a significant way. Monitoring interface - In order to provide mechanism coherent with the current JBI management layer, the JMX technology has been chosen. The flexibility of JMX connectors implies that it is currently the most compatible way of exposing the monitoring and management interface. Logging - Chosen Java Commons Logging provided the widest range of possible underneath logging frameworks. It gives the flexibility of adopting to ServiceMix and OpenESB, which use different logging libraries. Additionally, GlassBus maintains the best practices of logging. Strict separation to different experiments is the reason for blurring the complete architecture, which encompasses all discussed mechanisms. The next chapter tries to fill this gap by depicting a synthetic GlassBus view and pointing out the compliance of the mechanisms discussed above.

145 Chapter 6 GlassBus Implementation For the things we have to learn before we can do them, we learn by doing them. Aristotle, Ancient Greek Philosopher, Scientist and Physician. 6.1 Preamble The previous chapter went through each element of the GlassBus solution, composing a strictly analytical view of a system. Possible options, motivations and chosen realizations have been outlined, which provides a good basis for the synthetic presentation of the GlassBus s implementation details enclosed in this chapter. GlassBus implementation is opened with an overview of the architecture, which gathers all elements discussed in the previous chapter and forms a complete distributed architecture of the GlassBus solution. The architecture diagram provides a good reference for the content of the next section, which focuses on the details of the particular GlassBus parts. Two main sections are related to a logical and physical structure. The more important logical structure describes various layers traversed by monitoring information. The information is enclosed in a domain model and is gathered at the level of the GlassBus Agent. It is further passed to GlassBus Engine for aggregation and analysis and is finally exposed by an external interface. This process is described throughout Sections 6.3.1, 6.3.2, 6.3.3, For the purpose of readability, the information throughout the logical structure is divided into topology attributes and messaging attributes. The whole process can be managed by means described in the management and selectivity model (6.3.5). For a better understanding of the logical responsibilities of every component, invocation flows have been provided as the summary of the Logical structure section. The less important physical GlassBus structure describes the organization of the code and the relations between artifacts of the built process and the system components. The two further Sections (6.5 and 6.6) are related to GlassBus configuration and installation. Fortunately, these processes are fairly straightforward allowing for a compact and informative description. The chapter closes with an implementation status that points out the high complexity of JBI monitoring subject and discusses its influence on the architecture and implementation. Finally, GlassBus performance tests that prove usability for both ServiceMix and OpenESB JBI implementations are introduced.

146 6.2. Architecture Overview Architecture Overview The Diagram 6.1 shows the complete architecture of GlassBus, which combines the division into functional modules with their deployment and intercommunication. GlassBus encompasses two components: GlassBus-Agent; GlassBus-Engine; In order to monitor the cluster of JBI containers, the Agent has to be installed into each container, while the Engine is deployed only once either to the Application Server or by being embedded into the JBI bus (for details cf. Chapter 5). As depicted in the diagram, the components are interconnected by the Network of ActiveMQ brokers, which provides loosely-coupling, security and is extendible to any amount of monitored JBIs. It could be conceived that the GlassBus middleware is a bus constructed in parallel to the federation of the monitored JBI containers. Not every GlassBus-Agent is aware of other Agents connected to the networks of brokers, but an intelligent correlation built into the Agent s interceptors is able to track intra- and inter-containers in JBI business transactions (described in correlation heuristics from Chapter 5). The information gathered and correlated by interceptors is further enclosed in domain classes (presented in Section 6.3.1) and forwarded to GlassBus-Engine using the JMS Sender, which employs the GlassBus middleware. The information is received and processed by the JMS receiver module, which passes it to the Persistence module. Then, the data is properly prepared and verified with the information already stored. The abstraction layer - JPA - is used to persist the validated data. Persisted information is exposed by an external JMX interface. In parallel to transferring and processing the chain of monitoring data, an interaction between the Engine s and Agent s management layers is continuously performed. At the beginning, the Agent has to register in the Engine by means of one of its MBeans. The Engine is then capable of forwarding the Agent s JBI MBeans and exposing them to the Monitoring Administrator. Additionally, the Agent and the Engine keep querying each other to assure mutual accessibility. The whole management is realized through the JMS over JMX - Lingo framework.

147 6.2. Architecture Overview 135 Figure 6.1: GlassBus complete architecture

148 6.3. Logical Structure Logical Structure The Diagram 6.2 depicts a high level view on the GlassBus logical structure. The JBI data, which has been identified as important for the monitoring purposes, has been modeled in the Domain model. The model is used for two purposes: Enclosing data for further transferring between the Agent and the Engine. Persisting data which is the basis for further statistics calculation. The Agent and Engine deployment complexity has been delineated in the Architecture Overview. Fortunately, the logical structure is not so complex - the Agent has a mechanism of acquiring data, while the Engine exposes the data to an external user. In order to make the exposition possible, a Monitoring interface had to be modeled. The monitoring interface defines the abstraction, which can be the subject of queries performed by the Monitoring administrator. The following section discusses each model depicted in the Diagram 6.1, providing a synthetic insight into the GlassBus monitoring system. Figure 6.2: General logical structure Domain model This section covers the definition of JBI monitoring domain, which is in the GlassBus scope. The model is divided into two parts: topology and messaging. Topology refers to artifacts deployed to the JBI container and their inter-containment relations. Messaging refers to MessageExchanges intercepted in the bus, which are used in a later calculation of various statistics.

149 6.3. Logical Structure 137 Topology model Figure 6.3: JBI Topology The Diagram 6.3 depicts logical relations between the elements compounding a JBI container. Each lower element is simply aggregated by an element of a higher level, apart from: Service- Unit, ServiceAssembly and Component. The specificity of those elements is implied from the JBI deployment design. The deployment is realized directly by means of ServiceAssemblies, which are zip archives aggregating one of the many ServiceUnits. A particular ServiceUnit is deployed to only one component, which results in the ServiceAssembly being spread through some, but not necessarily all, components. An experiment related to Hierarchical data (5.9) justified the use of the XML Document Object Model. The following XML file has been designed for the purpose of topology enclosure: 1 <c l u s t e r name="clustername"> 2 <i n s t a n c e name="name"> 3 <component name="name"> 4 <su name="name"> 5 <endpoint name="name"/>... 6 </su>... 7 </component>... 8 <sa name="name"/> 9 <s u f l a t name="name"> 10 <s u f l a t name="name"> </sa> </ i n s t a n c e> </ c l u s t e r> The operation is omitted in the topology information because it cannot be easily extracted from the ServiceUnit. Its support can be added later on. GlassBus persists XML in a regular file but thanks to the HierarchyDao abstraction it is not difficult to employ some hierarchical-relational DBMS. Messaging model In order to justify the messaging model, JBI Message Exchange Patterns (MEP) presented in the Diagram 6.4 will be discussed. Patterns are necessary for explaining the timing path introduced in Section

150 6.3. Logical Structure 138 Figure 6.4: JBI Message Exchange Patterns. The image is taken directly from JSR-208. The introduced timing path is recalled again below 1 : Consumer Provider send() ---> > accept() 3 +< send() 4 accept() <--+ 1) Time spent in sender (first time is costly, others very cheap) 2) Waiting in queue for accept() (provider specific delay) 3) Processing time (provider) 4) Waiting in queue for accept() (consumer specific delay) As stated before, the diagram defines important points in the MessageExchange path. It has been decided that GlassBus will focus on the most important part of the timing path - part 3, because it informs about the real performance of the business logic (or the binding forwarding) exposed by the provider. Part 3 was incorporated into GlassBus as the responsetime. The design of JBI MEPs introduces fault and status propagation (acknowledges the last exchanged message). It implies that the introduced timing path is relevant not only for sending the 1 The diagram is downloaded from OpenESB wiki:

151 6.3. Logical Structure 139 response by provider, but also in the following cases: provider sending the response or fault providersresponsetime (prt); provider sending the status providersstatustime (pst); consumer sending the status consumersstatustime (cst); consumer sending the fault consumersresponsetime (crt); All the above-listed time measurements are constantly related to part 3 of the timing path presented earlier. Measurements start along with the invocation of the accept method and finish when the send method is invoked, giving the estimation of processing a particular logic: response time (consumer or provider), status time (consumer or provider). The details of MEPs presented in 6.4 allow to conclude the relation between a particular MEP and possible timing paths - it is depicted in Table 6.1. Table 6.1: Timing path - MEP relations InOnly Robust InOnly InOut InOptionalOut pst pst prt, cst pst prt, cst prt, cst prt, crt, pst The table has multiple lines because some MEPs have several possibilities of flows (for a complete understanding of MEPs please refer to Section of the JBI specification [2]). GlassBus measures every described timing, thus allowing for a complete analysis of each Message Exchange Pattern. Besides part 3 of the timing path, GlassBus measures the cumulative time of part 2 and 4, which are called timeinnmr. This parameter is accumulated through all the timing paths of a particular MEP. Part 1 is out of the GlassBus s scope because the design of the interceptors presented in Section 5.4 does not allow to measure the time spent in the sender. Having discussed the timing aspects, a class diagram of the messaging model is introduced in Figure 6.5. It encloses information related to the primary JBI messaging entity - Exchange, which is the only information used for the statistics calculation. The Exchange encompasses: Type of MEP; ExchangeTargets which uniquely define the consumer and provider of an exchange. Glass- Bus permits simplification by assuming that Operations, Endpoints, ServiceUnits, Service- Assemblies and components have unique names in the scope of one JBI instance. Therefore, the name of the instance and the endpoint is sufficient for uniqueness. Operation is an optional endpoint compound (but alone is also unique) - its enclosure enhances precision of monitoring and allows to introduce ME SUB FLOW PATTERN (introduced in Section 4.4). Providing Instance and Endpoint (optionally operation) seems to be sufficient, but unfortunately sometimes the consumer s endpoint cannot be acquired - it is a common case when the endpoint is the starting point of a JBI transaction. The consumer s endpoint cannot be directly retrieved from the JBI pointcuts - it is acquired through the context provided by

152 6.3. Logical Structure 140 correlation heuristics. It is the reason for introducing fallbackcomponentname, which can always be supplied in case the consumer s Endpoint is impossible to acquire; ExchangeStatus providing JBI s DONE, ERROR and specific to GlassBus INTERRUPTED which is set in case of MessageExchange being interrupted by some JBIException. ExchangeTimePath which encloses the timing explained earlier. It is associated with 2 to 4 TimeIntervals (a pair of beginning and ending timestamps). Interval of the exchange s whole timing is always provided. The remaining timings depend on a particular MEP and its messaging course type. As presented in Table 6.1, the number of these partial timings can fluctuate between 1 and 3. The GlassBus Endpoint terminology always refers to the ServiceEndpoint from the JBI specification. To make matters simple, the Endpoint appearing in GlassBus is a regular string which was built by a concatenation of the ServiceEndpoint s servicename and endpointname. Figure 6.5: GlassBus Dao The Exchange is designed at an abstraction level, but the GlassBus persistence realization allowed to use it directly for defining the JPA Entity Beans. It resulted in a straightforward solution of sending to GlassBus-Engine, messaging information (acquired by the interceptors) already enclosed in the Entity Beans defined by the model. The process of tracing all details of the Exchange is presented in the next three sections GlassBus Agent The GlassBus Agent s essential role is to intercept events in the JBI container, which is conducted by means of the JBI pointcuts (cf. 5.4), as well as to send them to the Engine. The Agent s class diagram presented in Figure 6.6 will be used to explain the structure and functional blocks which allow for providing these roles.

153 6.3. Logical Structure 141 Figure 6.6: GlassBus Agent The bottom part of the diagram refers to classes related to acquiring information. There are four aspects which crosscut JBI. Two JBITopology* aspects are related to the bus s topology change. The specificity of the topology events allows to forward them directly to the Engine without any additional processing. As soon as one of the aspects intercepts an event, a method on TopologyListenerMBean is invoked and the event is propagated to the Engine through a JMX over JMS layer (cf. Architecture overview). An aspect related to messaging interception - JBIMessagingInterceptor has not been designed to forward directly messaging events. GlassBus Agent is equipped with the intelligence of buffering messaging events and performing a send-out to the Engine while all information is being acquired. In order to perform a send-out to the Engine, the information defined in the Messaging model has to be combined. It includes: correlationid, timeinnmr, providersfault, consumers- Fault, error, jbiexception, sent and receivedrequests, sent and receivedreplies, the consumer s and provider s exchangetarget. The first attribute - correlationid - is provided by means of an XML or Thread heuristic (refexp:correlation). Each intercepted exchange of ME is passed to the MessagingProcessor, which tries to incrementally gather the required attributes. The processor chooses one of the heuristics (on the basis of results) and saves partial attributes in properties of the MessageExchange. The attributes are accumulated in properties until the final event of a particular MEP, where all information is gathered, is enclosed in GlassBus s Exchange and sent to the Engine by means of JmsSender. The consumer s exchangetarget is extracted by the heuristics

154 6.3. Logical Structure 142 but, as mentioned in the previous section, it cannot be always determined - sometimes a fallback to the consumer s component name is performed. The Agent s intelligence was implemented on the basis of the JBI s Message Exchange Pattern presented in the previous section. The Agent knows exactly what messages are employed in each pattern, the relations between MEPs and all timings - that knowledge is embedded into the MessagingProcessor. The Agent uses JmsSender for the purpose of the JMS s connection to server. The JMX over JMS communication is leveraged by other MBeans: TopologyListenerMBean, GlassBusEngineRegisterMBean and GlassBusAgentControlMBean. The first two are part of the Engine s code and are only exposed to the Agent. The latter MBean is exposed by the Agent itself and serves the Engine with the functionalities of management and accessibility checking (it is presented in Listing A.11). The GlassBus Agent cannot be classified as a regular component of JBI - Binding Component or Service Engine. The Agent has the ability to crosscut various JBI entities including DeliveryChannels of all the bus s components. Such capability exceeds those assigned by JSR-208 to SE and BC. The GlassBus Agent should be perceived as a functional building block which integrates tightly with the container s environment and allows for acquiring the necessary monitoring information (details of mechanism are described in Sections 5.3 and 5.4).

155 6.3. Logical Structure GlassBus Engine Figure 6.7: GlassBus Engine The GlassBus Engine is the most important element of the GlassBus solution. It allows to manage all Agents of a particular cluster, monitor and further process the data sent by them. The Engine provides three essential functionalities: the Agent s registration, topology processing and messaging processing. Registration is covered by GlassBusRegister. Topology is acquired by TopologyListener performing initial processing, after which the data is persisted (in HierarchicalDao) and notifications are propagated to MonitoringAdministrator by means of the GlassBusStatistics. The GlassBusStatistics is the heart of the Engine, which combines all information acquired by means of JMS and JMX over JMS. The combined information is further exposed through an external interface described in the next section. GlassBusStatistics can be perceived as a Complex Event Processor mechanism employed by the Business Activity Monitoring (cf. Chapter 3). It is equipped with local in-memory storages which allow to keep track of the notifications for which the Monitoring Administrator is regis-

156 6.3. Logical Structure 144 tered. The GlassBusStatistics combines all events and decides which of them have to be translated to some external notifications and which change the state of its intermediate storages. The Engine s management functionalities are provided by the GlassBusManagementMBean, which can be used by the Monitoring Administrator to manage not only the Engine, but the entire GlassBus infrastructure. The detailed Engine s logic is presented in Invocation Flows Section External interface model The model of external interface is the GlassBus s final element, which encloses all monitoring parameters and statistics available to the Monitoring administrator. Analogically to the domain model, the external interface has also been divided into messaging and topology part. The messaging part is further divided into an interface related to: Pattern messaging - related to ME FLOW PATTERN, ME TRANS PATTERN etc. (these concepts have been introduced in Section 4.4); Point Messaging - related to regular JBI elements: Endpoints, Components etc. It has already been decided in Section 5.11 that the interface is exposed through JMX MBeans. Logical parts of external interfaces have been mapped to MBeans in the following way: Topology GlassBusTopologyMBean; Point messaging GlassBusPointStatsMBean; Pattern messaging GlassBusPatternStatsMBean. Topology GlassBus encloses topology in the hierarchical XML DOM model (cf. the Domain Model in Section 6.3.1). It has been decided that in order to simplify the interface, the topology will be accessible through an XML file presented earlier. Various topology elements can be queried by the MBean topology: 1 public interface GlassBusTopologyMBean { 2 String gettopologycluster ( boolean recursive ) ; 3 String gettopologyinstance ( List<String> instancenames, boolean recursive ) ; 4 String gettopologycomponent ( List<String> componentenames, String instancename, boolean recursive ) ; 5 String gettopologyserviceassembly ( List<String> sanames, String instancename, boolean recursive ) ; 6 String gettopologyserviceunit ( List<String> sunames, String instancename, boolean recursive ) ; 7 void registerforclustertopology ( NotificationListener listener, boolean recursive ) ; 8 void registerforinstancetopology ( NotificationListener listener, 9 List<String> instancenames, boolean recursive ) ; 10 void registerforcomponenttopology ( NotificationListener listener, 11 List<String> componentenames, String instancename, boolean recursive ) ; 12 void registerforsatopology ( NotificationListener listener, 13 List<String> sanames, String instancename, boolean recursive ) ;

157 6.3. Logical Structure void registerforsutopology ( NotificationListener listener, 15 List<String> sunames, String instancename, boolean recursive ) ; 16 void deregistertopologylistener ( NotificationListener listener ). 17 } The Get methods allow for querying topology in a recursive or non-recursive manner. The resulting string is always a topology XML file (cf. the Domain Model in Section 6.3.1). The semantics of the recursive query means reporting any compounding JBI elements, including the endpoints. Therefore, for example if some component is queried in a recursive manner, the result will be an XML file containing serviceunits, each provided with a list of the enclosing endpoints. The nonrecursive component query results only in a list of serviceunits (endpoints are not included). The topology query can refer not only to one JBI element, but to whole lists - for example, a querying topology of several JBI instances from the whole cluster. Register methods provide exactly the same semantical information, but by means of a publishsubscribe pattern. The Monitoring administrator can subscribe for a particular topology part, which will be notified by means of JMX notifications as soon as that part of the topology changes. Topology information is enclosed in userdata the attribute of Notification 2. Messaging In order to discuss the messaging part of the external interface, abstraction model of statistics was introduced in Figure 6.8. The parameters are defined in a hierarchical approach grasping the relations between Pattern and Point messaging. 2

158 6.3. Logical Structure 146 Figure 6.8: External monitoring interface The primary entity is the ArtifactStats, which encloses statistics common to every JBI element - attribute names are meaningful, therefore further explanation is omitted. The elements of TimeThroughput and TimeRates have been postponed for future GlassBus works, but have been grasped in the diagram for completeness. TimeThroughput will be related to the speed of MessageExchange traversing between various JBI elements. TimeRates will relate to: MTTF (mean-time-to-failure), MTBF (mean-time-between-failures) and MTTR (mean-time-to-react). The ArtifactStats is primarily related to single JBI endpoint measuring requests, replies, faults, errors, etc., which appear in the context of that endpoint. GlassBus further extends such an endpoint statistics allowing to measure them in other JBI elements: Operation it is a compound of endpoints. Its ArtifactStats relates to messages consumed or provided by the operation. ServiceUnit, ServiceAssembly, Component, Instance, Cluster these elements always enclose several endpoints, which is why their ArtifactStats relates to accumulated statistics of all endpoints in the scope of a particular entity.

159 6.3. Logical Structure 147 EndpointTiming is related to the timing path introduced earlier. A particular endpoint (or operation) can serve as the endpoint s consumer or provider. Therefore, 2 instances of EndpointTiming are associated with the ArtifactStats. It allows to cover all timings: providerresponsetime, providerstatustime, consumerresponsetime and consumerstatustime. Enriched attributes: min. max. avg. provide a simple and possibly precise information, which can be easily aggregated over many endpoints - in the case of bigger JBI elements: SU, SA, etc. GlassBus allows to aggregate statistics of every JBI element. It has been modeled by two separate classes - IndividualStats and AggregatedStats. An additional division between containerspecific and artifact-specific elements has been introduced. It resulted in the following classification: ArtefactIndividualStats It can describe a single Operation, Endpoint or ServiceUnit. Additional data about uptime and activationtime is supplied. ArtefactAggregatedStats It can describe a set of Operations, Endpoints, SUs with detailed uptime information. ArtifactStats and EndpointTiming are aggregated over endpoints in scope the of a particular artifact. ContainerIndividualStats It can describe a single ServiceAssembly, Component, Jbi instance or the whole Cluster. Additional start, stop, startup, uptime information is provided. ContainerAggregatedStats It can describe a set of ServiceAssemblies, Components, Jbi instances. ArtifactStats and EndpointTiming are aggregated over endpoints in the scope of particular artifact. Detailed startup and uptime information is provided. In order to comply with this statistics abstraction, GlassBusPointStatsMBean has been equipped with following methods: 1 String getoperationstats ( List<String>operationNames, String Role, String instance ) ; 2 String getendpointstats ( List<String>endpointNames, String Role, String instance ) ; 3 String getsustats ( List<String> sunames, String Role, String instance ) ; 4 String getsastats ( List<String> sanames, String Role, String instance ) ; 5 String getcomponentstats ( List<String> componentnames, String Role, String instance ) ; 6 String getinstancestats ( List<String> instancenames, String Role ) ; 7 String getclusterstats ( String Role ) ; 8 void registerforoperationstats ( List<String> operationnames, 9 NotificationListener listener, String Role, String instance ) ; 10 void registerforendpointstats ( List<String> endpointnames, 11 NotificationListener listener, String Role, String instance ) ; 12 void registerforsustats ( List<String> sunames, NotificationListener listener, String Role, String instance ) ; 13 void registerforsastats ( List<String> sanames, NotificationListener listener, String Role, String instance ) ; 14 void registerforcomponentstats ( List<String> componentnames, 15 NotificationListener listener, String Role, String instance ) ; 16 void registerforinstancestats ( List<String> instancenames, 17 NotificationListener listener, String Role ) ; 18 void registerforclusterstats ( NotificationListener listener, String Role ) ; 19 void deregisterstatslistener ( NotificationListener listener ).

160 6.3. Logical Structure 148 Analogically to the case of TopologyMBean, information is available by means of regular method calls and by means of JMX Notifications. Another analogy with the topology is that method s result (sent as userdata of Notification) provides statistics also through an XML file. A sample result is provided in Listing (A.12). All methods can be invoked with Role an argument that can restrict the scope to particular consuming or providing MessageExchanges. It allows to measure the performance of the business logic provided by some component. The Role argument can have values of: CONSUMER, PROVIDER or null, which includes both roles. Pattern Messaging GlassBus introduced new terminology of Flow and Trans Pattern (detailed information is provided in Section 4.4: ME FLOW PATTERN, ME TRANS PATTERN). The primary entity of pattern messaging is ME SUB FLOW PATTERN Stats, which defines exchanges between two operations of some endpoints. It introduces only one new attribute - totalinvocations (which informs about the total amount of consumer-provider messages invocations), but it also changes the semantics of ArtifactStats and EndpointTiming. In this context they relate only to message exchanges between ME SUB FLOW PATTERN s consuming and providing operations. It is a different situation than aggregating these two operations by means of the getoperationstats method. ME FLOW PATTERN Stats has the same semantics as ME SUB FLOW PATTERN, but it relates to endpoints, not operations (cf. Section 4.4). Therefore, the diagram grasps the fact that ME FLOW PATTERN Stats can be associated with several ME SUB FLOW PATTERN Stats. ME TRANS PATTER relates to the JBI transaction, which can be perceived as several ME SUB FLOW PATTERNs interconnected with each other - it is the reason for the one to many association. ME TRANS PATTERN Stats can be used to enclose the statistics of both: the whole business operation (BUSS TRANS PATTERN concept from Section 4.4) and transaction (ME TRANS PATTERN). Pattern messaging is supported by the following methods of GlassBusPatternStatsMBean: 1 String querytrans ( String partialtransactionpath ) ; 2 3 String listtransincluster ( ) ; 4 String listtransstartingininstance ( String name ) ; 5 String listtransstartingincomponent ( String name, String instance ) ; 6 String listtransstartinginsa ( String name, String instance ) ; 7 // a l s o provided : InSU, InEndpoint, InOperation 8 9 void registerfornewtransincluster ( NotificationListener listener ) ; 10 // a l s o provided : InInstance, InComponent, InSA, InSU, InEndpoint, InOperation and d e r e g i s t e r String listbusinessopincluster ( ) ; 13 String listbusinessopstartingininstance ( String name ) ; 14 // a l s o provided : InComponent, InSA, InSU, InEndpoint, InOperation void registerfornewbusinessopincluster ( NotificationListener listener ) ; 17 // a l s o provided : InInstance, InComponent, InSA, InSU, InEndpoint, InOperation and d e r e g i s t e r String getsubflowstats ( Map<String, String>consumerProviderOperations, Map<String, String>fallbackComponentConsumerProviderOperations, String instance ) ;

161 6.3. Logical Structure String getflowstats ( Map<String, String>consumerProviderEndpoints, Map<String, String>fallbackComponentConsumerProviderEndpoints, String instance ) ; 21 String gettransstats ( String transactionpath ) ; 22 String getbusinessopstats ( Map<String, String>rootEndpointOperationPairs, List< String>fallbackComponentConsumer, String instance ) ; void registerforsubflowstats ( Map<String, String> consumerprovideroperations, Map< String, String>fallbackComponentConsumerProviderOperations 25 NotificationListener listener, String instance ) ; 26 void registerforflowstats ( Map<String, String> consumerproviderendpoints, Map< String, String>fallbackComponentConsumerProviderEndpoints 27 NotificationListener listener, String instance ) ; 28 void deregisterflowlistener ( NotificationListener listener ) ; void registerfortransstats ( String transactionpath, NotificationListener listener ) ; 31 void registerforbusinessopstats ( Map<String, String> rootendpointoperationpairs, List<String>fallbackComponentConsumer, 32 NotificationListener listener, String instance ) ; 33 void deregistertranslistener ( NotificationListener listener ) ; The methods have been designed in a manner similar to those from other MBeans of the monitoring interface model - regular methods and notifications, which result in same XML-embedded statistics (with the addition of totalinvocations parameter). However, there are new list methods, which allow to acquire information about transactions (ME TRANS PATTERN) and business operations (BUSS TRANS) starting in a particular JBI element. The result of listbusinessop encloses all root starting points of business operations. It is provided in XML with the following format: 1 <endpoint name="name1" operation="optionaloperation1" i s t a n c e ="name"> <! operation i s optional > <fallbackcomponentname name="name1" i n s t a n c e ="name"> <! I t i s used in the case of not a c q u i r a b l e consumer endpoint > 4... The result of listtrans encloses transaction paths provided in XML with the following format: 1 <j b i t r a n s a c t i o n> 2 <fallbackcomponentname name="name1" i n s t a n c e ="name"/> 3 <! I t i s o p t i o n a l l y used in the case of not a c q uirable consumer endpoint > 4 <endpoint name="name1" operation="optionaloperation1" i n s t a n c e ="name"/><! operation i s optional > <j b i t r a n s a c t i o n /> 7 <j b i t r a n s a c t i o n> 8... The method querytrans also allows to acquire a list of transaction paths, but the beginning of a transaction must be the same as the partialtransactionpath provided as argument of an XML file with the same format as the jbi transaction node presented above. The method allows to query transaction the starting part of which goes through some particular endpoints supplied in the query.

162 6.3. Logical Structure 150 The methods registerfornew* provide a listener notification in case of new transactions or business operations appearing in a particular cluster scope. The methods getsubflowstats and getflowstats (also their notification equivalents) allow to acquire statistics about particular ME SUB FLOW PATTERNs and ME FLOW PATTERNs. Consumer of ME FLOW PATTERN can be also specified by means of the componentfallback name (used when the consumer s endpoint is not acquirable), but it is optional. Method s Map arguments allow to supply multiple values and therefore conduct an aggregation of the related ME SUB FLOW PATTERNs or ME FLOW PATTERNs. The getbusinessopstats method (and its notification equivalent) allows to query statistics of whole business operations. The method provides the possibility of aggregation (thanks to Map and List arguments) and the option of supplying the consumer by means of the component s name. The only method not equipped with aggregation is gettransstats (along with its notification equivalent), which allows to acquire statistics of a particular transaction (ME TRANS PATTERN). Transactions can only be aggregated by means of business operations that enclose them. Unfortunately, a transaction has to be specified through all compounding endpoints (and possibly through fallbackcontainername at the beginning) enclosed in an XML argument of the same format as the jbi transaction node presented above (reference to the schema) Management and selectivity model As it has been delineated before, GlassBus is not only the Monitoring solution. Its architecture allowed for providing a wide range of management features. Management, analogically to monitoring, is realized at several levels providing flexibility of adjusting GlassBus to particular needs and tuning the selectivity of gathering monitoring information. The problem with management comes from the distributed GlassBus architecture. It is easy to imagine a federation of fifty JBI containers instrumented with GlassBus. The management of such a widespread cluster is difficult not only in terms of adjusting the GlassBus Agent mechanism, but also in terms of managing the containers themselves. The GlassBus management architecture solves both problems. GlassBus Agents can be managed in two ways: by JMX management interface; by configuration files. Both mechanisms allow to control similar elements, but JMX management is equipped with broader functionalities, which is the reason for discussing it in the first place. The discussion of configuration is presented in one of next sections (6.5). In order to provide selectivity of the information gathering mechanism, several levels of data acquiring have been introduced: TOPOLOGY only the information about the bus topology is gathered by the Agent and forwarded to the Engine. The topology is additionally divided into inner levels of selectivity: COMPONENT only information about components is acquired (employing JBITopologyComponentInterceptor);

163 6.3. Logical Structure 151 SERVICE UNIT information about components, service assemblies and service units is acquired (employing JBITopologyComponentInterceptor and part of JBITopologySUInterceptor); ENDPOINT information at all levels including endpoints and operations is acquired (employing both JBITopologyComponentInterceptor and JBITopologySUInterceptor); MESSAGING BASIC the topology information is enriched with messaging events, but no inter-message correlation is applied (employing all topology interceptors and JBIMessagingInterceptor); MESSAGING COMPLETE all acquirable information is gathered, including: topology, messaging along with correlation into jbi transactions (employing all topology, messaging interceptors and additionally correlation heuristics). The lowest possible monitoring level is TOPOLOGY with an inner COMPONENT level. JMX management of the Agent is realized by means of GlassBusAgentControlMBean, which is equipped with the following methods: getglobalselectivitylevel; gettopologyselectivitylevel; setglobalselectivitylevel; settopologyselectivitylevel; The methods allow for a flexible adjustment of the GlassBus information monitoring. Another functionality of the GlassBusAgentControlMBean is forwarding JBI MBeans, which are defined in JSR-208. It means that a reference to those MBeans can be obtained through invocation of the GlassBusAgentControlMBean methods (the code is provided in Listing A.11). The following MBeans are obtainable: AdminServiceMBean; DeploymentServiceMBean; InstallationServiceMBean; Thanks to the exposition of GlassBusAgentControlMBean to the Engine, all the above-listed selectivity levels and JBI MBeans can be managed from one central point - the GlassBus Engine. As depicted in the Engine s class diagram (6.7), the GlassBusAgentControlMBean of each Agent is accessed by the GlassBusManagementMBean, which allows the Monitoring administrator to manage every Agent. As it has been mentioned at the beginning of this section, such a solution combines the capabilities of managing not only GlassBus Agent, but also the JBI Container itself. The GlassBus Engine can be used to install new components or deploy new service assemblies on distant JBI containers (for the functionalities of JBI MBean pleas refer to the JBI Status Quo 3.3 provided earlier).

164 6.3. Logical Structure 152 Aspect weaving adjustment GlassBus installer instrumentation of the Agent is accompanied with the incorporation of all aspects into the JBI container. It implies that in the course of the container initialization all aspects are woven into the bus s code. Therefore, setting a lower selectivity cannot disable pointcuts already woven. Instead, each pointcut is equipped with checking the selectivity level and on its basis performs a logic or abandons it. Such a solution results in the minimal performance impact in the case of lower selectivity levels. It has been decided that this mechanism is sufficient for the purpose of performance impact minimization, however the user of GlassBus may still be concerned about disabling weaving of the unwanted aspects. It can be simply achieved by means of a modification aop.xml file (A.8) which is embedded in the following jar artifacts: glassbusagent-smix and glassbus-agent-openesb. In the course of the modification modification, aspects dependencies have to be preserved: the heuristics rely on messaging, the messaging relies on topology Invocation flows This section is devoted to discussing end-to-end flows of the three main GlassBus functionalities: registration, messaging and topology propagation. Every functionality is delineated in a separate UML activity diagram. The diagrams allow for a clear expression of dependencies between the logical structure elements and their interactions.

165 6.3. Logical Structure 153 Agent registration Figure 6.9: The Agent in the Engine registration sequence diagram Agent registration, depicted in Figure 6.9 is crucial for the whole GlassBus operation. Neither messaging nor topology propagation can be realized without a successful registration. In order to join the monitoring domain of a particular Engine, the Agent needs to know its name a priori. The default name of the Engine is JbiMonitoringEngine, but it can be adjusted in the component s properties files (presented in Section 6.5). The Agent invokes registration in the GlassBusEngineRegisterMBean interface. The Engine verifies the Agent s accessibility by means of GlassBusAgentControlMBean. Both interfaces are provided by Lingo s JMX over JMS and are not released during the further Engine s and Agent s lifetime. They are used for a constant accessibility verification allowing for instant identification of problems with the Agent-Engine communication.

166 6.3. Logical Structure 154 Messaging propagation Figure 6.10: Messaging propagation sequence diagram Messaging propagation, depicted in Figure 6.10, starts from the JBIMessagingInterceptor, which intercepts the sending and receiving of MessageExchanges. The Interceptor verifies the crosscut delivery channel in order to assure that it belongs to the component that has started, otherwise the event is ignored. After a successful verification, the event is passed to correlators for the transaction correlation. The correlators are incorporated by CorrelatorsExecutor (it is not depicted in the diagram), which allow for an easy future addition of new correlators. Further event and correlation information is passed to the Agent s most important element - MessagingProcessor. The Processor checks the event context (sending, receiving) and enriches the MessageExchange s properties with the information defined in the Domain model (6.3.1). If the event is not final for a particular MEP, the ME is returned to the interceptor and released. In the case of a final event which receives the JBI s status (DONE or ERROR), the MessagingProcessor embeds the ME data (extracted from its properties and internal attributes) into the Messaging model and sends it out through the JMSSender. The JMS layer draws a line of demarcation between the Agent and the Engine, which is traversed from the JMSSender to the ReceiverMDP. The latter receives messages, verifies the sending Agent (checks whether the Agent has previously successfully registered and is available), notifies the statistics module and finally persists the data in GlassBusDao. The GlassBusStatistics maintains a registry about the external users registered for changes of particular monitoring pa-

167 6.4. Physical Structure 155 rameters and informs the users when it is appropriate. Topology propagation Figure 6.11: Topology propagation sequence diagram The topology propagation, depicted in Figure 6.11, is less complicated than the messaging propagation. Only six main entities are involved - the three that are left are of the Agent and the rest belong to the Engine. The diagram captures the fact that the interception of the component and ServiceUnit events are not related - they can even take place concurrently. The JBITopologyComponentInterceptor updates the register of the DeliveryChannelInfo (containing bindings between the components and their delivery channels), which has previously been presented in Messaging propagation the diagram. The rest of the flow is the same for both the JBITopology- ComponentInterceptor and the JBITopologySUInterceptor. The border between the Agent and the Engine is traversed by means of the JMX over JMS layer - the methods of TopologyListenerMBean allow to propagate the intercepted topology changes. The TopologyListener performs further verification, notification of statistics module and finally persists the information in HierarchicalDao. By analogy to the messaging propagation, the GlassBusStatistics informs the external users when it is appropriate. 6.4 Physical Structure It has been decided that Maven [43] will be used for the purpose of the GlassBus implementation structuring and building. Its hierarchical model allowed for a flexible arrangement of the Glass- Bus artifacts. Thhe Glassbus s physical structure has been delineated in Figure 6.12 by means of a UML package diagram. The diagram provides a namespace of each artifact, which can be useful for the purpose of logging configuration.

168 6.4. Physical Structure 156 Figure 6.12: Physical structure of GlassBus implementation It was predictable that two most important logical entities: the Engine and the Agent will also be major elements of the physical structure. Each artifact s purpose is depicted in the following list: glassbus-common JMX MBean interfaces shared by the Agent and the Engine: Glass- BusEngineRegisterMBean and GlassBusAgentControlMBean. glassbus-dao JPA entity beans used for the purpose of persisting the information in the Engine. Dao is also used for transferring information between the Agent and the Engine. glassbus-agent It is divided into the core part, which covers all implementation, and additional artifacts, which personalize the Agent for a particular JBI bus. Currently, the openesb and smix artifacts enclose only the aop.xml file, which properly configures the aspects provided in the core artifact. In order to support an additional bus, a new artifact of glassbus-agent-jbibus has to be added. glassbus-engine similarly to the Agent, the core part is identified, which covers all the Engine s implementation. The following two artifacts provide support for deployment scenarios: out-of-the-box and application server. Depending on the requirements, the box or war variant has to be used for installation.

169 6.5. Configuration 157 glassbus-installer the installer combines all other artifacts and allows for an easy installation in any kind of a deployment scenario. glassbus-externaliface it is a small library which encloses external interfaces of the MBean statistics: GlassBusPatternStatsMBean, GlassBusPointStatsMBean and GlassBusTopologyM- Bean. It can be employed by the Monitoring administrator s JMX client for easier access to the monitoring interface. The clear-cut separation of the artifacts functionalities provides the ease of extending Glass- Bus to other JBI compliant buses. The hierarchical structure allows to perform incremental builds of parts of the system without influencing other elements. All complexity of compilation - processing the aspect files and assembling them into jars along with the rest of the code - is covered by maven, which also allows for a complete specification of each artifact s dependencies. Project configuration is enclosed in each artifact s pom.xml file and does not require any modification. In order to build a project after code modifications, it is sufficient to invoke the mvn install command, which takes care about everything. The created glassbus-installer has to be propagated to desired JBI containers. After that, the GlassBus monitoring system is fully functional and ready to use. 6.5 Configuration As mentioned earlier, configuration files are another way of managing the Agent. Being more precise, the configuration files are used not only on the Agent, but also on the Engine. They are related not only to management, but also to some regular parameter supplication. GlassBus is configured through two files: general properties and detailed broker XML. General configuration General configuration is enclosed in the properties files, which have the following names in the Agent and the Engine respectively: glassbus-agent.properties and glassbus-engine.properties. They are accessed through the Configuration class, which has been presented on the Agent s and the Engine s class diagrams. The glassbus-agent.properties file has the following content: 1 glassbus. clustername=jbicluster 2 glassbus. monitoringenginename=jbimonitoringengine 3 glassbus. instancename=jbiagent1 4 glassbus. startingownbroker=false 5 6 glassbus. globalselectivity=messaging\_complete 7 glassbus. topologyselectivity=endpoint 8 9 activemq. brokername=localhost lingo. jmxjmsresponetimeout=20000 The Agents and the Engine belonging to the same JBI cluster have to be configured with the same clustername. The monitoringenginename is used for addressing the initial in-engine registration, where the instancename is provided as the Agent s name. The startingownbroker and

170 6.5. Configuration 158 activemq.brokername are related to particular deployment scenarios - they will be discussed in the next section. The lingo.jmxjmsresponetimeout allows to tune the timeout of JMX over JMS invocations. It defaults to 20 seconds. The selectivity parameters (described in 6.3.5) allow to set up its initial values. Configuration of the GlassBus Engine is similar but it obviously does not have the monitoringenginename and selectivity values can be configured separately for each Agent s name: 1 glassbus. AGENT1. globalselectivity=value 2 glassbus. AGENT1. topologyselectivity=value 3 glassbus. AGENT2. globalselectivity=value 4 glassbus. AGENT2. topologyselectivity=value It allows to override settings from the Agent s configuration. As soon as the Agent registers in the Engine, its selectivity is set to the values provided in the glassbus-engine.properties. Broker configuration Broker configuration is related to the middleware realized by the ActiveMQ messaging solution. Configuration of all broker s usage cases is out of the scope of this thesis and can be found in the ActiveMQ documentation. This section is restricted to the configuration of the GlassBus s deployment scenarios presented in Section Out-of-the-box This scenario is realized by means of only one broker instance. Therefore, no additional configuration is needed. The tags transportconnectors and networkconnectors can be commented out because every GlassBus element accesses the broker internally through JVM. In a general configuration, the activemq.brokername can be set to any value (it defaults to localhost) and the value of the startingownbroker has to be set accordingly to the specific bus: ServiceMix has its own broker, which is why there is no need for starting another one, contrary to OpenESB, which is not equipped with a broker. Distributed cluster The distributed cluster allows for several instances of Agents and one Engine instance. Each broker has to set up its own transportconnector, which has to be referred to in the networkconnectors section of all other cluster nodes. A sample configuration with two Agents could have the following shape: 1 AGENT1: 2 <amq: transportconnectors> 3 <amq: transportconnector u r i ="tcp://localhost:agent1port"/> 4 </ amq: transportconnectors> 5 <amq:networkconnectors> 6 <amq:networkconnector u r i ="static://(tcp://enginehost:engineport)"/> > 7 <amq:networkconnector u r i ="static://(tcp://agent2host:agent2port)"/> > 8 </amq:networkconnectors> 9 AGENT2: 10 <amq: transportconnectors> 11 <amq: transportconnector u r i ="tcp://localhost:agent2port"/> 12 </ amq: transportconnectors> 13 <amq:networkconnectors> 14 <amq:networkconnector u r i ="static://(tcp://enginehost:engineport)"/> >

171 6.6. Installation <amq:networkconnector u r i ="static://(tcp://agent1host:agent1port)"/> > 16 </amq:networkconnectors> 17 ENGINE: 18 <amq: transportconnectors> 19 <amq: transportconnector u r i ="tcp://localhost:engineport"/> 20 </ amq: transportconnectors> 21 <amq:networkconnectors> 22 <amq:networkconnector u r i ="static://(tcp://agent2host:agent2port)"/> > 23 <amq:networkconnector u r i ="static://(tcp://agent1host:agent1port)"/> > 24 </amq:networkconnectors> In a general configuration, each node has to have unique value of activemq.brokername. The same uniqueness is required for the instancename parameter. The property startingownbroker has to be analogical to the one in the out-of-the-box scenario. 6.6 Installation Assumptions about GlassBus Installer realization were provided in Section This section covers detailed installation instructions. As delineated in Section 5.10, the GlassBus installer is provided as an executable jar. It encloses every GlassBus components apart from the glassbus-engine-war, which is a separate WAR artifact. The GlassBus WAR package can be downloaded and deployed to the Application Server independently from the GlassBus installer. Detailed instructions of the WAR deployment depend on a particular Application Server and are simple enough to be covered by the user himself/herself. In order to use the GlassBus installer, its jar has to be downloaded and executed. The installer should be executed in a root directory either of the ServiceMix or a particular domain of the OpenESB s application server. Alternatively, a destination path can be supplied with the -p argument. After executing the installer, an interactive process of choosing various options begins. It is depicted in the following listing: 1 I. Provide a JBI container which is glazed by GlassBus: 2 1) ServiceMix 3 2) OpenESB This step is often omitted because the installer can automatically detect a particular bus thanks to the analysis of its root directory. The next steps are as follows: 1 II. Instrument the bus with GlassBus Agent? [ y/n ] 2 III. Instrument the bus with GlassBus Engine out of the box scenario? [ y/n ] Answering these simple questions allows to complete the installation process. A particular JBI container is glazed by GlassBus and after some simple configuration (in the out-of-the-box scenario no additional configuration is needed), the JBI monitoring solution is ready to be used. The installation process can be performed in a non-interactive manner by the -c argument provided, which is followed by a path to the installation property file configuration. The file has to have the following format: 1 jbicontainer=container

172 6.7. Implementation status glassbusagent=yes/no 3 glassbusengine=yes/no It allows to perform GlassBus deployment on a large scale (big JBI clusters) in an automated manner. If the installer is invoked with the -u argument, uninstallation of a particular GlassBus component can be performed. The interactive and non-interactive uninstallation process is analogical to the installation one. 6.7 Implementation status In the course of the work on this thesis, it turned out that creating a complete architecture for the JBI monitoring solution is not an easy task. The contradiction between the requirement of a fully JBI compliant bus and the requirement of ServiceMix and OpenESB support in terms that are beyond the JBI scope (i.e. clustering), resulted in an architecture of a high complexity level. The wide range of solved problems discussed in Chapter 5 provides a good complexity estimation, which is verified by the completeness of the final functionalities described in Chapter 6. Such a GlassBus evolution resulted in restricting the implementation of a system with a production quality. The GlassBus design presented in this thesis has been based on a complete set of Proof Of Concept [106] implementations, which allowed to verify every architectural decision. As the fully functional implementation of GlassBus with a production quality is out of the scope of this thesis, it has been decided to provide only the research version of GlassBus. This version focuses on validating both the out-of-the-box and the distributed scenarios, but not in all cases. End-to-end flow of information is provided in each case (registration, topology propagation, messaging propagation) but without a complete implementation of the Engine s Glass- BusStatistics class, which comes to simple operations on the persisted data. The RCP GUI has been identified as a complex feature that was not essential. The implementation of the JMX external monitoring interface was entirely sufficient for the research, therefore a GUI has not been provided. It can be conceived that the research version of GlassBus is a fully functional skeleton but not a complete production system. The skeleton solves all crucial problems in an innovative way, leaving the regular implementation of well-known elements in scope of future GlassBus s work. 6.8 Performance Tests In order to validate the GlassBus solution, some performance tests have been conducted. The crucial issue, which has driven the architectural decisions, is the influence on the JBI container s performance. That was the reason for decoupling Agent instrumentation from Engine processing and analysis. The deep analysis of JMSSender performance and choosing high-performance messaging middleware solution - ActiveMQ allowed to minimize the overhead on the Agent s side. The research version of GlassBus does not provide all functionalities of the Engine s statistics, but it is not an obstacle for performance tests which focus on the GlassBus Agent s influence on ServiceMix and OpenESB.

173 6.8. Performance Tests 161 The tests verified the difference between regular non-instrumented bus operation performance and the performance after GlassBus installation. The following GlassBus s elements were in the scope of the performance measurement: topology interceptors; messaging interceptor; thread correlation heuristic; agent s MessagingProcessor; agent s JMSSender. In order to perform tests that give good performance estimation, the business logic of services deployed to the JBI container has been minimized. A small logic overhead allowed to focus on the performance of communication, which is the main subject of the GlassBus influence. In the cases of both ServiceMix and OpenESB, the business logic was a simple echo application exposed by the Binding Component. During the tests, the selectivity of GlassBus Agent was set to the highest possible value: MES- SAGING COMPLETE. The test has been performed in two deployment scenarios (cf. 5.10), which are further divided into sequential and concurrent cases: Out-of-the-box The Agent and the Engine are embedded into the non-distributed bus, which allows to share a common broker instance and avoid external communication. Sequential sequential invocations of endpoint exposed in BC; Concurrent invocations of BC s endpoint in 10 concurrent threads; Distributed There is one instance of a non-distributed bus instrumented with the Glass- Bus Agent and a separate node with the GlassBus Engine. The nodes are in the same local network. This scenario is analogically divided into the sequential and concurrent case. The goal of a particular test was to measure the mean duration of business operation execution. Business operation is exposed by a binding component, which is invoked by an external code of the tester. The tester performs appropriate measurements: calculation of mean and its standard deviation over all invocations. The number of invocations that will be the subject of the mean calculation was already provided: invocations in the sequential case and invocations performed by 10 threads in the concurrent case Test environment The following hardware-software environment was used for the purpose of the performance tests: Node for Box scenarios and for the GlassBus Agent in the distributed scenario: Processor: Intel(R) Core2 Duo T GHz, 2MB Cache L2, FSB 667MHz Ram: 2GB DDR II Hardware: 100 GB SATA rpm

174 6.8. Performance Tests 162 Networking: LAN 100 MBit OS: Ubuntu 7.10 (Gutsy Gibbon) Server version Node for the GlassBus Engine in the distributed scenario: Processor: Intel(R) Core Duo T GHz, 2MB Cache L2, FSB 667MHz Ram: 1GB DDR II Hardware: 100 GB SATA rpm Networking: LAN 100 MBit OS: Ubuntu 7.10 (Gutsy Gibbon) Server version In order to minimize interferences, these two nodes have been disconnected from the global Internet network. For the purpose of the distributed scenario, the nodes were interconnected with a regular 100 MBit Local Area Network. The operating system installed on both nodes was properly prepared: all not related daemons and applications were disabled, leaving only the basic networking functionalities ServiceMix Simple ServiceAssembly with HTTP binding component and JSR-181 service engine were used in ServiceMix. SA s topology has been delineated in Figure In the distributed scenario, the first node was equipped only with the GlassBus Agent, while the second node encompassed the GlassBus Engine embedded into another ServiceMix instance (using the out-of-the-box feature). Figure 6.13: ServiceMix topology for performance tests The results of the ServiceMix performance tests are depicted in Table 6.2. The results provide information about mean duration of a single BC invocation, given in milliseconds. Table 6.2: Performance of glazed ServiceMix Scenario Sequential Concurrent Mean Std. Dev. Mean Std. Dev Without GlassBus ms 2.78 ms ms 2.91 ms GlassBus Box ms 4.58 ms ms 0.74 ms GlassBus Distributed ms 2.60 ms ms 1.74 ms The results show that the GlassBus instrumentation implies some overhead. The exact overhead percentage is presented in Table 6.3. The overhead was calculated according to the following DurationW ithglassbus DurationW ithoutglassbus formula: Overhead = DurationW ithoutglassbus 100%

175 6.8. Performance Tests 163 Table 6.3: GlassBus overhead in ServiceMix Scenario Sequential Concurrent Box 18.2% 26.5% Distributed 12.9% 19.9% It can be observed that the distributed scenario tends to outperform the out-of-the-box scenario. Another observation related to the concurrency influence is that, as suspected, it leads to a higher GlassBus overhead. Further conclusions are provided in the summary OpenESB A simple ServiceAssembly has also been used in the case of OpenESB. It encompassed a HTTP binding component and a J2EE service engine. The SA s topology is delineated in Figure The distributed scenario was covered similarly to ServiceMix - the GlassBus Engine was embedded into a second OpenESB instance (out-of-the-box feature), installed on the second node. In both scenarios, OpenESB was executed in the Application Server s environment (GlassFish). Figure 6.14: OpenESB topology for performance tests The results of OpenESB performance tests are depicted in Table 6.5. The results provide information about mean duration of a single BC invocation, given in milliseconds. Table 6.4: Performance of glazed OpenESB Scenario Sequential Concurrent Mean Std. Dev. Mean Std. Dev Without GlassBus 6.69 ms 1.42 ms ms 1.78 ms GlassBus Box 7.62 ms 1.33 ms ms 1.48 ms GlassBus Distributed 7.14 ms 0.52 ms ms 2.12 ms Similarly to ServiceMix, there is an overhead implied by GlassBus. The exact overhead percentage is presented in Table 6.5. The overhead was calculated in the same way as in ServiceMix. Table 6.5: GlassBus overhead in OpenESB Scenario Sequential Concurrent Box 13.9% 19.9% Distributed 6.7% 14.0%

176 6.8. Performance Tests 164 The OpenESB performance tendencies are similar to those of ServiceMix: the distributed case outperforms the out-of-the-box one, concurrent invocations imply more overhead than the sequential ones Summary Comparison of the ServiceMix and OpenESB overhead is depicted in Figure Figure 6.15: Comparison of GlassBus performance overhead The following tendencies have already been noticed earlier: The distributed case outperforms the out-of-the-box scenario - It is suspicious at first glance, but a deeper analysis confirms it. In the case of the box scenario, the node is additionally loaded with the GlassBus Engine logic, which is moved to the second node in the distributed scenario. While distributed, the Agent simply passes the information about MEP to the ActiveMQ broker for send-out. It is less demanding for the processor. Concurrent invocations are the reason for a higher overhead - The reason for this lies in the bottleneck of the broker. Suddenly, there are more messages forwarded to the Engine, which all have to go through one broker of the GlassBus Agent. The analysis of the chart presendted above allows to make further conclusions. The GlassBus overhead measured in OpenESB is lesser than that of ServiceMix. It is difficult to point out the reason for that - probably the internal implementation of OpenESB is better suited for the AOP crosscuting. The most important observation is that the maximal GlassBus overhead is only 25%