PERFORMANCE COMPARISON OF COMMON OBJECT REQUEST BROKER ARCHITECTURE(CORBA) VS JAVA MESSAGING SERVICE(JMS) BY TEAM SCALABLE

Transcription

1 PERFORMANCE COMPARISON OF COMMON OBJECT REQUEST BROKER ARCHITECTURE(CORBA) VS JAVA MESSAGING SERVICE(JMS) BY TEAM SCALABLE TIGRAN HAKOBYAN SUJAL PATEL VANDANA MURALI

2 INTRODUCTION Common Object Request Broker Architecture (CORBA) is a communication standard defined by the Object Management Group (OMG), which mainly enables software components written in multiple languages to interact and communicate with each other. It forms the basic principle behind developing inter- operable distributed applications. Java Messaging Service (JMS) is a Java based protocol that is used for establishing communication between remote systems. Here, communication is viewed as a session where there is an as an exchange of messages. JMS has an intermediary message broker component which allows loosely coupled, reliable, and asynchronous communication between remote components of distributed system, i.e., the message sender does not have to have the knowledge of its receiver(s). Asynchronous and loosely coupled messaging based communication reduces system bottleneck and increases scalability. The main idea behind this team research investigation is to compare the performance of these two communication methodologies, using a simple benchmark like the Bubble Sort program. The rest of the report is organized as follows- First we present the hypothesis statement that forms the basis of this investigation. We then go on to describe the approach taken in this investigation to prove/disprove the hypothesis. Following this we give a brief overview of the research papers that serve as a reference though the course of this investigation, followed by the actual high- level software design. The data collected and also the analysis of the data performed is described. The final sections of this report talk about the inferences from the analysis, future work and the references used for this investigation. STATEMENT OF HYPOTHESIS Both the communication techniques compared in this investigation rely heavily on the communication between the networks, using the underlying protocol. In case of CORBA the underlying communication protocol is Internet Inter- Orb protocol (IIOP) and in case of JMS, TCP/IP forms the base for the network communication. The main idea in CORBA' is to achieve interoperability between communicating objects through language specific mappings to the IDL. This is a highly time intensive process. On the contrary, JMS being asynchronous in nature would definitely offer better performance. This idea formed the basis of our following hypothesis. Mean response time in a JMS based distributed system is less than the mean response time in a CORBA based distributed system.

3 APPROACH TAKEN In this course of this research investigation we have implemented CORBA based and Java JMS based application, using the Bubble Sort program as a benchmark to evaluate the performance of the two communication methodologies. In both cases the client program takes as input from user, the number of elements to be considered in the array that is passed to the program, to be sorted. This array of numbers is then sent to the server side program, where the input array is sorted using the Bubble Sort algorithm. This sorted array is then sent back to the client program where it is displayed to the user. We have used a variety of input array sizes that would simulate the transfer of a wide range of data through the communication protocol. This also served as a means to account for the protocol overhead in both techniques, which ensured that the simulation is more realistic and close to the possible real- world application of these two techniques. We have measured the response time of both communication techniques under two conditions- i. The client and the server program running on the same computer (in our case local host connection). ii. The client and server program running on different computers, i.e., the client establishes a remote connection to the server to execute the operations. We have made use of two computers on the Computer Science department s lab, (glados and joplin) to simulate remote execution. For the purpose of this investigation, the response time taken is defined as the time elapsed from the point where the request to sort, (method call for sort) leaves the client application till the results are returned and displayed to the user. We have defined a set number of trials for the simulation experiment (in the investigation the number of trials is 10), and in each run of the simulation, for each input size array, we measured the response time of both CORBA based and JMS based application. We also calculated the mean response time of CORBA and JMS for each of the various input array sizes. We then analyzed the performance of both applications, by plotting a graph of array input sizes versus the mean response time. Once we gathered the data from the simulation trials, we performed a statistical analysis on the data to evaluate the correctness of the hypothesis. ANALYSIS OF RESEARCH PAPER 1 Author: Wang ZhaoShun, Yingjian An and Li Tao Title: Research of software component development based on CORBA

4 Overview: This paper gives the basic idea behind the concept of component based technology, and its relevance to software development as a whole. The paper presents an detailed overview of the CORBA architecture specification and describes MICO- an open source project that implements CORBA features and standards. Conference: The 2nd IEEE International Conference on Information Management and Engineering (ICIME) Publication year : 2010 URL- Approach and Main ideas in the paper i. The paper summarizes the key features of component based software development, highlighting its advantages such as increased reusability of code, better maintenance, loose coupling of modules, increased efficiency and fault tolerance etc. ii. A Component is defined as certain specific features of the software module that has clear boundaries and can communicate with other modules through its Interfaces, to perform a logical flow of operation. Component based development typically separates the interface from implementation, providing for more code reusability and also for changes across a system to be incorporated more efficiently. Here the interface defines the basic signature of the functionality the component provides, and any module using the component can use the functionality as is- or can change parameters to provide different functionality to the code. This also provides for inter- operability across interacting systems. iii. CORBA architecture definition can be viewed as a compilation of three basic participation modules. The Interface Definition Language (IDL) provides the means to describe the interface definitions of the functionality being implemented. Irrespective of the languages used to invoke/develop the system. The Object Request Broker is the backbone of the technology. It provides language specific mappings from IDL to various popular languages used for

5 software development. It also serves as the middleman communicator for various objects to interact on the network. The Internet Inter- Orb Protocol is the underlying communication protocol that uses TCP/IP packets to transfer operation calls and parameters between different ORBs. iv. The various steps in component based development using CORBA are- 1). To write object interfaces, generates IDL file. 2) Use the ORB product IDL compiler to compile the IDL file, generate client code and framework of server code. 3) Implementation of the objects 4) To write a service program code 5) To write client- side code 6) To create and deploy applications 7) Start object request broker, start the server, start the client Contribution of paper to the investigation: This paper provided us with a basic understanding of what component based development is, and also helped us understand the architectural components of CORBA and their use. The steps to develop an application using components, served as the primary guideline to develop the CORBA based Bubble sort application, for the investigation. ANALYSIS OF RESEARCH PAPER 2 Author: Pengwei Zhang and Jingxia Chen Title: Research and Application Of Asynchronous Message Transmission on JMS Overview: This paper talks about the asynchronous Java Messaging System in detail and gives in depth information of the JMS API is structure and implementation. The target audience for this paper is developers who want to build messaging oriented middleware systems using JMS. Conference: nd International Conference on Computer Engineering and Technology (ICCET). Publication year: 2010 URL: Approach and Main ideas in the paper

6 i. In this paper, the authors have described the architectural components and the API capabilities of JMS in details. The paper also outlines the functionality of each component, by taking the example of an online- job seeking portal, developed using JMS. ii. The paper talks about the various components of Java JMS API- 1) JMS Provider: The middleware used for implementing the JMS API interface. 2) JMS Client: Java programs that send/receive messages 3) Messages: Object communicated between JMS clients 4) Messaging Domains: JMS supports two domains (Point to point, Publish/Subscribe) 5) Administered Objects: Pre- configured objects generated by management tools 6) Native Client: Local client API in messaging system 7) Session: Sessions are created for each connection ii. The paper also talks about the two messaging domains in JMs and presents the features of the messages and the communication technique used from sender to receiver in each case

7 Contribution of paper to the investigation: This paper provided us with a basic understanding of JMS is and how it its API functionality is implemented. We also made a choice to used the point- to- point message domain for this simulation. The example of the Job seeking system listed in the paper served ad our primary reference, for developing the JMS based Bubble sort application. ANALYSIS OF RESEARCH PAPER 3 Author: Markus Aleksy,Martin Schader and AlexanderSchnell Title: Design and implementation of a bridge between CORBA's notification service and the Java message service. Overview: This paper talks about the role of messaging in object- oriented distributed systems and describes the design and working of a bridge system, implemented by using the functionalities of CORBA's CNS (Corba Notification Service) and Java JMS. The authors also talk briefly about the advantages to developing such a bridge architecture and the usage of such a system in building distributed applications. Conference: Proceedings of the 36th Annual Hawaii International Conference on System Sciences. Publication year:2003 URL: Approach and Main ideas in the paper i. The paper describes the basic characteristics of both messaging services, CORBA s CNS and Java JMS and then proceeds to describe the architecture of the CNS- JMS bridge. The working of CORBA's CNS is comparable to the publish /subscribe messaging pattern available in JMS. In describing the features the authors talk about the various types of messaging models and various message types supported in both services. ii. iii. The paper highlights specific issues in communication that these messaging services do not address, when implemented individually. This is the motivation for the author's to present the CNS- JMS bridge, that captures the essence of both schemes to handle messaging. The software structure of the bridge can be described as follows- 1) Front- end tasks include initializing objects, loggers, parsing input to establish both CNS and JMS connections. 2) Back- end, this layer is responsible for the connection between the Channel and the Topic, for selection of suitable convertor, message delivery and reception and also handling the configuration parsing at the start of the connection.

8 3) Converter- The task of the converter object is to convert messages form one service architecture to another. Contribution of paper to the investigation: Though this paper did not have a direct contribution to the research investigation, it provided us with a lot of insight into the working of both CNS and JMS publish/subscribe architecture. The idea of a bridge between the two systems is definitely worth exploring in a future time instant. SOFTWARE DESIGN The design section describes the high- level design of the Bubble sort application using CORBA and JMS, with focus on the specific modules and their working in the entire system.

9 BUBBLE SORT USING CORBA The diagram represents the various modules and their interaction within the system. Following the guidelines mentioned in the first research paper, to building the application, the middleware components (ORB and IIOP ) are provided by the CORBA architecture itself. We have developed the Bubble Sort client and server side code in Java, so the CORBA architecture is also responsible for providing the mappings from IDL to Java. The client code basically takes the range of the input array size as command line from the user and then passes this as a parameter to invoke the SortArray() function. The main work of the CORBA middleware then begins, where it resolves the signature of the function call and uses object reference mappings to invoke the corresponding correct implementation of the SortArray() method. At the server end, once the client call arrives, the input array is sorted using Bubble sort that is implemented in the SortArray() function. The running time of this algorithm is 0(n^2) where n is the number of elements to be sorted. Once the array is sorted the results are then returned to the invoking method i.e., back to the client side where the results are displayed. In the case where the client ns server are running on the same machine, this process is a simple resolution of a method call as the object reference is stored locally. However, in the case of Remote method execution, the method call is resolved and passed thought he network

10 (through IIOP) to interact with the correct ORB on the server end and invoke the corresponding function. In both cases, CORBA achieves interoperability and also a separation of interface and implementation. The client side is only aware of the interface definition of the SortArray() method, i.e., only the method signature and the parameters. The actual implementation details remain obscured. BUBBLE SORT USING JMS We have developed the JMS version of Bubble Sort using the API definitions defined in Apache's ActiveMQ distribution, which provides an implementation of the JMS message oriented middleware paradigm. We have chosen the point- to- point model for comparison with CORBA architecture. The Embedded Broker here acts as a server for the middleware communication system. Both the client and the server side code establish connections with this broker to make up the two end points of the system. The Embedded Broker holds the central queue in the system. to which the client submits request and the server side code processes requests. Once the embedded broker and the connections are set up, the client and server establish a session for a series of message exchanges. The client code takes input from user the range of the input array size, initializes the array using random values and pushes this array on to the central queue, encapsulating it as a parameter in the Sort() method call. The server is up and

11 running initially and once the method request arrives on the queue, it receives it processes it and sends back the results to the client side application. The presence of a central queue implements the feature of asynchronous communication, where the server and client need not be connected at the same time instance. The client could submit a request and when the server arrives, it receives and processes it. Till the time a server is processing the request, the method call resides on the queue on the Embedded broker, waiting to be processed. In case of local host execution, this is a trivial matter as once the host and port are specified the communication is established almost instantly and the request starts processing. In the case where the server and the client are on different machines, the wait time in the queue is far more evident, when the server and client are not up and running at the same time. DESIGN OF THE T- TEST PROGRAM To verify the validity of the hypothesis statement, we have used the statistical t- test measure. Once we compute the mean response times of CORBA and JMS for various input array sizes, we have stored these measurement values in a text file and used this text file as input to the t- test program. Each line of text in the file is composed of 4 values, representing the following in the same order 1) Size of the input array. 2) Mean response time of CORBA application when executing on a local machine 3) Mean response time of CORBA application when executing as a remote call 4) Mean response time of JMS application when executing on a local machine 5) Mean response time of CORBA application when executing as a remote call Once the t- test program executes, the t- statistic and p values are computer for the pair of execution times of CORBA and JMs on local host, and on remote host execution.

12 DEVELOPER'S MANUAL In our research investigation, we have developed all out code files in Java using Java version 1.6.To compile and run the files, Java 1.6 needs to be installed in the system. this can be downloaded from the following URL The following libraries also need to be downloaded and installed in order to set up the simulation environment and execute the code on the Computer Science department machines. Computer Science course library: Parallel Java Library: STEPS TO COMPILE CORBA APPLICATION: 1) Download the source code zip file from 2) Extract the JMS directory and set it to current directory. 3) The tool idlj reads OMG IDL files and creates the required Java files. Use - fall option when running the idlj compiler to generate client- side bindings and server- side skeletons. The command here is idlj - fall BubbleSort.idl 4) To compile the Java Files, the command is javac *.java To compile the code on remote machine, the steps are- 1) Create and compile the BubbleSort.idl file on the CLIENT machine: idlj - fall BubbleSort.idl 2) Create HelloClient.java on the CLIENT machine. Compile the *.java files, including the stubs and skeletons (which are located in BubbleSortApp folder): javac *.java BubbleSortApp/*.java 3) Create and compile BubbleSortServer.java on the SERVER machine. 1) javac BubbleSortServer.java STEPS TO COMPILE JMS APPLICATION: 5) Download the Apache ActiveMQ library version5.8.0 from the following URL 6) Download the source code zip file from 7) Extract the JMS directory and set it to current directory.

13 8) Set the CLASSPATH in order to include CSCL.jar, PJ.jar,activemq- all jar to your compilation. To achieve this, the command to be typed into terminal is- export CLASSPATH=.:<path of cscl.jar>:<path of pj.jar>:<path of activemq- all jar> 9) To execute the source files in the JMS directory, the command is javac *.java When executing the JMS application on remote machines, i.e., the server and client on different machines, a single line of code needs to be changed in order to specify the ip address of the host machine that the connection is established to. This change is in line 15 of the EmbeddedBroker.java file. Once this is done, save and recompile the code using steps 4 and 5 specified above. STEPS TO COMPILE T- TEST CODE: 1) Download the source code from the URL: 2) Extract the t- test directory and set it to current directory. 3) Set the CLASSPATH in order to include CSCL.jar and PJ.jar to your compilation. To achieve this, the command to be typed into terminal is- export CLASSPATH=.:<path of cscl.jar>:<path of pj.jar> 4) To execute the source files in the JMS directory, the command is javac *.java USER'S MANUAL The instructions given below are the series of steps to execute the application and observe the results. The assumption here is that all software source code has been successfully complied using the instructions specified in the Developer's manual. STEPS TO RUN CORBA BASED BUBBLE SORT APPLICATION: 1) Compile the source code as instructed in the developer's manual 2) LOCAL HOST EXECUTION: i) Start orbd (Naming Server) :To start orbd from a UNIX command shell, please enter: orbd - ORBInitialPort <port> - ORBInitialHost localhost ii) Start the BubbleSort server : To start the server from a UNIX command shell, enter: java BubbleSortServer - ORBInitialPort <port> - ORBInitialHost localhost iii) Run the client application. The command is java BubbleSortClient - ORBInitialPort <port> - ORBInitialHost localhost <elcount> <seed>

14 3) REMOTE HOST EXECUTION: i) Start orbd (Naming Service) on the SERVER machine. orbd - ORBInitialPort <port> - ORBInitialHost servername ORBInitialPort and - ORBInitialHost are required arguments on the orbd command line. The ORBD must be run on the same machine, as the servers will be activated. ii) On the SERVER machine, start the BubbleSort server: java BubbleSortServer - ORBInitialPort <port> - ORBInitialHost servername iii) Run the client program on the CLIENT machine: java BubbleSortClient - ORBInitialPort <port> - ORBInitialHost servername <elcount> <seed> servername is the host on which the IDL name server is running. In this case, it is the server machine. The name server and server will continue to wait for invocations until they explicitly stopped by the user. STEPS TO RUN JMS BASED BUBBLE SORT APPLICATION: 1) Compile the source code using instructions from Developer's manual 2) LOCAL HOST EXECUTION: i. Start the message Broker service by executing the embedded Broker. The command is- java EmbeddedBroker ii. In a new terminal window, set class path and execute the Server side code by running the Sorter program. The command is- java Sorter iii. Run the client side code on a new terminal window, by executing the command java Requester <size> <seed> where, <size> is the size of numbers (range of input array size)to be generated for sorting. <seed> is some random seed. 3) REMOTE HOST EXECUTION: i. Here both the server and client run on different machines. After making the change in the code as specified in the Developer's manual, execute the commands in the same sequence as above i.e., ii. Start the message Broker service by executing the embedded Broker. The command is- java EmbeddedBroker

15 iii. iv. In a new terminal window, set class path and execute the Server side code by running the Sorter program. The command is- java Sorter Run the client side code on a new terminal window, by executing the command java Requester <size> <seed> where, <size> is the size of numbers (range of input array size)to be generated for sorting. <seed> is some random seed. STEPS TO RUN T- TEST APPLICATION: 1) Follow the instructions given in the developer's manual to compile the t- test program. 2) Execute the code by using the command java VarianceTest <inputfile> where, <inputfile> is the input data file containing the response time data. DATA COLLECTED FROM EXPERIMENTS We ran both applications for different array sizes, calculated their response times and the mean response for each input array size. We fixed the number of trials as 10. CORBA LOCAL HOST EXECUTION: Array size : 5000 Response time (ms): Mean response time(ms): 169 Array size: Response time(ms): Mean response time(ms): Array size : Response time(ms): Mean response time(ms): Array Size: Response time(ms): Mean response time(ms):

16 Array Size: Response time(ms): Mean response time(ms): CORBA REMOTE HOST EXECUTION: Array size : 5000 Response time (ms): Mean response time(ms): 69 Array size: Response time(ms): Mean response time(ms): 5145 Array size : Response time(ms): Mean response time(ms): Array Size: Response time(ms): Mean response time(ms): Array Size: Response time(ms): Mean response time(ms): JMS LOCAL HOST EXECUTION: Array size : 5000 Response time (ms): Mean response time(ms): 86 Array size: Response time(ms): Mean response time(ms): 2733 Array size : Response time(ms): Mean response time(ms): Array Size: Response time(ms): Mean response time(ms): 48588

17 Array Size: Response time(ms): Mean response time(ms): JMS REMOTE HOST EXECUTION: Array size : 5000 Response time (ms): Mean response time(ms): 407 Array size: Response time(ms): Mean response time(ms): 3470 Array size : Response time(ms): Mean response time(ms): Array Size: Response time(ms): Mean response time(ms): Array Size: Response time(ms): Mean response time(ms): Once the response times were calculated, we compared the array sizes with the mean response times of both CORBA and JMS in both local and remote host executions. The resultant data obtained is as follows- INPUT ARRAY SIZE MEAN RESPONSE TIME- CORBA- LOCALT HOST MEAN RESPONSE TIME- JMS- LOCAL HOST MEAN RESPONSE TIME- CORBA- REMOTE EXECUTION MEAN RESPONSE TIME- JMS- REMOTE EXECUTION

18 GRAPH PLOT- CORBA and JMS execution on LOCAL host GRAPH PLOT- CORBA AND JMS ON REMOTE HOST

19 ANALYSIS OF DATA COLLECTED FROM EXPERIMENTS Preliminary Analysis: By looking at the tabular data obtained from the experiments, it is evident that the CORBA mean response time is higher than that of JMS mean response time, i.e., CORBA takes longer than JMS to process requests, which is in accordance with the proposed hypothesis statement. Statistical Analysis: In order to statistically analyze the data obtained, the unequal variance t- test is used as a benchmark. The mean response time information is fed to the t- test program and the results obtained is as follows- Local Host Execution: T- test for Response Times on localhost T- Statistic: p- value: Here we can observe that the t- statistic value is positive, which indicates that the mean response time of CORBA is greater than that of JMS. Also as the difference between the mean response times of both communication methods (CORBA and JMS) decreases, the p- value move away from zero. Remote Host Execution T- test for Response Times on Remote Machine T- Statistic: p- value: Since the p- value is larger than zero, the difference between mean response times of two protocols is very less. This fact is illustrated in the graph plot as well. Moreover, since t- value is positive the response time of CORBA is higher, than that of JMS, but the difference is comparatively less when compared to local host execution. FUTURE WORK In this investigation, we chose to perform a performance comparison of CORBA with the point- to- point messaging domain model of JMS. These two technologies could be compared and contrasted on various other benchmarks like protocol overhead,

20 TCP packet traffic, file data transfer rate etc. that might be of importance to analyze performance. A comparison of the publish subscribe model of JMS with CORBA notification service, would present for an interesting investigation in itself. Instead of choosing Bubble Sort as a benchmark program for comparison, we could choose some other industry- defined benchmark to analyze performance. CONCLUSION The analysis of the data presented above evaluates the validity of out hypothesis as stated below- Mean response time in a JMS based distributed system is less than the mean response time in a CORBA based distributed system. The statistical tests performed also strongly support the correctness of the above stated hypothesis statement. LEARNING FROM THIS INVESTIGATION This research investigation helped us understand the process of developing a hypothesis and designing simulation experiments to come up with data, and also analyze the data to prove the validity of the hypothesis. We also learnt, in great detail about the working of both CORBA and JMS architectures, and their use in the development of distributed applications. INDIVIDUAL CONTRIBUTION Though the investigation was a collaborative team effort, we decided to allocate specific parts of the task execution among ourselves to achieve the results in the allotted period of time. Tigran was responsible for the development and testing of the CORBA application and the task of developing the JMS part in the investigation was handled by Sujal. The data collection and statistical analysis of part of the investigation was handled by Vandana. The team presentations and the final report was a group effort, with all inputs being presented together as a comprehensive document.

21 REFERENCES [1] Parallel Java Library by Professor Alan Kaminsky, Rochester Institute of Technology. [2] Computer Science Course Library by Professor Alan Kaminsky, Rochester Institute of Technology [3] Apache ActiveMQ [4] Paper 1: Research of software component development based on CORBA [5] Paper 2: Research and Application Of Asynchronous Message Transmission on JMS [6] Paper 3: Design and implementation of a bridge between CORBA's notification service and the Java message service.