AHP Based Comparison of open-source BPM Systems

AHP Based Comparison of open-source BPM Systems Dragan Mišić *, Milena Mišić **, MilanTrifunović *, Tanja Arh, Pipan Matić *** * Faculty of Mechanical Engineering, Niš, Serbia ** Student at the Faculty of Electronic Engineering, Niš, Serbia *** Jozef Stefan Institute, Ljubljana, Slovenia misicdr@gmail.com, misicmilena3@gmail.com, milant@masfak.ni.ac.rs, tanja@e5.ijs.si, matic@e5.ijs.si Abstract Although Business Process Management in companies can be done without using software tools, it is much more efficient to use corresponding software systems as help. Those systems are called Business Process Management systems (BPMS). Today there are many various tools available in the market, which differ in technologies that are used to develop them and which are based on different business models. In order to facilitate the process of choosing the most appropriate tool, in this paper we compared a few solutions, which are based on opensource business model. The comparison was done based on about 20 criteria which are evaluated and ranked by using the AHP method. I. INTRODUCTION The world economy has been experiencing crisis for quite a while. Therefore, there is not enough money and many companies have collapsed. This particularly applies to Serbian economy which was not in a good condition even before the crisis. How is it possible to find money for a software that could improve the functioning of the company under those circumstances? One of the possible solutions is using a software which comes from the vendor with open source philosophy. When someone mentions open-source software, the first thing that comes to mind is that such software is free. That can be, but it is not always true. The most important characteristic of this software is that people get it along with the program code. That code can be adjusted to suit specific needs, and it can further be distributed under the same conditions. The number of both profit and non-profit organizations that use the advantages of the open-source industry is rising rapidly [1]. Open-source software has become a serious competitor to commercial programs [2]. Even though we are not talking about a free software, the amounts which are paid for this kind of program are much smaller than those that are needed to purchase a commercial software of the same category. That is one of the reasons for which we focused our analysis on systems that are from the category of open-source software. Today, the process orientation of a company is something that is hardly ever questioned [3] [4]. There are some empirical researches that indicate the undoubtful connection between BPM (Business Process Management) and customers' satisfaction [5]. When we are talking about BPMS it is clear that the first thing to do is to define what the term 'process' implies, and then to define what is these systems' task in regard to that process. According to [6], a business process is "a set of one or more linked procedures or activities which collectively realize a business objective or policy goal, normally within the context of an organizational structure defining functional roles and relationships". Processes that are defined in this manner must be managed. That management is the task of BPM. BPMS are systems which represent the evolution of Workflow Management Systems (WfMS), which first appeared in the nineties. According to Workflow Management Coalition [6], WfMS is "A system that defines, creates and manages the execution of workflows through the use of software, running on one or more workflow engines, which is able to interpret the process definition, interact with workflow participants and, where required, invoke the use of IT tools and applications". BPMS originate from the evolution of WfMS. For example, van der Aalst, one of the world's leading researchers in this area, defines BPMS as "Supporting business processes using methods, techniques, and software to design, enact, control, and analyse operational processes involving humans, organizations, applications, documents and other sources of information" [7]. According to the two definitions above, it can be seen that the main difference between WfMS and BPM system is the possibility of analysing that exists in BPM systems. In addition to the use of BPMS in process management in organizations, these systems can also be used to

integrate software components and build a more complex software. Further in this paper, alternatives and the way they are evaluated are explained in more detail. The structure of the paper is the following: In the second chapter we briefly describe systems which were evaluated. The third part explains the AHP method which was used to evaluate the alternatives, and gives the explanation of the criteria used to evaluate systems. Results are shown in the fourth chapter, and the conclusion is presented in the fifth part. II. ALTERNATIVE SYSTEMS FOR BUSINESS PROCESS MANAGEMENT Today, there is a large number of various products which are related to computer management of business processes. That number is approximately a few hundred. Some of those products are independent, and some are embedded in some other software products. It is clear that it is very difficult for the organization to pick one product out of many that are available, and be sure that the one they picked will suit their needs in the best way possible. That task requires a detail analysis, but also the knowledge in the area of BPM by ones that perform the analysis. The goal of this paper is to provide users with some recommendations and help with choosing the right alternative. Due to limited space and time, we decided to restrict the analysis to a few tools. We chose those tools based on our experience and the analysis of users' experiences which are posted online. One of the conditions was that the products exist for a certain number of years, in other words that they have matured. It was also required that there is a community which uses those products and monitors their development. We have already mentioned that the comparison is focused on open-source products, because we estimated that the price of these products is something that our organizations and companies are able to pay. In accordance with all aforementioned we made the comparison of five BPM tools. Those are: Intalio, jbpm, ProcessMaker, Bonita i Yawl. Some of the chosen tools are fully available under open-source licenses, and some of them have versions which are distributed as open-source, and also as commercial versions. Intalio (www.intalio.com) is one of the first BPM projects. This tool is available under open-source license, with limited capabilities, and under commercial license which offers a full set of characteristics. Intalio consists of a visual process designer, machine for execution, and also a number of additional tools that you would expect to find within a modern BPM system. Some of those tools are Business Activity Monitoring (BAM), support tool for business rules, document management system, various integration tools etc. It is written in Java programming language. jbpm (www.jbpm.org) is also a very mature product which has been being developed for a number of years. It is probably one of the first references in the area of opensource BPM tools. jbpm is a tool which is fully available in open-source version. According to its authors, one of its advantages is that it is designed for both non-technical users and developers that intend to adjust the tool to their own needs. It also has a visual designer (two types of it Eclipse based for developers and web based for business users), support tool for monitoring of business activities, a possibility of defining rules, support tool for managing documents etc. It is also written in programming language Java. ProcessMaker (www.processmaker.com) is another BPM tool that is often mentioned as one of the best in the category of open-source software. Unlike other products that we mentioned, this one is written in PHP. Visual tool for creating this process is completely web-based, which is one of the advantages of this product.it is fairly easy to integrate it and connect it to the other systems, such as SugarCRM etc. It is also not completely open-source; there is a community version, and also an enterprise version which costs a certain amount of money, and has additional options. Bonita (www.bonitasoft.com), which comes from company called BonitaSoft, offers both community and commercial version. It is also developed in Java. It is a product which has existed for years, and it comes with a lot of features that are usual for quality and mature BPM systems. Process Designer is a high quality product, it complies with the standards (BPMN2), and also offers the possibilities of process simulation, integration with other tools, monitoring the activities etc. Yawl (www.yawlfoundation.org) is the last one of the tools that we compared in this paper. It is also a completely open-source product, and some of the leading experts in the area of BPM systems, such as van der Aalst, Holfstede and others, participate in its development. It is based on modeling language Yawl, which is probably one of the best tools in the aspect of support for workflow patterns. Yawl was also developed in Java programming language. III. ANALYTIC HIERARCHY PROCESS AND CHOSEN CRITERIA In our everyday lives, we make various decisions on daily basis. Sometimes we do that intuitively, and sometimes our decisions are based on detail researches. We use the similar algorithm when we choose BPMS that will suit our needs in the best way possible. There are lots of factors that should be taken into account when making the decision. In order for that to be done in a consistent manner, we decided to use systematic, mathematically proven process of choosing between alternatives AHP (Analytic Hierarchy Process) [8,9]. The AHP is a decision support tool which can be used to solve complex decision problems. It uses a multi-level hierarchical structure of objectives, criteria, subcriteria, and alternatives. The pertinent data are derived by using a set of pairwise comparisons. These comparisons are used to obtain the weights of importance of the decision criteria, and the relative performance measures of the alternatives in terms of each individual decision criterion. If the comparisons are not perfectly consistent, then it provides a mechanism for improving consistency. Various authors have various ideas about criteria that a good BPMS must meet. As far back as 2006, Gartner listed some of criteria in his report [10]. Nonacademic press and research institutions such as Gartner and Forrester regularly release reviews on BPM tools, but their analysis are mainly focused on commercial products,

which are not appropriate for the conditions prevailing in Serbia, mostly because of their price. Some other authors also tried to evaluate the quality of BPM system based on their criteria. For example, in [11] the comparison was done based on workflow patterns that a certain system can support. In [12] the case of a BPM tool selection in an Australian government agency is reported, where 10 products from major vendors were evaluated using a weighted scoring model with 47 criteria grouped into six main categories. Based on the review of the literature and our experience in working with BPMS, we came up with about 20 criteria, which we divided in several categories. Criteria based on which we did the software evaluation are divided in 8 categories (clusters). Some of these categories are further divided into sub-categories, while others are not. The first category refers to some external aspects of the system, instead of basic tool characteristics. This includes: Documentation Quality of the documentation, both for end users and developers Installation Simplicity of installation, and the duration of it Database management systems that are used for data storage Whether the system can function and be connected to various database management systems Licenses and price Although they are opensource, some of the tools have a commercial version, and this criteria helps to take that into account Vendor support What kind of support a user gets from the vendor, and how great that support is. Whether there are forums, blogs, e-mail support etc. Business templates and models Whether there are business process template that users can use to improve their processes, and what their quality is. In the second category, we presented criteria related to compliance with standards and possibility to connect to other systems. This includes: Compliance with standards This refers to compliance with modeling standards (BPMN2), support for XPDL, BPEL and other standards in the area of BPM systems. Communication with other systems BPMS are not used in isolation. It is often needed for the model, which was made in a certain tool, to be transferred to some other system, and vice versa. Integration capabilities These systems are often connected to other elements of information systems in organizations. This criterion refers to capability of interoperating, and the simplicity of it. Third category of criteria refers to abilities of the tool in the aspect of simulation and process and activity monitoring, and it also refers to system's ability to respond to exceptions. These criteria are: Simulation Capability of process simulation, based on given parameters BAM (Business Activity Monitoring) Capability of the tool to monitor the progress of the process, analytical capability, historical event analysis etc. Exceptions handling support This criteria evaluates possibilities of the system to react to exceptions which occur during the process execution The fourth group of criteria is related to users. This includes: User interface This refers to potential portals, document management system, search, capability of creating portlets etc. Quality of modeling tool This criterion compares visual designers that are used for process modeling. Team collaboration support Whether the process can be shared, whether there is a support tool for discussions, an access to shared work lists, potential support for group decisions making systems etc. Along with the aforementioned criteria, which represent categories with sub-criteria, we used criteria which we did not further divide. This includes the following criteria: Support for user management This criterion is used to make comparisons between the tools based on the routing method (work assignment), their capability of describing roles, capability of importing organizational structures from other systems etc. Support for automatic task and integration This criteria evaluates capabilities of the tool which are related to sending messages, access to data and its transformation, interoperability with various platforms, creating of web service, creating of complex services, finding services etc. Support for defining and managing the rules Various capabilities to present the rules (decision tables, pseudolinguistic rules etc.), capability of dynamic rule defining (separately from the machine), testing the rule order, the rule version etc. Human task support This criterion compares the tools based on their support for manual activities (activities that are executed by human). This refers to whether it is possible to define manual tasks, whether they can be divided into levels, the capability of changing the workflow on the fly, adaptive representation of the workflow etc. IV. COMPARISON RESULTS After having defined the criteria, the next thing to do was to put a procedure that AHP method implies into practice. In order to do that, we needed a sample process. We have been working on a project 'Virtual human osteoarticular system and its application in preclinical and clinical practice at the Faculty of Mechanical Engineering. One of the goals of that project is defining the process of designing and manufacturing customized osteofixation material. Since we needed a sample process for comparison of the BPM tools, we decided to choose exactly that one. This process is very complex, so we simplified it for the purposes of this paper. That was done in such a manner that the majority of the elements that could be used to evaluate the tools remain in the process. Process mainly consists of manual activities, but there are some system activities, too. This process contains various data, such as textual, numerical, and it also contains files

(patients' images). We will not further explain this process in this paper, it is only important to note that the same process was used as a sample process. The first step when comparing tools is defining the importance of certain criteria. In AHP, this is done by pairwise comparisons between criteria. Based on that, we get a list which shows the importance of certain criteria (in this case clusters). Clusters and their importance are shown in table 1. Evaluation of the clusters is done based on our experience and literature recommendations. Within each one of the clusters, importance of subcriteria should be further defined. That is done for the purposes of this comparison too, but those tables will not be shown here. After we defined the importance of the clusters and subcriteria within clusters, we defined the quality of the tool based on defined criteria. Within the AHP method, every alternative is compared to every other, in regard to every criterion. Since there are a lot of pairwise comparisons in regard to a single criterion, it is important to provide the lowest possible value of inconsistency. Officially, the inconsistency index has to be below 10% in order to consider the result useful. The inconsistency index for pairwise comparisons between clusters given below is 5.38%. Based on comparisons we made, the following results are obtained. Those results are shown in table 2, and graphic display is given in figure 1. Criteria clusters TABLE I. CLUSTER WEIGHT Normals Human task support 0.33605837 User management 0.167295041 System tasks and integration 0.167294942 Human interface support 0.11007005 Defining and managing the rules 0.069340688 Simulation 0.064086442 Standards 0.059900098 Documentation 0.02595437 TABLE II. RESULTS OF COMPARISON Alternatives Normals 1 Bonita 0.296282 2 ProcessMaker 0.200709 3 Intalio 0.179399 4 YAWL 0.166032 5 jbpm 0.157578 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0.296282 0.179399 Alternatives 0.157578 Figure 1: Results of comparison As can be seen, the best rated tool is Bonita, which is followed by ProcessMaker, the third one is Intalio, and the last ones are Yawl and jbpm. These results have perhaps been expected, because the tools which got the highest grades also have commercial versions, which show that a large group of people is constantly working on their development. Naturally, this does not mean that nobody works on the developments of jbpm and Yawl, it only means that their development runs at a slower pace. It needs to be noticed that all of the mentioned tools work well when it comes to basic things, such as designing the process and its execution, which is in fact the reason BPMS exist. The difference we got is made by additional options, which are related to interface quality, possibility of simulation etc. V. CONCLUSION 0.200709 0.166032 Bonita Intalio jbpm ProcessMaker YAWL Based on obtained results, we can give recommendations to Serbian organizations about choosing the right BPM tool. The author's opinion is that it is illusory to expect that our companies and organizations raise enough money to invest in purchasing some of the leading commercial products in this area. The main reason is the price of those products, which is estimated at hundreds of thousands dollars. It is more realistic to expect that some of the opensource variants will be used here in Serbia, either completely open-source or some of the commercial versions. Although we did not mention prices in this paper, they are also one of the criteria, but maybe not so important for Serbian organizations. Commercial versions of aforementioned tools are cheaper than those of commercial world's leaders, and they start with about ten thousand Euros a year. Naturally, the prices vary depending on the license type and received support. jbpm and Yawl, which were evaluated as worst of the tools, are maybe the best solutions for us, because all of the options that the tools offer are available in open-source variants. Again, that does not mean that everything is completely free, because certain amount of time must be spent on exploring and adjusting these tools, but this is probably the most appropriate option for Serbian organizations and companies. Since all of the tools are open-source, it should be kept in mind that there are additional possibilities of adapting the code to specific needs. A certain amount of money must be spent on this, since programmers who are able to adapt the code must be paid. Regardless of the above-mentioned problems, the author's opinion is that it is most profitable to use some of the open-source BPM tools, because they offer the satisfying ratio between invested assets and their usefulness for companies and organizations.

ACKNOWLEDGMENT The paper is part of the project III41017 - Virtual Human Osteoarticular System and its Application in Preclinical and Clinical Practice, sponsored by Republic of Serbia for the period of 2011-2014. REFERENCES [1] TDG, Open Source Software: Case Studies Examining Its Use, The Dravis Group, 2003 [2] Peeling, N. and J. Satchell, Analysis of the Impact of Open Source Software, 2001, QinetiQ. [3] Harmon, P., Business Process Change: A guide for business managers and BPM and six sigma professionals, 2nd ed., (Amsterdam: Elsevier/Morgan Kaufmann Publishers), 2007 [4] Hammer, M., Deep Change, Harvard Business Review, 2004 [5] Kumar, V., Smart, P.A., Maddern, H. and Maull, R.S., Alternative perspectives on service quality and customer satisfaction: the role of BPM, International Journal of Service Industry Management, 19 (2): 176-187, 2008 [6] Coalition, W., Workflow Management Coalition Terminology and Glossary. Technical report, Workflow Management Coalition, 1999 [7] van der Aalst, W., Business Process Management: A personal view. Business Process Management Journal, 2004 [8] T. L. Saaty, The Analytic Hierarchy Process. New York: McGraw Hill, 1980. [9] T. L. Saaty, Decision Making for Leaders The Analytic Hierarchy Process for Decisions in a Complex World, 3 ed. Pittsburgh: RWS Publications, 2001. [10] Selection Criteria Details for Business Process Management Suites, Gartner BPM report, 2006 [11] P. Wohed and N. Russell and A.H.M. ter Hofstede and B. Andersson and W.M.P. van d er Aalst, Patterns-based evaluation of open source BPM systems : The cases of jbpm, OpenWFE, and Enhydra Shark, Information & Software Technology, vol. 51, no. 8, pp. 1187 1216, 2009. [12] I. Davies and M. Reeves. BPM Tool Selection: The Case of the Queensland Court of Justice. Handbook on Business Process Management 1, pages 339 360, 2010