ON THE ROAD TO BENCHMARKING BPMN 2.0 WORKFLOW ENGINES

Transcription

1 ON THE RO TO ENHMRKING PMN 2.0 WORKFLOW ENGINES Marigianna Skouradaki, ieter H. Roller, Frank Leymann Institute of rchitecture and pplication Systems University of Stuttgart Germany Vincenzo Ferme, esare Pautasso Faculty of Informatics University of Lugano (USI) Switzerland

2 What is a Workflow Engine? Workflow Engine Task ispatcher Process Navigator Users Job Executor ore Engine Service Invoker Web Service Transaction Manager Persistent Manager Instance atabase pplication Server MS 2

3 Many usiness Process Modeling/Execution Languages EP XPL PEL YWL PNML PMN

4 PMN 2.0: Widely dopted Standard PMN 2.0 Jan 2011 ISO/IE PMN Jan

5 Why do we need a benchmark? companies, developers 5

6 Why do we need a benchmark? companies, developers 1. How to choose the best engine according to the company requirements? 2. How to choose the best engine according to the company business process models? 5

7 Why do we need a benchmark? companies, developers 1. How to choose the best engine according to the company requirements? 2. How to choose the best engine according to the company business process models? 3. How to evaluate performance improvements during the engine development? 4. How to find out the engine bottlenecks? 5

8 Main hallenges in enchmarking PMN 2.0 Workflow Engines 6

9 Main hallenges in enchmarking PMN 2.0 Workflow Engines WORKLO HRTERIZTION ENHMRK EXEUTION 6

10 Main hallenges in enchmarking PMN 2.0 Workflow Engines 20% WORKLO HRTERIZTION 80% 1. efine the Workload Mix 2. efine the Load Functions ENHMRK EXEUTION 6

11 Main hallenges in enchmarking PMN 2.0 Workflow Engines 20% WORKLO HRTERIZTION 80% 1. efine the Workload Mix 2. efine the Load Functions Engine Users lient Engine x Engine Web Services ENHMRK EXEUTION 3. eal with engine-specific interfaces and PMN 2.0 customizations Instance atabase 4. synchronous execution of business processes 5. efine meaningful and reliable KPIs 6

12 1. efine the Workload Mix 7

13 1. efine the Workload Mix ontrol Flow ata Flow Events ctivities Task Types Execution ehavior G H E F I 7

14 1. efine the Workload Mix NUMER OF REL-WORL MOELS NUMER OF ENGINES SUPPORTING THE FETURE 8

15 1. efine the Workload Mix NUMER OF REL-WORL MOELS NUMER OF ENGINES SUPPORTING THE FETURE 8

16 1. efine the Workload Mix NUMER OF REL-WORL MOELS NUMER OF ENGINES SUPPORTING THE FETURE 8

17 1. efine the Workload Mix NUMER OF REL-WORL MOELS F NUMER OF ENGINES SUPPORTING THE FETURE 8

18 2. efine the Load Functions Start Events Workflow Engine Users Users Start Web Service pplication Server Instance atabase Web Services MS 9

19 3. eal with engine-specific interfaces and PMN 2.0 customizations Loading river Workflow Engine pplication Server Users Web Service Instance atabase MS 10

20 4. synchronous execution of processes Start Loading river Workflow Engine pplication Server Users Web Service Instance atabase MS 11

21 4. synchronous execution of processes Start Loading river End Workflow Engine Users Web Service pplication Server Instance atabase MS 11

22 The enchflow Project esign the first benchmark to assess and compare the performance of Workflow Engines that are compliant with usiness Process Model and Notation 2.0 (PMN 2.0) standard 20% 80% Engine Users lient Engine Engine Web Services Instance atabase 12

23 1. efine the Workload Mix E F G H REL-WORL PROESSES What we need: even more (anonymized) real-world PMN 2.0 process models 13

24 1. efine the Workload Mix Skouradaki et al. [SOSE2015] Graph Matching a1 a2 E F G a3 a6 a5 H a4 REL-WORL PROESSES REOURRING STRUTURES What we need: even more (anonymized) real-world PMN 2.0 process models 13

25 1. efine the Workload Mix Skouradaki et al. [SOSE2015] Graph Matching Selection riteria a1 a2 E F G a3 a6 a5 H a4 REL-WORL PROESSES REOURRING STRUTURES What we need: even more (anonymized) real-world PMN 2.0 process models 13

26 1. efine the Workload Mix Skouradaki et al. [SOSE2015] Graph Matching Selection riteria a1 a2 omposition riteria 50% a1 a2 E F G a3 a6 a5 50% a3 a5 H a4 REL-WORL PROESSES REOURRING STRUTURES WORKLO MIX What we need: even more (anonymized) real-world PMN 2.0 process models 13

27 Enabling the enchmark Execution Loading river Workflow Engine pplication Server Users Web Service Instance atabase MS 14

28 Enabling the enchmark Execution harness Faban Faban rivers Workflow Engine Users pplication Server Web Service Loading Functions Instance atabase MS 15

29 Enabling the enchmark Execution Faban harness Faban rivers Workflow Engine MS Web Service Loading Functions 1. Flexible deployment 2. Flexible HW Resources 3. Frozen Initial ondition Servers ocker ontainers 16

30 Enabling the enchmark Execution harness Faban rivers Workflow Engine MS Faban + 17

31 Enabling the enchmark Execution harness Workflow Engine Faban rivers MS Faban + 1. utomatically deploy and start the benchmark environment; 17

32 Enabling the enchmark Execution harness Workflow Engine Faban rivers MS Faban + 1. utomatically deploy and start the benchmark environment; 2. utomatically deploy the workload mix; 17

33 Enabling the enchmark Execution harness Workflow Engine Faban rivers MONITOR MS Faban + 1. utomatically deploy and start the benchmark environment; 2. utomatically deploy the workload mix; 3. etermine when the benchmark ends; 17

34 Enabling the enchmark Execution Workflow Faban rivers harness MONITOR Engine MS OLLETORS Instance atabase Faban + 1. utomatically deploy and start the benchmark environment; 2. utomatically deploy the workload mix; 3. etermine when the benchmark ends; 4. ollect the execution and process logs. 17

35 The enchflow Project Next Steps Release the first prototype of the enchmark environment» Yes: bstract the Interaction with the Engines; utomatic eploy and Undeploy of the S.U.T.; Execution and Process Log Gathering» No: utomatic Generation of rivers; Users, Web Services and External atching usiness Events Release the first prototype of the Workload Mix synthesizer First Experiments with KPIs efinition and omputation ollect More Process Models and Process Execution Logs enchflow Project: 18

36 KUP SLIES ited Works; Related Works. Marigianna Skouradaki, ieter H. Roller, Frank Leymann Institute of rchitecture and pplication Systems University of Stuttgart Germany Vincenzo Ferme, esare Pautasso Faculty of Informatics University of Lugano (USI) Switzerland

37 ited Works [SOSE2015] Skouradaki, Marigianna; Goerlach, Katharina; Hahn, Michael; Leymann, Frank. pplication of Sub-Graph Isomorphism to Extract Reoccurring Structures from PMN 2.0 Process Models. In Proceedings of 9th International IEEE Symposium on Service-Oriented System Engineering (SOSE 2015). San Francisco ay, US, March 30 - pril 3, (to appear) MS 20

38 Related Works ctive Endpoints Inc. ssessing ctivevos performance, content/developers/ technical_notes/assessing_activevos_performance.pdf.. ianculli, W. inder, and M. L. rago. SOench: Performance evaluation of service-oriented middleware made easy. In Proc. of ISE 10 - Volume 2, pages , J. ardoso. usiness process control-flow complexity: Metric, evaluation, and validation. International Journal of Web Services Research, 5(2):49 76, G. in, K.-P. Eckert, and I. Schieferdecker. workload model for benchmarking PEL engines. In Proc. of ISTW 08, pages , M. umas, L. Garćıa-añuelos, and R. M. ijkman. Similarity search of business process models. IEEE ata Eng. ull., 32(3):23 28, J. Gray. The enchmark Handbook for atabase and Transaction Systems. Morgan Kaufmann, 2nd edition, G. Hackmann, M. Haitjema,. Gill, and G.-. Roman. Sliver: PEL workflow process execution engine for mobile devices. In Proc. of ISO 06, pages Springer, S. Harrer, J. Lenhard, and G. Wirtz. PEL conformance in open source engines. In Proc. of SO 12, pages 1 8, MS 21

39 Related Works K. Huppler. The art of building a good benchmark. In Performance Evaluation and enchmarking, pages Springer, Intel and ape lear. PEL scalability and performance testing. White paper, F. Leymann. Managing business processes via workflow technology. In Proc. of VL 2001, pages 729, Liu, Q. Li, L. Huang, and M. Xiao. Facts: framework for fault-tolerant composition of transactional web services. IEEE Trans. on Services omputing, 3(1):46 59, J. Mendling. Metrics for Process Models: Empirical Foundations of Verification, Error Prediction, and Guidelines for orrectness. Springer, I. Molyneaux. The rt of pplication Performance Testing: Help for Programmers and Quality ssurance. O Reilly, M. Z. Muehlen and J. Recker. How much language is enough? theoretical and practical use of the business process modeling notation. In Proc. of ise 08, pages , Röck and S. Harrer. Literature survey of performance benchmarking approaches of PEL engines. Technical report, Otto-Friedrich University of amberg, H. Roller. Throughput Improvements for PEL MS Engines: Implementation Techniques and Measurements applied in SWoM. Ph thesis, University of Stuttgart,

40 Related Works N. Russell, W. M. van der alst, and. Hofstede. ll that glitters is not gold: Selecting the right tool for your PM needs. utter IT Journal, 20(11):31 38, Schumm,. Karastoyanova, O. Kopp, F. Leymann, M. Sonntag, and S. Strauch. Process fragment libraries for easier and faster development of process-based applications. SSI, 2(1):39 55, M. Skouradaki,. Roller,. Pautasso, and F. Leymann. PELanon: nonymizing PEL processes. In Proc. of ZEUS 14, pages 9 15, Sun Microsystems. enchmarking PEL service engine, Wiki.jsp?page=pelPerformance.html.. Wetzstein, P. Leitner, F. Rosenberg, I. randic, S. ustdar, and F. Leymann. Monitoring and analyzing influential factors of business process performance. In Proc. of EO 09, pages , MS 23