Methods and Technologies for Business Process Monitoring

Transcription

1 Methods and Technologies for Business Monitoring Josef Schiefer Vienna, June 2005

2 Agenda» Motivation/Introduction» Real-World Examples» Technology Perspective» Web-Service Based Business Monitoring» Adaptive Business Monitoring with Sense & Respond» Modeling Sense & Respond» Business Monitoring with Senactive» Diplomarbeitsthemen in the area of business process management & monitoring» Q&A 2

3 Motivation 3

4 Motivation "What one can measure, shall be measured; what cannot be measured, shall be made measurable." Schmalenbach (1963) "If you cannot measure it, you cannot control it. And if you cannot control it, you cannot manage it." Harrington (1991) "Large scale production workflow systems with thousands of interactive tasks per day will generate megabytes of audit trail data for analysis." What do you do with this information? McLellan (1996) 4

5 Why Business Monitoring?» Automatically extracts and generates performance data from enterprise processes» Uncovers weaknesses in process handling» Optimize throughput times, resource assignments» Warning system by monitoring business processes» Benchmarking based on measurable process indicators -> identification of best practices» -driven analysis of business data (e.g. break down metrics by business activities)» Past activities can be used to drive prospective interactions with the customer» Continuous process improvement, closed loop analysis 5

6 Introduction 6

7 Business Management Lifecycle Business Modelling Implementation Business Execution BPM End-to-End Tooling Business Analysis & Control 7

8 How well is Performing a Business? M1 Executive Objectives M2 Vice Pres./Director Objectives - Fully exploit customer potential - Improve profitability - Customer satisfaction - Shorten delivery and supply time - Increase Return on Investment - Increase delivery performance M3 M3 M3 M2 M1 M1 External M4 M4 M4 M4 M4 Internal Task 1 Task 2 Task 3 Task 4 Task 5 Outcomes Customer Sub 1 Sub 2 M3 Manager Objectives M4 Workgroup Objectives - Reduce service costs - Reduce order processing time - Improve service quality - Optimize resource assignments 8 - Reduce defects/waste/delays - Improve customer rating - Automate processing steps - Early error detection

9 Related Business Questions Sample Customer Request ing 1/3 Customer Contact ing Customer Request Create Customer Offer Create Contract Business Questions Analysis of the process (time, quality, costs...) and its activities: How many customer requests result in a contract? Which activities are the bottlenecks in the customer request processing? How satisfied are customers with the current process? Show the customer requests with the longest processing time? Show the average processing time of customer requests for the company XY? How did the processing time and process cycle time change compared to the last quarter / last year? How did an introduction of an CRM software package impact the processing of the customer requests? 9

10 Related Business Questions Customer Request ing 2/3 No Need for Further Contact Export Not Permitted Customer Req. Is Profitable Customer Contact Customer Contact ing Customer Request from Contact Customer Request Offer Must Be Created From Customer. Req. Business Questions Customer Request received Analysis of the control flow and the events: Customer Req. Is Canceled Create Customer Offer How many customer contracts originate from customer contacts and how many are direct customer queries? Contract is Which Offered customer Items or requests couldn't be processed becausecanceled of export Service Cannot restrictions? Be Delivered Customer Offer Is Sent Which customer requests were turned back because of less profitability? Which customer requests were canceled? Why were they canceled? Which customer requests were not processed because the items or Contract Must Create services could not be delivered? Be Created Contract From Offer Which customer requests were canceled during the contract processing? 10 Contract Confirmation is Sent Fulfillment Request Created

11 Related Business Questions Customer Request ing 3/3 Sales Team No Need for Further Contact Sales Team Export Not Permitted Customer Req. Is Profitable Customer Contact Customer Contact ing Customer Request from Contact Customer Request Offer Must Be Created From Customer. Req. Customer Request received Business Questions Sales Data Customer Req. Is Canceled Sales Team Create Customer Offer Offer Data Analysis of the process context (organization, customer, channel...) Which sales teams initiated and processed the most customer requests? Which sales Contract is teamsoffered areitems initiating or customer contract withcontract a contract volumecanceled of more than $? Contract Service Cannot ing Confirmation is What are Be Delivered the Top/Bottom Customer 10 contract Offer volumes Departmentby channels which were used Sent for the Is Sent customer contacts. Which sales teams do the most profitable customer request processing? Which customer requests result in long-term contracts? Contract Must Create Fulfillment Which customer requestsbe cannot Created be fulfilled Contract and why? Request From Offer Created How long is the average customer request processing per sales team? How much time does the contract processing take with a contract volume of more than $ compared with last quarter / year? 11 Contract Data

12 Real-World Examples Real-World Examples 12

13 Example: Ordering a Book from Amazon Shopping Checkout Post-Checkout ing 13

14 Example: Ordering a Book from Amazon Events from Website User Session Started Book Search Add Book to Shopping Cart Remove Book from Shopping Cart Checkout Identify User Shipping Information Captured Payment Information Captured Order Placed Payment Cleared Order Confirmation Sent Business Monitoring 14

15 Management Portal for Supply Chains 15 (c) IBM Watson Research

16 Management Portal for Supply Chains Detailed Exception View 16 (c) IBM Watson Research

17 Technology Perspective of Business Monitoring 17

18 Technology Evolution for Gaining Insight into the Business Approaches for Insight into Business Data Technology Evolution Data Warehousing Business Warehousing Real-Time Business Monitoring Adaptive Monitoring Environment 18

19 Business Monitoring Technology Landscape Lombardi IBM, FileNet, Staffware (Tibco) SAP, Oracle, Microsoft Business & Workflow Management IDS-Scheer, CaseWISE, Popkin Business Software ERP, CRM, SCM Business Objects, Cognos, Hyperion Business Modeling & Analysis Business Intelligence, Data Analysis Network Systems Management Business Monitoring Technology Mix Data Warehouse, Databases BMC, CA, HP Integration Broker Suites, EAI Tibco, Vitria, IBM, SeeBeyond Data Integration Suites, ETL Informatica, Ascential, Data Mirror Oracle, IBM, Microsoft 19

20 Web-Service Based Business Monitoring 20

21 Monitoring of a Distributed Supply Chain Monitoring of intraorganizational business processes Shared monitoring infrastructure for business partners Monitoring of business processes as on demand service 21

22 Business Monitoring: Four Web Service Types Define Web Service Log Web Service Analyze Web Service Monitor Web Service Setup of monitoring infrastructure Identifies web services for monitoring and their owners Defines the web services states and state transition rules Defines performance objectives Defines security policies Captures audit trail data from various source systems Logs state changes of a business process Calculates business process metrics and stores them in a database Correlation of process events Query interfaces for retrieving performance data The performance data includes web service enactments process metrics service level violations Access to analytical agents Dependent on policies defined by the web service owner Senses performance data for exceptional business situations Responds to exceptional business situations (e.g. by calling another webservice) Manages service level agreements 22

23 Monitoring BPEL4WS es Business Partner Business es Organisation with Monitored Business es by SMWS Monitoring Infrastructure... <receive> <reply> <invoke> <receive> <invoke> audit... Log Web Service BPEL4WS <invoke> BPEL4WS log item <receive> Port Type Event Stream or... <reply> <receive> <receive> <reply> <invoke> audit... Real-time Data Store BPEL4WS BPEL4WS Data Warehouse 23

24 Monitoring Managed By Workflow Management Systems 24

25 Monitoring with Probes in Operational Systems Observed Business System 1 System 2 System 3 System 4 Business Operations Observable Business Operations Probes Log Files, Log Tables Audit Data Flows Logging Daemon/ SMWS Proxy Monitoring Infrastructure log item Log Web Service Event Stream or Real-time Data Store Data Warehouse 25

26 Event Stream or Log Web Services Event Stream or BPEL4WS Log Web Service 1 Receive/Unify Events Event ing Evaluation/Alerting BPEL4WS Log Web Service 2 BPEL4WS BPEL4WS Log Web Service 3 Log Web Service 4 Raw Logging Items Services for Event ing Event Standization Event Correlation Event Condensation 26 Event Stream Querying Metric Calculation Event and Metric Persistence Event Flow Management Metric Evaluation Notifications & Alerting

27 Adaptive Business Monitoring with Sense & Respond 27

28 Business Management and Sense & Respond Sense & Respond adds intelligence to when and how a business operation is executed Monitored Environment Events Business Operations IT Systems Business es Sense & Respond adds intelligence when the execution of a business operation depends on» State of a business process (e.g. delays of process activities)» Complex event pattern (e.g. a fraud event patterns)» Assessment of opportunities, risks, capacity utilization, quality of service» Real-time key performance indicators, which can be highly aggregated business metrics (e.g. cycle times, processing costs, response time to customer)» Predictable indicators (e.g. demand forecasts, production plans) 28 Adapt

29 Sense & Respond Loops Sense Interpret Analyze Decide Respond» Extracts and collects relevant event data from the business environment» Giving the event data a meaning» Transforms event data into information (e.g. KPIs, business situations and exceptions)» Determines root causes of identified business situations» Predicts the performance and assess the risks for changing the business environment 29 Accelerate» Selects the best option for improving the business situations» Determines most appropriate action for a response to the business environment» Communicating the decision as a command or suggestion» Executing the business actions in target systems Sensing Interpret Sense Analyze Respond Decide Responding

30 Business Activity Monitoring with Senactive InTime Business Activity Monitoring Real-time Business Performance Data Production Management Metrics 4 Notifications for order confirmations 1 Sense Events of order process Sense Events of production processes 3 2 Response Updates of production plan 5 Response Trigger re-ordering of products and materials E-Commerce System Production System ERP System 30

31 Sense & Respond and Data Warehousing Traditional approach for integrating data into the data warehouse Sense Interpret Analyze Decide Respond 31

32 Sense & Respond Logistics & Transportation Management Sense (Shipment Events) Sense Interpret Analyze Decide Respond Response & Notifications (Change route, transporation mode, priorities,...) Facility to Facility Transportation 32

33 Modeling Sense & Respond 33

34 Event Driven Chains and Sense & Respond es Outside View of a Sense & Respond 34

35 Event Driven Chains and Sense & Respond es Inside View of a Sense & Respond BP Event 1 35

36 Event Driven Chains and Sense & Respond es Example for Inside View of a Sense & Respond Filtered Order Event Select Orders that should be examed Order Event Order Request Analyze Customer Payment History Customer Payment History Accepted Deny Order Event Deny Order External Credit Check Credit Rating Accepted Deny Order Event New Customer Rating Decide on Discount Customer Discount Applicable Give Discount Update Customer Rating 36

37 Web-Service Based & Adaptive Business Monitoring 37

38 Senactive InTime Sense & Respond Loops Monitored Environment Transform Events into Detect Situations or 4 Business Indicators 5 Exceptions Steps to Go IT Systems Probing / Sensing Unify Events Transmit Events InTime S&R Service 6 Analyze Situations or Exceptions Business es 7 Make Decisions 8 Put Response into Action 38

39 Supply Chain Monitoring of BPEL4WS es 39

40 Monitoring of BPEL4WS es with Senactive Service Provider Type Service Provider Type Monitoring Service Provider Type... <receive> <reply> BPEL4WS <invoke> <receive> <invoke> <invoke> audit BPEL4WS... Log Web Service log item Port Type <receive> Sense & Respond System... <reply> <receive> <receive> <reply> <invoke> audit <invoke>... BPEL4WS BPEL4WS Warehouse 40

41 Monitoring of Non-BPEL4WS es with Senactive Observed Business System 1 System 2 System 3 System 4 Business Operations Observable Business Operations Probe Points Audit Data Flows Web Service Proxy Response Monitoring Service log item Log Web Service Sense & Respond System Warehouse 41

42 Sense & Respond System Log Web Services Sense & Respond System BPEL4WS BPEL4WS Log Web Service 1 Log Web Service 2 Receive/Unify Events Event ing Event Adapters Event Service Evaluators Evaluation/Alerting BPEL4WS BPEL4WS Log Web Service 3 Log Web Service 4 Raw Logging Items S&R Services Event Standization Event Correlation Event Condensation Event Stream Querying Metric Calculation Event and Metric Persistence Event Flow Management Metric Evaluation Notifications & Alerting 42

43 Sense & Respond Monitoring with Senactive InTime Business Environment Sense & Respond Model Senactive InTime Key Features Business es Computer Systems Production Logistics, Transportation Sense Interface Respond Interface Adapter Adapter Adapter... Adapter Adapter Adapter... Hub Event Service Event Service Event Service Event Service Event Service... Hub Event Service Hub Sense Interpret Analyze Decide Respond» Visualization of Sense & Respond ing» Modular and Service-Oriented Architecture» Generation of Real-Time Key Performance Indicators» Real-Time Data Analysis for Controlling Business es» Instant Response on Business Situations and Exceptions External Systems, Business Partners 43

44 Research Topics for Bakk.- Arbeiten, Diplomarbeiten etc. 44

45 Research Topics (see also Business Management» Web Service Based Auditing of BPEL4WS es ( Web Services, BPEL4WS, Messaging)» Scalable Messaging Infrastructure for Monitoring Business es ( Messaging, MSMQ, MQSeries, JMS, )» Analysis Services with XMLA for Business Monitoring ( OLAP, Data Mining, SQL Server 2005) Event Management» Event Models for Monitoring Business es ( XML Schema for events)» Event Simulation Model for Business es ( Generating consistent XML events)» Discovering Patterns in Event Streams, Event-Driven ECA Rules ( Rule Engines, XML Rules)» Correlation and Synchronization of Event Streams ( XPath, Concurrency, Distributed Computing) Data Management» Data Management for Event Streams ( Star Schemas, Data Warehousing, SQL Server 2005)» Real-time Analytics & Data Stream Analysis with OLAP, Neuronal Network, Decision Trees, Clustering Techniques, Association Rules ( Data Warehousing, Business Intelligence, SQL Server 2005) User Interfaces» Visualization/Modeling of Sense & Respond es (data flows, control flows, dependencies) ( Visualization and manipulation of Graphs, GUI design, Visual Studio 2005, C#) 45

46 Q&A 46

47 Methods and Technologies for Business Monitoring Josef Schiefer Vienna, June 2005