BPM, EDA and SOA: How the Combination of these Technologies Facilitates Change. Dr. Neil Thomson, Head of Group Development, Microgen plc

Transcription

1 BPM, EDA and SOA: How the Combination of these Technologies Facilitates Change Dr. Neil Thomson, Head of Group Development, Microgen plc

2 What are we trying to do? The aim is survival everything else is a bonus To achieve survival we have to make our businesses more competitive c.f. the UK car and motor bike industry We need flexibility and responsiveness now known as agility with stability and low costs The other frequently cited homily is that we must give the business users some control over those things which are their responsibility Unfortunately, one of the greatest obstacles to change is perceived to be IT although in theory it should be the opposite.

3 How we are trying to achieve agility Integrate if you cannot replace legacy systems Utilise integration products such as EAI Suites, ESBs and messaging systems to reduce multiple disparate sources of data Replace hard-wired business processes with BPM and Workflow software Create composite applications from existing Silo programs Implement tools that allow business users to manage and control those areas for which they are responsible No taxation without representation Embrace SOA for reuse, lower cost of ownership and consistency across the organisation

4 Past and Present Stable and conservative Slow to change Business Practices wired into the organisation Reports were historical Silo applications Batch systems for efficiency of processing Low Transaction Volumes Responsive and radical Fast to change Business Practices flexible and automated BAM What s happening now Integrated applications EDA with STP for speed of response High and ever increasing transaction volumes

5 How we might achieve survival through agility and integration Through the use of a combination of these technologies: Technology Description Business Gain BPM and Workflow Business Process Management EDA Event Driven Architecture BAM Business Activity Monitoring XTP Extreme Transaction Processing SOA Service Oriented Architecture Automation of the modelling, deployment and execution of Business Processes and Rules Software which responds to events when they arrive and processes them immediately A User Interface that provides real-time information about volatile events Software that can cope with transaction volumes in excess of 5,000 / second with transactional control Writing commonly used software routines as services which can be utilised by any program regardless of location Responsiveness to change - Agility Coping with the T+0 world and the need for immediate updates for information To keep Management in touch with what is happening NOW Most business data is still transactional and the volumes are ever increasing Reduction of IT costs through maximisation of Software Reuse. Better consistency of processes

6 Purpose of this Presentation To show how these disparate technologies can work together to facilitate change EDA SOA BPM & Workflow BAM XTP

7 Business Process Management (BPM)

8 BPM What is it BPM, Business Rules and Workflow all aim to soften business processes and the differences between them are less important than their similarities Workflow must handle long lasting transactions, human -> human interactions and those situations where there is no response It may be data centric as in the case of exception handling, reconciliations and error correction or control centric as for authorisation and checking Most BPM suites now incorporate this functionality Business Rules can be treated as part of a Business Process or as a separate application but the former allows for context effects and integration. Therefore we will treat these three areas as one for the purposes of this presentation; they are all necessary to produce a flexible and responsive organisation where processes are not hard-wired.

9 Modelling Business Processes What is clear is that Business Processes require information that is the domain of the business and therefore should be capable of being written and changed by business users. Semiotic Programming and a Data Driven Architecture allows processes and rules to be modelled graphically and then deployed and executed directly there is no separate build stage so maximum business flexibility and agility is achieved This approach differs from other graphic approaches: It does not need to stay close to IT concepts and models, e.g. UML as it does not generate code. It can therefore utilise business concepts It is designed for a graphic approach and does not need to resort to the controversial near natural language approach Because the model can be directly deployed there is no need to write code and no opportunity for misunderstandings between the Business and IT This approach caters for those differences between BPM, BR and WF and allows them to be integrated.

10 Business Processes A Business Rule embedded in a Business Process

11 Business Rules Rules can have different scopes to cope for the effects of context: Local to a project or application; Global across a set of projects, e.g. for a particular legal entity Global to the organisation, e.g. within an SOA, it may be accessed by other applications as a service Rules are not restricted to simple conditional statements and therefore their numbers can be kept manageable removing performance issues

12 Workflow As with BPM and BR, WF can be modelled using Semiotic programming and a data driven model Workflow embedded in a Business Process

13 Event Driven Architecture (EDA)

14 Event Driven Architecture (EDA) The Elusive Feature Event Driven Architecture A system that responds to and processes an event when it occurs. Well known in the Real-Time world where we talk about Interrupt Driven Programs. The alternative is Polling which is inefficient, slower and unpredictable Main() /*Main Program*/ External Integer Ticks; Interrupt Vector = 5; /* Clock */ Interrupt Function TickTock(); Do while Ticks < 1000 { Step 1 Step 2 Step 3 }; End Function TickTock(); /*Interrupt Routine for Clock*/ External Integer Ticks; Interrupt Vector = 5; /* Clock */ Start: Ticks := Ticks + 1 Return An Interrupt Driven Architecture

15 Events have no Friends Event-centric programming means Each process receives and publishes events It has no knowledge of other processes, cf SOA External Enterprise Service Bus

16 Event Driven Architecture (EDA) There is increasing pressure to process events as they arrive rather than in batches. Business Processes and hence BPM Suites have to support this. Investment Banks are moving to the T+1 (Transaction Date + 1 Day) world and then to T+0. This is also true in the Retail Banking sector where payments systems are having to conform to new standards such as PEDD and Faster Payments In manufacturing, sales and stock systems transactions have to process events in real-time to support Just-in-Time (JIT) environments. Integration of BPM and EDA EDA has to be built into a BPM Suite it cannot be added as an after-thought; semiotic programming and a data driven model can provide the means of integration

17 EDA and Semiotic Programming READ extract 1 extract 2 evaluate STORE WRITE Operational States Not able to execute Able to execute Executed

18 EDA and Semiotic Programming For Business Processes, Data Flow is from Left to Right An event presents data at the process source node As each node s input port has data presented, it will start the execution of the function represented by the node. The resulting data will be placed at the node s output port.. and so on through the process For Rules, Data Flow is from Top to Bottom

19 So Far. We have integrated BPM and EDA EDA SOA BPM & Workflow BAM XTP

20 Business Activity Monitoring (BAM)

21 Business Activity Monitoring (BAM) If you only know what happened at the end of every month, you will lose to a company that knows what happens at the end of every day. Even if you have an EDA architecture for your BPM software, you need reporting tools that can show you data in real-time. BAM tools are products which connect to data streams and allow real-time displays of data. They sit over Business Processes and connect to their data flows as well as to stored data which can be used for comparison. EDA architectures are best equipped to support BAM but the associated BPM tool needs facilities to access stored data through adaptors which can then be integrated with the real-time data. This data can be used on Forms as part of BAM.

22 Business Activity Monitoring EDA also provides the foundation for real-time reporting and this can be realised using BAM tools where data streams can be connected to controls on forms Better suited to data driven models A BAM Screen fed by a Business Process

23 So Far. We have integrated BPM, EDA and BAM EDA SOA BPM & Workflow BAM XTP

24 Extreme Transaction Processing (XTP)

25 Extreme Transaction Processing (XTP) Some businesses operate in high transaction processing environments and others are feeling squeezed by the increasing amount of data that requires processing. In addition, SOA will place increasing processing loads on systems Either way, more and more organisations are realizing the need for high or extreme transaction processing (XTP) due to increased regulatory requirements, the merging of global operations or simply increased customer data requirements. XTP has to be integrated with the BPM Suite and needs an EDA architecture XTP Requires: Fast processing of transactions > 5,000/sec or 18 million/hour Low latency which implies EDA Transactional Control

26 XTP This normally requires an architecture which includes: In-memory processing Event Driven processing Scaleable multi-threaded/multi-processor architecture Distributed Caching Transactional Control Support for Clusters, Server Farms and Grid Computing (?)

27 The Profile of an XTP Tool Fast processing of transactions > 5,000/sec, even for complex rules and calculations Scaling over multiple threads and processors Low latency which implies EDA Transactional Control Transactions per Second 40,000 35,000 30,000 25,000 20,000 15,000 10,000 5,000 0 Extreme Transaction Processing Performance Number of threads Simple ETL mapping Rule mapping - 5 rules Rule mapping - 40 rules Complex expressions - 34 rules Rule mapping - 1 rule Rule mapping - 10 rules Block mapping - 5 rules Mapping

28 So Far.. We have integrated BPM, EDA, BAM and XTP EDA SOA BPM & Workflow BAM XTP

29 Service Oriented Architecture (SOA)

30 Service Oriented Architecture (SOA) A Service Oriented Architecture allows reusable functions to be externalised into services which can be utilised by multiple programs. These functions can be in the form of web services, function libraries (DLL s under Windows), and other programs (BPEL tools only allow these to be in the form of web services). SOA has been around for some time but the issue has always been utilising services which are not present on the same machine as the calling program. Previous solutions were very technical and quite complex such as Remote Procedure Calls and ORBS Web Services provides a very simple methodology with a standard set of interfaces including WSDL, SOAP and XML. SOA is an important tool for achieving our goals of agility and integration.

31 Putting BPM and SOA Together If an application is seen as a set of services (Web or Others), then the main application needs to Orchestrate the services This integration is often achieved by means of a BPM Engine that works with a script which describes the sequence of service calls. This script is frequently written in XML to a standard called Business Process Execution Language (BPEL) BPEL is limited to Web Services and defines them in a prescribed sequence and contains instructions for the synchronisation of the calls and processes. However, to obtain full integration, SOA should not be limited to Web Services otherwise: Data Integration can be achieved within an SOA framework but frequently needs a common data model that can be used across services Transaction processing is difficult to achieve Workflow processes which involve people are not properly addressed

32 A Problems with Web Services Only SOA Task: Transport 4,000 Sugar Cubes from A to B B Transport Mechanism Batch Size 40lb 0.5 lb 2,000 lb 10 lb No. of Batches 50 4, Data Format 4 Boxes 1 Cube 2 Crates 1 Box BPEL has no means of coping with the translation required among types of service carriers and cannot provide transactional control

33 True SOA integrated with BPM Single Data Model across all Services Data Structure Transformation Call to Manual Process Full Transactional Control Call to Web Service Call to an External Library Call to an External Program

34 To Conclude. We have integrated BPM, EDA, BAM, XTP and SOA SOA EDA BPM BAM XTP

35 Summary To achieve competitiveness and responsiveness, we need to implement a set of technologies that includes BPM, SOA, EDA, BAM and XTP Together these enable us to have flexible and automated business processes (BPM), straight through processing (EDA) and reporting (BAM), and fast transaction processing (XTP) all achieved with lower cost software development and uniform practices (SOA) Each of these technologies is represented by different products which address one or two of these areas A BPM Suite should address all these areas and should be able to integrate them within applications built using the Suite Microgen Aptitude is a BPMS which incorporates all these technologies in a single integrated suite.