From message passing to service orientation. EAI, SOA, BPM Timo Itälä T-86.5141 4.10.2006 Contents Business and technology views Process maps, need for integration (EAI) Integration topology options Enterprise Application Integration stack Business integration Evolution of transactional systems architecture Service oriented architecture SOA Business Process Management (BPM) After this: Case METSO 15:00 @ Paavo Kotinurmi & Timo Itälä 2006 2
EAI, SOA, BPM: Business and technology views ENTERPRISE ARCHITECTURE - A FRAMEWORK TM DATA What FUNCTION How NETWORK Where PEOPLE Who When MOTIVATION Why TIME SCOPE (CONTEXTUAL) List of Things Important to the Business List of Processes the Business Performs List of Locations in which the Business Operates List of Organizations Important to the Business List of Events Significant to the Business List of Business Goals/Strat SCOPE (CONTEXTUAL) Planner ENTITY = Class of Business Thing Function = Class of Business Process Node = Major Business Location People = Major Organizations Time = Major Business Event Ends/Means=Major Bus. Goal/ Critical Success Factor Planner Business Side ENTERPRISE MODEL (CONCEPTUAL) e.g. Semantic Model e.g. Business Process Model e.g. Business Logistics System e.g. Work Flow Model e.g. Master Schedule e.g. Business Plan ENTERPRISE MODEL (CONCEPTUAL) Owner Ent = Business Entity Reln = Business Relationship Proc. = Business Process I/O = Business Resources Node = Business Location Link = Business Linkage People = Organization Unit Work = Work Product Time = Business Event Cycle = Business Cycle End = Business Objective Means = Business Strategy Owner SYSTEM MODEL (LOGICAL) e.g. Logical Data Model e.g. Application Architecture e.g. Distributed System Architecture e.g. Human Interface Architecture e.g. Processing Structure e.g., Business Rule Model SYSTEM MODEL (LOGICAL) Designer Ent = Data Entity Reln = Data Relationship Proc.= Application Function I/O = User Views Node = I/S Function (Processor, Storage, etc) Link = Line Characteristics People = Role Work = Deliverable Time = System Event Cycle = Processing Cycle End = Structural Assertion Means =Action Assertion Designer TECHNOLOGY MODEL (PHYSICAL) Builder e.g. Physical Data Model Ent = Segment/Table/etc. Reln = Pointer/Key/etc. e.g. System Design Proc.= Computer Function I/O = Data Elements/Sets e.g. Technology Architecture Node = Hardware/System Software Link = Line Specifications e.g. Presentation Architecture People = User Work = Screen Format e.g. Control Structure Time = Execute Cycle = Component Cycle e.g. Rule Design End = Condition Means = Action TECHNOLOGY MODEL (PHYSICAL) Builder Technology Side DETAILED REPRESEN- TATIONS (OUT-OF- CONTEXT) e.g. Data Definition e.g. Program e.g. Network Architecture e.g. Security Architecture e.g. Timing Definition e.g. Rule Specification DETAILED REPRESEN- TATIONS (OUT-OF CONTEXT) Ent = Field Reln = Address Proc.= Language Stmt I/O = Control Block Node = Addresses Link = Protocols People = Identity Work = Job Time = Interrupt Cycle = Machine Cycle End = Sub-condition Means = Step Sub- Contractor Sub- Contractor FUNCTIONING ENTERPRISE e.g. DATA e.g. FUNCTION e.g. NETWORK e.g. ORGANIZATION e.g. SCHEDULE e.g. STRATEGY FUNCTIONING ENTERPRISE John A. Zachman, Zachman International (810) 231-0531 @ Paavo Kotinurmi & Timo Itälä 2006 3 Contents Business and technology views Process maps, need for integration (EAI) Integration topology options Enterprise Application Integration stack Business integration Evolution of transactional systems architecture Service oriented architecture SOA Business Process Management (BPM) After this: Case METSO 15:00 @ Paavo Kotinurmi & Timo Itälä 2006 4
Process map Supplier Functions Customer Core processes @ Paavo Kotinurmi & Timo Itälä 2006 5 Example: Paper sales and distribution Core business processes Sales forecasting and budgeting Order entry and delivery Functions Administration Production Shipments management Sales/shipments Customer Information flows within a business process between functions Source: Jouko Hannus: Prosessijohtaminen, 1993 @ Paavo Kotinurmi & Timo Itälä 2006 6
Example: Integration of information flow Applications are functionally developed have interfaces exchange messages use common data codes and formats Business processes cover several applications need seamless information flow Enterprise Application Integration? Source: Jouko Hannus: Prosessijohtaminen, 1993 @ Paavo Kotinurmi & Timo Itälä 2006 7 Integration of functional applications Choices ERP: Enterprise wide application in one package Enterprise Applicaton Integration (EAI): Integration of multiple applications using standard interfaces and messages Enterprises use several applications ERP, PDM, SCM, CRM, HR and others Business partners need also integration We need EAI anyway, both within and between enterprises @ Paavo Kotinurmi & Timo Itälä 2006 8
Contents Business and technology views Process maps, need for integration (EAI) Integration topology and stack Business integration Evolution of transactional systems architecture Service oriented architecture SOA Business Process Management (BPM) After this: Case METSO 15:00 @ Paavo Kotinurmi & Timo Itälä 2006 9 Integration topology options Point-to-point Hub and spoke Common definitions @ Paavo Kotinurmi & Timo Itälä 2006 10
EAI Stack: Integration needs Integration is needed: Communications protocol, network, routing, addressing, message wrappers Authentication, authorization, security Applications protocol, messages, syntax Semantic interoperability, codes, vocabularies Business process protocol Goals, objectives, metrics Procurement, share of costs Implemantation, training, support @ Paavo Kotinurmi & Timo Itälä 2006 11 Integration options Adapter Point-to-point n(n-2)/2 connections Hub and spoke n communications connections n(n-2)/2 semantic connections Common definitions n communications connections n semantic connections n(n-2)/2 business connections @ Paavo Kotinurmi & Timo Itälä 2006 12
EAI Stack EAI Stack integrates Communications protocol, network, routing, addressing, message wrappers Authentication, authorization, security Applications protocol, messages, syntax Semantic interoperability, codes, vocabularies Business process protocol Goals, objectives, metrics Procurement, share of costs Implemantation, training, support Key decisions The roles and responsibilities of the Hub and the Spokes @ Paavo Kotinurmi & Timo Itälä 2006 13 EAI services include Platform, protocol management.. Asynchronous,Synchronous messaging support.. Security services, encryption,non-repudiation..etc Routing services, publish/subscribe.. Directory services.. Metadata management & repository services.. Event management, topology & synchronization.. Data mapping & transformation services.. (not always unified data model behind) @ Paavo Kotinurmi & Timo Itälä 2006 14
Business to business integration (B2Bi) EAI EAI Company A EAI B2Bi services Different requirements within enterprises and between enterprises Different approaches within enterprises and between enterprises EAI EAI Company B Company D @ Paavo Kotinurmi & Timo Itälä 2006 15 Technology framework for EAI/B2Bi Business process Integration Private EAI Data Integration B2B Trade Public Application integration Services Native data Multiple Platforms Diverse Protocols Partner management Services Standard data (XML) Platform independence Primarily HTTPS, SOAP @ Paavo Kotinurmi & Timo Itälä 2006 16
Contents Business and technology views Process maps, need for integration (EAI) Integration topology and stack Business integration Evolution of transactional systems architecture Service oriented architecture SOA Business Process Management (BPM) After this: Case METSO 15:00 @ Paavo Kotinurmi & Timo Itälä 2006 17 EDI background The creators of Electronic Data Interchange (EDI) were mainly concerned about the size of their messages. EDI messages are very compressed and use codes to represent complex values. The metadata is stripped from an EDI message makes the message difficult to read and debug (but they are meant for computers not human) EDI programmers are hard to train and expensive to keep. This complexity drives cost. No one global EDI standard (UN/CEFACT, X.12) @ Paavo Kotinurmi & Timo Itälä 2006 18
EDI (EDIFACT order) UNH+0002771776+ORDERS:D:99A:UN:FI0084 BGM+105+40063000277177602748497+9 DTM+4:20000705:102 DTM+2:20000706:102 DTM+9:20000705:102 NAD+BY+003709895955:100++TRADEKA OY NAD+SE+003702134547:100++OY HARTWALL AB NAD+CN+40063000::92++VALINTATALO HERVANTA+LINDFORSINKATU 2+TAMPERE++33720 LIN+1++6413600001584:EN IMD+F+8+:::HTW NOVELLE ORANGE LIME 1,5 L QTY+21:144 LIN+2++6413600000280:EN IMD+F+8+-:::VICHY 0,33L/HARTWALL QTY+21:430 UNS+S CNT+2:2 UNT+17+0002771776' Source: TIEKE (EDI standards implementation guidelines document) @ Paavo Kotinurmi & Timo Itälä 2006 19 Motivation for integration & e-business standards Data transfer needs high, volume high, probability of human errors high, frequency of data transfer Integration is needed between systems XML has been central element in recent B2B standardization but XML alone is not enough Although following are all human readable/interpretable, computers have problems in these things <e-business/> <date>4.10.2006</date> <ebusiness/> <date>10/4/2006</date> <E-business/> <xs:date>2006-10-04</xs:date> XML provides a syntax way to represent information Need standard to define commonly understood business documents B2B standards often don t just standardize the business documents, but define also the inter-company business processes and how the business documents can be securely transported over the Internet @ Paavo Kotinurmi & Timo Itälä 2006 20
Automatic process Business Document is standardized message encoded using XML or EDI Reliable messaging = postal service @ Paavo Kotinurmi & Timo Itälä 2006 21 RosettaNet messaging principle Company A Company Specific processing Internet & XML Company B Company Specific processing SAP ERP I2 APS Translate from RosettaNet standards to Company A system data set. RosettaNet defines processes and a framework for how data gets passed over the Web and certain handshake criteria. Translate from RosettaNet standards to Company B system data set. source: RosettaNet @ Paavo Kotinurmi & Timo Itälä 2006 22
Problem that RosettaNet tries to solve Standards needed to enable system-to-system B2B collaboration. Helps solving some semantic problems in specific industry by providing message guidelines, dictionaries and unique identifiers Defines standard business processes Reliable messaging over Internet @ Paavo Kotinurmi & Timo Itälä 2006 23 B2B Process Integration Customer Private process (Company -specific) Process PO Receive PO Request Public process (Standard) Send PO Customer Public process (Standard) Send PO Supplier Supplier Private process (Company -specific) Process Sales Order Receive PO Figure provided by Vitria Systems P O Select Supplier Generate RFQ Send RFQ Select RFQ Response Send PO Receive PO Acknowledge Receive PO Response Send PO Send PO Response HELSINKI Close UNIVERSITY OF TECHNOLOGY Acknowledge Receive PO Check Customer Check Send PO Credit Acknowledge Check Availability Send PO Create Sales Response Order CRM SCM Send PO Receive PO Response Response @ Paavo Kotinurmi Acknowledge & Timo Itälä 2006 Close 24 ERP
ebxml Provides a Standard Way to: Exchange business messages Conduct trading relationships Communicate data in common terms Define and register business processes source: www.ebxml.org @ Paavo Kotinurmi & Timo Itälä 2006 25 What does ebxml do? Lowers the cost/complexity of electronic business Facilitates global trade and puts SMEs and developing nations in the picture Expands electronic business to new and existing trading partners Converges current and emerging XML efforts Eliminates dependence by supporting any language, any payload, any transport @ Paavo Kotinurmi & Timo Itälä 2006 26
ebxml Specs--Modular Suite Technical Architecture Business Process Registry and Repository Collaboration Protocol Profile and Agreement Message Services Core Components @ Paavo Kotinurmi & Timo Itälä 2006 27 ebxml Example 1 Usage Example 3 4 Request ebxml specification Send ebxml specification Build System Register company profile 2 11 DO BUSINESS! 8 TPA Accepted 7 Submit TPA 5 9 Query about Company X Send Company X s Profile Request Company X s Scenario Send Company X s Scenario 6 10 Specifications Profiles Scenarios ebxml BO Library ebxml BP Model @ Paavo Kotinurmi & Timo Itälä 2006 28
ebxml Registry & Repository Human Interface Business Application Interface PUBLISH TO API SECURITY LAYER Registry Repository ebxml specification Schemas TPP Objects w. MetaData Reference Content POINTS AT @ Paavo Kotinurmi & Timo Itälä 2006 29 ebxml Registry & Repository Request Industry Domain Business Process Detail Constraints Response Details Content Action Status Transport Layer Access Syntax in XML Classification & Ownership ebxml conformant XML object Collections & Versioning Interface Information Model Remote ebxml Registry Registry Services Associated References Registration of Domain Registry Service Interface Registry Access Index XML Content Referencing Other Registry Service Interface(s): UDDI, CORBA Compatibility Wrappers Repository Content @ Paavo Kotinurmi & Timo Itälä 2006 30
Contents Business and technology views Process maps, need for integration (EAI) Integration topology and stack Business integration Evolution of transactional systems architecture Service oriented architecture SOA Business Process Management (BPM) After this: Case METSO 15:00 @ Paavo Kotinurmi & Timo Itälä 2006 31 Paradigm shift Service orientation Service oriented architecture (SOA) Service oriented computation (SOC) Enterprise service bus (ESB) @ Paavo Kotinurmi & Timo Itälä 2006 32
Tiers 1,,n; where are we coming from Terminal PC Client PC Client PC Client UI App UI Client App Browser Client Apps Components Mainframe Mini Server App Server Server Apps Servers UI User Interface File server Application DB server DB Database DB server DB servers 1-Tier 2-Tier 3-Tier n-tier @ Paavo Kotinurmi & Timo Itälä 2006 33 Middleware User Interface Transaction Processing Monitor Application logic Middleware @ Paavo Kotinurmi & Timo Itälä 2006 34
Read or update the database? To Read To Update Are very different types of applications @ Paavo Kotinurmi & Timo Itälä 2006 35 Transactions Transaction Processing Monitor helps the applications to achieve ACID properties Transaction is an action that changes the state of an enterprise is a collection of actions that are governed by "ACID" properties ACID Atomicity All or nothing Consistency Achieve a stable end state or abort Isolation The changes that the transaction makes to shared resources become visible only after the transaction commits Durability Transaction s effects are permanent after it commits. @ Paavo Kotinurmi & Timo Itälä 2006 36
Distributed transactions Root node Subordinate node Subordinate node Distributed transactions Can span multiple nodes Can update different types of databases from different vendors Can span different tp-monitors Confirm to ACID properties Coordination Commit coordinator (root node) Subordinate Nodes Two-phase commit protocol 1. Prepare-to-commit 2. Commit @ Paavo Kotinurmi & Timo Itälä 2006 37 Distributed transactions in practice Component based programming Everything is made of components Reusable components Application server platforms manage components transactions and ACID properties "component containers" Hide complexities of transaction management Distributed transactions Difficult to implement Difficult to manage Difficult to use for B2Bi @ Paavo Kotinurmi & Timo Itälä 2006 38
Client - Server applications Example: Material system for health care GUI: SQL Windows Application and database platform: Tandem Nonstop (today HP NonStop) Client components support business functions Customer and organization management Product and inventory management Inventory management Order management Server components offer services Insert, update, delete, query Customers Products Orders Deliveries Services Expose interfaces Hide internal data structures Invoked by name Location independence Check for authorization to use Version control from very start @ Paavo Kotinurmi & Timo Itälä 2006 39 Contents Business and technology views Process maps, need for integration (EAI) Integration topology and stack Business integration Evolution of transactional systems architecture Service oriented architecture (SOA) Business Process Management (BPM) After this: Case METSO 15:00 @ Paavo Kotinurmi & Timo Itälä 2006 40
Web services Web services acronyms html over http Web server Middleware xml over http soap over http html pages Servlets Java components XML Standard Language for denoting information content and process control SOAP - (Simple Object Access Protocol) XML based Messaging Protocol UDDI - (Universal Description, Discovery and Integration) Yellow Pages for Web Services WSDL - (Web Services Description Language) Data Tier @ Paavo Kotinurmi & Timo Itälä 2006 41 Service oriented architecture (SOA) New buzzword in Enterprise systems architecture SOA expresses a software architectural concept that defines the use of services to support the requirements of software users Most definitions of SOA identify the use of Web service technologies (SOAP and WSDL) in implementation. However, SOA can use any service-based technology SOAs comprise loosely joined, highly interoperable application services (platform independency) E.g. Wikipedia (www.wikipedia.org) good for current information @ Paavo Kotinurmi & Timo Itälä 2006 42
Enterprise service bus (ESB) ESB refers to a category of Web services standards based middleware infrastructure products or technologies that enable a service-oriented architecture via an event-driven and XML-based messaging engine Key benefits faster and cheaper accommodation of existing systems increased flexibility: easier to change as requirements change standards-based @ Paavo Kotinurmi & Timo Itälä 2006 43 SOA and ESB Source:www.ibm.com/developerworks/ web/library/wa-soaesb/. @ Paavo Kotinurmi & Timo Itälä 2006 44
Contents Business and technology views Process maps, need for integration (EAI) Integration topology and stack Business integration Evolution of transactional systems architecture Service oriented architecture SOA Business Process Management (BPM) After this: Case METSO 15:00 @ Paavo Kotinurmi & Timo Itälä 2006 45 Process map Supplier Functions Customer Core processes @ Paavo Kotinurmi & Timo Itälä 2006 46
Modeling the business process Actors, participants in the process: Swimlanes Task: Input, process, output Flow: Sequence of tasks Control: Decisions that affect the sequence Events: Start, End, Exceptions Messages: Message exchange between actors Compositets: Groups of tasks @ Paavo Kotinurmi & Timo Itälä 2006 47 @ Paavo Kotinurmi & Timo Itälä 2006 48
From process model to process exection BPMN (Business Process Modeling Notation) Human understandable version of the process BPEL (Business Process Execution Language) Machine understandable version of the process @ Paavo Kotinurmi & Timo Itälä 2006 49 Orchestration of a process Execution of the business process (BPEL code) BPEL engine Web Services - interfaces Business services Core data @ Paavo Kotinurmi & Timo Itälä 2006 50
BPMS addresses the business process lifecycle Design Graphical modeling Business & process rules Simulation and testing Execute Process Workflow Automation Integration Business rules Portal/forms BPMS Business Process Management Suite Monitor/Manage Real time dashboards Reporting Analyze/Optimize Dashboards Historical analytics Performance mgmt Simulation Source: Forrester @ Paavo Kotinurmi & Timo Itälä 2006 51 The confusing BPMS vendor landscape Pure-Play BPM Integration Lombardi, Metastorm Pegasystems, Savvion TIBCO, Vitria, webmethods Enterprise Application SAP, Oracle BPM Functionality Traditional B2B Sterling Commerce, Inovis, GXS Application Platform Enterprise Content Mgmt IBM, BEA, Microsoft, Sybase, SUN EMC/documentum, Open Text, FileNet Source: Forrester @ Paavo Kotinurmi & Timo Itälä 2006 52
Two different kinds of BPMS Enterprise Application Integration Integration-centric Business Process Integration Business Process Management P R O C E S S Human-centric Workflow Business Process Management Expanded into BPM F O C U S Document imaging Separate market Mid-late Mid-late 1990s 1980s 2002-05 2006+ Source: Forrester @ Paavo Kotinurmi & Timo Itälä 2006 53 Match BPMS to process characteristics Process Characteristics System intensive People intensive Decision intensive Example processes Order fulfillment HIPA transactions Straight thru processing Claims processing Employee on-boarding Underwriting Loan origination Required features Integration tools Transaction management Partner profile mgmt Task list/workflow portal UI development Organization management Forms management Business rules engine Business intelligence Document intensive Contract management Document management Accounts payable integration Source: Forrester Claims dispute @ Paavo resolution Kotinurmi & Timo Itälä 2006 54
Business Process Management (BPM) @ Paavo Kotinurmi & Timo Itälä 2006 55 Processes and services Referral Assesment Execute care Plan care Make orders Assess results Discharge Need for care? Care needed? Acute care Book service Execute service Record results @ Paavo Kotinurmi & Timo Itälä 2006 56
BPM & SOA, business meets technology Business processes Business services Composite services Atomic services Applications @ Paavo Kotinurmi & Timo Itälä 2006 57 Questions? @ Paavo Kotinurmi & Timo Itälä 2006 58