ist tot! Lang leben Services! 24.04.2010 ist tot! Lang leben Services! Nicolai Josuttis 2010 by Nicolai Josuttis, IT-communication.com 1 Independent consultant continuously learning since 1962 Nicolai M. Josuttis Systems Architect, Technical Manager finance, manufacturing, automobile, telecommunication ti experience for multiple years Focus: bringing into operation Telco: >50 international systems >500 services >10 million service calls per day Automotive: Business (top-down) Project (bottom-up) Co-Author of the Manifesto http://www.soa-in-practice.com 2010 by Nicolai Josuttis, IT-communication.com 2 Copyright 2010 by Nicolai Josuttis 1
ist tot! Lang leben Services! 24.04.2010 2010 by Nicolai Josuttis, IT-communication.com 3All Contents (c) 2009 Burton Group. All rights reserved. What is? The longer I am involved in, the more confused I am about its definition. [Stefan Tilkov, W-JAX 2008] The concept of business-driven interfaces Contract-oriented management of system interaction Distributed business process modeling BPM-driven development System interaction management with or without central data model(s) Enterprise architecture management including domain modeling Management of product line services 2010 by Nicolai Josuttis, IT-communication.com 4 Copyright 2010 by Nicolai Josuttis 2
ist tot! Lang leben Services! 24.04.2010 Globalization We scale more and more Business is highly distributed Harmonization is not an option The Context of the 21 st Century We are on the road towards ULS (ultra-large-scale) systems Internet Health care systems Traffic and transportation... 2010 by Nicolai Josuttis, IT-communication.com 5 Characteristics of Large Systems Characteristics of large systems: Decentralization Heterogeneous, inconsistent, and changing elements Continuous evolution and deployment Redundancies Inherently conflicting, evolving, and diverse requirements Normal failures Erosion of the people/system boundary These characteristics may appear in today s systems can become critical in system landscapes dominate in ultra-large-scale systems 2010 by Nicolai Josuttis, IT-communication.com 6 Copyright 2010 by Nicolai Josuttis 3
ist tot! Lang leben Services! 24.04.2010 Distributed Systems to Deal with Millions of Customers Thin Client Smart Client (Call Center Desktop) Mobile Phone / Fax Frontends Portal Processes Order Fulfillment rule read write rule read write rule read write Backends CRM Billing Logistics 2010 by Nicolai Josuttis, IT-communication.com 7 Sign Contract Distributed Systems to Deal with Millions of Customers with Mobile Phone Thin Client Smart Client (Call Center Desktop) Mobile Phone / Fax Frontends Portal Process Order Processes Order Fulfillment AddContract( ) SendBill( ) ShipMobilePhone( ) rule read write rule read write rule read write Backends CRM Billing Logistics 2010 by Nicolai Josuttis, IT-communication.com 8 Copyright 2010 by Nicolai Josuttis 4
ist tot! Lang leben Services! 24.04.2010 Distributed Systems to Deal with Millions of Customers Thin Client Frontends Portal Wireless Business Unit Smart Client (Call Center Desktop) Mobile Phone / Fax Processes Order Fulfillment rule read write rule read write rule read write Backends CRM Billing Logistics 2010 by Nicolai Josuttis, IT-communication.com 9 Distributed Processes Over Multiple Companies Wired Business Unit Wireless Business Unit Billing CRM Insurance Company Property CRM Life 2010 by Nicolai Josuttis, IT-communication.com 10 Copyright 2010 by Nicolai Josuttis 5
ist tot! Lang leben Services! 24.04.2010 We moved houses Our new Address is: Distributed Processes Over Multiple Companies Wired Business Unit Wireless Business Unit Billing CRM Insurance Company UpdateAddress( ) UpdateAddress( ) Property CRM Life 2010 by Nicolai Josuttis, IT-communication.com 11 Reality: Systems in Multiple Countries, BU s, Companies Austria International USA UPS FedEx ESB B2B Gateway ESB Vendor Wired Germany UK 2010 by Nicolai Josuttis, IT-communication.com 12 Copyright 2010 by Nicolai Josuttis 6
ist tot! Lang leben Services! 24.04.2010 Dealing with Business Requirements findparcelstatus() Austria International USA UPS FedEx ESB B2B Gateway ESB Where is my phone? Vendor Wired checkphonestatus() findshippingdata() Germany findcustomer() UK findprocess() 2010 by Nicolai Josuttis, IT-communication.com 13 New World of IT System Development Maintenance of System Landscapes 2010 by Nicolai Josuttis, IT-communication.com 14 Copyright 2010 by Nicolai Josuttis 7
ist tot! Lang leben Services! 24.04.2010 Stack Enterprise Architecture Management (EAM)? No! Business Process Management (BPM) => BPEL Interoperability and loose coupling => Web Services, ESB Business-driven self-contained interfaces => Services 2010 by Nicolai Josuttis, IT-communication.com 15 Services Elements of IT representation of business functionality encapsulates implementations details High Interoperability An Enterprise Service Bus (ESB) connects all the systems ESB Loose Coupling Reduce dependencies (e.g. no common object model) But there is always a price for it 2010 by Nicolai Josuttis, IT-communication.com 16 Copyright 2010 by Nicolai Josuttis 8
ist tot! Lang leben Services! 24.04.2010 Forms of Loose Coupling [Krafzig04] with extensions: Tight Coupling Loose Coupling Possible Price physical point-to-point intermediator 3 rd party involved communication style synchronous asynchronous late and unordered responses data model common complex types common basic types mapping type system strong weak runtime instead of compile-time errors interaction pattern navigate through complex data-centric, self-contained type system limitations? object trees message control of process logic central control distributed control problem to find out state of a process binding statically dynamically runtime instead of compile-time errors platform strong dependencies platform independent no platform specific optimizations transactionality 2PC (two-phase commit) compensation extended fault handling deployment simultaneous at different times versioning, soft migration versioning explicit upgrades implicit upgrades many versions 2010 by Nicolai Josuttis, IT-communication.com 17 Forms of Loose Coupling [Krafzig04] with extensions: Find the right amount of lose coupling Tight Coupling Loose Coupling Possible Price physical point-to-point intermediator 3 rd party involved communication style synchronous asynchronous late and unordered responses data model common complex types common basic types mapping type system strong weak runtime instead of compile-time errors interaction pattern navigate through complex data-centric, self-contained type system limitations? object trees message control of process logic central control distributed control problem to find out state of a process binding statically dynamically runtime instead of compile-time errors platform strong dependencies platform independent no platform specific optimizations transactionality 2PC (two-phase commit) compensation extended fault handling deployment simultaneous at different times versioning, soft migration versioning explicit upgrades implicit upgrades many versions 2010 by Nicolai Josuttis, IT-communication.com 18 Copyright 2010 by Nicolai Josuttis 9
ist tot! Lang leben Services! 24.04.2010 Motivation for a Service Bus up to 25 individual connections no central management 8 connections to the bus central management possible 2010 by Nicolai Josuttis, IT-communication.com 19 Enterprise Service Bus (ESB) Infrastructure to provide high interoperability in a (heterogeneous) distributed environment Tasks: mandatory: Providing connectivity Technical data transformation (Intelligent) routing Monitoring and logging Dealing with security optional ( value value-added added services ): Dealing with reliability Service management Business data processing ESB 2010 by Nicolai Josuttis, IT-communication.com 20 Copyright 2010 by Nicolai Josuttis 10
ist tot! Lang leben Services! 24.04.2010 ESB Approaches in Practice just a common protocol: some central intelligence (monitoring, decoupling, security, ): with business logic (mapping of interfaces): Consumer Contract Consumer Contract Consumer Consumer Consumer Consumer Contract Contract Contract Contract Provider Provider Provider Distributed ESB, native Web Services, Internet as ESB possible EAI approach 2010 by Nicolai Josuttis, IT-communication.com 21 Scaling in a Global World Business Object Models (BOM s) don t scale Object models will differ Communication patterns don t scale You will have to support small and huge service calls You will need synchronous and asynchronous communication Standards don t scale You will have different versions of Web Services and other standards ESB s don t scale You will have different technologies for service calls QA does not scale A central service police will become a bottleneck 2010 by Nicolai Josuttis, IT-communication.com 22 Copyright 2010 by Nicolai Josuttis 11
ist tot! Lang leben Services! 24.04.2010 Conway s Law Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure. Melvin Conway, 1968 in How Do Committees Invent? http://www.melconway.com/law/index.html 2010 by Nicolai Josuttis, IT-communication.com 23 and the Organization Before : With : CRM Inventory Production Billing Client Client Client Client CRM Inventory Production Billing based on: Eberhard Wolff, Interface21 @JAX2006 2010 by Nicolai Josuttis, IT-communication.com 24 Copyright 2010 by Nicolai Josuttis 12
ist tot! Lang leben Services! 24.04.2010 leads to new Organizational Structures Split monoliths into channel specific frontends domain specific backends New departments for new systems e.g.: systems for common business processes over multiple backends Project culture in addition i to department culture Solution Managers 2010 by Nicolai Josuttis, IT-communication.com 25 Distribution Costs Many people involved Solution Manager System Analysts System Designers System Developers System Testers Solution Integrators Solution Testers Delivery Managers Incident Managers 2010 by Nicolai Josuttis, IT-communication.com 26 Copyright 2010 by Nicolai Josuttis 13
ist tot! Lang leben Services! 24.04.2010 Lessons Learned Distribution costs Distributed design (multiple parties involved) Needs organizational culture (collaboration, trust) and structures Distributed testing Might turn out to be the biggest day-to-day problem Distributed debugging Your infrastructure/esb is your debugger Distributed quality Overall quality depends on individual qualities Loose Coupling costs Any form of indirection and heterogeneity has a price You can t buy Tools can only help Main tasks are Architecture, Processes, Culture 2010 by Nicolai Josuttis, IT-communication.com 27 Performance Example: Reuse of a Broad Service 2010 by Nicolai Josuttis, IT-communication.com 28 Copyright 2010 by Nicolai Josuttis 14
ist tot! Lang leben Services! 24.04.2010 The Performance Reusability Tradeoff You will find: Reused services/interfaces Redundant services/interfaces Custom-made made services/interfaces Unused services/interfaces 5 4 Average number of consumers per service 3 2 1 Years 1 2 3 4 5 2010 by Nicolai Josuttis, IT-communication.com 29 Key Elements of Security in Distributed Systems Authentication Who is sending a message? Authorization Is it allowed to access a resource or see a result? Confidentiality/Privacy Remains the data confidential while in transit or in storage? Integrity Is the data not modified while in transit or in storage? Availability Is it possible to become inoperable due to an immense number of messages? Accounting Keep track of consumed resources (e.g. for billing) Auditing Evaluate security concepts and their implementation to detect security holes/attacks 2010 by Nicolai Josuttis, IT-communication.com 30 Copyright 2010 by Nicolai Josuttis 15
ist tot! Lang leben Services! 24.04.2010 What is special in : /Integration and Security High Interoperability has the goal to support connectivity, security has the opposite goal Heterogeneity Backends have different identity mechanisms and policies Many Layers of Abstraction End-to-end security over multiple hops Multi Tenancy German: Mandantenfähigkeit Technology specific threads 2010 by Nicolai Josuttis, IT-communication.com 31 Security can become very expensive weak: usr/password strong: + HTTPS Web Server gettoken() SAML Token IdP allowservicecalls() mediatetoken() ESB SAP Logon Ticket propagateuserids() SAML Token Username Token LTPA Token HTTPS Token Ctx local roles reverse proxy reverse proxy 2010 by Nicolai Josuttis, IT-communication.com 32 Copyright 2010 by Nicolai Josuttis 16
ist tot! Lang leben Services! 24.04.2010 Iterative and Incremental Establishment Landscape Repositories Processes and Policies Infrastructure (incl. Meta Model and MEPs) Project 1 Project 2 Refactoring Project 1 Refactoring Project 1&2 Project 3 Further Projects... 3-6 months 6-12 months 1-2 years 2010 by Nicolai Josuttis, IT-communication.com 33 You will recognize Iterative and Incremental Establishment when you need a repository Landscape Processes and Repositories policies evolve Each Project Processes and Policies brings a business requirement in production Infrastructure (incl. Meta Model and MEPs) Concepts and tools grow Project 1 Project 2 Refactoring Project 1 Project 3 Refactoring Project 1&2 Maturity takes time never touch a running systems Further Projects... 3-6 months 6-12 months 1-2 years 2010 by Nicolai Josuttis, IT-communication.com 34 It takes time Copyright 2010 by Nicolai Josuttis 17
ist tot! Lang leben Services! 24.04.2010 Central Integration Team You need a central team to decide about technology and restrictions setting-up processes and frameworks serve as central contact for business teams maintain infrastructure, architecture, processes 2010 by Nicolai Josuttis, IT-communication.com 35 Things to Decide Which (initial) protocol / ESB approach? e.g.: Just Web Services Initial guidelines: e.g.: document literal wrapped e.g.: with correlation ID e.g.: some naming conventions e.g.: initial meta model Which amount of loose coupling? e.g.: synchronous calls, one data model per provider,... Which first project(s)? Which first project(s)? e.g.: some useful solution, not mission-critical, multiple systems involved, only basic services... 2010 by Nicolai Josuttis, IT-communication.com 36 Copyright 2010 by Nicolai Josuttis 18
ist tot! Lang leben Services! 24.04.2010 Central Integration Team You need a central team to decide about technology and restrictions setting-up processes and frameworks serve as central contact for business teams maintain infrastructure, architecture, processes But note This is a central team that has the challenge to provide decentralization It s hard to find the right balance (central teams tend to centralization) ti 2010 by Nicolai Josuttis, IT-communication.com 37 Summary is a concept for system landscapes under maintenance But: There is no such thing as a quick fix Organizations are complex business systems, within which a change in any one component is likely to have an impact on other components. Andrew Spanyi in Business Process Management is a Team Sport: Play it to Win! is a strategy Therefore: Support top-down Establish bottom-up 2010 by Nicolai Josuttis, IT-communication.com 38 Copyright 2010 by Nicolai Josuttis 19
ist tot! Lang leben Services! 24.04.2010 Q&A Nicolai Josuttis www.it-communication.com josuttis@it-communication.com Gaussstr. 29 D - 38106 Braunschweig Germany Tel.: +49 531 / 129 88 86 +49 700 / 5678 8888 +49 700 / JOSUTTIS 2010 by Nicolai Josuttis, IT-communication.com 39 Manifesto Photo with permission by Mike van Alst Ali Arsanjani Grady Booch Toufic Boubez Paul C. Brown Dave Chappell John devadoss Thomas Erl Nicolai Josuttis Dirk Krafzig Mark Little Brian Loesgen Anne Thomas Manes Joe McKendrick Steve Ross-Talbot Stefan Tilkov Clemens Utschig-Utschig Herbjörn Wilhelmsen 40 Copyright 2010 by Nicolai Josuttis 20
ist tot! Lang leben Services! 24.04.2010 Das kommentierte -Manifest Nicolai Josuttis www.it-communication.com i ti josuttis@it-communication.com Gaußstr. 29 D - 38106 Braunschweig Germany Einzelexemplare kostenlos bei: wittmer@dpunkt.de Tel.: +49 531 / 129 88 86 +49 700 / 5678 8888 +49 700 / JOSUTTIS 41 Copyright 2010 by Nicolai Josuttis 21