Case Studies in Solving Testing Constraints using Service Virtualization Rix.Groenboom@Parasoft.NL 2/21/14 1
Introduction Paraso& is supplier automated tes1ng solu1ons Since 1984, Los Angeles (US) and Krakow (EU) Development tes1ng API tes1ng Service Virtualiza1on Solu1on Architect for Paraso& Nordics Since 1992 in Valida1on & Verifica1on (PhD) Joined Paraso& 2004, focus Benelux and Scandinavia Member of Dutch Testnet since 1999 Also, supervisor of Faris Nizamic PhD project on tes1ng in distributed systems University of Groningen (NL) SW project now with University of Linnaeus (S) 2
We are living in a Distributed World Cloud challenges ingrained concepts about software development Mobile devices force the industry to re-think the user experience APIs drive composite apps and interconnecting multiple dependencies 3
Testing in our Distributed World Bus ERP Partner Bus Partner Cloud Resource Bus Bus Modern IT systems are a collec1on of many components, o&en exposes as services 4
Testing in our Distributed World For tes1ng this: We need this: Bus ERP Bus Partner Partner Cloud Resource Bus Bus 5
Testing in our Distributed World Service- oriented systems characteris1cs: Func1onality spread across the network Interdependent services with no single owner No full control over a (sub)system Test automa1on - sa1sfying maturity level Test tools capable suppor1ng automa1on process Test automa1on techniques available to test teams E R P Bus Partner Partner Cloud Resour ce Bus Bus Bus YET, Agile so&ware development is being delayed due to of number of constraints imposed by SOA. 6
Survey on Constraints in Testing A survey conducted in 2012, asking so&ware developers, testers, and performance test engineers a series of ques1ons about their access to test environments. The survey results show: 1. Applica1ons rely on an average of 8 dependent services 2. 30% of 1me is spend on configura1on on test environment 3. 71% of developers/testers need to schedule test environment access 4. Average 1me block for access is 4 hours 5. 50% of the test plan is actually executed 6. 20% of the dependent services' func1onality is required for dev / test tasks 7
Survey on Constraints in Testing Constraints Availability Schedule Data Other limited access to a sub- system no access to a sub- system unfinished subsystem conflic1ng delivery schedules of sub- systems not available data not appropriate data corrupted data fees of invoking third- party system regulatory controls not sufficient dev/test environments non- instant availability of a sub- system limited usage permission of a sub- system not appropriate delivery coordina1on necessary test- data not in sub- systems Not allowed data corrup1on of a sub- system 8
Other industries: Simulation Tes1ng facili1es in Aircra& industry simulate real- 1me effects of wind, load and pressure, and repeat the standard movements over and over MeridianBUF 9
Other industries: Simulation 10
How to simulate environment in SOA For Service Oriented Systems the use of Virtual Services is the current technique to obtain simulated environments. Service virtualiza-on emulates the behavior of so5ware components to remove dependency constraints on development and tes-ng teams (Wikipedia). Virtual Services can be modeled or generated from traffic coming to and from real services. 11
Service Virtualization: How does it work? Monitor & Capture Model & Deploy Provision & Consume Initiated from the system under test, the user has the ability to capture detail from a live monitor that analyzes system traffic, from analyzing transaction logs or by modeling virtual behavior within the Parasoft Virtualized interface. After the virtualized artifact has been captured, users can now instruct the details of the virtualized asset behavior. This includes: performance, data sources and conditional response criteria. The virtualized asset is then provisioned for simplified uniform access across teams and business partners. The virtualized asset can now be called for unit, functional and performance tests. The virtualized asset can be leveraged by any test suite including Parasoft Test. 12
How does it work? Monitor and Model Application Under Test 1 Define Monitors Database 2 Capture Mainframe Application Service Traffic Logs 3 Create Server Desktop 4 Deploy Virtualize SME 13
How does it work? Provision and Consume Application Under Test Database 6 Consume Mainframe Application 5 Provision Environment Manager Service Server 14
Case Studies in Service Virtualization Real Virtualize back- end services Server (constrained) (full control) Virtual Web Service 1 Service Under Test Virtual Web Service 2 Virtual Web Service 3 Virtual Service Manager 15
Case study 1: Telecom platform Service Under Test Back- end service Back- end service In-appropriate test-data 16
Case study 1: Telecom platform Constraints: In- appropriate test- data Backend systems are from 3 rd party Test cases ( orders ) need to be added manually Solu1on: Virtual service to simulate the 2 backend systems Simulate business processes (asynchronous transac1ons) Results: Development includes regression tes1ng Prepara1on of testcase from a week to several minutes Increase test- coverage: from ca 10 to unlimited ( Stable access to backend in test- environment) 17
Case study 1: Telecom platform Back- end Virtual Service service Automated tests Service Under Test Back- end Virtual Service service Automated service virtualizafon management 18
Case study 1: Telecom platform 19
Case study 2: SOA (e-banking) Back- end service Service Under Test E S B Back- end service Back- end service Back- end service Back- end service Limited Available sub systems Back- end service 20
Case study 2: SOA (e-banking) Main constraint: Unavailable backend services Service might be down for load- test environment Services are in wrong version Solu1on: Virtual services of 70 back- end systems Data in Virtual Services synchronized with Loadtest tool Full automated deployment & mapping of endpoints Results: Load- tes1ng is now part of Agile- cycle Efficiency: >50% reducfon in performance test- 1me Possibility to experiment with response 1mes (SLA) Performance measurements of ESB infrastructure 21
Case study 2: SOA (e-banking) Back- end Virtual Service service Back- end Virtual Service service Automated tests Service Under Test ESB Back- end Virtual Service service Back- end Virtual Service service Automated service virtualizafon management Back- end Virtual Service service Back- end Virtual Service service 22
Case study 3: Student administration 23
Case study 3: Student administration New interfaces by central government body No test environment available (DUO) Created a collec1on of Virtual Assets Including asynchronous services to simulate address modifica1ons etc Contains over 4000 student records, including school history Available for three different test- environments Via SSL available over the Internet Used by all Dutch universi1es and polytechnics. 24
Case study 3: Student administration 25
Case study 3: Student administration 26
Case study 3: Student administration 27
Case study 3: Student administration 28
Case study 4: Pension application Service Under Test E S B Back- end service Back- end service Back- end service Back- end service Back- end service Back- end service Cost of Staged Environments 29
Case study 4: Pension application New implementa1on pension applica1on: Outsourced development 6 releases a year Around 20 backend systems MQ middleware Many staged environments: Development, Test, Acceptance, Pre- produc1on, Training, Produc1on (6) External connec1ons (GBA) without data- synchroniza1on Applied 80 / 20 rule (not full simula1on of all back- ends) Used Service Virtualiza1on to: Simulate (remove) external sub- systems Automated provision data in the several databases Load- tests without impac1ng other parts of organiza1on 30
Case study 5: Validation environment Energy market introduced New XML protocol for gas transport From EDI to XML based (EdiGAS) AS2 protocol (and new SHA2 cer1ficates) Needed a test tool (cer1fica1on environment) to cer1fy over 120 market par1es, both na1onal and interna1onal Had to act like a chameleon Use Virtualize (in combina1on with ESB and Database) to have fully automated cer1fica1on environment Dynamic responses based on role of the business partner Automated verifica1on of the test- scenario s SSL Cer1fica1on and authen1ca1on 31
Case study 5: Validation environment Remote Party Hosted TestTool 32
Summary Service Virtualiza1on solves constrains in tes1ng distributed systems Availability Scheduling Data Benefits depends of the situa1on: Cost reduc1on in infrastructure for staged environments Speed up the overall tes1ng 1me Solve the test- data management problem Enabler of test- automa1on E R P Bus Partner Partner Bus Bus Bus 33
Summary Enabler for Con1nuous Tes1ng and Delivery in distributed environments Key for Agile methodologies, DevOps Integra1on with Jenkins etc. Allows Environment Based Tes1ng: Not only the front door, also the back door Project with University of Groningen and Linnaeus 34
Further reading and information Email: Mobile: Paraso& Proprietary and Confiden1al Rix.Groenboom@Paraso&.nl +31-62- 240-1968 35