Shift Left with Service Virtualization: Rational Test Workbench & Test Virtualization Server

Transcription

1 Shift Left with Service Virtualization: Rational Test Workbench & Test Virtualization Server Khurram Nizami IBM Corporation

2 IBM Corporation

3 Only one-tenth of an iceberg is above water IBM Competitive Project Office Traditional UI testing: Automated Integration Testing and Service Virtualization: Presentation layer Integrations, data and business logic 2 The majority of risk in modern systems is seldom tested as it is unseen.

4 Complexity increases as your points of integration increase. 4 systems 6 interactions 5 systems 10 interactions 12 systems 66 interactions Types of systems and interfaces? Availability of systems? Collectively? Cost to interact with the systems? Difficulty in setting up test systems? Lets just deal with it later.. 3

5 Today, integration of enterprise applications still follow the big-bang approach Integrate Integrate Big Bang! Big Bang! Mobile component Web component Mainframe component typ. 6 Weeks 1 week typ. 6 Weeks 1 week 4

6 Relative Cost to Repair IBM Competitive Project Office Software quality fact: errors caught early cost less to fix... Bugs/failures are exponentially more expensive the longer they go uncaught Extends to analysis, requirements, development, and everywhere across the application lifecycle 34% of software projects are deemed successful costing $300B annually 30% of project costs are due to rework and poor execution of requirements 50% of outsourced projects are expected to under perform Stage in which Error is Discovered 200x 50x 20x 1.5x 5x 10x Analysis Design Coding Unit Test Acceptance Maintenance 5 5

7 Shift Left focus on testing high risk areas earlier in the development lifecycle Development Testing Staging Production Automated Integration Testing Traditional UI Tools Testing only at the UI level provides little confidence for deployment of complex distributed systems UNIT SERVICE UI Testing must be shifted left with focus on testing the highest areas of risk Shift Left Service Virtualization Service virtualization can be applied at all phases even allowing integration testing to begin in parallel with Unit Testing. 6

8 With virtual testing, the big bang is eliminated from the process Requirements Unit Tests Big Bang Accelerates testing Reduces costs Lowers risk Integration Tests System Tests System Tests UAT OAT Performance 7 7

9 We could just write up some stub code Not a bad idea, for short lived apps, simple scenarios and limited 8 sets of integrations public class TestStockAnalyzer { public class TestStockAnalyzer { public void TestMethod1() public void TestMethod1() { { // Arrange: // Arrange: var analyzer = new StockAnalyzer(); var analyzer = new StockAnalyzer(); // Act: // Act: var result = analyzer.getcontosoprice(); var result = analyzer.getcontosoprice(); // Assert: // Assert: Assert.AreEqual(123, result); // Why 123? Assert.AreEqual(123, result); // Why 123? } } } } public class StockAnalyzer { public class StockAnalyzer { public int GetContosoPrice(){ public int GetContosoPrice(){ var stockfeed = new StockFeedStub(); var stockfeed = new StockFeed(); return stockfeed.getshareprice("cooo");}} return stockfeed.getshareprice("cooo");}} } } public class StockFeed { public class StockFeedStub { //this stuff isn t ready, isn t available yet, is expensive to use, or public float GetSharePrice(string stock) { some other reason return ; } } } Hybrid Cloud DevOps

10 Shift Left with Service Virtualization Service virtualization can replace components and databases in complex test scenarios. 9 Service virtualization can work with any type of test manual, automated or performance

11 Stubs or virtual components allow developers and testers to start integration tests early The idea of stubs is not new Developers used them for decades to simulate interfaces and behaviour of the components that were not available for testing Components are introduced into the continuous integration cycle in a prioritized, controlled fashion. Components not yet built or components that cannot be accessed in the Development can be simulated and tested against. actual component Incremental Integration Testing 10 a stub that replaces a component that does not exist yet or cannot be accessed in dev and test

12 Transports and Formats IBM Rational Test Virtualization Server supports multiple platforms and protocols The building blocks of an integration test or virtualized service The transport is the technology that carries your message. E.g. HTTP Several transports have advanced support e.g. sift and pass through PAYLOAD The format is the type of message that is carried on the transport e.g. XML Formats are presented in an easily readable and configurable tree structure The payload is the contents of the message fields Fields can be parameterized using data sources 11

13 Testing and Virtualizing Web Technologies Accelerator WSDL SOAP JSON Accelerator XSD XML REST WADL RECORD HTTP(s) TCP UDP 12 Record transactions Or use accelerators such as WSDLs to short cut the process Build tests and stubs across a range of supported protocols and formats

14 13 Supporting technologies the business depends on IBM Rational Test Virtualization Server supports multiple platforms and protocols Protocols, message formats, and technologies Transport/Protocol Message Formats Technologies ActiveMQ (SMTP, IMAP) Files FTP/S HTTP/S WebSphere MQ Java RMI JMS SAP IDoc, BAPI, RFC Software AG s IB & IS Sonic MQ TCP/IP TIBCO Rendezvous, Smart Sockets & EMS CICS TG CICS IPIC IMS Connect MQTT.Net Objects Bytes COBOL Copybook ebxml ASC X12 UN/EDIFACT DFDL Fixed Width Java Objects MIME SOAP Software AG Broker Docs SWIFT CHIPS FedWire TIBCO ActiveEnterprise HL7 v3 XML (DTD, XSD, WSDL) ISO8583 Custom CentraSite Oracle Fusion SCA Domain Software AG IS, Broker, BPMS WebSphere Application Server WebSphere Message Broker WebSphere MQ Websphere SI Bus Sonic ESB System z (CICS and IMS) TIBCO ActiveMatrix UDDI Web Services WebSphere RR WSDL BPM Databases Log Files

15 Integration testing is interrelated and connected to your other aspects of application testing Maintenance Analysis Requirements Release to Production Testing Acceptance System Performance Integration Testing Functional Testing Coding Unit Testing 14 All of these tests must be planned. Artifacts created by tests need to be managed, linked to other development artifacts, and analyzed. 14

16 Rational also offers a complete suite of testing tools Rational Functional Tester Record, enhance, and execute automated tests Java and/or Visual Studio generated test scripts Rational Test Workbench Rational Performance Tester Root cause analysis Performance loads Automated script capture and playback Rational Integration Tester Web service visual editors WS-BPEL testing Code-free testing Adapters 10 Rational Quality Manager Test Planning: Comprehensive test plan, Shared objectives, Scope, Timeline, Resources, Risk assessment Test Execution: Manual test execution, Use test automation tools, Record test results, Submit & track defects Team Collaboration: Process enactment and enforcement, reviews, task management, Rapid team member on-boarding Test Construction: Requirement-driven testing, Test environments coverage, Manual test authoring, Test lab management Reporting & Dashboard: Status & progress tracking, Customizable live dashboard, Real-time metrics and reports, Compliance and quality audit Jazz Team Server 15

17 Service Virtualization Rational Quality Manager Test Management and Manual Testing Rational Test Virtualization Server Webservices Databases MQ Mainframes Middleware and others Urban Code Automated Deployment Version Control RTC and others APM ITCAM and others Automated Testing Rational Test Workbench and others Test Data Optim and others Continuous Integration Jenkins and others 16

18 Continuous Combining Testing continuous from build testing to with production Reliable, repeatable, & traceable deployment automation Orchestrate and automate the deployment and testing of applications, middleware configuration, and database changes into development, test and production environments. IBM UrbanCode Deploy Test Environments virtual components integrated with Databases Internal Messages Third-party Services IBM InfoSphere Optim Test Data Mgmt Deploy private, secure right sized test data IBM Rational Test Virtualization Server IBM Rational Test Workbench Deploy what is ready, virtualize the rest Continuously test in production-like env. Dev QA Simultaneously test across multiple test stages IBM Rational Quality Manager UC Deploy Plugins Manage quality across the delivery lifecycle Dynamic Infrastructure 17

19 Continuous testing and continuous deployment Continuous Deployment Continuous Testing = RIT/RTVS = URBANCODE UrbanCode Automated Deploy Integration Testing 1 2 Service Virtualization UrbanCode Deploy enables continuous deployment of available environments into the targeted infrastructure 2. RTW/RIT can automatically test the deployment and integration of that environment - do the components communicate correctly? 3. Service Virtualization allows the testers to: a. Stand up virtual services in place of components that are unavailable (not built yet, too expensive, resource contention, 3 rd parties etc.) b. Plan for test conditions, such as negative testing that may not be possible with the real components.

20 Continuous testing with virtualized services Rational Team Concert Build Deploy Jenkins Automating deployment and testing to production-like environments Dev QA Application changes being tested IBM UrbanCode Deploy RTW plugin RTVS plugin Test Rational Test Workbench Databases Mainframe applications Third-party Services Virtualized Services Rational Test Virtualization Server 19

21 RQM and Service Virtualization Invocation RQM RTCP / RTVS RTW / RIT Create Test Case Publish Virtual Service Create Virtual Test Service Associate Test with Virtual Service Start Virtual Service Run Test Stop Virtual Service 20 Rational Quality Manager 20 Virtual Service Logs linked to Test Case Result Rational Test Control Panel Rational Test Virtualization Server RTW: Rational Test Workbench RIT: Rational Integration Tester

22 The Coca-Cola Company decreased production defects by 26-50% with IBM Business Challenges: Quality issues/defects found in production Insufficient or unavailable test environments The Solution: The Coca-Cola Company chose IBM s service virtualization and automated integration testing solution over other offerings for the following reasons: broadest support for the technologies in their environment; cost of the solution; realizing increased developer and tester productivity. Software: IBM Rational Test Virtualization Server, IBM Rational Test Workbench 26-50% Decreased Defects in production 26-50% Accelerated Project delivery 26-50% Reduced Cost of testing 26-50% Improved Team productivity 21

23 DEMO 22

24 Approach to get started with Virtualization Discover Configure Virtualize Model Deploy Execute Identify testing pain points Understand IT technologies Review use cases Initiate ROI analysis Understand testing methodology Install and configure SV software Configure transactions / protocols Configure services Build additional AV test cases Execute specified test cases Listen and capture transactions Build negative test case Refine ROI Model behavior of transactions Model negative test cases Model perfect storm test cases Apply metrics to virtual models Deploy models to virtual environment Configure SUT connections Execute specified test cases Capture test results Capture and measure results Recommend next steps 23

25 24