Business white paper Simulated tests, real results These companies are using HP Service Virtualization to overcome testing challenges, accelerate delivery, and improve quality. So can yours.
Table of contents 3 Executive summary 3 The pressing need for smarter testing 4 Service virtualization: a real solution 4 Customer example #1: TTNET Applications Testing Unit 5 End-to-end automation: from pipe dream to reality 5 From POC to 95% virtualization in 10 weeks 6 100% payback in three months, 100% efficiency gain 7 Customer example #2: EWE TEL 7 Solving the issues of testing customer-built applications 9 More than a test optimization solution 10 Making the Dev/Ops vision a reality 11 Lessons learned and practical advice 11 How to jumpstart your move to service virtualization 12 Summary 12 HP: We re here to help
Executive summary The cloud. Mobility. Social media. Connected devices. Collaborative computing. All the latest technology trends and advancements share two attributes: They re great for users, creating new capabilities and conveniences and efficiencies. And they can be difficult for development and testing teams, imposing new challenges, complexities, and obstacles. How can companies deliver new applications on time and on budget when there are multiple interdependent components from multiple sources, when those components are not always available for testing, and when teams and toolsets are distributed across multiple locations? Service virtualization offers a solution. Service virtualization tools simulate software components so end-to-end testing can proceed even when dependent components are not available. That means teams can perform integration tests sooner and more often, accelerating the delivery of high-quality, thoroughly tested applications. There s no question that service virtualization holds considerable promise. The question is, who has transformed the potential into actual results? This paper provides two compelling examples from the HP Service Virtualization customer base. Following a brief summary of the advantages of the service virtualization approach, it presents the real-world experiences of TTNET, Turkey s largest Internet service provider, and EWE TEL, a major German telecommunications company. The pressing need for smarter testing Multiple factors are making it more difficult to meet development and testing deadlines without compromising on quality: Changing expectations. Users now expect enterprise applications to be as easy and convenient as consumer-based apps and to evolve quickly to meet new requirements. That puts pressure on development and testing teams to accelerate both innovation and velocity. Composite applications. Apps that combine existing functions into a new application promise agility but actually introduce new impediments. As new composite application functionality is developed, it can be difficult to get all of the pieces ready and available for testing. Moreover, each component may have a different service-level agreement (SLA) or performance requirement; sometimes there may be parallel development and delivery requirements; some components may be behind a firewall, making testing difficult; or there may be access fees for components, contributing to budget overruns. Connected devices. The sheer number and interconnectedness of devices is mushrooming and the devices are built from multiple software variants running on a diverse range of hardware platforms over multiple networks. That, in turn, increases the complexity of the testing environment: more carriers, data sources, security considerations, toolsets, and so on. Non-stop testing. Iterative or Agile development replaces one long development cycle with many shorter ones, making testing a non-stop exercise aimed at a moving target. However, teams cannot afford to wait for all dependent components to be available before they begin integration testing. That is why, in the words of Forrester Research, as Agile adoption matures and release cycles shorten, testing becomes the weak link. 1 The net result is that testing is now a bottleneck for many projects and initiatives, a source of delay and unexpected expense rather than a source of business value and strategic advantage. Moreover, today s testing practices can actually decrease quality. Developers often build ad hoc stubs that don t mimic production precisely or they may forego some aspects of testing altogether. Development and testing teams can t stop any of these trends. But they can respond and tame the complexities with service virtualization. 1 Source: Service Virtualization and Testing, Forrester Research, January 15, 2013. 3
Service virtualization: a real solution Service virtualization tools can remove the obstacles to fast delivery of high-quality applications. They give development and testing teams building composite applications the ability to virtualize services within existing environments both on site and in the cloud. Service virtualization solutions will enable the delivery of greater business value from software. voke Research 2 Adopters of service virtualization and testing solutions improve quality and increase speed of delivery by testing better, faster, and continuously. Forrester Research 3 By enabling parallel development and early functional testing, service virtualization can eliminate wait times for teams using Agile methods. And they can reduce costs by eliminating the requirement for access to constrained business-critical infrastructure or pay-per-use cloud components for testing. Additional benefits include: Fast access to all components: Service virtualization allows for the simulation of all components or system dependencies to ensure a complete and accurate test of the composite application. Higher quality: Service virtualization increases code coverage and reduces the number of defects released to production, so overall quality improves. Earlier functional and load testing: Service virtualization facilitates performance and load testing by virtualizing platforms and making them available to test for performance and scalability. Support for apps with complex interdependencies: Today s apps are often complex systems with multiple contributors. Service virtualization makes it practical to share assets and virtualize an entire system of software from any member of the supply chain. More collaboration: Service virtualization provides a way for teams across the lifecycle to collaborate and deliver higher quality software. With this brief summary of the value of service virtualization providing context, let us now turn our attention to the actual experiences of two HP Service Virtualization customers. Customer example #1: TTNET Applications Testing Unit As the largest Internet service provider in Turkey, TTNET (a subsidiary of Türk Telekom), is under constant pressure to roll out innovative new services to its six million subscribers and to cut costs and delivery timeframes at the same time. The company has found a solution in HP Service Virtualization. TTNET deploys more than 200 applications per year, primarily related to customer relationship management (CRM), and many of these applications are mission critical. Before implementing HP Service Virtualization, TTNET had to use test infrastructures from third parties in its test cases. This was creating issues such as lack of accessibility, burdensome authorization requirements, inability to access private data, even downtime. We needed to virtualize our test systems and we needed automation to accelerate deployment, said Hasan Yükselten, test and release manager at TTNET. And of course, we needed to reduce our cost. We decided to solve the problems of the company by implementing HP Service Virtualization. 2 Source: Market Snapshot Report: Service Virtualization, by voke Research, December 11, 2012. 3 Source: Service Virtualization and Testing, Forrester Research, January 15, 2013. 4
End-to-end automation: from dream to reality Previously, automation was not an option for TTNET because dependent software components were in discrete locations and it was difficult to reach the systems where they resided. For example TTNET systems had to interact with the Turkish National Identity System through telecom infrastructure such as public switched telephone network (PSTN) systems. TTNET could automate functional test cases, but end-to-end automated testing was impossible. TTNET took its first step toward full automation by implementing HP Service Virtualization to virtualize the third-party systems, putting HP Service Virtualization between its infrastructure and the third-party infrastructure. The TTNET team could then use HP Service Virtualization instead of the third-party infrastructure. After that, the TTNET team was able to use automation tools by integrating HP Unified Functional Testing (UFT) and HP Service Virtualization tools. Now we can run automation test cases and end-to-end test cases on HP Service Virtualization, said Mr. Yükselten. Pleased with this initial success, the TTNET team then implemented HP Service Virtualization for the development systems as well. From POC to 95% virtualization in 10 weeks According to Mr. Yükselten the implementation of HP Service Virtualization was fast and easy. Including the time needed to do the proof of concept, we could use this tool in six weeks, he said. We spent first two weeks for the POC and after four weeks, we were successfully using the tool. For the first six weeks after implementation, TTNET was using HP Service Virtualization for 45% of end-to-end test cases. Within 10 weeks, 95% of its test cases could be run on HP Service Virtualization, according to Mr. Yükselten. In all the team is now using three HP Service Virtualization systems: One for development, one is used exclusively for campaigns, and one is for end-to-end testing. Figure 1. In a total of 10 weeks, TTNET moved from POC to 95% virtualization. 6 weeks 4 weeks POC Making decision for tool HP SV set up Tool customization Additional service virtualization 45% virtualization 95% virtualization 10 weeks from start of POC to 95% virtualization 5
TTNET needed customization of the HP Service Virtualization tool to meet its specific requirements, and found that HP Professional Services was quick to respond. We were also using HP Application Lifecycle Management tools before Service Virtualization, and we had a good relationship with HP Software, said Mr. Yükselten. With the HP Rapid Deployment service we were able to use the tool very quickly. Figure 2. At TTNET 166 services were virtualized successfully, and 48 are now being used actively. 100% gain in efficiency 100% payback in three months, 100% efficiency gain TTNET achieved the cost savings and efficiency gains it had expected quickly. In less than three months we got the investment back, said Mr. Yükselten, and for the campaign test cases we gained 100% efficiency. Previously, we could run just seven campaigns in a month, but after HP, we managed to run 14 campaigns in a month. For basic test scenarios, TTNET could previously run all test cases in 112 hours. After implementing HP Service Virtualization the team ran them all in just 54 hours. We managed to automate 52% of our campaign test cases, and this meant a very big efficiency gain, said Mr. Yükselten. Since deploying HP Service Virtualization, TTNET has drawn the attention of other divisions and companies that want similar results. We have a branch company called AVEA, and they also want to use this tool, said Mr. Yükselten. Eight companies visited us in Turkey to learn more about our experiences using this tool. Many companies want this tool in their test systems. 6
Mr. Yükselten has advice for other companies looking to benefit from service virtualization. Know your needs first, he said. For example, in our company we have three blockage systems for third parties and the other systems don t change every day. So it was easy to implement HP Service Virtualization in our systems and virtualize the other systems. We don t need to do virtualization day by day, because the other systems don t change every day. Once a month, we consult and change our systems, update our web services on Service Virtualization, and that is enough for us. But if the other parties systems change day by day or frequently, it may be difficult to do virtualization every day. In addition, said Mr. Yükselten, companies should think about automation, not just virtualization. They are different aspects of increasing efficiency, and they should both be considered individually. What s next for TTNET? Our first goal is doing more automation with HP Service Virtualization and Unified Functional Testing and the next is using HP Service Virtualization in development sites, said Mr. Yükselten. We plan to find early defects in development sites and get more high-quality products into testing faster. Customer example #2: EWE TEL When more than a million people depend on your company for electricity and gas, your network and applications had better be up and running reliably. EWE TEL operates the network for EWE Group, one of the largest energy suppliers in Germany, and offers consumers a wide range of telecommunications services such as high-speed Internet access. EWE TEL has kept the quality of its applications high by deploying HP Service Virtualization. The company has used the tool to reduce performance complexity across an enterprise billing process and has significantly improved application quality while also gaining predictive insights in the behavior of composite application services. Solving the issues of testing customer-built applications EWE TEL depends on its software and IT systems to interact with its end users, so these applications are mission critical. The company has about 40 applications, including customer self-service applications, activation components, and the billing system. Because the telecommunications business is very specialized, EWE TEL develops many of its applications on its own. In fact, nearly half of its applications are developed in house; for example, the customer self-service portal and the customer care system are internally developed. To test its home-grown applications, EWE TEL created a team, recruiting personnel from the operating departments, adding IT staff and certifying them as testers. This made it easy to agree on roles, tasks, processes, and so on, concerning the tests. When we created this new team, we faced the problem of testing the systems end to end, said Bernd Schindelasch, leader for quality management and testing at EWE TEL. When you have 40 applications and have to test an end-to-end process over all of those applications, all the contributing applications have to be available and have a certain level of quality to be useful. However, what the team discovered was that the order interface of another service provider was often unavailable and responses from that system were faulty. So end-to-end testing was a difficult challenge. 7
Figure 3. The complex composite application landscape at EWE TEL made end-to-end testing extremely difficult. Legend DB Own development External systems/ External development To overcome the obstacle, the team initially developed diverse simulations based on traditional mockup scripts. These were very useful for developers to do unit testing, but they weren t configurable to enable testers to create the right situations for positive and negative tests. Creating these mockups was also a major undertaking, and sometimes the effort to create the mockup would have been greater than the real development effort. In 2012, the EWE TEL teams searched for products that might be able to help with the composite application testing problem. They requested a proof of concept of HP Service Virtualization to further evaluate the product. We found that it supported all the protocols we needed, and came with a rule set to predict the responses, said Mr. Schindelasch. During the POC we found that HP Service Virtualization delivered benefits both for developers and testers. Even our architects found it to be a good solution. So in the end, we decided to purchase the software. EWE TEL first implemented service virtualization in a pilot project. The team integrated service virtualization as a proxy between its customer care system and the order system. It s in the middle, between your systems, and records all messages and their responses, said Mr. Schindelasch. Afterward, you can just replay this message response or you can improve the rules manually. For example, you can add data tables so you can configure the system to work with the actual test data you are using for your test cases to be able to support positive and negative tests. 8
Figure 4. EWE TEL created an efficient five-step virtualization process. Plan and analyze Design Implement Improve Use More than a test optimization solution While the primary objective for implementing HP Service Virtualization was to find a way to do end-to-end testing to optimize applications, EWE TEL quickly found that additional benefits were possible. We ve reduced the effort to create simulations by 80%, which is a huge amount, and have been able to virtualize services that were still under development, said Mr. Schindelasch. Figure 5. A closer look at the design components and capabilities of HP Service Virtualization used by EWE TEL Performance model Data model Current status Performance model jmsx queue configuration Protocol used We have been able to uncouple the tests of the self-service application from a new technical feasibility check. Therefore, we ve been able to test earlier in our processes, he continued. That reduced our efforts and costs in development and testing and it s the basis for further test automation at low testing cost. Equally important, EWE TEL was able to boost application quality while cutting testing timeframes. In the end, we ve improved quality, said Mr. Schindelasch. It s even better for our customers, because we re able to deliver faster and cut our time to market for new products. 9
Figure 6. The intuitive interface makes it easy to find useful server and performance data quickly. Service name and host Logged messages Current service status Protocol used Data model/ performance model Making the Dev/Ops vision a reality As development shifts from traditional waterfall methods to Agile methodologies, better integration is needed between development and testing tools. According to Mr. Schindelasch, HP Service Virtualization can help accelerate that integration and foster greater cooperation between teams. I think it s crucial to increase cooperation between development and testing, because there wouldn t be a real benefit to virtualize the service after development already mocked up in an old-fashioned way, said Mr. Schindelasch. We brought them together. We had the training for a lot of developers. They started to see the benefits and started to use HP Service Virtualization the way the testers already did. Now EWE TEL developers and testers are working together more closely and earlier in the process, according to Mr. Schindelasch. What s coming in the future is that the developers will start to use service virtualization for their continuous integration, because service virtualization has the potential to change the performance model, so you can let your application answer slower or faster. With HP Service Virtualization, developers and testers at EWE TEL are now working together closely and collaboratively. 10
Lessons learned and practical advice Based on his experience with HP Service Virtualization, Mr. Schindelasch has some advice to pass along to others who are on the road to virtualized, automated testing. Results at a glance TTNET Set up in six weeks including POC Achieved 95% virtualization in 10 weeks 166 services successfully virtualized Increased campaign test efficiency 100% Saved 147 man-days in testing Saved three man-months via automated testing EWE TEL 80% reduction in time to create simulations Able to test earlier in the process Cut costs through test automation Improved application quality Developers and testers collaborating effectively Created an efficient virtualization process It s important to work together with development and testing, he said. To gain maximum benefit from HP Service Virtualization, you have to design your future solutions. What services do you want to virtualize, which protocols will you use, and where are the best places to intercept? Do I want to replace real systems or create the whole environment as virtualized? In which way do I want to use performance model and so on? It s also very important to really understand what your needs are before you start using the tools and just virtualize everything. It s easy to virtualize, but there is no real benefit if you virtualize a lot of things you didn t really want. As always, it s important to think first, design your future solutions, and then start to do it. How to jumpstart your move to service virtualization When your company is ready to make the transition to service virtualization, HP Software Professional Services is ready to assist you with end-to-end solutions including: HP Service Virtualization foundation services for product installation, configuration, and deployment Educational services for customer enablement Virtual services development using HP Service Virtualization Test data management for virtual services Best practices consulting for HP Service Virtualization Testing and test environment management using HP Service Virtualization Setting up and operating a service virtualization center of excellence (CoE) In addition, the HP Service Virtualization QuickStart Service provides hands-on mentoring and implementation of HP Service Virtualization in your specific environment. An HP-certified consultant will be on site to accelerate adoption and use of the tool by reducing the learning curve and assisting with real use case scenarios. In just two weeks, HP Software Professional Services can provide your testing team with the knowledge and experience to create, deploy, configure, and exercise virtual services. Installation of the HP Service Virtualization software by a trained HP Software Professional Services specialist is included in this service. HP Software Professional Services offers outcome-based services focused on generating specific outcomes that are guaranteed through service-level agreements (SLAs). For more information, click here. Early adopters of SVT [services virtualization and testing] show that they are debunking many of the testing bottlenecks, allowing early integration testing, more automation, and smarter testing coverage. Forrester Research 4 4 Source: Service Virtualization and Testing, Forrester Research, January 15, 2013. 11
Summary Service virtualization is a complex, multi-layered topic and this paper can only address a small fraction of the issues and opportunities. However, the intent of this paper is simply to answer one simple question: Are the benefits of service virtualization real? By highlighting the actual experiences of two real-world companies with HP Service Virtualization, we hope we ve made the case that there is a better, smarter way to test composite applications from end to end and to deliver composite applications with less waiting, reduced risk, and lower costs. In the process, we hope we have opened your eyes to additional possibilities and benefits for service virtualization within your company. HP: We re here to help HP is ready to provide assistance in your transition to service virtualization, and to help you extract maximum business value from your HP Service Virtualization deployment. For more information, please visit hp.com/go/sv. Sign up for updates hp.com/go/getupdated Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. 4AA5-1188ENW, February 2014 This is an HP Indigo digital print.