Addressing Software Testing Costs, Complexity and Challenges: Sogeti UK s Annual TestExpo Survey

Transcription

1 Addressing Software Testing Costs, Complexity and Challenges: Sogeti UK s Annual TestExpo Survey Industry survey highlights demand for high-quality training, tools and techniques as businesses prepare for rising complexity of software testing TestExpo is one of the longest-serving and most prestigious events for software testing and Quality Assurance professionals in the UK. Since its launch in 1996, it has grown into an indispensable forum for debate, discussion and networking as well as a respected and consistent source of sentiment and opinions from and for members of the Testing and QA industry. This year s TestExpo event explored a series of themes surrounding its primary focus: Transforming Testing: Identifying Solutions that Work. The day highlighted the aspiration of testing professionals to improve quality, reduce time-to-market and lower testing costs. Recurrent themes included: test transformation; mobile testing; performance testing; testing budgets; the testing organisation; test environments and test data management; the cloud; and, most prominently, agile testing. Rigorous software testing is crucial for businesses to survive, thrive and to maintain their standing let alone get ahead of their competition. A lack of preparedness, resourcing and execution on QA processes will cause a substantial loss of revenue and reputation. To uncover some of the challenges that businesses face from both a QA and software testing perspective, Sogeti UK conducted a detailed in-person survey of TestExpo 2013 delegates to reveal attitudes, actions and challenges relating to the transformations faced by today s software testing teams. The responses help to gauge the potential impact they see for their businesses, as well as plans to help their business address these imminent issues. Key findings of these conversations include: 92% of respondents say the cost and complexity of software testing is increasing 7 admit that their business or organisation is not prepared for future testing challenges Training, transformation, recruitment and tools are key in preparing and meeting future challenges use cloud-based testing, whether it s via a specific platform, service or resource. And many voiced concerns on the complexity of cloud-based test environments and cloud-based access to data (1), as well as security (16%) 49% conduct mobile testing, while a further plan to do so in the next 12 months Functionality is ranked higher (43%) than security (31%) as a priority for mobile testing Cost and challenges Successful businesses are driven by the constant need to reduce time to market, improve the quality of all deliverables and applications, and the need to harness technology to meet the requirements of their organisation. Yet as software, applications and technology become more intricate, timelines shorten and user expectations increase, there is a consequential impact on underlying processes including software testing and QA. The potential reach of this impact is reflected in our opening question to TestExpo delegates, with 92% stating that the cost and complexity of software testing is increasing (figure 1). This forewarning of change and its impact on business is likely to be exacerbated by the lack of preparedness, with around 7 admitting that their own businesses are not yet completely ready to face such challenges (figure 2). Figure 1: Future complexity and cost expectations Page 1

2 Percentage 8 7 Do you feel the cost and complexity of testing is increasing? 73% Yes, both Yes, but just the cost Yes, but just complexity Neither Figure 2: How prepared are businesses to meet anticipated future complexity 8 7 How equipped do you feel your current test team is to overcome cost and complexity challenges? % We feel comfortable and prepared We feel partly prepared We are not prepared at all Today s multi-vendor application landscape has produced a highly complex testing environment. Add to this picture an increasing reliance of large businesses on specialised, interconnected legacy systems, and one Page 2

3 can begin to grasp the difficulties modern day testing. Software testing and QA professionals are tasked with supporting their businesses to be ready and to adapt to an ever-changing landscape. When asked how they expected to do this, Sogeti UK found that quality training, tools and techniques are perceived as critical to these challenges. From a human resources perspective, the balance between recruiting appropriately skilled testers and QA professionals in house and training existing testers remains delicate, with implications impacting bottom line costs as well as quality What do you feel you need to do in order to keep up with this increase? 51% 61% Recruit Training our appropriate existing testers skilled testers and QA and QA professionals professionals in house 51% Identify and purchase the right testing tools 56% Identify new methods and techniques to innovate and transform testing fuction 32% Secure more budget from management 14% Engage with specialist test provider to benefit from external expertise Engage with a Managed Testing Service test fuction to allow the test fuction to benefit from external expertise To keep pace with this rapidly evolving complexity and overcome rising costs, the leading test professionals questioned said that their highest priorities were to train their existing test and QA teams (61%), identify innovative ways to transform their testing function (56%), recruit testers and QA pros with new skills (51%) and purchase new, relevant testing tools (also 51%). It is also interesting to note that only one-third of respondents feel they need to secure more budget to keep up with increases in cost and complexity - highlighting a desire to do more with less - while 22% would like to engage with either a specialist test provider or a Managed Testing Service to benefit from external expertise. Our next line of questioning addressed delegates thoughts on Managed Testing, Testing Platform as a Service (TPaaS) and mobile testing. Respondents provided their thoughts on Managed Testing, looking ahead to the next 12 months. Many responses fell in line with Sogeti, Capgemini and Hewlett-Packard World Quality Report findings that lean towards the use of more centralised structures in order to achieve efficiency and cost optimisation. Within the report we notice that the practice of engaging external vendors to manage the delivery function a Managed Testing Service has grown from 13% to over the last year. Page 3

4 45% 4 39% Thoughts on Managed Testing 35% % 5% 17% Centralise testing internally Already engaging with a Managed Test Service provider Looking to engage with Do not have that level of an external Managed requirement Testing Service Provider It comes as a little surprise to see that the majority of respondents either centralise testing internally (39%), or already engage with an external managed test service provider (17%). Furthermore, will look to outsource to a Managed Test Service in future, and the remainder of respondents are not yet operating at a level that requires Managed Testing Services Cloud based testing When it comes to data security and performance, cloud adoption and cloud-based testing adoption has been somewhat sluggish across the industry. Overall, however, executives remain positive about its growth, and any challenges are perceived as temporary as cloud adoption becomes more mainstream. While security remains a top priority for testing and IT. Sogeti UK turned to TestExpo 2013 delegates to gauge whether or not organisations currently, or would consider, using Testing Platform as a Service (TPaaS) and if not, why not. Page 4

5 Percentage 45% 4 35% 3 15% Testing Platform as a Service (TPaaS) 31% 41% 5% 5% Heavy use of cloud based testing platforms, services or resources Some use of cloud based testing platforms, services or resources No use currently, but plan to within the next 12 months No use currently, and no plans within the next 12 months The survey responses reflect the perception that many organisations have been slow to migrate or host software applications in the cloud largely due to security and infrastructure concerns with a clear split between those who do not use some form of cloud-based testing platform, service or resource (72%) and those who do (). Among the majority of delegates who indicate they do not currently use TPaaS, a range of reasons why were offered. The most popular responses focus around the three areas of complexity of test environments and access to data (1), security (16%) and a lack of benefit to the business (). 1 16% 14% 6% 4% 2% 16% It's not secure If you do not use TPaaS, why not? It wouldn't give any benefits to our business 9% 9% It wouldn't give financial return It would mean disruption - would have to change existing testing set-up We don't have required skills or resources 1 Test environments are complex to set up and we want ongoing access to our data Have never considered it Page 5

6 Mobile testing Our survey findings reflect the fact that mobility is redefining the rules of communication and business. While offering an extra tool for business users and customers alike, mobile applications also raise many challenges and demands from users in terms of quality and usability. The importance of mobile testing has caused an increase in mobile testing activity; however efficiency and performance remain a primary focus. Thorough mobile testing is intended to help organisations to deliver high quality customer experiences, better execution and consistency across networks and devices, and to ensure brands are represented correctly through their related applications at all times. Close to 75% of all TestExpo delegates said that they already performed mobile testing, or planned to in the near future: 6 5 Do you currently perform any mobile testing? 49% % 3% Yes, mobile testing is important to us No, but plan to in the next 12 months No, we have mobile applications but don't have mobile testing No, we don't have any mobile applications Nearly half of organisations (49%) currently perform mobile testing, while a further plan to in the next 12 months. Mobile testing adoption has increased every year (World Quality Report ); however, there is much room for improvement for businesses to fully embrace mobile testing methods and tools. Just 3% have mobile applications but do not test them. Page 6

7 45% Areas of focus for your mobile testing activity 43% 4 35% 3 31% 33% 29% 15% 22% 14% 5% 3% Testing efficiency Security Functionality Portability Certification Compatability / regression testing Integration with other services Functionality (43%) is the top focus for mobile testing, followed by compatibility (33%) and security (31%). Mobile Functional Testing is a priority for many organisations and assesses criteria such as interoperability, application lifecycle, and network/display management. Testing generally involves use case models, complemented by exploratory testing. Compatibility testing is an understandable area of focus for a third of all TestExpo delegates, because it provides a practical solution to the challenge of testing hundreds of mobile possibilities, clearly defining the broad spectrum of platforms and specific devices available. Both manual testing and highly efficient test automation tools can be applied to validate quality across pre-defined devices and provide peace of mind. Mobile Security Testing is increasing in importance, with organisations keen to reveal and resolve critical security holes and threats, particularly for applications that feature some transaction of money or data. Typically mobile testing efforts would validate six common security aspects: confidentiality, integrity, authentication, authorisation, availability and non-repudiation. Because mobile is now responsible for a great deal of business value within the majority or organisations, it is particularly reassuring to see that the understanding of the importance of mobile testing is certainly increasing. Business priorities in 2013 We asked our TestExpo 2013 attendees to rank the following in order of priority for their own businesses in 2013: centralisation, cost, time-to-market or speed, quality, innovation, flexibility and scalability, and delivering a better customer experience. Page 7

8 7 Priority for your business in % % 36% 15% 23% 34% Centralisation Cost Speed Quality Innovation Flexibility / Scalability Delivering a better customer expereince While each of these factors is important for any organisation to consider, respondents cite improving quality of testing to the business (65%) as their top priority for the year ahead of other drivers; including reducing timeto-market (36%), delivering a better customer experience (34%), reducing costs (29%) and improving flexibility and scalability (23%). The final question we asked TestExpo attendees was what hoped to take, or had taken away from the event. Respondents stated that they walked away with valuable takeaways from the event, with three-quarters (74%) stating they left with fresh ideas for best practices, techniques and methods that would help them to overcome the key business challenges they are facing in software testing. Page 8

9 8 Key takeaways from TestExpo % % 3% Ideas for best practices Information on relevant techniques and methods tools and technologies Networking New services Sogeti s strong partnerships, in-depth expertise and continuous research efforts ensure that it can address each and every testing priority highlighted in the survey. Its broad range of business-driven testing solutions, and industry standard methodologies including Sogeti s PointZERO Vision, Test Process Improvement (TPI), Test Management Approach (TMap) and Agile Testing underpin the full range of test services available, and are trusted by some of the world s leading organisations. In summarising the success and the findings of this revealing survey, Brian Shea, CEO of Sogeti, said: Quality has never been a higher priority for businesses as they work hard to attract customers in highly competitive markets. Continual re-evaluation of test process and capability is necessary to ensure the smooth roll out of transformational projects that will ultimately ensure they stay at the top of their game. Through the adoption of pioneering test tools and innovative methodologies, these organisations can gain confidence in the services they deliver and be assured that their testing activity is successfully aligned with broader business objectives and stakeholder values. The overall survey findings echo Sogeti s World Quality Report, published annually by Capgemini, Sogeti and Hewlett Packard Software, which also reveals a renewed focus on quality, customer experience and streamlining of the in-house Testing function, as well as impressive growth in mobile testing, and increased challenges around test environments. View or download a full copy of the World Quality Report here. Page 9