Growth Market Software-Testing Market Trends, Service Providers, and Success Factors

Transcription

1 Growth Market Software-Testing Market Trends, Service Providers, and Success Factors A study by Pierre Audoin Consultants GmbH on behalf of SQS Software Quality Systems AG June 2011

2 2 Contents I. TABLE OF CONTENTS 1. Introduction Concept of the Study Trends in Software-TestING and QualitY management Boom in Optimization of Test Activities Increasing Acceptance of External Testing Support Development of Demand for Dedicated Testing Services Development of Demand for Blended Service Delivery in the Scope of Testing Services Success Factors in Software Testing Factors for Successful Collaboration with Testing Service Providers Important Factors in the Selection of a Testing Service Provider SQS: Successful in the MarkEt as an Independent Testing Specialist Overview of SQS Competency Profile of SQS Service Pledges of SQS Blended Service Delivery Transparent Attainment of Objectives and Performance-related Billing Summary... 28

3 3 Figures II. TABLE OF FIGURES Fig. 1: Distribution of Size Classes within the Surveyed Target Group... 5 Fig. 2: Fig. 3: Planning Status of the Companies Surveyed in respect of the Optimization of Testing Activities... 7 Reasons Given by the Companies Surveyed for Optimizing Testing Activities... 8 Fig. 4: Organization of Testing Activities in the Companies Surveyed Fig. 5: Fig. 6: Benefits in Software Testing through Collaboration with External Specialists Planning Status of the Surveyed Companies in the Collaboration with Testing Centers Fig. 7: Reasons Given by Companies Surveyed for the Use of Testing Centers 15 Fig. 8: Fig. 9: Expected Flexibility and Availability of Testing Team Factors for Measuring the Achievement of Objectives of the Testing Team Fig. 10: Significance of Results-oriented Remuneration of Testing Services Fig. 11: Significance of Linguistic Capability in Testing Projects Fig. 12: Prioritized Competences of the Testing Service Provider... 21

4 4 1. INTRODUCTION Companies are focusing increasingly on the quality assurance of the IT application landscape. It is pivotal for all IT operations and contributes considerably to profitability and efficiency. Testing and quality management enable companies to address key challenges, such as incident-free support of business processes and efficient working practices for software users. Today companies around the world invest more than 50 billion per year in applications testing and quality assurance. This area is of high strategic importance as the results of this study underline: the vast majority of the managers surveyed 91 per cent confirm that software testing and quality assurance are among the most important IT disciplines in their company. Yet what form of software-testing and quality management lends itself to achieving the greatest possible added value? How can expenditure and quality of testing be harmonized? What standards do companies expect from their testing activities and external testing service providers and how can such collaboration best be organized? The survey conducted by Pierre Audoin Consultants GmbH (PAC), market analysis and strategy consultancy, and commissioned by SQS Software Quality Systems AG (SQS), provider of software-testing and quality management services, answers these questions. The introduction to this study, entitled Growth Market Software Testing: Market Trends, Service Providers and Factors for Success, analyzes the trends and dynamics on the demand side. The studyʼs second section shows the opportunities and success factors in collaboration with external testing service providers. The study also has a practice-oriented focus: the positioning of SQS is then compared with the survey results and market analyses. The study is intended to help companies take decisions relating to organizing and performing their testing activities to generate the maximum added value and to properly evaluate the potential of permanent collaboration with providers of software-testing and quality management services.

5 5 2. CONCEPT OF THE STUDY This study is based on a survey of 309 managers and IT decision-makers conducted by the independent market analyst PAC on behalf of SQS, service provider for software-testing and quality management, between February and May The participants in the survey provided information about the organization of their testing activities and set out the pros and cons of integrating qualified partners. They gave their assessment of how to optimize the performance of tests and measure the progress achieved. Finally, the respondents identified key criteria that a service provider needs to meet for software-testing and quality management and specified the significance of managed services for testing. The target group comprises companies from all sectors from Europe and North America with a minimum size of 1,000 employees and an IT team consisting of at least 100 employees. Just under half of the organizations surveyed currently employ between 1,000 and 4,999 employees, and a further 52 per cent have over 5,000 employees. Composition of sample by number of employees 1,000 to 2,499 2,500 to 4,999 5,000 to 9,999 10,000 and more Don t know/no answer Share in percentage of companies, n = % 15% 24% 23% Fig. 1: Distribution of Size Categories within the Surveyed Target Group

6 6 86 companies are headquartered in the DACH region (Germany, Austria, Switzerland), and more than sixty companies are headquartered in Great Britain, the USA, and in Scandinavia (Denmark, Sweden, Norway, Finland) respectively. A further 27 companies are based in the Netherlands. 52 per cent of all decision-makers were heads of IT, Chief Information Officers (CIOs), Chief Operating Officers (COOs), or Chief Technology Officers (CTOs), and 27 per cent were members of the management or managing board. A further 20 per cent were responsible for their companyʼs testing activities as heads of IT departments. This report is aimed primarily at IT and corporate decision-makers who are responsible for the area of software-testing and quality management or deal with the handling and optimization of the test activities and processes and the selection of and collaboration with testing service providers on the strategic and operational level.

7 7 3. TRENDS IN SOFTWARE-TESTING AND QUALITY MANAGEMENT The awareness of the commercial added value of flawless, fail-safe corporate applications is increasing. Alongside quality and security, the cost factor plays a central part at a time of limited budgets. Companies are actively looking for opportunities to improve both the software itself and the organization of testing at the same time as optimizing the cost aspect Boom in Optimization of Test Activities The majority of the companies surveyed are currently involved in optimizing their testing activities. At present around a third of companies find themselves in the optimization phase. A quarter of interviewees confirm that optimization will commence within the next twelve months, and a further eleven per cent within the next 24. Optimization of testing activities is on the radar of a further eight per cent, but they have not yet set the date for implementation; and 13 per cent are considering optimizing testing activities, but have made no concrete plans as yet. The remaining companies are not currently considering optimization. Readiness to Optimize Testing Activities Yes, optimization is already in progress Yes, optimization to start in the next 12 months Yes, optimization to start in the next 24 months Yes, optimization is planned but starting point is uncertain Yes, readiness exists but no plans yet No Share in percentage of companies, n = % 13% 8% 11% 25% 32% Fig. 2: Planning Status of the Companies Surveyed in respect of the Optimization of Testing Activities

8 8 Reasons for Optimizing Testing Activities The study investigated the motives that are prompting increasing numbers of companies to opt for the optimization of their testing activities. The results show that companies have identified software-testing and quality management as a valuable instrument in improving performance. At the forefront is desire to improve quality: in 61 per cent of companies the aim of optimizing testing activities is to enhance product quality; and 58 per cent hope to increase quality and transparency within the testing processes. Almost half of respondents expect that the level of testing automation will increase as a result of the improvement measures. For 35 per cent of companies, optimization was about increasing the objectivity of testing activities. What is interesting is that comparatively few respondents just around a quarter stated reducing costs as a reason for optimizing testing activities. It is clear that companies see quality as more significant. Reasons for Optimizing Testing Activities Increasing product quality... Increasing quality and transparency of testing processes... Increasing testing automation... Increasing objectivity of testing activities... Reducing costs of testing activities Share in percentage of companies that are currently preparing optimization, n = 279 Fig. 3: Reasons Given by the Companies Surveyed for Optimizing Testing Activities The interviewees explained that benefits such as greater objectivity in the company s testing activities can be best achieved in collaboration with a dedicated, independent testing service provider. An increase in the automation of testing can be realized via long-term managed test services engagements.

9 9 Reasons Hindering the Optimization of Testing Activities The majority of companies have the optimization of testing activities on their IT agenda. Only a small number of the companies surveyed have made no provision for adjustments in this area. They state that the organization is already managing its testing optimally (37 per cent), currently has different priorities (27 per cent), or is not ready for optimization for economic or organizational reasons (13 per cent) Increasing Acceptance of External Testing Support There is currently a lot of interest in collaboration with professional service providers for software-testing and quality management. The integration of a professional service provider, either on a by-case or more permanent basis, is a way of achieving high quality whilst also increasing efficiency and effectiveness in testing. Testing is therefore one of the worldʼs fastest-growing areas within IT services. According to PACʼs estimates, companies will spend a total of 30 million on testing services in That corresponds to a proportion of around five per cent of the total expenditure on IT services. PAC is of the view that the demand for professional testing support from service providers will further increase. The company survey shows consensus in the market: the vast majority of those surveyed agree that a suitable service provider can be used to optimize testing activities. Longer-term testing engagements in the form of managed testing services partnerships based on strictly-defined service level agreements are becoming increasingly important. According to PACʼs estimates, this form of selective outsourcing is experiencing double-digit annual growth and is thus one of the most dynamic segments within IT services. More than 90 per cent of respondents consider a longer-term, partnershipbased collaboration in testing to be indispensible and view the agreement of performance indicators, such as are standard in managed test services agreements, to be crucial for success. The high demand is a result of three main trends: the significance of qualityoriented software-testing and quality management has risen with increasing complexity in IT and testing. Long-term collaboration with service providers is also popular among companies that want to improve the organization and performance of their testing activities. Additionally, strategic considerations are gaining in significance. Long-term external support is particularly attractive for companies that

10 10 want to focus more intensively on their core competences and deploy their own IT resources for innovative projects. The survey conducted in the scope of the current study underlines the high acceptance of testing services: two thirds of the companies surveyed involve external service providers in their testing activities. The intensity of the collaboration varies and ranges from occasional collaboration on projects through to permanent cooperation in the scope of long-standing managed testing services agreements, even if only a small proportion mainly large companies outsource all testing activities to an external service provider. Where longer-term engagements are involved, the service provider, often also assumes responsibility for results and risks, for example in respect of how fail-safe the applications are alongside fixed tasks, the performance of which is ensured by service level agreements. The goals of a long-term collaboration usually also include the optimization and standardization of test processes, increasing the level of testing automation and the use of consistent and proven methods, best practices, and suitable testing tools. ;401/.E1+.7/*76*C,-+./0*( ,- ())*+,-+./0* ,-*14,*5,46748,9*./+,4/1)): ;221-.7/1)*<547=,2+ >1-,9?*./37)3,8,/+*76*,@+,4/1)*-,43.2,*5473.9,4- A,0B)14*<547=,2+ >1-,9?*./37)3,8,/+*76*,@+,4/1)*-,43.2,*5473.9,4- C,-+* ,-*5,46748,9*,@+,4/1)):*./*514+*<)7/0 +,48*104,,8,/+-? ())*+,-+./0* ,-*14,*5,46748,9*,@+,4/1)):*<)7/0 +,48*104,,8,/+-? ;3,41))*./ D7B-,*1/9*,@+,4/1)*+,-+./0*-5,2.1).-+-*B-,9 ID14,*./*5,42,/+10,*76*27851/.,-J*/*K*!$L!" %" #$" &'" #!"!!" F*G(H*#$&& Fig. 4: Organization of Testing Activities in the Companies Surveyed

11 11 Added Value through External Testing Services The study examined the benefits that companies associate with the testing-services model. For companies that already use external support, a clear advantage is improved access to qualified testing specialists. Companies that seek to reduce their internal IT expenditure also opt for external testing services. The clear majority of 85 per cent confirms that collaboration with external service providers for software-testing and quality management cuts costs and reduces the burden on internal IT teams, freeing them up for innovative projects. More than 80 per cent of respondents confirm that collaboration with an external testing-service provider reduces testing times and increases flexibility in the testing process. The clear majority of respondents also identified benefits in the increased standardization of testing processes. Benefits in Software Testing through Collaboration with External Specialists Large benefit Small benefit No benefit Improved access to... qualif. workforce Reduction of testing... costs Reduction of burden on internal... IT/testing specialists Reduction of testing... times Greater flexibility in... testing process Standardized testing approach Share in percentage of companies that deploy external testing specialists, n = 207 Fig. 5: Benefits in Software Testing through Collaboration with External Specialists Averaged out, the specified factors are considered to be both large and small benefits in roughly equal measure. Users of testing services stated that certain benefits are produced, in particular in the scope of longer-term managed testingservices relationships. In many cases objectives such as the standardization of testing processes are a firm feature of corresponding contracts.

12 Development of Demand for Dedicated Testing Services Most expenditure is incurred for testing activities that are part of more comprehensive IT service agreements. As well as testing, these often incorporate the development, operation, and maintenance of IT systems. In contrast to this are dedicated contracts for testing and quality management, which today accounts for more than a third of the budget for external testing services, a trend that is on the increase. PAC anticipates that the collaboration with specialized partners in the field of software testing will rise as customers increasingly demand independent tests of the application landscape. The survey provides clear evidence of this: 91 per cent of managers surveyed consider the independence of the testing team and the product development team to be an important or very important success factor. The dedicated award of contracts, particularly in the scope of longer-term managed test services agreements, is increasing in significance as the market matures. Companies that separate out software development and testing benefit in particular from a clear allocation of roles, improved transparency of services and test quality, and also the objectivity of testing performance that is guaranteed by the fact that the testing specialist is independent of the development team. As experts in their field, focused testers also bring special competences to the table. Their routine and their use at an early stage in the development process contribute to the efficiency and effectiveness of testing performance Development of Demand for Blended Service Delivery in the Scope of Testing Services For many companies today, global service provision is a factor that impacts quality, flexibility, and price of the testing engagement. The study shows that blended service delivery, i.e. the use of local resources in combination with near- and offshore resources in testing centers, has a high importance: the clear majority (76 per cent) of companies that have outsourced part or all of their testing activities to an external service provider currently perform software tests in collaboration with (near-/ offshore) testing centers.

13 13 Optimum Resource Distribution in Blended Service Delivery There is no optimum composition of resource allocation that applies to all companies. It is dependent both on individual objectives that organizations pursue by using blended service delivery and the maturity of the company in terms of collaboration and control of globally-allocated resources. In order to categorize the use of near- and offshore resources into companiesʼ current IT strategy nonetheless, the interviewees were asked about the ideal distribution of testing activities. The survey concluded that the number of companies that desire a high proportion of near- and offshore activities is currently relatively low: only 14 per cent of those surveyed can imagine more than half of the testing activities being performed nearand offshore. What is also interesting here is that no interviewee wants to outsource all testing tasks to a testing center. At the same time almost all companies consider it important to continue to perform a certain proportion of testing activities internally.

14 14 Future Usage of External (Near-/ Offshore) Testing Centers The study underlines the fact that testing centers are continuing to increase in significance for performing testing services. Asked about future planning, three quarters of all interviewees confirm that they are considering the usage of external (near-/ offshore) testing centers. While for the majority this is due in the short- or mid-term, the rest of the companies have no precise schedule. However, the further development of demand can be clearly plotted: interest in the use of blended service delivery is constantly D,E..*58C)F,G)*1/0A,H)01)C ()*+,-)./0/1)23,4/15/0,15),0)61,$",78015*, ()*+,-)./0/1)23,4/15/0,15),0)61,"',78015*, ()*+,9:1,1/7),;8/01,/*,:0<0840, ()*,9:1,08,-)./0/1),;2=0*, >8, L5=C),/0,;)CM)01=A),8.,M87;=0/)*+,0,N,!K% $$# "'# $&# $%#!"# Fig. 6: Planning Status of the Surveyed Companies in Collaboration with Testing Centers

15 15 Drivers for the Use of Testing Centers For the companies surveyed, the use of external (near-/ offshore) testing centers, and thus of blended service delivery, serves a range of purposes. For more than half reducing costs (61 per cent) and increasing the quality of service provision (54 per cent) plays a more important role in their considerations. 48 per cent stated the increasing availability and flexibility that can be achieved e.g. through global distribution and across different time zones as a further motive. Around a quarter of those surveyed consider access to additional competences to be an important argument for the integration of near- and offshore resources into their testing activities. '(610-1/;05/O1,-. 6/=P(65 IQ;;1?05(A/R(12,-./N(-2(5 '()*+,-./+0121/ (61,-./2?(/B*68,2:/8(7(8/,-/1(57,+(/)(8,7(5:/ (61,-./676,869,8,2:/6-)/ ;8(<,9,8,2:/=>650*-)/2?(/+80+@>A/ C++(11/20/6)),2,0-68/ +0DE(2(-+,(1/ &% $% #$!" L/MCN/&F"" F "F &F GF $F #F!F H?65(/,-/E(5+(-26.(/0;/+0DE6-,(1/2?62/65( E86--,-./20/*1(/6/=-(65 I0;;1?05(A/2(12,-./+(-2(5J/-/K/&G! Fig. 7: Reasons Given by Companies Surveyed for the Use of Testing Centers Reasons for not Using Testing Centers Companies that do not currently use an external (near-/ offshore) testing center gave the reasons for not doing so: a good third of companies will prefer to continue to perform testing services locally in the future. A quarter of companies currently have other priorities. 14 per cent of companies operate their own testing center. Further companies stated that they have not yet found a suitable partner for implementation or already collaborate with a fixed partner.

16 16 4. SUCCESS FACTORS IN SOFTWARE TESTING Testing activities are closely linked to the challenges that companies currently face: flexible reaction to changes on the market and customer side, high speed of introduction of new products through optimized time-to-market, and greater efficiency in delivering services. The results of the study show that for most companies today it is not a case of whether to optimize their testing activities in collaboration with an external service provider, but rather the challenge of identifying the optimum approach for doing so and selecting the correct partner.

17 Factors for Successful Collaboration with Testing Service Providers Availability and Flexibility of Testing Teams Speed and flexibility are at the forefront for companies in terms of their testing activities. Demand is particularly high as a result of the intensive competition landscape. The majority of companies surveyed 60 per cent expect high to very high availability and flexibility from a testing team. Maximum availability is permanent, around the clock readiness (24/7). The use of globally-distributed resources helps guarantee maximum availability and flexibility. Availability and Flexibility Expected of Testing Team Maximum (24/7) High (10/7) Medium (10/5) Low (8/5) Don t know/no answer Share in percentage of companies that deploy external testing specialists, n = % 6% 4% 18% 42% Fig. 8: Expected Flexibility and Availability of Testing Team Close harmonization and seamless communication are equally significant. Almost all interviewees demand direct and immediate feedback to queries from the testing teams.

18 18 Evaluation of the Achievement of Objectives by Measuring Success The measurement of success plays a key role in the scope of testing services. Two thirds of the companies surveyed regularly measure whether the testing team is achieving its agreed objectives by using selected key performance indicators (KPIs). Of these, 76 per cent monitor whether deadlines are met. 72 per cent evaluate the number of test cases performed and test coverage. A further 66 per cent actively measure errors in production. Factors Used to Measure Testing Team s Goal Attainment Yes No Don t know Measurement of adherence to deadlines... Cost and number of test cases performed... Active measurement of test coverage... Active measurement of errors in production Detailed KPIs Share in percentage of companies that measure the testing team s goal attainment, n = 208 Fig. 9: Factors for Measuring the Achievement of Objectives of the Testing Team However, Figure 9 shows that around a quarter of companies collect a detailed set of KPIs. This result of the study shows that there is further optimization potential in managing testing quality. Collating a selection of pre-defined performance indicators is usual for managed test services. They provide for transparency and increase the transparency of the testing activities performed.

19 19 Results-oriented Remuneration of Testing Activities Companies that already use external support in testing have strict requirements in terms of professional performance of services and fulfillment of objectives. 97 per cent state technical competence of the testing team to be an important or very important criterion. 89 per cent of managers surveyed consider flexible billing of testing services based on results actually achieved to be important or very important. Only a few interviewees stated that this aspect is less important. <)=)>23?),0@,A-==-3B,0@,C);1-3B,D)*>-?); 03,<);E=1; 0*-)31)F,A2;-; ()*+,-./0* ,-./0* /0* ,8309:30,23;9)* DM2*),-3,/)*?)312B),0@,?0./23-);,1M21,F)/=0+ )N1)*32=,1);1-3B,,;/)?-2=-;1;O,3,P,K"& $%# &# '#!"# G,HIJ,K"LL Fig. 10: Significance of Results-oriented Remuneration of Testing Services

20 20 Significance of Linguistic Capability in Testing Projects Language provides an additional facet to communication. Multi-lingual capability is indispensable, above all in globally-operating companies and for international testing activities. 39 per cent of companies surveyed consider this aspect to be very important, and for a third it is important. The principal requirement is for contacts who can speak the language of the respective country and English. 7(8(912:(+/;+<680, 8,2=618+>1.1?,8,0*,2+@(A0,2=+B)/C(:0A '()*+,-./) /)0120 4/5+62,-./)0120 HI1)(+,2+.():(201=(+/;+:/-.12,(A5+2+J+!F" %&#!$#!"# D+BE>+%FGG Fig. 11: Significance of Linguistic Capability in Testing Projects

21 Important Factors in the Selection of a Testing Service Provider Successful collaboration with a provider of testing services confronts companies with the challenge of finding a suitable partner. This demanding task is dependent above all on an evident core competence where the deciding factor is that the service provider concentrates on testing. Equally important is for the service provider to have an evident quality orientation, which just under two thirds of the companies surveyed for the study consider to be highly important, as the figure below illustrates. Companies that already work with an external testing service provider were considered. Further necessary competences for the majority of respondents include industry and business-process knowledge. Around half of companies expect a provider of software-testing and quality management to have detailed knowledge of the underlying technologies and the products to be tested. Methodological competence is key for 44 per cent of respondents in choosing a service provider. Priority of Criteria Selecting a Service Provider for Testing Activities Highest Priority Medium Priority Low Priority Core competence/ focus on testing Quality orientation Industry/ process knowledge... Technological/ product knowledge (e.g. SAP) Methodological competence Share in percentage of companies that deploy external testing specialists, n = 207 Fig. 12: Prioritized Competences of the Testing Service Provider

22 22 5. SQS: SUCCESSFUL IN THE MARKET AS AN INDEPENDENT TESTING SPECIALIST The previous chapter looked at the trends and success factors in relation to testing services. The results of the study show that the majority of companies are working on optimizing their test activities and are aware of the added value that professional external support offers. Dedicated testing services providers are being commissioned increasingly frequently on the basis of managed test services agreements to achieve maximum leverage of the benefits. Chapter five presents the competences, service approach, and differentiation features that characterize SQS in the view of the PAC market analysts Overview of SQS The IT software service provider SQS specializes in software-testing and quality management. SQS currently generates revenues of over 160 million. The company has its head office in Germany and operates globally. True to the motto wherever our customers are, SQS is represented at locations in ten countries across Europe and in the USA, Egypt, India, and South Africa. SQS positions itself in the growing software-testing market as a specialized and independent provider of testing with a particular focus on managed test services. The company has almost 30 yearsʼ experience in the testing context. An important differentiating feature over the competition is its exclusive focus on services in this area. With a resource strength of currently 2,000 employees, SQS is one of the leading specialists in testing services in Europe and globally.

23 Competency Profile of SQS The results of this study show that competence and reliability of the testing team are the ultimate in seamless service provision. Further, the objectivity of the testing team and its independence from the product development team are highly regarded. In PACʼs view, SQS meets these requirements. The scope of the company s services includes comprehensive software-testing and quality management services that cover all requirements of the customer in this area. SQS also offers its own software tools for quality assurance in software development projects. Considering the services along the value-creation chain, SQS offers end-to-end services, beginning with testing consultation and the performance of testing audits and assessments of testing specifications, performance, automation, and training up to software quality assurance and controlling the testing activities. The tool box known as SQS-TEST / Professional Suite consists of proven modules for testing management, test design and performance, and the automation of testing processes. The solution suite also comprises a platform for integrating various applications in software testing. From the point of view of PAC, SQS is marked out due to its in-depth competence, methodology, and custom tools in software-testing and quality management as well as its high level of professionalism. Customers choose SQS as a partner for demanding tasks, e.g. testing of sector-specific and business critical applications as well as strategic issues and the re-conception of the overall organization of testing. Its technological knowledge and the evident industry and process knowledge enable the company to cater for both trends in IT and the commercial challenges of its customers. Finally, its innovative orientation and the development and deployment of state-of-the-art software tools and methods, enable SQS to contribute to driving forward the development of the market for testing services.

24 24 A special feature of the solution approach of SQS is that the company has grouped its custom methodologies, which are offered to the customer as standardized services or as tailored solutions under the keyword of SQS PractiQ. The solutions packages cater for e.g. performance testing or test automation as managed services or a health check, i.e. the analysis and checking of testing processes and procedures of companies with the aim of improving them. The standardization of the SQS solutions approach allows the customer to procure pre-defined, frequently recurring services more cost effectively. SQS, as a specialized provider of software-testing and quality management, fulfills the requirement for independence. The clear focus on testing services and the associated independence and objectivity in performing services are features that differentiate SQS from the competition. This has enabled SQS to position itself with a clear profile in the IT services market Service Pledges of SQS Although SQS performs standardized, performance-oriented testing services to ensure efficiency and quality, elements of services can be put together for individual customers and delivered on the basis of the preferred model case-bycase. Depending on the customerʼs situation, SQS will performs testing activities tailored to the customerʼs needs, partial or complete testing projects, or all testing activities under managed test services agreements. As this study shows, SQS has positioned itself appropriately in relation to the main trends in the growth market of software testing. The testing services providerʼs response to the sustained demand for managed test services is continual strengthening of its resources and abilities in this area. Successful reference projects underline the attractiveness of the concept: as early as 2009 the St. Galler Kantonalbank (Switzerland) entered into a managed test services agreement with SQS over several years under which SQS was responsible for testing the core bank system. The outcome to date has been positive: after just a short time into the contract, the most important objectives reached in collaboration with SQS include automation of testing and increase in efficiency.

25 25 Testing Automation at St. Galler Kantonalbank The St. Galler Kantonalbank is a bank for private and business clients that operates globally. The institute deploys the Avaloq core bank application to support its central business processes. Regular software updates and error correction mean that reliable and efficient software tests are indispensable for the Kantonalbank. In order to achieve a high degree of automation of the Avaloq test cases and establish optimum processes, the bank entered into a five-year contract for managed test services with SQS in In the first year of the contract SQS implemented a large number of automated test cases that today are fully performed and evaluated within a few days. The significant reduction of manual tests means that the potential for savings is high. The testing activities have been seamlessly integrated into the existing IT set-up such that implementation occurs without friction losses. The test cases are conducted at the customerʼs premises while the test cases implemented are prepared and maintained in the SQS testing center in Cairo. The current managed test services agreement means that the Kantonalbank St. Gallen will rely on SQSʼs support until at least The fixed-price model in combination with defined performance indicators guarantees that the bankʼs objectives in terms of efficiency and quality will be achieved. References assist in evaluating the competences and experience of a service provider. In PAC s view satisfied customers are an important indication that a company fulfills its service pledges Blended Service Delivery SQS achieves cost and quality benefits in software testing for its customers by means of blended service delivery. The company deploys a mix of resources from the customerʼs local unit in conjunction with its global delivery units. While SQS manages all testing jobs locally, the services are performed in various countries via a best-shore approach. This saves costs as the tests are performed around the clock and thus faster across different time zones. Blended service delivery also enables SQS to meet demand peaks if a customer requires many resources and specific competences at the same time.

26 26 Contacts from the local unit are available to SQS customers at all times. The customers benefit from this, in particular in phases of service performance requiring intensive communication. The global delivery locations of SQS are distributed around the world and are located in Görlitz (Germany), Cairo (Egypt), Durban (South Africa), Belfast (Northern Ireland), and Pune (India). Today around a third of SQS employees are deployed at these locations. Within the international services organization SQS scores points with well-educated, multilingual teams, low fluctuation at all locations, including near- and offshore locations, and with a high performance orientation as well as intensive communication between the teams within the delivery organization as a whole. SQS is thus in a position to fulfill further needs of its customers: maintenance of high quality standards, the acceleration of test performance, and access to supplementary resources. The blended service delivery approach is widespread in todayʼs IT services environment. The large systems integrators in particular have near- and offshore units in numbers that considerably exceed those of SQS in terms of resource strength and number of locations. Conversely, the global delivery unit of SQS is comparable to that of the global IT generalists in terms of the number of dedicated testing specialists. However, not every company wants the integration of near- and offshore resources for the delivery of testing services. This illustrates the issue of the ideal resource mix that was explored in section 3.4. If required SQS realizes projects or project phases on a wholly local basis Transparent Attainment of Objectives and Performance-related Billing SQS is particularly concerned with transparency in respect of delivering services and attaining objectives for its customers. To ensure that the benefits of collaboration can be verified transparently, service delivery and objectives are set down with measurable indicators and time axes.

27 27 Active budget control and measurement of success criteria enables the customer to establish precisely not only the efficiency of the testing services delivered, but also their quality and effectiveness. This can be measured e.g. based on the performance and failure-safety of the tested application. What is significant for SQS is not so much the control of the individual test specialists: what is much more important is the management of the overall testing team and the transparency of the testing activities in terms of quality and efficiency. Active and consistent measurements ensure early intervention and remedy in the event that the objectives cannot be fulfilled at any point. The approach in the delivery of the testing services and the tools used to do so are selected by SQS teams in consultation with the customer and in accordance with its requirements. In this way a verifiable result can be achieved in the conflicting priorities between different objectives such as increasing quality and efficiency. SQS bills its services on the basis of various pricing models. The desire for fair, results-oriented billing is clearly increasing and is met by SQS.

28 28 6. SUMMARY The survey, conducted with more than 300 decision-makers from large companies in Europe and the USA, showed that most companies today use external support for their testing activities and benefit from doing so. Optimization of Testing Activities The organization of testing activities and their optimization is closely associated with the increasing awareness of the added value of flawless, fail-safe corporate applications. The majority of the companies surveyed are currently involved in optimizing their testing activities. Currently around half of companies surveyed are in the optimization phase, while 36 per cent are planning to start in the coming twelve to 24 months. Optimization enables the managers surveyed to aim at increasing the quality and transparency within testing processes. Companies that have made no provision for adapting their testing processes mostly justified this by saying that their testing activities were already optimized. Trends in the Market for Testing Services Software-testing and quality assurance is one of the most important IT topics for companies, as 91 per cent of the managers in the survey conducted for the study confirmed. At the same time, there is currently a lot of interest in collaboration with professional service providers for software-testing and quality management: two thirds of companies surveyed already integrate external service providers into their testing activities, in the majority of cases regularly or on the basis of long-term managed test services agreements. Companies associate different opportunities with the testing services model. More than 80 per cent of the companies surveyed see the greatest benefits in improved access to qualified test specialists, shorter testing times, the reduction of testing costs, relieving the burden on IT teams to free them up for innovative projects, and in greater flexibility in the testing process.

29 29 Development in Demand for a Blended Service Delivery Today global service delivery is hugely significant. The vast majority of companies that have already outsourced part or all of their testing activities to an external service provider use resources distributed across the globe for software tests, i.e. they work with a mix of local and near- and offshore resources. The study shows that the importance of external (near-/ offshore) testing centers for delivering testing services continues to increase. Three quarters of interviewees confirm that they are considering using testing centers in future. The companies surveyed use testing centers to pursue a variety of objectives. For the majority of respondents the most important reasons are reducing costs and increasing quality of service provision. What is noticeable is that none of the IT decisions makers surveyed was in favor of outsourcing all testing activities to a testing center. Successful Collaboration with Testing Service Providers Customers demand high competence in service delivery from their testing service provider. 97 per cent of interviewees stated the technical competence of and instant feedback from the testing team as an important or very important criterion. The majority of companies surveyed 60 per cent further expect high to very high availability and flexibility from a testing team. Maximum availability means permanent readiness around the clock. The majority of respondents also classify multilingual capability as important or very important in testing projects. The principal requirement is for contacts who can speak the language of the respective country and English. Measuring success on the basis of KPIs for testing services is important for the majority of interviewees: around two thirds of companies regularly measure whether the testing team is achieving the agreed objectives. What is crucial is the adherence to deadlines and the number of test cases performed as compared to the expenditure incurred. Only a small number of companies surveyed 26 per cent measure a set of dedicated KPIs. A clear majority of respondents 89 per cent consider the flexible billing of testing services based on results actually achieved to be important. Success-critical Factors in the Selection of a Testing-Service Provider In summary it becomes clear that the demand for quality-oriented services is increasing in the growth software testing-market, while performance is subjected to closer scrutiny than ever before. Companies demand a lot of their software-testing partners. The main issue in selecting a testing service provider is its focus on

30 30 testing and quality orientation, which just under two thirds of companies surveyed for the study consider to be particularly significant. The majority of managers surveyed have high requirements in terms of industry and business-process knowledge and demand technological and product expertise. Testing Specialist SQS SQS positions itself clearly in the market as a specialized provider of softwaretesting and quality management within a particular focus on long-term, successoriented customer relationships is correspondingly well-placed in respect of the trends in the software-testing market. These include above all the increasing demand for long-term contracts for managed test services and the use of a balanced blended service delivery. The company meets the high demands of the target group by pursuing a quality- and performance-oriented service approach combined with outcome-oriented remuneration and has the necessary competence and experience. The company provides evidence of this via credible references and its active measurement of the service delivery and attainment of objectives on the basis of KPIs. These are criteria that are essential in the selection of a testing service provider, as the company survey has shown. Further factors come into the mix for companies that seek a suitable partner for collaboration in software testing and quality management, depending on the specific circumstances. The requirements are manifold. This study cannot replace a full analysis of either the specific requirements or the services of the testing service providers. However, PAC as the author of the study and SQS as its commissioner, hope that they have provided the reader with useful background information on the growing testing market and that it provides orientation in planning professional managed test services.

31 31 About SQS Software Quality Systems AG SQS is the largest independent provider of services for software quality management, quality assurance and testing, with its focus in Europe. Founded in Cologne in 1982, SQS employs 2,000 employees. Alongside a strong presence in Germany and the UK, SQS has subsidiaries in Egypt, Finland, India, Ireland, the Netherlands, Norway, Austria, Sweden, Switzerland, South Africa, and the USA. SQS also has a minority stake in a Portuguese company and a joint venture with a company in Spain. In 2010 SQS generates sales of million. SQS was the first German company to undertake a primary listing on AIM (Alternative Investment Market) in London. Further, SQS has a secondary listing on the Open Market of the Deutsche Börse in Frankfurt/Main. With over 5,000 concluded projects, SQS has a strong customer base, including half of the DAX 30 companies, almost a third of STOXX 50 companies and 20 FTSE 100 companies. These include Allianz, Beazley, BP, Centrica, Daimler, Deutsche Post, Generali, JP Morgan, Meteor, Reuters, and Volkswagen and other companies across all sectors. For further information: About Pierre Audoin Consultants GmbH PAC provides focused and objective answers to the growth demands of actors in the information and communication technology market (ITC) from strategy through to implementation. Pierre Audoin Consultants was set up in 1976 and is an independent marketanalysis and consultancy company for the software and ITC services market. We support ITC service providers with quantitative and qualitative market analyses and strategic and operative consultation. We advise CIOs and financial investors on the evaluation of ITC service providers and solutions and provide support for their investment decisions. Public and voluntary organizations use our analyses and recommendations as a basis of their ITC policies. For further information: and

32 Contact Editor: Pierre Audoin Consultants (PAC) GmbH Holzstraße 26 D München Germany Telephone: +49 (0) Fax: +49 (0) Publisher: SQS Software Quality Systems AG Stollwerckstraße Köln Germany Telephone: +49 (0) Fax: +49 (0)