The Value of QA from the CIO s Perspective

Transcription

1 The Value of QA from the CIO s Perspective Joe Desuta SCQAA-OC May 17, 2011 jdesuta@hotmail.com

2 Overview QA defined for purposes of discussion Traditional challenges for QA Value of QA based on a Company s Size and Type Review of Industry Standards of Tester per Developer Ratios Trends in QA What QA Managers and Staff do to add value QA vendors and Tools

3 Quality Assurance Defined Quality assurance, or QA for short, is the systematic monitoring and evaluation of the various aspects of a project, service or facility to maximize the probability that minimum standards of quality are being attained by the production process. QA cannot absolutely guarantee the production of quality products. -Wikipedia

4 Question for the Group Which one of our SCQAA members wrote this definition for Wikipedia? (ice-breaker attempt at QA humor insert laugh tracks) Serious Note I will discuss later how this managerial courage from QA is a value add for CIOs

5 QA as an IT function Generally thought of as Software Quality Assurance (SQA) Commonly understood as part of SDLC Considered a requirement for Level 2 (Repeatable) of Capability Maturity Model (CMM) Includes any technology related testing falls under the IT QA function Examples include: Stress testing of corporate website; Smoke testing of the enterprise after a change to the environment Traditional QC, History, Deming principles, etc.. Out of Scope for the presentation and discussion

6 Traditional challenges for QA Hard to Quantify the ROI What is the value of found bugs? How much did we save by not have X defect go to production? Nearly impossible to quantify hypothetical What If scenarios Testing takes TIME and delays releases Testing is the first thing that gets short changed when invariable development delays occur Software testing as a Function and as a Career Path isn t as visible as other parts of the SDLC There are few specific university degrees focused on it Poorly Defined Career Path (except in the largest, mature shops)

7 41% of respondents see SQA as a nice-to-have or are not interested in it at all Source: Original Software Survey

8 Value of QA based on a Company s Size and Type How QA is viewed varies significantly on the size and the type of company Maturity of an organization s IT function has even more to determine the value that is derived from QA GMAC (Ditech) and The Irvine Company Paragon, All-Safe Pool Safety, Northwood

9 Industry Standards of Tester per Developer Ratios Development Type Average Best in Class Lagging ISV 1:3 1:1 1:5 Web/E-commerce 1:3 1:1 1:5 Enterprise Custom Development 1:5 1:3 1:10 Departmental Custom Development 1:10 1:3 1:20 Package Implementation 1:10 1:3 1:15 Source: Gartner (August 2010)

10 Tester per Developer Ratios Analysis Spending on testing and quality is driven by the perceived risk involved Software companies know the value of software it is their business so they invest more in testing resources New technologies, government regulations and competitive advantage factor in the level of risk/investment Does this matrix and ratios hold true for your organizations?

11 What does this mean for you? Seek out organizations that are have lower ratios from this chart: example - mature software vendors, large shops OR Recognize the opportunity from the laggards to bring in your expertise and to be flexible, expanding your skill set in the process

12 Trends in QA and a Value Driven Approach QA is getting involved early in the development process QA can validate if requirements can be testable More relevant concept with the proliferation of Agile vis-a-vie Evolutionary Prototyping

13 Trends in QA and a Value Driven Approach QA is getting involved early in the development process QA can wear many hats and be able function in an analyst role or even demonstrate subject matter expertise Develop communication skills Be able to work with the customers Demonstrate knowledge of the business to gain credibility with stakeholders

14 Trends in QA and a Value Driven Approach Regarding Credibility QA needs to be regarded as a pillar of objectivity QA adds value by being an independent voice QA Managers/Leaders need to present objective testing results but also offer their interpretation to the state of a project Leadership requires management courage CIOs rely on you to give it to them straight

15 Trends in QA and a Value Driven Approach Regarding Credibility examples Very evident in highly regulated organizations Government Contractors Medical/Bio/Pharma Edwards Life Sciences has a small corporate QA department 5 and large 40+ QA department that is mandated to R&D projects

16 QA Team of the future CIOs need to build a balanced team The future QA team is more than just testers looking for bugs The future team is multidisciplined team that is closely integrated with the project, development and operations function of the IT group

17 QA Team of the future Career Path Example Junior Tester Senior Tester Software Test Engineer Software Performance Analyst Test Architect Software Security Analyst

18 Trends in QA Tools/Vendors New script-free and model-driven testing environments Tools for testing "in the cloud" Tools are more supportive of Agile leading tool support integration between software requirements and test plans

19 New Opportunity for QA CIOs are open to new tools that address the traditional challenges New tools offer a new opportunity for QA to educate IT Leadership and the business community Promotes the trends of Agile so QA can be more into the process

20 Magic Quadrant for Integrated Software Quality Suites

21 Questions? Thank you for your attention and the opportunity to present to you this evening References Gartner Research - Thomas E. Murphy