A Poit of View Implemetig TMMi for Agile Projects A Ratig Criteria ad Mappig Framework Abstract Over the past few years, both Agile ad Test Maturity Model Itegratio (TMMi) have see growig iterest from the software idustry. Software compaies are takig to Agile, sice it eables iterative workable software deploymet, ad accelerates time to market. Eve as Agile makes its way ito the IT ladscape, aother disruptive chage is brewig i the assurace space - TMMi. With emphasis o improved product quality through effective ad efficiet testig processes, TMMi is drivig sigificat improvemets. Ca TMMi-defied processes match up to Agile's flexible ad iterative approach? Are these two models compatible? Ca TMMi ad Agile co-exist ad complemet each other? This paper attempts to bust myths by mappig Agile methods to TMMi processes. It explais, with practical examples, how Agile projects ca adopt the TMMi framework. Agile ad TMMi: Two Sides of the Same Coi? With its simplified ad flexible developmet processes, Agile helps icrease visibility, miimize risks, ad accelerate time to market. I its exteded avatar DevOps, it also improves collaboratio betwee busiess ad IT. This improved collaboratio, i tur, improves workforce productivity. With these bottomlie beefits, Agile ad DevOps have established a strog foothold i busiess IT ladscapes. LEVEL 5: OPTIMIZATION Alog with Agile ad DevOps, software Defect Prevetio developmet teams have also adopted TMMi to Test Process Optimizatio drive early defect detectio, improve product Quality Cotrol quality ad performace, ad reduce the cost of LEVEL 4: MEASURED IT ad product owership. Over the years, the umber of TMMi certified busiesses has icreased expoetially. These busiesses are also adoptig Agile methodologies to accelerate developmet cycles ad time to market. LEVEL 3: DEFINED TMMi Overview 1 Developed by the TMMi Foudatio as a guidelie ad referece framework for test process improvemet, TMMi, with five maturity levels, complemets the Capability Maturity Model Itegratio (CMMI) Versio 1.2 (ow updated to CMMI V1.3). It addresses importat issues faced by test maagers, test egieers, ad software quality professioals. Testig, as defied i TMMi, is applied i its broadest sese - to ecompass all software product-quality related activities. Test Measuremet Product Quality Evaluatio Advaced Reviews Test Orgaizatio Test Traiig Program Test Lifecycle ad Itegratio No-fuctioal Testig Peer Reviews LEVEL 2: MANAGED Test Policy ad Strategy Test Plaig Test Moitorig ad Cotrol Test Desig ad Executio Test Eviromet LEVEL 1: INITIAL Chaotic process Depedet o heroes No uderstadig of the cost of quality Figure 1: TMMi Levels [1] TMMi Framework (Release 1.0), accessed Sep 01, 2015, http://www.tmmi.org/pdf/tmmi.framework.pdf
Agile Overview The Agile methodology comprises software developmet methods i which requiremets ad solutios evolve through collaboratio betwee self-orgaizig, cross-fuctioal teams. The methodology promotes adaptive plaig, evolutioary developmet, early delivery, ad cotiuous improvemet. It also ecourages rapid ad flexible respose to chages. Throughout the costructio phase, Agile teams sped sigificat effort i performig cofirmatory testig - a combiatio of Static testig at the desig level, ad Agile Acceptace Testig at the requiremets level. STLC Product Requiremet Aalysis Product Test Strategy/Plaig Requiremet & Aalysis Test Strategy Plaig/Estimatio Desig Test Desig Codig/ Build Test Automatio Scripts Testig Test Executio & Completio User Acceptace Test UAT Test Support Beta Testig Beta Test Support Pre-Productio Deploymet shakedow Testig Pre-Productio Deploymet shakedow Test Support Outside Sprit Sprit 1 SDLC Figure 2: Testig i Agile Applyig TMMi to Agile: The Ratig Criteria ad Framework The applicability of TMMi to Agile ca be measured usig ratig criteria (see Table 1), which provide a framework for mappig Agile processes to TMMi. The ratig represets a percetage score based o the applicability ad fitmet of a TMMi level process to Agile. Ratig N Not Applicable P Partially Applicable F Fully Applicable Criteria Little or o evidece of compliace. The process fids margial (0 15%) applicability i Agile Some evidece of compliace, but the process may be icomplete, or icosistetly applied (15 ) Cosistet (80 ) evidece of compliace Table 1: Ratig criteria for TMMi level-wise mappig with Agile 2
TMMi level-wise mappig with Agile TMMi Level 1: Iitial With ad-hoc testig, chaotic processes, little or o process orietatio ad o uderstadig of Cost of Quality that characterize TMMi level 1, this level is ot ameable to mappig with Agile. The maturity mappig therefore starts from Level 2, with defied goals ad practices. TMMi Level 2: Maaged Level 2 is about test policy ad strategy, test plaig, test moitorig ad cotrol, test desig ad executio, ad test eviromet. Test Policy ad Strategy ivolves the defiitio ad deploymet of the test policy, test strategy, ad test levels. To measure test performace ad the accomplishmet of test (improvemet) objectives, test performace idicators are defied ad implemeted. While Agile does ot have a separate test policy or test goals, it follows a defied test strategy ad measures effectiveess through key performace idicators (KPIs) such as the umber of defects, defect removal time, cofidece delivery, story cycle time, product ad release bur dow charts. As such, test policy ad strategy are partially applicable (P) to Agile. Buildig o the give test strategy, test plaig helps defie the best approach based o risks idetified by the product ower. It ivolves establishig ad maitaiig well-fouded plas for performig ad maagig testig activities. Etry ad exit criteria for each activity help verify activity completio ad istitute adequate process rigor. Test suspesio ad resumptio are ot applicable i Agile, but testig ca be moved to aother iteratio uder certai circumstaces such as uavailability of the test eviromet. Methods such as plaig poker facilitate Agile estimatios. Give this sceario, the test plaig process is fully applicable (F) to Agile. Test Moitorig ad Cotrol provides a uderstadig of test progress ad product quality, facilitatig appropriate corrective actios i case of sigificat deviatios from the plaed schedule or quality levels¹. Agile moitorig ad cotrol measures such as frequet deliveries, daily team meetigs, ad time-boxed review meetigs help eable test moitorig ad cotrol, both i terms of test progress ad product quality. Hece, this process area is also fully applicable (F) to Agile. Test Desig ad Executio establishes test desig specificatios usig test desig techiques. This helps perform structured test executio. The structured approach helps maage test icidets to closure, sigificatly improvig the test process capabilities. I Agile, test cases are prioritized as specified i the geerate test case procedure documet. Agile tools eable traceability. Itake test, test executio, writig logs, defect detectio, defect prioritizatio, defect retest, ad closig are all practiced i Agile as well, makig the test desig ad executio process area fully ameable ad applicable (F) to Agile. Test Eviromet ivolves provisioig adequate ifrastructure, icludig test data, to execute tests i a maageable ad repeatable way. I Agile, release maagers ow test eviromet maagemet. The testig team performs eviromet itake tests by checkig the release otice o the versio of the build, ad the performig smoke testig. The teams also ow test data creatio ad maagemet. Give this practical applicatio sceario, this process area is also fully applicable (F) to Agile. LEVEL 2 9 7 5 3 2 1 Test Policy ad Strategy Test Plaig Test Moitorig ad Cotrol Test Desig ad Executio Figure 3: TMMi Level 2 3 Test Eviromet
TMMi Level 3: Defied Level 3 is about test orgaizatio, test traiig, test life cycle itegratio, o-fuctioal testig ad peer review. Let's examie each i detail. Test Orgaizatio helps idetify ad orgaize a group of highly skilled people who are resposible for testig. I additio to testig, the test group, based o the stregths ad weakesses of the orgaizatio's curret test processes ad assets, drives ad maages test process improvemets¹. While TMMi provides for a idividual test orgaizatio with defied test fuctios ad testig career paths, there is o separate tester role i Agile. I Agile, testig is part of the team member or developer roles. A separate testig orgaizatio is provided for oly i some hybrid methodologies. As such, this process area is ot applicable (N) to Agile. The Test Traiig Program helps develop testig courseware, which facilitates the developmet of kowledge ad required testig competecies to eable people to perform their tasks effectively ad efficietly. Test traiig is applicable to agile projects, ot as a idividual test orgaizatio traiig, but at the project level, i lie with project eeds. This sceario makes the process area fully applicable (F) to Agile. The Test Life Cycle ad Itegratio establishes work eviromet stadards, ad itegrates the testig ad developmet lifecycle, brigig them i syc with each other. The itegratio results i the early ivolvemet of testig through a defied coheret test approach that spas multiple test levels. Based o the idetified risks ad a defied test strategy, the approach facilitates the defiitio of the overall test lifecycle. Test life cycle itegratio is achieved i Agile through practices such as Cotiuous Itegratio Cotiuous Developmet (CICD) ad Test Drive Developmet (TDD) that eable the early itegratio of testig with the developmet lifecycle. However, i Agile, test process assets are maitaied at the project level ad ot at the orgaizatio level. This is differet from TMMi which recommeds maitaiig orgaizatioal test process assets. As such, this process area is partially applicable (P) to Agile. By icludig No-fuctioal Testig i test plaig, test desig, ad executio, test process capabilities ca be improved. No-fuctioal testig is based o a defied test approach, which idetifies o-fuctioal product risks, establishes o-fuctioal test specificatios, ad facilitates the executio of structured test methods focused o o-fuctioal testig. No-fuctioal testig is also applicable to Agile, through No Fuctioal Requiremets (NFRs) give as user stories. Agile implemets these o-fuctioal tests all at oce, or i a icremetal, story by story, path. Therefore, this process area is fully applicable (F) to Agile. Peer Reviews facilitate early defect detectio ad removal, thus esurig work products ad deliverables meet their specified requiremets¹. The peer review process ca also be well implemeted i Agile XP or pair programmig, where there is costat review ad feedback. Agile teams comprise subject matter experts (SMEs), developers, ad techical ad busiess specialists who are ivolved i costat review ad feedback. As such, this process is fully applicable (F) to Agile. LEVEL 3 9 7 5 3 2 1 Test Orgaizatio Test Traiig Program Test Life No fuctioal Cycle Itegratio Testig 4 Peer Reviews
TMMi Level 4: Measured Level 4 is about test measuremet, product quality evaluatio, ad advaced reviews. Test Measuremet idetifies, collects, ad aalyzes KPIs that eable objective evaluatio of the effectiveess ad efficiecy of the test process, the productivity of testig staff, the resultig product quality, ad the results of test improvemet. These measures ad aalyses, i tur, eable the test orgaizatio to develop ad sustai test measuremet capability, ad support maagemet iformatio eeds. Reports like burt dow charts ad velocity are prepared i Agile projects to measure productivity ad track the schedule. Agile also ivolves the collectio ad aalysis of testig metrics such as test effectiveess ad test coverage. Test measuremet reports are created through Agile tools such as JIRA, HP ALM, ad HP Agile maager. I view of this, the Test Measuremet process area is fully applicable (F) to Agile. Product Quality Evaluatio covers project level practices for developig a quatitative uderstadig of the product ad achievig defied ad measurable product quality goals. Both fuctioal ad o-fuctioal quality attributes are cosidered while defiig goals ad practices for this process area¹. Product Quality evaluatio is applicable i Agile, with the Product Ower resposible for ot just product quality goals ad/or eeds, but also cotiuous quality measuremet through SPRINT Review meetigs ivolvig all stakeholders. Therefore, this process area is fully applicable (F) to Agile. Advaced Reviews measure product quality early i the lifecycle, ad ehace the test strategy ad test approach by aligig peer reviews (static testig) with dyamic testig1. Cotiuous reviews are doe ad review commets updated i the Agile tool itself. Review meetigs ad retrospective meetigs are coducted to assess the product quality ad completeess of the workig software to esure the effectiveess of advaced reviews. Give this, the Advaced Reviews process area is fully applicable (F) to Agile. LEVEL 4 9 7 5 3 2 1 Test Measuremet Product Quality Evaluatio Advaced Reviews TMMi Level 5: Optimizig Level 5 is about defect prevetio, quality cotrol ad test process improvemet. Defect Prevetio helps idetify ad aalyze commo causes of defects across the developmet lifecycle. It also facilitates the deploymet of actios to prevet the recurrece of defects i the future1. Agile too, through Sprit ad retrospective review meetigs, determies commo causes of defects, prioritizes them, ad defies actios to elimiate their future recurrece. The meetigs ivolve discussig impedimet reasos ad resolutios. As such, this process area is fully applicable (F) to Agile. Quality Cotrol helps statistically maage ad cotrol the test process. At this level, test process performace is fully predictable ad stabilized withi acceptable limits. Project level testig is performed usig statistical methods based o represetative samples to predict product quality ad make testig more efficiet¹. While Agile establishes statistically cotrolled test processes such as maitaiig test performace measures, ad base-liig ad moitorig them, it does ot ivolve testig drive by statistical methods. Though test cases are derived from user stories, statistical techiques are ot used to make test stop decisios. Therefore, this process area is partially applicable (P) to Agile. 5
Test Process Optimizatio cotiuously improves the orgaizatio's existig testig processes through ew testig techologies, tools, ad methods. It trasitios these ew approaches ad chages ito the orgaizatio i a orderly, plaed maer. Test Process Optimizatio also supports the reuse of test assets across the orgaizatio. The improvemets support the orgaizatio's product quality ad test process performace objectives that are derived from the orgaizatio's busiess objectives¹. Agile's project retrospective meetigs discuss ad evaluate SPRINT issues ad improvemet methods, future plas, ad test tools ad methods. However, the reuse of test assets across the orgaizatio is ot widely practiced i Agile projects. Therefore, this process area is partially applicable (P) to Agile. LEVEL 5 9 7 5 3 2 1 Defect Prevetio Quality Cotrol Test Process Improvemet Overall TMMi Levels mappig with Agile TMMi Vs Agile 92% 77% 75% 2 Level 2 Level 3 Level 4 6 Level 5
Coclusio The Agile-TMMi mappig exercise shows how Agile, with mior tailorig of processes, ca co-exist with TMMi, helpig achieve the overall busiess ad techology goals. For example, while Agile maitais project level assets, TMMi's Level 3 recommeds orgaizatio level assets. However, i the same level, Agile processes such as itegratio of the test lifecycle ad developmet models, ad master test pla defiitio fit well ito the Level 3 framework. Similarly, i Level 5 too, select test process improvemets, ad ew testig techologies are evaluated to determie their impact o the testig process. Test improvemets ad the reuse of high quality test process assets are idirectly achieved through retrospective meetigs. Retrospective meetigs also help improve the SPRINT performace of people, relatioships, process, ad tools. While Agile drives faster delivery, reduced time to market, ad facilitates icorporatig chage all the time, TMMi focuses o better cotrol of the product quality usig statistically drive testig, thus improvig the reliability of the uderlyig software. Hece, the TMMi-Agile mappig makes software more reliable, ad relevat to chagig eeds. The key to success is parterig with a TMMi assessor with Agile kowledge ad capability. This combied skill-set eables viewig Agile activities ad processes withi the spirit of TMMi. With Agile adoptio ad TMMi certificatio are both gaiig pace, we eed to fid a way to fit Agile projects ito TMMi. Igorig TMMi or excludig it from Agile is ot a optio. This mappig framework ot oly prevets force-fittig, but also esures seamless itegratio of TMMi with Agile, eablig busiesses to reap rewards such as improved product quality, effective test processes, test egieerig, ad reduced cycle time. About the Authors Gayathri Ekambaram Gayathri Ekambaram is a Audit ad Compliace Maager with the Assurace Delivery ad Risk Maagemet team at Tata Cosultacy Services (TCS). With more tha 11 years of IT experiece, Gayathri has worked i the area of Quality Assurace (QA) across idustries such as healthcare, fiacial services, ad telecom. I additio to TMMi ad Scrum Master Accredited certificatios, she has experiece i test, project, ad defect maagemet. Rabidramoha Pothireddy Rabidramoha Pothireddy is a Cluster Delivery Assurace Maager with the Assurace Delivery ad Risk Maagemet team at TCS. He has 14 years of IT experiece spaig delivery, process ad project maagemet, ad is a certified Oracle DBA, TMMi Professioal, SarbaesOxley, ITIL (Foudatio) ad ISTQB (Foudatio). 7
About TCS' Assurace Services Uit With oe of the most comprehesive portfolios of idepedet test capabilities o offer, TCS addresses both busiess ad quality challeges for its global cliets. We empower orgaizatios across domais to optimize overheads, realize first mover advatage ad improve customer satisfactio. TCS offers assurace services across the testig value cycle, icludig test cosultig ad advisory, test services implemetatio, ad maaged services for test eviromet ad test data maagemet. We cotiually redefie testig ad QA paradigms to help our cliets stay ahead of the curve. Our library of domai-based reusable busiess fuctios ad prove egagemet model fouded o the twi pillars of product ad process quality eable us to deliver certaity to our cliets. Over 28,000 testig cosultats, strategic alliaces ad parterships with key product vedors, more tha 60 dedicated test ceters of excellece ad our iovatio labs power our tailormade solutios, testig assets ad accelerators. With specialized test eviromets ad labs, TCS drives the delivery of assurace i a o-disruptive, agile, ad automated maer, makig the etire developmet lifecycle more efficiet. Cotact For more iformatio about TCS Assurace Services Uit, visit: http://www.tcs.com/assurace Email: global.assurace@tcs.com Blog: #ThikAssurace About Tata Cosultacy Services Ltd (TCS) Tata Cosultacy Services is a IT services, cosultig ad busiess solutios orgaizatio that delivers real results to global busiess, esurig a level of certaity o other firm ca match. TCS offers a cosultig-led, itegrated portfolio of IT ad IT-eabled ifrastructure, egieerig ad assurace services. This is delivered through its uique Global Network Delivery ModelTM, recogized as the bechmark of excellece i software developmet. A part of the Tata Group, Idia s largest idustrial coglomerate, TCS has a global footprit ad is listed o the Natioal Stock Exchage ad Bombay Stock Exchage i Idia. IT Services Busiess Solutios Cosultig All cotet / iformatio preset here is the exclusive property of Tata Cosultacy Services Limited (TCS). The cotet / iformatio cotaied here is correct at the time of publishig. No material from here may be copied, modified, reproduced, republished, uploaded, trasmitted, posted or distributed i ay form without prior writte permissio from TCS. Uauthorized use of the cotet / iformatio appearig here may violate copyright, trademark ad other applicable laws, ad could result i crimial or civil pealties. Copyright 2016 Tata Cosultacy TCS Desig Services I M I 01 I 16 For more iformatio, visit us at www.tcs.com