Process Improvement Reviews

Transcription

1 Process Improvement Reviews Why your team should have regular Process Improvement Reviews Leanne Howard, Agile Practices Consultant Planit Software Testing Abstract We all should be continuously challenging ourselves to make improvements in efficiency and our effectiveness. This article discusses different processes that can be used to do this. Planit Software Testing in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 1

2 With the increase in popularity of Agile, one of the ceremonies undertaken by many Agile teams is the retrospective. This regular meeting investigates what the team is doing well and thus should continue doing, as well as which areas could stand to be improved, prompting the team to make suggestions on how they think these areas could be improved. Even with retrospectives in place, continuous improvement is not guaranteed. For instance, the team may try to make too much change at once without giving the new ideas time to become embedded, or suggested improvements may never be implemented if no-one takes ownership of that initiative. In traditional projects this is even worse as the Post Implementation Review (PIR) does not happen until the end of the project when it is too late to implement improvements. In such instances, the team often gets disbanded which means the lessons learnt may never be applied in the following project and the same bad or inefficient practices can persist. Often when you are heavily involved in the day-to-day delivery of a project it is difficult to see the wood for the trees, so an independent viewpoint is helpful. Also, if you move from project to project or manage a number of projects it is often difficult to make available the time needed to do the root cause analysis required in order to look at more efficient ways to work. If you have been working for the same company or even within same domain, there may be better ways to do activities or new techniques to use that you may not know about. An outside person can also give additional credibility to ideas that you have had but have not had sufficient resources in order to change or even document standard policy. Once you have established that a Process Improvement Assessment (PIA) is required, you must then discern which option is the right choice for your situation. But first, as with any project, it is important to understand your requirements and their associated priorities. You must then work out what the costs and benefits will be for each activity, being sure to account for indirect costs and benefits. Assessing Costs Effort and costs can vary depending on: The assessing organisations costs i.e. their charge rate The size and scope of the assessment i.e. is it an organisation, sector of the business, program or project? The number of projects typically running, as it is important to assess against a percentage of live projects The number and location of interviews that need to be conducted Whether it is a formal or informal assessment Very often only direct costs and benefits are considered, especially given the potential difficulty in quantifying indirect costs. Nonetheless, these should be assessed as their impact can be substantial. Direct Costs Monetary Work hours Training and education External consultancy Assessing Benefits Like costs, benefits are also direct or indirect, with indirect benefits often being more substantial than their more directly associated counterparts. Direct Benefits Rise in productivity Early defect detection therefore cheaper to fix Fewer costly issues in production Indirect Benefits Improved staff experiences, morale and motivation leading to a better quality work Increased customer loyalty Better opportunities for employee movement within projects Improved working environments Recognises testing as a profession and integrated in the development process Indirect Costs Time spent in training Learning curve through job rotations Productivity lost due to opposition to process changes in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 2

3 When examining the process improvement options available, it is important to understand the costs, advantages and disadvantages of each as well as their differing level of formality. These factors should be matched with your organisational structure and needs. In order to help decide which Process Improvement approach to follow, a summary of each one has been included below. Test Maturity Model Integration (TMMi) The Test Maturity Model Integration (TMMi) is the only process improvement option that provides a certificate of maturity. It aims to introduce a structured / controlled set of test processes, moving to a level of optimisation through increased rigour. Developed by the TMMi Foundation as a guideline and reference framework for test process improvement, this model is positioned to complement the Capability Maturity Model Integration (CMMi), being aligned with v1.3. TMMi has a staged architecture for process improvement, with organisations passing through a number of stages/levels as its testing process evolves from one that is ad-hoc and unmanaged to one that is managed, defined, measured, and optimised. Achieving each stage ensures that an adequate improvement has been laid as a foundation for the next stage. The internal structure of the TMMi is rich in testing practices that can be learned and applied in a systematic way to support a quality testing process that improves in incremental steps. There are five levels in the TMMi Model that prescribe a maturity hierarchy and an evolutionary path to test process improvement. Each level has a set of process areas that an organisation must focus on to achieve maturity at that level. Experience has shown that organisations do their best when they focus their test improvement process efforts on a manageable number of process areas at a time, and that those areas require increasing sophistication as the organisation evolves. in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 3

4 Because each maturity level forms a necessary foundation for the next level, trying to skip a maturity level is usually counterproductive. At the same time, it must be recognised that test process improvement efforts should focus on the needs of the organisation in the context of its business environment and the process areas at higher maturity levels may better address the current needs of an organisation or project. Benefits Improved organisational effectiveness and efficiency Improved product quality Improved testing productivity potentially leading to shorter delivery time frames Suited to ANY development lifecycle model Shifted focus from defect detection to defect prevention There are two types of TMMi assessment: Formal Assessments Formal assessments must be conducted by an assessment team led by an accredited lead assessor and at least one other accredited assessor. Formal assessments require a strict level of evidence for the achievement of specific and generic goals in the relevant TMMi process areas. One mandatory point of evidence is in the form of staff interviews, which must be corroborated with the findings from the document study. One of the results of formal assessment is a full gap analysis showing the strengths and weaknesses of an organisation against the TMMi model. This gap analysis can be used as the basis for future improvement projects. A formal TMMi assessment typically acknowledges the following phases: 1. Planning 2. Preparation 3. Interviews 4. Reporting Informal Assessments Alternatively, informal assessments can be conducted, following a less rigorous process than is required for formal assessments, resulting in faster and cheaper delivery, but also one that is less precise. Informal assessments are led by an experienced assessor, designed as an initial indicative view and quick check to evaluate the current state of the test process against TMMi. Implementing TMMi First and foremost, TMMi is a list of best practices or a description of a mature test process. TMMi does not offer a standard approach to a change program in an organisation. To support the implementation of models such as CMMI, the Software Engineering Institute (SEI) has developed a model for change processes IDEAL. This model has also proved to be very useful when implementing TMMi. IDEAL offers an extensive and practical reference standard for change processes and also shows what needs to be done when implementing TMMi improvements in an organisation. The model contains a five phase improvement cycle as shown below: Phase Initiating Diagnosing Establishing Acting Learning Goal Establishing the initial improvement infrastructure for a successful improvement process. Determining the organisation s current state as opposed to what it wants to achieve. Planning and specifying how the chosen situation will be established. Executing the plan. Learning by experience and improving the abilities to implement changes. in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 4

5 Organisations are free to choose the appropriate improvement approach for their implementation of TMMi. In addition to IDEAL, there are several other models for the implementation of process improvement. In general these models are based on Edward Deming s plan-do-check-act cycle. The Deming cycle starts with making a plan that determines the improvement goals and how they will be achieved (plan). Then the improvements are implemented (do) and it is determined whether the planned advantages have been achieved (check). Based on the results of this assessment, further actions are taken as needed (act). Each of these change management processes can also be used with any of the Process Improvement activities outlined in this whitepaper, or other alternatives that may not be covered. Test Process Improvement (TPI) Test Process Improvement (TPI) acts as an independent audit of current testing practices versus testing best practices to assist organisations in bolstering the likelihood of project success and to improve productivity and cost efficiencies in testing. The key areas covered in a TPI include but are not limited to: Examining the strategy and approach to testing Cataloguing the roles, responsibilities and capabilities of the existing test team Analysing of training needs Reviewing existing test procedures and standards for testing Reviewing existing test assets Assessing third party vendor testing and software release management Assessing test environment set-up, configuration and management Reviewing regression testing and the regression test pack Reviewing performance testing, including the use of testing tools Evaluating test tools to support the testing process The TPI audit model is specifically designed to measure testing processes, skills and practice across an entire organisation and their interactions with other teams throughout the project s entire lifecycle, addressing 20 key areas. The power behind this model is to specifically identify the current stage of testing 'maturity', identifying where the organisation is headed or needs to be and then mapping activities for improving this status as well as checkpoints to ensure that the end result is achieved. Assessment Criteria: Test Strategy Life-cycle model Communication Reporting Evaluation Low-level testing Metrics Test automation Test environment Office environment Commitment and motivation Test functions and training Scope of Methodology Test specification techniques Static test techniques Defect management Testware Management Test process management Moment of involvement Estimating and planning This improvement review can be conducted across divisions or projects to provide a benchmark and guide organisations as to which assets are performing well and thus should be re-used. This could also be completed for one project to help them reach their milestones in an optimal manner. Part of the deliverable is a comparison graph which clearly and easily highlights areas to be improved. An updated version of this approach has been coined by Sogeti called TPI NEXT, following the same process only with a slight change to the assessment criteria. These key areas examined are as follows: Degree of Involvement Metrics Test Strategy Test Organisation Communication Reporting Test Case Design Test Tools Stakeholder Commitment Defect Management Testware Management Methodology Practice Tester Professionalism Estimating and Planning Test Process Management Test Environment in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 5

6 Score Project X vs Best Practice Industry Best Practice Project X Key Areas Agile Process Improvement (API) As the name suggests, Agile Process Improvement is focussed on Agile teams or organisations moving to Agile. It focuses on the team interactions and collaboration, product quality and the sharing of testing responsibility amongst the whole team. The key areas to be covered in an API include but are not limited to: Examining the strategy and approach to Agile Planning at Release and Iteration level Cataloguing the roles, responsibilities and capabilities of the existing Agile team Team collaboration and interactions, including with the Business Analysing of training needs Reviewing existing Agile procedures and any standards/compliance requirements for the Agile team Reviewing requirements gathering and grooming Reviewing existing Agile assets Assessing Agile environment set-up, configuration and management Reviewing regression testing, including automation capability Reviewing non-functional testing, including the use of testing tools Evaluating tools to support the Agile process The key deliverable from an API assignment is a report assessing the current Agile process and procedures adopted, mapped against leading practice regardless of the framework/methodology used by the organisation. It highlights areas of potential improvement, steps required to achieve these improvements in productivity, effectiveness and cost efficiency, which are all well known benefits for transforming to agile. It is surprising to note that as many organisations struggle with their agile transformation that there are very few recognised process improvement frameworks for agile specifically. Those that are available focus on the developer and their activities. Planit s API is unique in that it has been designed to measure Agile processes, skills and practice across an entire project lifecycle, including the Business interaction which is often one of the key challenges. It assesses the whole team collaboration and moving to a cross functional team. There is an emphasis on whole team ownership of quality and what activities the team members can complete in order to optimise the benefits of agile and deliver product that gives the business value. The API is designed to allow Planit to get to know your organisation in more detail, which then enables us to provide an accurate overview of opportunities to improve the Agile team. The API report outlines the findings of the review, desired future Agile maturity and suggestions for achieving this. This will provide a baseline roadmap of the desired Agile process. in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 6

7 Test Process Health Check A Test Process Health Check similar to the Test Process Improvement only less formally, utilising a questionnaire that focusses solely on the testing team. The Health Check is typically shorter in duration and is best suited to smaller organisations or those who do not require highly formalised reviews, allowing them to achieve and implement quick wins. Test Framework Risk Management Test Reporting Test Environments Test Automation Test Estimation Test Preparation Test Execution Defect Management Test Metrics Testing within the SDLC Regression Testing Test Career Path & Organisation Change & Release Management Test Improvement Process The Health Check consists of two different questionnaires; one focussed at the test management level and the other at the test analysis level. Based on the required scope, the topics covered in the Test Process Health Check may include: The Health Check process and activities are split into four distinct phases: Initiation and preparation Interviews Review of test artefacts Completion and reporting Implementing Improvements Regardless of which areas you are targeting for improvement, it is important that you allocate the right resources in order to realise the true benefits. You are about to embark on a change management programme which needs to be managed and supported as such. Early successes through the implementation of quick wins will help to gain momentum although introducing too much change at once that cannot be absorbed by the team can be disastrous and derail the good before it has ever begun. Change Curve Enterprise change, or even change at the project level, evokes emotional reactions that may be categorised using Elizabeth Kubler-Ross s (1970) Model of Emotional Reaction to Change. Self esteem Integration Anger Denial Bargining Discovery Experimentation Shock Acceptance Depression Time While death and dying is the ultimate trauma for many people, similar emotional upsets can be experienced when dealing with many of life's challenges. This is especially the case if confronting something difficult for the first time or if the challenge happens to threaten an area of psychological weakness, which we all possess in different ways. Of course as individuals we all react differently and this will also depend on the size of the change. Often if people are included in the definition of the change, plus if they can see the benefits for them and ultimately the organisation, which are articulated consistently, it is more likely that you will get buy-in or at least acceptance. What you are really aiming for is enthusiasm for the change, introducing a culture of continual process improvement where challenging ineffective and inefficient activities is commonplace. If you are trying to implement these new practices within teams particularly new ones, it is also important that you understand some team dynamics. Team Dynamics When looking at introducing change you also need to understand team dynamics. The Forming Storming Norming Performing Model of group development was first proposed by Bruce Tuckman in 1965, who posited that these four phases are necessary and inevitable in order for the team to grow, face up to challenges, tackle problems, find solutions, plan work and in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 7

8 deliver results. If you understand where you team falls in this model, you will better be able to recognise the acceptance or disruption that change is going to bring to them. In summary the phases are: Forming In the first stages of team building, during the forming phase, individual behaviour is driven by a desire to be accepted by the others, and avoid controversy or conflict. This is a comfortable stage to be in, but the avoidance of conflict and threat means that not much actually gets done. Storming Groups next enter the storming stage in which different ideas compete for consideration. The team addresses issues such as what problems they are really supposed to solve, how they will function independently and together, and what leadership model they will accept. Team members open up to each other and confront each other's ideas and perspectives. Norming At this stage, the team has come together with one mutual goal and plan, with all team members working towards ensuring the success of these goals. To reach this stage, some team members may have to concede their own ideas and follow team consensus in order to be a functional unit. Performing Teams at this stage are high-performing, able to function as a unit as they find ways to get the job done smoothly and effectively without inappropriate conflict or the need for external supervision. Such teams are characterised by interdependent team members. Conclusion With the increasing adoption of Agile, the concept of continuous improvement has become a regular practice for many teams, with the principles of inspect and adapt helping to improve feedback systems. Even those working on more traditional projects are adopting continuous improvement or at least regular checks on how the project is tracking with the goal of finding potential gains in efficiency. But when deciding on how to review processes and implement improvements, it is critical that you evaluate your organisational or project needs, as this will dictate which model will best fit your circumstances. Each of the approaches listed has its benefits, but also an associated cost. If you understand your requirements but are unsure of which model is best for you or how you should go about implementing this model, you should discuss this with an internal expert or experienced senior consultant rather than risk heading down the wrong path. Planit s highly experienced consultants are able to tailor a solution to fit within your scope, budget, timeframes and availability. By pursuing an independent assessment, you are able to maximise the benefits of this activity: Avoiding bias and internal politics, gaining an accurate view of the true state of testing Benefiting from extensive knowledge of industry practices based on accredited training and cross-industry experience Adding further credibility to the recommendations Gaining the time to dedicate to the root cause analysis Ensuring focus on implementing improvements by having a third party dedicated to keeping the project on track Also there is no point doing this exercise of evaluating where you are currently and then making improvement suggestions, if you are not willing to put in place resources to make it happen. It will not be implemented if it is seen as a something on the side activity or project. Often it has been found that having an external consultant as part of the improvement team helps to keep the project on track, but it is important that the organisational team make the decisions aided by the experience that the consultant can bring. in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 8

9 The key focus of testing should be to move from defect detection to defect prevention. Adhoc processes often only produce results due to the hero mentality of a few within your team which is not sustainable over a period of time. The aim is to move through just finding defects (debugging), checking against requirements, proving fit for purpose, good feedback on product quality to optimising the test process. Testing accounts for at least 21% of overall project spend (Planit Testing Index 2014) and this can rise significantly if a project has to test across multiple browser and/or mobile devices combinations. Therefore if efficiencies can be made in testing this could result in large savings to the project budget or all wider coverage in testing thereby reducing risk even further. References TMMi Foundation TPI and TPI Next Sogeti Elizabeth Kubler-Ross s (1970) model Software Engineering Institute (SEI) IDEAL Edward Deming s plan-do-check-act cycle Team Dynamics by Bruce Tuckman Planit Testing Index 2014 About the Author Leanne Howard is an Agile Practices Consultant with over 20 years experience working in software testing. Leanne is a specialist in the implementation of Agile practices and testing methods with a strong focus on client satisfaction. She enjoys sharing her knowledge through mentoring and coaching others to promote continuous learning, and is active in the testing profession both locally and internationally. Leanne is an ANZTB Member and sits on the Australian Standards IT Committee. in any form or by any means, electronic, mechanical, recording, photocopying, or otherwise, without the express written permission of Planit Software Testing. 9