SPICE, CMMI & Beyond: Industry Perspectives and Challenges for Software Process Management Dr. Klaus Hoermann Principal, Partner SEI-certified SCAMPI Lead Appraiser and CMMI Instructor intacs certified SPICE Principal Assessor & Instructor Keynote for ICSP 2010 (International Conference on Software Process) Paderborn, July 8-9, 2010 KUGLER MAAG CIE GmbH Leibnizstr. 11, 70806 Kornwestheim / Stuttgart, Germany Klaus.hoermann@kuglermaag.com www.kuglermaag.com Seite 1
The Agenda Lessons learned from process improvement projects Suggestions for future research Questions, discussions Page 2
Lessons learned from process improvement projects Suggestions for future research Questions, discussions Page 3
Where you want to go? You re not really sure where you are Page 4
Page 5
Potential elements of a good vision What has market leadership to do with processes? If you want to be among the leaders you have to gain advantage over competitors: Better and more innovative products Shorter product development time Better performance at lower cost Less problems in the field causing customer dissatisfaction and high cost of nonquality In order to gain these advantages you have to cope with: Get quality under control - flawless engineering Manage efficiency improvement programs successfully Make outsourcing/offshoring and internationalization of your operations work satisfactory Keep your strongest assets happy: your people Page 6
Why is the strategy you propose making sense? Example argumentation for a CMMI-based strategy: There are many case studies showing benefits from using CMMI: Reduction of product development time and cost Lower cost of quality and lower cost of non-quality Higher productivity Better product and service quality Better customer and employee satisfaction Good ROI This is achieved by: Frontloading : Defects are avoided by better processes/tools or they are found and removed very early. This reduces the amount of rework and other non-value add time and improves quality. Better quality through better processes (like requirements engineering, design, review, and test) More reliable project estimations More efficient people: high quality inputs for their work needed from others is timely available Page 7
Page 8 The frontloading effect
An example from BMW Data by BMW from 26 projects developing electronic control units: Page 9
An example from BMW Data by BMW from 26 projects developing electronic control units: Page 10
Be careful with concrete numbers You are telling your executives that you have evidence from industry that the following improvements are possible: We will reduce overhead rate by 10%! We will increase software development productivity by 50%! We will reduce the cost for finding and fixing defects by 22%! Wow. Which organization was it? they will ask: Lockheed Martin Integrated Systems & Solutions 15,000 employees with an estimated $5 Billion in annual revenues It took them18 years doing process improvements: 1993-1995: SW-CMM Level 2 1996-1997: SW-CMM Level 3 1998-1999: SW-CMM Level 4 2000-2001: SW-CMM Level 5 2002-2010: CMMI Level 5 Source: By Peter J. McLoone and Sharon L. Rohde: Performance Outcomes of CMMI -Based Process Improvements Page 11
Be careful with concrete numbers The problem with this argumentation is The situation of another organization is hard to compare to your own organization. Gaining the success took far to long (as in the example). The success story often looks exaggerated. Page 12
Page 13 If you don t know where you are, a map won t help. (Watts S. Humphrey)
The problem The situation is often seen as much more positive than it is in reality especially when talking to executives. The reality in many large organizations is: There is no precise inventory of projects. Decisions are not based on facts such as precise measurements or KPIs. People are more interested in their own advantage than in the organization s. Page 14
The problem The reality in many large organizations is: Project reporting to management trivializes the real problems nobody wants to be guilty. Page 15
The problem The reality in many large organizations is: There have been so many initiatives with threeor four letter acronyms most of them dozed off and had little or no effect. People got used to it and do not take new initiatives seriously. There are always several initiatives running in parallel and sponsored from different senior management representatives. The initiatives are never coordinated and, even worse, jeopardize each other. Why should the new initiative be any better? And how should it fit to the others already running? Page 16
The problem Source: Standish Group, Chaos Report Page 17
What to do against it Perform a study, an assessment or similar Find those people who are not happy with the situation. Find the real problems. Find disaster stories: what went terribly wrong and what were the reasons Perform a process assessment using a state-of-the-art method against acknowledged standards such as CMMI, SPICE, Automotive SPICE. Utilize customer pressure: Invite customers to give feedback on what went wrong in the cooperation and what expectations they have. Page 18
Example We developed the Health Check method to evaluate an organization to which degree the success factors for process improvement are fulfilled. Pre-structured interviews with representatives of different roles such as senior management, lower management, project managers, developers, QA, etc. Interviews focus on typical success factors for process improvement. Results are used to gain top management attention and create a sense of urgency. Page 19
Training (Not) All roads lead to Rome. Teamwork Great Place to Work Institute Groupware Workflow management Certification programs Incentives Improving cooperation (e.g., integrated teams ) Agile Development IPPD (CMMI Addition) Development tools IT technology Maturity Models (CMMI.) Product technology SOA Six Sigma Continuous Improvement Process modeling tools Integrating development tools and processes TQM Lean Page 20
What organizations do Organizations can invest in different kind of improvements. Process improvement is just one of them. Many organizations invest in improvements in several sectors (People, processes, technology). When it comes to process improvement maturity models such as CMMI, SPICE, or Automotive SPICE are a good choice. There is a trend to combined improvement initiatives using multiple models/paradigms, such as CMMI + Lean Six Sigma. Possible reason: The quest for faster ROI Page 21
What organizations do There is a trend to certificates instead of real improvements which is particularly dominant in Asia. This development is well known from ISO 9000. Capability Target threshold Assessments Benefit Good: Taking it seriously Achieving capability quickly Building trust by planning improvements and actually achieving it on time Time Capability Assessments Benefit Bad: Taking it NOT seriously: starting too late, not investing enough manpower No change in attitude over time Investments focus purely on passing assessments. Page 22 Time
Trends Example CMMI and taking CMMI Appraisals as an indicator: Strong growth Time to move up one level is becoming shorter: currently 18 month in average Application area is shifting: Services strongly increasing as opposed to development 2010 (2009): 71% (53%) of all Appraisals conducted in services organizations Regional focus is shifting (Asia is growing stronger than other regions.) Regional distribution and growth Page 23
Trends Interest seems to shift from models requiring longterm invest to those promising quick results, preferably with a clear return on invest Example statistics for selected keywords ( Google insights for search) CMMI Contradiction: Public interest in CMMI seems to decrease while number of appraisals is growing. Possible explanations: A new model variant CMMI for Services was published February 2009 which might have increased the number of appraisals among existing CMMI users. The changing number of CMMI users might be not correlated with the public interest in CMMI. The increasing number of CMMI users might be due to factors such as competitive pressure, customer pressure, marketing reasons, etc. Page 24
Trends ISO 9000 Automotive SPICE EFQM Page 25
Trends Lean Six Sigma Agile Development Agile Project Management Page 26
Problems in planning Organizations underestimate the effort; Our experience is: Reaching a level cost in average 5% of your development budget with 40% of this needed for external consultants. Keeping a level cost in average 3% of your development budget. Organizations underestimate the time needed SEI data show that reaching the next level takes in average 18 months. (In my experience this is rather the lower limit.) Organizations underestimate the resistance of the people Many improvement projects fail because management fails to convince their people that this is worthwhile. Once top management understands what it requires they often loose interest. Organizations do not treat it as an ordinary project with Clear objectives, a capable project manager, an agreed budget and resources, milestones, management reporting, management attention. Page 27
Problems in planning Organizations fail to establish and proof an attractive relationship between business objectives and the processes. Processes Business objectives Page 28
Unique resource aspects in improvement projects For sure you need everything an ordinary project needs (see before). Some unique aspects in improvement projects are: You need a project organization which appropriately reflects the variety of your people. Business objectives Page 29 EPG Process Teams You need the best people Processes Make sure that they understand that paperwork is not enough: they need to change the habits of people.
Unique resource aspects in improvement projects An example from a CMMI-based improvement project Project mgmt. Steering group EPG Engineering Process Group Core team QAG Quality Assurance Group Process Action Teams PO PO PO PA PA PA Process architecture Process Action Teams PO PO PO PA PA PA Process definition Process imprvmt Training Design Integration Test PM Project coach Organizational Process Assets HR PM Project coach Page 30 PM Project coach
We don t have a knowledge problem, we have a implementation problem. (Klaus Hoermann) Even the longest journey begins with a single step. (Lao Tzu) So many ideas, so many plans, but so little energy to actually get moving. Many initiatives get stuck in this phase. There are many attractive improvement opportunities (people technology processes). The ROI of the initiative is questionable. No top management sponsor who wants to take responsibility What helps: Prioritization of what you want to do compared to other initiatives Clear connection to business objectives Plausible ROI calculation Page 31
Many process improvement projects run dry Overengineering Processes are trying to stipulate every detail. Processes are more oriented towards model compliance than usefulness or user friendliness. Focus on the perfect process, not on successful deployment Endless debate and struggle for the perfect process rather than transferring it to the people and making sure they find it useful and take responsibility for the process. Successful processes are created in practice, not in an ivory tower. Good processes mature over the years when users feel responsible for improving them. Loosing business perspective The improvement project is focused on levels, not on business benefit. Achievement of any business benefit can never be demonstrated. Loosing your people s interest People are not convinced or do not want to change their attitude Not thorough enough Processes are not consistently used everywhere, but just where managers/qa/customers pay attention. Page 32
Many process improvement projects run dry Not enough patience Some executives seem to want to make the race by setting aggressive objectives ( Level 3 in one year from scratch ). You need patience to achieve sustainable cultural changes by convincing and motivating your people and make them change their attitude and behavior. Halfhearted investment the problem of the Therapeutic Threshold Resources Therapeutic Threshold Time Page 33
Many process improvement projects run dry Achieving cultural change is hard Behavioral effectiveness Flash in the pan Sustainable cultural change Flop Compulsory exercise Sustainability Cultural change needs to succeed against elements of the organizational culture (habits, values, ) Page 34
Many process improvement projects run dry Practitioners do not understand the purpose of a process. The purpose is To provide guidelines to avoid pitfalls and inefficient or ineffective procedures To achieve quality of results through quality of the process To reduce risks To reduce ever repeating discussions on how to do the work To train new staff members To achieve better performance of the community in average The purpose is not To please everybody To make everybody more efficient Performance Staff members, teams Page 35
The living culture as opposed to the bureaucratic culture The living culture Processes mirror how people work. Users define and maintain their processes. Processes are adhered to even under stress. Little management pressure and little effort needed to maintain the culture Users stand behind their processes and find them useful. The bureaucratic culture Processes specify how people should work. Processes are defined and maintained from the ivory tower without real participation from users. There are always plenty of reasons not to follow the processes. High management pressure and high effort needed to maintain the culture Users see processes as useless barriers to work. Low cost high benefit High cost little benefit Page 36
What is really important? The majority of staff members cooperates. Process adherence is checked regularly. Staff requires processes to be useful for their work. Processes are improved continuously. Targeted improvements are actually achieved. Synergies are obtained between people and projects (forums, newsgroups, ). Achievement of improvements can be measured. Management requires and monitors that processes are useful for business. Processes and tools are well integrated. Page 37
Lessons learned from process improvement projects Suggestions for future research Questions, discussions Page 38
Last not least: Potential research topics ROI drivers Studies to show that frontloading really works and why, e.g. using defect flow modeling What are the drivers for people s productivity and how can they be assisted? In general, neutral case studies on ROI calculation in improvement programs People issues What motivates people in improvement programs and what are the barriers? How is employee satisfaction influenced by processes? Understanding how cultural change develops over time in improvement programs and what are the influencing factors. How do different clusters of people (leaders, laggards, ) develop and how do they impact each other? Deployment and change management methods: How do they work in which situation and in which target group? Page 39
Last not least: Potential research topics Others How does customer satisfaction relate to improvements? Does this lead to more business? Relationship process maturity product quality in detail: Which process properties, methods, etc. are the drivers? How big is the impact of process maturity to other drivers? Do drivers impact each other? Multi-model and multi-method studies (CMMI/Lean, CMMI/Agile, ): Success factors, what are positive effects, do they support and complement each other? Conformity to several models: How to demonstrate conformance (traceability), how to manage evolution without jeopardizing conformity Effective use of KPIs: Why is it so hard and how can it be improved? Is it just a question of maturity? Page 40
Lessons learned from process improvement projects Suggestions for future research Questions, discussions Page 41
Contact KUGLER MAAG CIE GmbH Leibnizstrasse 11 70806 Kornwestheim Germany Tel +49 7154 1796 100 Klaus.Hoermann@KuglerMaag.com www.kuglermaag.com Page 42