Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Mid-Year Update August 2002 We could not have produced this report without the support of those organizations and lead appraisers who sent their appraisal results to the SEI. Our gratitude goes to them for their continued cooperation with our data collection and analysis efforts. Software Engineering Measurement and Analysis Team The is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon University 2002 by Carnegie Mellon University 1 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Outline Introduction Current Status Community Trends Organizational Trends Summary 2 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Introduction -1: Purpose and Source Characterize the process maturity of the software community This briefing uses information from reports of CMM Based Appraisals for Internal Process Improvement (CBA IPIs) and Software Process Assessments (SPAs) CMM, Capability Maturity Model and Capability Maturity Modeling are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University 3 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Introduction -2: Data Description CBA IPIs and SPAs conducted since 1987 through June 2002 and reported to the SEI by July 2002 2325 assessments 1840 CBA IPIs 485 SPAs 1756 organizations 512 participating companies 451 reassessed organizations 9632 projects Please refer to: Terms Used in this Report on page 30 4 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Introduction -3: Report Contents This briefing includes three primary sections: Current Status - Snapshot of the software community based on the most recent assessment, since 1998, of reporting organizations Community Trends - Global distribution of assessments - Growth in the number of assessments performed - Shifts in the maturity profile over time Organizational Trends - Analysis of Key Process Area (KPA) satisfaction - Time to move up in maturity 5 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Current Status Assessments conducted from 1998 June 2002 through 1124 organizations 388 participating companies 5538 projects 42.3% offshore organizations Please refer to: Terms Used in this Report on page 30 6 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Reporting Organization Types Commercial/In-house 70.5% DoD/Fed Contractor 24.9% Military/Federal 4.6% Based on 1124 organizations 0% 10% 20% 30% 40% 50% 60% 70% 80% % of Organizations 7 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Types of Organizations Based on Primary Standard Industrial Classification (SIC) Code Retail Trade 0.3% Wholesale Trade 0.5% Transportation, Communication, Electric, Gas and Sanitary Services 4.4% Public Administration (Including Defense) 4.6% Finance, Insurance and Real Estate 5.5% Primary Metal Industries 0.1% Chemicals And Allied Products 0.2% Printing And Publishing 0.4% Fabricated Metal Products 0.5% Industrial Machinery And Equipment 3.0% Offshore (SIC Code Not Applicable) 42.3% Manufacturing 15.4% Transportation Equipment 3.4% Electronic & Other Electric Equipment 3.4% Services 27.0% Instruments And Related Products 4.4% Educational Services 0.1% Business Services 19.3% Engineering & Management Services 6.9% Health Services 0.4% Services, Nec 0.1% Membership Organizations 0.2% Based on 1124 organizations 8 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Organization Size Based on the total number of employees primarily engaged in software development/maintenance in the assessed organization 1001 to 2000 1.9% 2000+ 0.9% 501 to 1000 6.4% 25 or less 9.8% 301 to 500 9.1% 25 to 50 14.7% 201 to 2000+ 28.5% 201 to 300 10.1% 1 to 100 48.0% 101 to 200 23.5% 51 to 75 12.4% Based on 1094 organizations reporting size data 76 to 100 11.1% 9 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Organization Maturity Profile August 2002 100% 90% 80% % of Organizations 70% 60% 50% 40% 30% 20% 19.3% 43.2% 23.4% 10% 7.3% 6.8% 0% Initial Repeatable Defined Managed Optimizing Based on most recent assessment, since 1998, of 1124 organizations. 10 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Maturity Profile by Organization Type 100% 90% 80% 70% % of Organizations 60% 50% 40% 30% 28.8% 45.7% 35.0% 50.0% 37.1% 20% 10% 0% 20.2% 19.2% 15.0% 13.5% 8.9% 8.1% 6.8% 5.8% 3.9% 1.9% Initial Repeatable Defined Managed Optimizing Commercial/In-house DoD/Fed Contractor Military/Federal Based on most recent assessment, since 1998, of 1124 organizations 11 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Maturity Profile by Organization Size Based on the total number of employees primarily engaged in software development/maintenance in the assessed organization 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 25.2% 61.7% 10.3% 0.9% 1.9% 24.2% 54.0% 17.4% 3.7% 0.6% 22.8% 45.6% 25.0% 5.1% 1.5% 23.1% 52.1% 18.2% 3.3% 3.3% 17.1% 37.7% 28.8% 9.3% 7.0% 12.6% 41.4% 28.8% 9.9% 7.2% 10.0% 38.0% 27.0% 12.0% 13.0% 12.9% 18.6% 25 or less 25 to 50 51 to 75 76 to 100 101 to 200 201 to 300 301 to 500 501 to 1000 40.0% 12.9% 15.7% 23.8% 14.3% 9.5% 14.3% 1001 to 2000 Initial Repeatable Defined Managed Optimizing 38.1% 10.0% 40.0% 2000+ 50.0% Based on 1094 organizations reporting size data The 1001 to 2000 and 2000+ categories are of a small percentage which will inflate the maturity level bars. Please see page 9 and take this into account. The purpose of this chart is to indicate that all size categories contain most, if not all, maturity levels. 12 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
USA and Offshore Organization Maturity Profiles 100% 90% 80% % of Organizations 70% 60% 50% 40% 30% 20% 10% 0% 47.4% 37.6% 23.3% 25.3% 22.0% 14.0% 13.2% 10.0% 5.3% 2.0% Initial Repeatable Defined Managed Optimizing USA Offshore Based on 645 U.S. organizations and 479 offshore organizations 13 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Community Trends Assessments conducted from 1987 through June 2002 2325 assessments 1756 organizations 512 participating companies 451 reassessed organizations 9632 projects Please refer to: Terms Used in this Report on page 30 14 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Countries where Assessments have been Performed and Reported to the SEI Argentina Australia Austria Barbados Belgium Brazil Canada Chile China Colombia Denmark Egypt Finland France Germany Greece Hong Kong Hungary India Ireland Israel Italy Japan Korea, Republic of Malaysia Mexico Netherlands New Zealand Pakistan Philippines Poland Portugal Puerto Rico Russia Saudi Arabia Singapore South Africa Spain Sweden Switzerland Taiwan Thailand Turkey United Kingdom United States Uruguay Venezuela Vietnam 15 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Country Carnegie Mellon University Number of Assessments and Maturity Levels Reported to the SEI by Country Number of Assessments Reported Maturity Level 1 Reported Maturity Level 2 Reported Maturity Level 3 Reported Maturity Level 4 Reported Maturity Level 5 Reported Country Number of Assessments Reported Maturity Level 1 Reported Maturity Level 2 Reported Maturity Level 3 Reported Maturity Level 4 Reported Argentina Less Than 10 Malaysia Less Than 10 Australia 27 Yes Yes Yes No No Mexico 11 Yes Yes Yes Yes No Austria Less Than 10 Netherlands 12 Yes Yes Yes No No Barbados Less Than 10 New Zealand Less Than 10 Belgium Less Than 10 Pakistan Less Than 10 Brazil 16 Yes Yes Yes No No Philippines Less Than 10 Canada 51 Yes Yes Yes No Yes Poland Less Than 10 Chile Less Than 10 Portugal Less Than 10 China 37 Yes Yes Yes Yes Yes Puerto Rico Less Than 10 Colombia Less Than 10 Russia Less Than 10 Denmark Less Than 10 Saudi Arabia Less Than 10 Egypt Less Than 10 Singapore 15 Yes Yes Yes No Yes Finland Less Than 10 South Africa Less Than 10 France 105 Yes Yes Yes Yes No Spain Less Than 10 Germany 23 Yes Yes Yes No No Sweden Less Than 10 Greece Less Than 10 Switzerland Less Than 10 Hong Kong Less Than 10 Taiwan Less Than 10 Hungary Less Than 10 Thailand Less Than 10 India 187 Yes Yes Yes Yes Yes Turkey Less Than 10 Ireland Less Than 10 United Kingdom 110 Yes Yes Yes No No Israel 28 Yes Yes Yes No No United States 1563 Yes Yes Yes Yes Yes Italy 24 Yes Yes Yes No No Uruguay Less Than 10 Japan 56 Yes Yes Yes Yes Yes Venezuela Less Than 10 Korea, Republic of Less Than 10 Viet nam Less Than 10 Maturity Level 5 Reported Please note that this information, as with all information in this report, is based on what assessments have been sent to the SEI. This does not necessarily mean every assessment performed is reported to the SEI. Of those countries with more than 10 assessments reported to the SEI, most, if not all, maturity levels are represented. A further breakdown is not possible without compromising confidentiality. In order to provide this breakdown, every assessment performed should be reported to the SEI. Please make sure the assessment artifacts reported are translated into English for proper processing. 16 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Number of Assessments Reported to the SEI by Year 350 300 Number of assessments 250 200 150 100 50 0 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 First Assessment Reassessment Based on 2325 assessments conducted through Jun 2002 and reported to the SEI by Jul 2002 17 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02 * *
Number of Assessments Reported by Organization Type and Year 250 200 Number of assessments 150 100 50 0 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 Commercial/In-house DoD/Fed Contractor Military/Federal Based on 2325 assessments conducted through Jun 2002 and reported to the SEI by Jul 2002 18 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02 * *
Trends in the Community Maturity Profile 100% % of Organizations 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 80.3% 64.6% 60.4% 54.4% 48.1% 42.2% 37.5% 35.8% 12.1% 21.8% 22.7% 27.0% 30.5% 32.8% 33.9% 34.9% 6.8% 11.9% 14.2% 14.7% 15.7% 17.4% 20.0% 19.5% 1.4% 2.2% 3.1% 3.7% 4.3% 4.6% 5.3% 1987-1991 1992-1996 1997 1998 1999 2000 2001 2002 Initial Repeatable Defined Managed Optimizing Based on a cumulative view of the most recent assessments of organizations up through the year indicated. This accounts for the difference from the figures on page 10. Year 0.8% 0.3% 0.5% 0.7% Orgs 132 632 787 956 1171 1413 1641 1756 2.0% 3.3% 4.1% 4.6% 19 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Organizational Trends Assessments conducted through December 2001 1227 Key Process Area (KPA) profiles - satisfaction of KPAs by maturity level for organizations assessed at levels 1 and 2 451 reassessed organizations - although some organizations conducted multiple reassessments, only the first and latest assessments were used in creating the charts on pages 23 & 24 Please refer to: Terms Used in this Report on page 30 20 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Key Process Area Profiles -1 Organizations Assessed at Level 1 Repeatable RM SPP PTO SSM SQA SCM Defined OPF OPD TP ISM SPE IC Managed Optimizing PR QPM SQM DP TCM Fully Satisfied Rated PCM 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Based on 496 CBA IPI assessments % of assessments Software Subcontract Management (SSM) is not applicable/not rated in many assessments. Please take that into account when interpreting its Fully Satisfied rating. 21 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Key Process Area Profiles -2 Organizations Assessed at Level 2 Defined OPF OPD TP ISM SPE IC PR Managed QPM SQM Optimizing DP TCM Fully Satisfied Rated PCM 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % of assessments Based on 731 CBA IPI assessments 22 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Maturity Level of First and Latest Assessments 100% 90% 80% 70% % of Organizations 60% 50% 40% 30% 52.1% 32.8% 31.0% 40.6% 20% 10% 0% 9.3% 8.9% 8.6% 10.4% 5.8% 0.4% Initial Repeatable Defined Managed Optimizing First Latest Based on 451 reassessed organizations using their first and latest assessment 23 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Reassessments Change in Maturity Level Level 2 to 5 0.9% Level 3 to 4 3.3% Level 3 to 5 3.3% Level 4 to 5 5.5% No Change 12.6% Moved Down 2.9% Level 1 to 1 7.1% Level 2 to 2 3.8% Level 3 to 3 1.3% Level 4 to 4 0.2% Level 5 to 5 0.2% Level 2 to 4 3.1% Level 2 to 3 23.3% Level 1 to 2 26.8% Level 1 to 3 16.0% Level 1 to 5 0.4% Level 1 to 4 1.8% Based on 451 reassessed organizations using their first and latest assessment 24 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Time to Move Up 100 Number of months to move to next maturity level 75 50 30 Recommended { time between appraisals 18 0 Time Period of Initial Assessment 39 40 Pre-1992 23 28 25 22 23 1992 to Present 30 17 17 All (1987 to Present) Largest observed value that is not an outlier 75th Percentile Median 25th Percentile Smallest observed value that is not an outlier Level Orgs 1 to 2 2 to 3 24 12 1 to 2 2 to 3 3 to 4 4 to 5 143 150 33 29 1 to 2 2 to 3 3 to 4 4 to 5 167 162 35 29 25 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Maturity Summary - 1 Current Status Increasing proportion of commercial and in-house organizations Of U.S. organizations, the services and manufacturing industries are conducting the most software process assessments Nearly half of the organizations reporting size have 100 or less software personnel 26 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Maturity Summary - 2 Community Trends Overall community profile continues to shift towards higher maturity Trend towards higher maturity profile for offshore organizations compared to U.S. organizations continues 27 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Maturity Summary - 3 Organizational Trends Software Quality Assurance is the least frequently satisfied level 2 KPA among organizations* assessed at level 1 Integrated Software Management is the least frequently satisfied level 3 KPA among organizations* assessed at level 2 Higher maturity has been reached among those organizations reporting reassessments *Adjusted for number of organizations rating the KPA. 28 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Maturity Summary - 4 Organizational Trends (continued) For organizations that began their CMM-based SPI effort in 1992 or later, the median time to move from: maturity level 1 to 2 is 23 months maturity level 2 to 3 is 22 months maturity level 3 to 4 is 28 months maturity level 4 to 5 is 17 months 29 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Terms Used in this Report Assessments - The assessment methods used in this report are the Software Process Assessment (SPA) and CMM-Based Appraisal for Internal Process Improvement (CBA IPI). We do request and receive other Software CMM-based appraisals such as Software Capability Evaluations (SCE SM ) and Interim Profiles SM. As our sampling size of these other methods increase, they will be reported here. Company - Parent of the organization A company can be a commercial or non-commercial firm, for-profit or not for-profit business, a research and development unit, a higher education unit, a government agency, or branch of service, etc. Organization - Appraised entity The organization unit to which the appraisal results apply. An appraised entity may be any portion of an organization including an entire company, a selected business unit, units supporting a particular product line or service, etc.. Offshore - An organization whose geographic location is not within the United States. The parent of the organization may or may not be based within the United States. SM Interim Profile and SCE are service marks of Carnegie Mellon University 30 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Feedback & Questions Welcome We are always interested in improving the maturity profile to serve you better. To do this, we need to know a little more about you. Please let us know: 1. How you use the information in this report 2. What additional information would you like to see presented in the maturity profile report 3. If there is a problem in your supplying us with the required data to create the information you would like to see As always, if you have questions or comments, we would appreciate hearing them. Please respond to: PAIS Include: Your Name Address 4500 Fifth Avenue Phone Pittsburgh, PA 15213 Fax or E-Mail to: pais@sei.cmu.edu E-Mail 31 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Submit Your Appraisal Data Visit our Web site for forms used to submit data and for future maturity profile reports: http://www.sei.cmu.edu/sema/packet.html Send the forms and your appraisal data to PAIS 4500 Fifth Ave. Pittsburgh, PA 15213 32 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
A Web Based Interactive Maturity Profile is available through the Software Engineering Information Repository http://seir.sei.cmu.edu Incorporate charts from the profile into your own presentation Create a particular Maturity Profile chart for a more specific segment within the software engineering community The segments are from our seven categories (Commercial, In-House, DoD or Federal Contractor, etc.) or from the Standard Industrial Classification Code. The Interactive Maturity Profile contains all releases of the Maturity Profile. Of the charts within a Maturity Profile, fourteen can be customized. 33 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02
Contacts for General SEI Information SEI Customer Relations (412) 268-5800 SEI FAX number (412) 268-5758 Internet Address customer-relations@sei.cmu.edu Mailing Address Customer Relations Carnegie Mellon University Pittsburgh, PA 15213-3890 34 2002 by Carnegie Mellon University Process Maturity Profile of the Software Community 2002 Update - SEMA.8.02