EMPIRICAL STUDY ON CAPABILITY MATURITY MODEL INTEGRATION (CMMI) FRAMEWORK IMPLEMENTATION INSTITUT PENGURUSAN PENYELIDIKAN UNIVERSITI TEKNOLOGI MARA 40450 SHAH ALAM, SELANGOR MALAYSIA BY: SHUKOR SANIM BIN MOHD FAUZI AUGUST 2009
EMPIRICAL STUDY ON CAPABILITY MATURITY MODEL TNTEGRATION (CMMI) FRAMEWORK IMPLEMENTATION BY SHUKOR SANIM BIN MOHD FAUZI AUGUST 2009
Tarikh : 3 OGOS 2009 No. Fail Projek : 600-IRDCISTIDANA 513IDst (5912008) Penolong Naib Canselor (Penyelidikan) Institut Pengurusan Penyelidikan Universiti Teknologi MARA 40450 Shah Alam Yang Berbahagia Prof., LAPORAN AKHIR PENYELIDIKAN "EMPIRICAL STUDY ON CAPABILITY MATURITY MODEL INTEGRATION (CMMI) FRAMEWORK IMPLEMENTATION" Merujuk kepada perkara di atas, bersama-sama ini disertakan 3 (tiga) naskah Laporan Akhir Penyelidikan bertajuk "Empirical Study On Capability Maturity Model Integration (CMMI) Framework Implementation". Sekian, terima kasih. Ketua Projek Penyelidikan MOHD FAUZI
PROJECT TEAM SHUKOR SANIM BlN MOHD FAUZI Project Leader
LIST OF PUBLICATIONS The progress of this research produced 5 papers, which provided the feedback to the research. 1. Shukor Sanim M. Fauzi, Nuraminah R., M.Hairu1 Nizam M. Nasir."The Extent of Software Process Improvement Models in Malaysia", International Multiconference on Engineers and Computer Sciencetist, 18-20 March 2009, Hong Kong, China. ISBN 978-988-17012-2-0 pp 1075-1080 (Indexing: ISI-Thomson, INSPEC, EBSCO, ISI-Thomson, Cambridge Scientific Abstract, DBLP, SCOPUS, Computer Science Bibliographies) 2. Shukor Sanim M. Fauzi,Nuraminah R. M. Hairul Nizam M. Nasir. "Software Process Improvement Models Implementation in Malaysia", Proceeding of The Third International Conference on Systems, Computing Science and Software Engineering. 3. Shukor Sanim M. Fauzi,Nuraminah R. M. Hairul Nizam M. Nasir. "Software Process Improvement Models Implementation in Malaysia", Book Chapter - Systems, Computing Science and Software Engineering, Springer-Verlag (Will be send by the conference organizer by August 2009) 4. Shukor Sanim M. Fauzi, et. al. "The Implementation of Software Process Improvement Models". Accepted for publication in International Review on Computer and Software. Appear online (May 2009). (Indexing: Cambridge Scientific Abstract, EBSCO, COMPENDEX, Copernicus) 5. Shukor Sanim M. Fauzi, Nuraminah R., M. Hairul Nizam M. Nasir (2008). "The Implementation of Software Process Improvement Models and Other Associated
Frameworks in Malaysia". 4th Malaysian Software Engineering Conference 2008, 16-17 December 2008, Terengganu,Malaysia. (ISBN 978-983-2888-82-6) pp 391-394
ACKNOWLEDGEMENT In researching and writing this report, the author has received support and assists from a lot of people. The author would like to truthfully extend his sincere gratitude to the followings: 1. To all the respondents from selected IT companies in Malaysia. They have giving me good feedback at all time during performing the research... 11. Universiti Teknologi Mara via Research Management Institute for giving the financial support in order for me to complete the research.
ABSTRACT Software Process Improvement (SPI) is important in the sense that it can help the Malaysian software organizations to enhance their capabilities in software engineering in order to compete with other organizations around the world. It also can help to achieve the vision of MSC Malaysia which is "to realize Malaysia as a global hub and preferred location for ICT and multimedia innovations, services and operations". There is a number of studies have been conducted on SPI in other continents such as Europe, Australia, New Zealand and North America. However, there is still lack of published studies on SPI or software best practices in Southeast Asia, especially Malaysia itself. Surveys in Malaysian software organizations which have an experience in initiating and conducting software process improvement initiative have been done. The surveys were conducted by using paper survey and also online survey. Two different type of survey was conducted because of the different risk or constraint of cost, time, and distance can be overcome. Most of the respondents were professionals who are directly involved in SPI initiative. In the data analysis, the respondents are characterized according to the attribution of weight to their answers. The research has found that the IT Malaysian organizations are employing CMMI, Prince 2, PMBOK and also Six Sigma. This study shows that Malaysian organizations have already take an initiative to adopt software process improvement models in their organization. Next, the research also found three main variables that are really important to ensure the success of CMMI implementation which are; commitment, continuous improvement and also management. Keywords: Software Process Improvement, Capability Maturity Model Integration, Software process, Software Quality
TABLE OF CONTENTS CHAPTER TITLE LIST OF PUBLICATIONS ACKNOWLEDGEMENT ABSTRACT LIST OF TABLES PAGE iv vi vii X LIST OF FIGURES LIST OF ABBREVIATIONS LIST OF SYMBOLS 1 INTRODUCTION 1.0 Introduction 1.1 Project Background 1.2 Problem Statement 1.3 Project Objectives 1.4 Project Scope 1.5 Significance of the Project 2 LITERATURE STUDY 2.1 Capability Maturity Model Integration 2.1.1 Categories of CMMI Process Area 2.2 Other Software Process Models in Place - IS0 - Six Sigma - Trillium 2.3 Conclusion 3 PROJECT METHODOLOGY 3.1 Flow Chart of Research Methodology 3.1.1 Literature Review 3.1.2 Participants 3.1.3 Data Gathering xi... Xlll xiv 1 1 2 3 3 3 4 5 5 6 9 9 12 14 16 17 17 18 18 18
CHAPTER TITLE - Section A - Section B - Section C - Section D 3.1.4 Analysis of Data PAGE 18 4 PROJECT DISCUSSION 4.1 Demographic Information 4.2 Statistics of Malaysian Companies with CMMI 4.3 Implementation of Service Management Framework 4.4 Survey Analysis on Other Framework 4.5 CMMI Initiatives and Progress 4.6 Implementation Progress in CMMI Functions/Process Areas 4.7 Factors Contributing to CMMI Implementation Success 4.7.1 Factors Contributing to CMMI Implementation Success - Organization Management 4.7.2 Factors Contributing to CMMI Implementation Success - Client/Customer Organization 4.7.3 Factors Contributing to CMMI Implementation Success - IT Department 4.7.4 Factors Contributing to CMMI Implementation Success - External Consultant 4.8 CMMI Effectiveness 4.9 Benefits Provided by CMMI 5 CONCLUSION 6 REFERENCES
LIST OF TABLES TABLE NO. TITLE PAGE Categories of CMMI Process Areas Q 1 (I) On Respondent Position Q2(I) On Respondent's Organization Ownership Q3(I) On Respondent's Organization Budget Q4(I) On Respondent's Organization Size Q5(1) On Respondent's Organization Operation Years Total Score of the Respondents Malaysian Companies with CMMI Level 2 Malaysian Companies with CMMI Level 3 Malaysian Companies with CMMI Level 5
LIST OF FIGURES FIGURE NO. 3.1 4.1 TITLE PAGE Flow Chart of Research Methodology 17 Statistics of Malaysian Companies with CMMI Certification 26 Implementation of IT Service Management Framework 33 Implementation of other Framework 34 Reason for CMMI Implementation 35 Roles in the CMMI Implementation 35 Total CMMI Implementation Budget 36 Percentage of the CMMI Budget Spent On 37 Percentage of the CMMI Budget Spent On 37 Percentage of the CMMI Budget Spent On 38 Implementation Progress for CMMI Level 2 39 Implementation Progress for CMMI Level 3 40 Implementation Progress for CMMI Level 4 40 Implementation Progress for CMMI Level 5 4 1 Factors Contributing to CMMI Implementation Success - Organization Management 42 Factors Contributing to CMMI Implementation Success - Client/Customer Organization 43 Factors Contributing to CMMI Implementation Success - IT Department 44 Factors Contributing to CMMI Implementation Success - External Consultant 45 CMMI Effectiveness 46
xii Benefits Gained from CMMI 47
LIST OF ABBREVIATIONS CMMI~~ Capability Maturity Model Integration
xiv LIST OF SYMBOL CMMI~~ Capability Maturity Model Integration
CHAPTER 1 INTRODUCTION 1.0 Introduction The processes for large-scale software development can themselves be quite large and complex which may involve many software engineers, programmers and designers. As a result, they are often hard to define, difficult to understand, and sometimes even harder to establish or standardize. Software process improvement (SPI) emerges to tackle these process-related difficulties. The underlying assumption behind SPI and the many SPI frameworks is that quality process will influence the quality of product. We should therefore focus our improvement efforts on the software process in order to improve software quality. The following definitions are given to describe quality and software process. Quality - Kitchenham mentions that quality "is hard to define, impossible to measure, easy to recognize" [I]. Most SPI frameworks specify a measurable quality indicator for the process.
Software process - "...a set of partially ordered process steps, with sets of related products, human and computerized resources, organizational structures and constraints, intended to produce and maintain the requested software products" [2]. There is a number of studies have been conducted on software process improvement by using above models and methodologies in other continents such as Europe [3], Australia, New Zealand [4] and North America [5]. However, there is still lack of published studies on software process improvement or software best practices in Southeast Asia, especially Malaysia itself. 1.1 Project Background IT Malaysian company must now move to the new era of software development by comprising the quality aspect in their software process. It can be done by taking several certifications such as CMMI, IS0 and any other IT Services management model that we have in place; Information Technology Infrastructure Library (ITIL), AS8018 ICT Service Management, HP ITSM (IT service Management Reference Model), MOF (Microsoft Operations Framework), IBM SMSL (Systems Manage Solution Lifecycle, CobiT (Control Objectives for Information and related Technology). In short coming years in Malaysia, we believe all the bidders for the government tender for information technology infrastructure should be recognized with at least one IT certification that has been mentioned above. There is a number of studies have been conducted on software process improvement by using above models and methodologies in other continents such as Europe [3], Australia, New Zealand [4] and North America [5]. However, there is still lack of published studies on software process improvement or software best practices in Southeast Asia, especially Malaysia itself.
1.2 Problem Statement The growing focus on software development has highlighted the importance of software process. Over the last few years, many large foreign organizations like Motorola, Thales, Lockheed Martin and other organizations have adopted the Capability Maturity Model Integration (CMMI) to provide effective software process management and control of the software they developed. What about our IT Malaysian company in Malaysia? This research aims to determine the extent to which Malaysian organisations have implemented the CMMI framework and associated processes, to gather practitioners' perceptions of the effectiveness of the framework, and to identify the critical success factors. 1.3 Project Objectives The objective of the project has been alienated into three (3). Project objective will confer a thought for the author on what the author will do. Three (3) project objectives that have been identified are: I. To determine the extent to which Malaysian organizations have implemented the CMMI framework and associated processes.. 11. To gather practitioners' perceptions of the effectiveness of the framework iii. To identify the critical success factors of the framework in IT Malaysian company 1.4 Project Scope The research will focus on the IT organization which already implemented and currently implemented CMMI framework in their organization.
1.5 Significance of the Project The significance of the project is, it can help to measure the level of IT Malaysian organizations in implementing the CMMI model in their organization. Besides, the result can be a first step to develop a "Malaysian-way" framework for software quality. Next, the research also can help the Asian region to grow or pay attention to the software process improvement research. There is still lack of papers from Asian region.
CHAPTER 2 LITERATURE STUDY 2.1 Capability Maturity Model Integration (CMMI) A model is a simplified representation of the world. CMMI contains the essential elements of effective processes for one or more bodies of knowledge. These elements are based on the concepts developed by Crosby, Deming, Juran, and Humphrey [5][6][7][8]. CMMI models provide guidance to use when developing processes. CMMI models are not processes or process. descriptions. The actual processes used in an organization depend on many factors, including application domain(s) and organization structure and size. In particular, the process areas of a CMMI model typically do not map one to one with the processes used in an organization. A process is a leverage point for an organization's sustained improvement. The purpose of CMMI is to provide guidance for improving an organization's processes and the ability to manage the development, acquisition, and maintenance of products or services. CMMI places proven approaches into a structure that helps an organization appraise its organizational maturity or process area capability, establish priorities for improvement, and implement the improvements.
CMMI contains and is produced from a framework that provides the ability to generate multiple models and associated training and appraisal materials. These models may reflect content from bodies of knowledge such as Systems Engineering, Software Engineering, Integrated Product and Process Development) in combinations most useful to an organization such as CMMI-SEISW, CMMI-SEISWIIPPDISS. 2.1.1 Categories of CMMI Process Areas An organization must be aware of the interactions that exist among CMMI model components to apply the model in a useful and productive way. Process areas can be grouped into four (4) categories. Table 2.1 describes the categories of CMMI Process Areas.
Table 2.1: Categories of CMMI Process Areas (Source: Software Engineering Institute) Process Management process areas contain the cross-project activities related to defining, planning, resourcing, deploying, implementing, monitoring, controlling, appraising, measuring, and improving processes. The Process Management process areas of CMMI are as follows: Organizational Process Focus. Organizational Process Definition. Organizational Training. Organizational Process Performance. anagement activities related to planning, monitoring, and ontrolling the project. The Project Management process reas of CMMI are as follows: Project Planning. Project Monitoring and Control. Supplier Agreement Management. Integrated Project Management for IPPD (or Integrated Project Management). Risk Management. Integrated Teaming. Quantitative Project Management.
Table 2.1: Categories of CMMI Process Areas (Source: Software Engineering Institute) maintenance activities that are shared across engineering disciplines such as systems engineering and software engineering. The six (6) process areas in the Engineering process area category have inherent interrelationships. These interrelationships stem from applying a product development process rather than discipline-specific processes such as software engineering or systems engineering. The Engineering process areas of CMMI are Requirements Development. Requirements Management. Technical Solution. Product Integration. Verification. Validation. oduct development and maintenance. The Support ocess areas address processes that are used in the context performing other processes. In general the Support ocess areas address processes that are targeted towards the oject, and may address processes that apply more nerally to the organization. For example, Process and roduct Quality Assurance can be used with all the process areas to provide an objective evaluation of the processes and
Table 2.1: Categories of CMMI Process Areas (Continued) (Source: Software Engineering Institute) 2.2 Other Software Process Models in Place IS0 (International Organization of Standardization) - Based on John P. Barrett, The International Organization of Standardization (ISO) is an international organization that is based in Geneva which originally released the IS0 standard in 1987. There are basically seven standard in the IS0 9000 series which are IS0 9000, IS0 9000-3, IS0 9001, IS0 9002, IS0 9003, IS0 9004, IS0 9004-2. The IS0 9000, IS0 9000-3, IS0 9004, IS0 9004-2 is about guidelines that can be used to understand how to develop a quality management system in an organization and how to understand the IS0 series standard. The IS0 9001, IS0 9002 and IS0 9003 are the standard that organization use to audit and access their quality management system.