An Empirical Study of Software Process Maturity, TQM Practices and Organizational Characteristics in Taiwanese Companies

Transcription

1 Total Quality Management Vol. 16, No. 10, , December 2005 An Empirical Study of Software Process Maturity, TQM Practices and Organizational Characteristics in Taiwanese Companies SHIANG-HENG CHEN, HOUN-GEE CHEN & DAVID C. YEN Department of Information Management, National Chung Cheng University, Taiwan, Institute of Technology Management, National Tsing-Hua University, Shinchu 300, Taiwan and Department of Decision Sciences and Management Information Systems, Miami University, USA ABSTRACT In this paper, the authors examine the relationship between TQM-based organizational culture, organizational characteristics, IS department characteristics, and the modified software maturity process (MCMM). The result shows that the use of data, continuous improvement, customer-orientation, and the job challenges are significantly correlated with the MCMM levels. The size of the organization, IS department size, and IS department roles also have impacts on MCMM level. KEY WORDS: Capability Maturity Model (CMM), Information Systems (IS), International Standards Organization (ISO), Software Engineering Institute (SEI), System Development Life Cycle (SDLC), Total Quality Management (TQM) Introduction The proliferation of information systems (IS) in an organization has been known for many decades. It becomes even more important for an organization to adopt IS and to use IS as a competitive weapon in today s global marketplace. It was reported that the quality of IS depends not only on the company s technical tools but also on how well the company is managed. To take full advantage of IS, the research on how to improve software (IS) quality has long been a hot issue. In 1986, the Software Engineering Institute (SEI) of Carnegie Melon University developed a software process maturity framework known as the Capability Maturity Model. The maturity model (CMM) provides a guiding framework to examine the whole software developing process and means to improve the quality in the process. It also describes an evolutionary improvement path for software organization from an ad hoc path to a mature path. Correspondence Address: David C. Yen, Department of Decision Sciences and Management Information Systems, Miami University, Oxford, Ohio 45056, USA. yendc@muohio.edu Print= Online=05= # 2005 Taylor & Francis DOI: =

2 1092 S.-H. Chen et al. The CMM defines five maturity levels, and each level contains several key process areas (KPA) to accomplish. The key processes define what goals to achieve in each level, so an organization can follow the content of KPA to establish the foundation to evolve to the next maturity level. With the aid of CMM, a company can continuously improve its software process. Many studies have found that the software process is one of the dominant factors influencing software quality. The software process is a set of activities, methods, practices, and transformations that people use to develop and maintain software and its associated products. Both the software engineers and the user should work together to ensure the success of the software process. SEI has also made several tools for software process maturity assessment like the maturity questionnaire, the CMM appraisal framework, etc. The stage structure of the CMM is based on principles of product quality, and it defines an evolutionary path for software process improvement (Paulk et al., 1993). Since the 1930s, the concept of product quality has evolved to the concept of quality inspection, quality control, total quality control, and finally total quality management (TQM). TQM was formerly known as Total Quality Control (TQC), which was a prevailing management practice in the organization. TQM is both a philosophy and a set of guiding principles that represent the foundation for a continuously improving organization. TQM is the application of quantitative methods and human resources to improve the material and services supplied to an organization, and the degree to which the needs of the customers are met. TQM integrates fundamental management techniques, existing improvement efforts, and technical tools under a disciplined approach focused on continuous improvement. Its strategy tries to achieve total quality throughout the entire business, not just in the product. After unfolding numerous successful stories in manufacturing, the TQM concept has recently been adapted in other areas such as service industries, government, and schools as well. In essence, TQM is defined by: (1) continuous improvement; (2) customer-driven quality standard; (3) management commitment; (4) company-wide involvement; (5) quality chain; and (6) prevention orientation. As such, both CMM and TQM are based on the concept of continuous improvement. Many studies reported that the organizational culture influenced the process of TQM implementation, and TQM could be an integrated element of an organizational culture. Researchers also suggested the adoption of the TQM concept in the system development life cycle (SDLC) and software development. Tata & Prasad s (1998) study revealed that organizations with low uncertainty avoidance and low power distance favoured the implementation of a TQM program while the companies with flexibility-oriented cultures and organizational structures resulted in better TQM programs. In addition, Zeitz & Johannesson s (1997) study showed that organizational culture dimensions of communication, trust, social cohesion, innovation, job challenge are significantly correlated with the implementation of TQM in the areas of management support, supplier relationship, supervision, customer-orientation, use of data, and continuous improvement. Organization size, manager s attitude towards information systems, the role of the information system department, the information system maturity level, and the user participation of information systems are shown to be significantly correlated with the software process maturity level (Chen, 1999; Hsieh, 1996). Paulk s (1995) study revealed that ISO-9001 can be mapped to some key process areas in CMM. Hsieh s (1996) research conducted an extensive study examining the CMM practices in Taiwan s companies and the results from that study suggested that

3 Software Process Maturity, TQM Practices and Organizational Characteristics 1093 CMM s five levels be further re-categorized into ten levels. This modified taxonomy (MCMM) in fact, provides a better differentiation in classifying the CMM practices and has been adopted in subsequent research such as the studies from Li et al. (2002, 2003)] and Wu et al. (2002). This proposed study aims at examining the relationship between TQM, information maturity, and MCMM. Organizational culture can be classified as three levels (Ott, 1989). The first level includes many tangible artefacts and behaviour in an organization such as uniform style, office ornamentation, management style, etc. The second level is about the core value or belief in a department or the whole organization. It can be assessed by interview and observation. The third level is the basic assumption in every member of the organization. All other organizational characteristics are based on this basic assumption. Organizational characteristics like organization size and organization type were found to have an impact on the software process capability (Chen, 1999). And the following factors such as the way people do things in an organization, the communication style in an organization, the resolution of conflicts, and the management of system development projects were also reported to affect the software process and the success of system development projects. Our study looks into the spectrum of the organization culture and examines its impacts on the software development process. Although organizational culture was often viewed as a qualitative construct, studies concerning quantitative analysis to measure organization culture have emerged. Der Post et al. (1997) reported a rather complete study on this. They summarized all aspects of organizational culture and classified it into 15 constructs. A questionnaire consisting of 97 questions was developed and was a good instrument tool for its measurement. Studies showed that the organization s culture and TQM are closely related (Zeitz & Johannesson, 1997; Tata & Prasad, 1998). The results indicated that the communication between the organization hierarchy, the trust of members, and the social coherence influenced TQM success (Zeitz & Johannesson, 1997). The CMM developed by SEI has been adopted as a basic framework in many software process tools. CMM is guidance for organizations to know what to do to improve their software process. TQM is mainly an organizational management technique, and its core concept is to continually improve the quality of products or services. TQM and CMM are similar in requiring evolutionary improvement on processes. Research Model This study hypothesizes that organizations with TQM experience should have a better software process maturity level. A TQM-based organizational model is used to describe the TQM practices and its embedded organizational culture. The organizational culture, organizational characteristics, and TQM practices will influence the software process maturity level as shown in Figure 1. The constructs of TQM practices and organizational culture comprise 12 factors suggested by Zeitz & Johannesson (1997). The organizational size includes organization size, the number of employees, and annual revenue; while the IS department size defines the annual IS budget, number of employees in the IS department, and number of software engineers in the IS department.

4 1094 S.-H. Chen et al. Figure 1. Organizational culture, organizational characteristics, and software process maturity levels Questionnaire Design The questionnaire used in this study is composed of two parts. The first part consists of the questions regarding the organizational culture and characteristics, the roles of the IS department, and TQM practices. The 75 questions are included in part one of the questionnaire and grouped into the following categories: 10 items for the company s background, 32 items for TQM practices and measures, 26 items for organizational culture, and 7 items for the roles of the IS department. Each item is measured using a 5-point Likert-type scale with a range from strongly agree to strongly disagree. The second part is developed to measure the CMM practices and assess the software process maturity levels. For part two of the questionnaire, items are drawn from the SEI s maturity model, which contain all yes/no questions measuring the whole 18 KPAs of the CMM to assess the software process maturity levels (Zubrow et al., 1994). Since the CMM implementation is a complicated and continuously improved project by nature, it becomes a common practice that a dedicated person (senior software engineer in most cases) is assigned to lead the CMM implementation team. As a result, the complete questionnaire (including part one and part two) was mailed to the IS department manager for his/her attention. The IS department manger will take care of the first part of the questionnaire and s/he was instructed to distribute the second part of the questionnaire to the right person who leads the CMM implementation team (in most cases, this leader is the senior software engineer) to complete the survey. It is reasonably assumed that the implementation team leader with an in-depth and hand-on experience in CMM practice should be able to provide more reliable/accurate answers for survey questions in part two. To observe the integrity of this proposed study, only the matched-paired questionnaires (parts one and two) received were compiled, and then analysed in this study. Sample The sample was drawn from the top 500 companies of the 1999 and 2000 CommonWealth magazine special editions. In the sample, 300 are manufacturing companies and 200 are service-related companies.

5 Software Process Maturity, TQM Practices and Organizational Characteristics 1095 Research Findings Fifty-nine matched questionnaires (IS department manager matched IS department software engineers) were received from the mail survey, which represented a response rate of 11.8%. The reliability test (Cronbach s alpha) for each construct is higher than 0.7: TQM (a ¼ ), organizational culture (a ¼ ), the role of IS department (a ¼ ), SEI maturity questionnaire (a ¼ ). Table 1 is the profile of companies and IS departments. Table 1. Profiles of companies and IS department Company characteristics Number % Industry type Manufacturing Service Number of employees Below Above ISO-9000 series certification Yes No Annual IS department budget (in NT dollars) Below 10 million million million Above 100 million IS department employee number Below Above Number of software engineers Below Above Software development and management staff average experience 15 years years Above 10 years Software integration and testing staff average experience 15 years years Above 10 years Experience of development tools of software development staff above 1 year Below 50% %75% %100% Note: In the IS departments profiles, some companies did not respond the questions so the total number is smaller.

6 1096 S.-H. Chen et al. Most of the companies are large companies with employee number Eightythree percent of companies had ISO-9000 certification. The average working experiences of software engineers is between 1 to 5 years. The annual budget of IS departments is below NT$100 million, and the average number of employees working in the IS departments is below 50. The percentage of yes items of the SEI maturity questionnaire was tallied and the original CMM five levels were further divided into ten levels with 10% each (MCMM). The taxonomy of the MCMM maturity level is given in Table 2 along with the results of two previous studies. Table 2 shows a steady improvement of software process maturity level in Taiwan s large companies. The results of the year 2000 survey indicate that about 75% of the companies depict a level 5 and above maturity level and these companies have achieved at least half of the key process areas of the MCMM. Spearman s correlation matrices of TQM practices versus the software process maturity level and organizational culture versus the software process maturity level are given in Tables 3 and 4, respectively. Three of the TQM practices customer orientation, use of data, and continuous improvement are significantly correlated with the MCMM maturity level. As IS departments are becoming more concerned about their customers needs, they will do more in improving the software process maturity level for quality service. In addition, the more the organization members practice the use of data in managing and controlling the processes, the higher the MCMM level the organization has. Continuous improvement is the foundation of TQM that also has a growing and evolutionary path. When applied to software projects, continuous improvement ensures the success of the projects. Spearman s correlation matrices of organizational characteristics, IS department characteristics and MCMM level are given in Tables 5 7. Table 5 shows that larger organizations have higher software process maturity levels. As for the role of the IS department, the contact with user departments, the relationship with user departments, and the alignment with the corporate goals are found to be significantly correlated with the MCMM level (Table 6). As for IS departments, the number of employees and software engineers are positively correlated with the MCMM levels (Table 7). The experience using the software development tools also influences the MCMM level (Table 7). A chi-square test is used to verify the correlation of the MCMM level and the ISO-9000 certification. In the test, we first regroup the MCMM levels into five categories to ensure sufficient samples in each category. The result indicates that there is no significant Table 2. MCMM level of large companies in Taiwan (percentage) MCMM level Year MCMM level average in 1996: 3.96, STD. Dev MCMM level average in 1999: 4.89, STD. Dev MCMM level average in 2000: 6.03, STD. Dev

7 Management support Table 3. Correlation matrix of TQM practices and MCMM level Employee suggestion Supervision Supplier relationship Customer orientation Use of data Continuous improvement MCMM level Correlation coefficient P-value Significance level Software Process Maturity, TQM Practices and Organizational Characteristics 1097

8 1098 S.-H. Chen et al. Table 4. Correlation matrix of organizational culture and MCMM level Communication Trust Job challenge Innovation Social cohesion MCMM level Correlation coefficient P-value Significance level correlation between MCMM level and ISO 9000 (Table 8). Since most of the participating companies are manufacturing firms, the motive to get an ISO-9000 certificate may be different from that of software process improvement. Conclusions Four dimensions (customer orientation, use of data, continuous improvement, and job challenge) of TQM practice and organization culture are actually correlated with MCMM level. Customer-centric and continuous improvement are two major concepts of TQM that also affect the software maturity process. The finding is, in fact, a starting point for future research to conduct similar research on organizational culture and software processes. In addition, proprietors should also push the concept of TQM to be implemented through the whole organization, and consequently let the TQM concept become part of an integrated organizational culture. As for the IS department roles, this study finds that a better relationship between IS departments and user departments may result in a higher software process maturity level. The results from this study indicate that working closeness with the user is a crucial factor for the success of CMM implementation. In IS departments, people should know the critical importance to continuously improve/enhance the quality of their services and products. Further, people in IS departments should understand that the improvement of the software and processes is a long-lasting effort, and little progress in each improvement stop needs to be achieved to reach a higher software process capability. As for the IS department characteristics, the results obtained supports the fact that dedicated and experienced personnel (e.g. software engineers) do play a significant role in CMM implementation. This evidence also suggests that a continuous training and development programme should exist/be maintained to provide the concept of skill building Table 5. Correlation matrix of organizational characteristics and MCMM level Number of employees Total capital Annual sales revenue Company lasting time MCMM level Correlation coefficient P-value Significance level 0.05.

9 Influence of information related decisions in companies Table 6. Correlation matrix of IS department role and MCMM level Management support to IS department Capability of negotiation with user departments Extent of contact with user departments Relationship with user departments Fitness with the corporate goal Extent of cooperation of user departments MCMM level Correlation coefficient P-value Significance level Software Process Maturity, TQM Practices and Organizational Characteristics 1099

10 1100 S.-H. Chen et al. Table 7. Correlation matrix of IS department characteristics and MCMM level IS department annual budget Number of employees in IS department Number of software engineers Average experience of software development and management staff Average experience of software integration and testing staff Experience of development tools of software development staff above 1 year MCMM level Correlation coefficient P-value Significance level 0.05.

11 Software Process Maturity, TQM Practices and Organizational Characteristics 1101 Table 8. Chi-square test of ISO-9000 certification and MCMM level Chi-Square df Chi-Square (a ¼ 0.05) Test result and knowledge sharing. External consultants could be a good source of help in the implementation process. The improvements of software processes in smaller organizations, however, are more difficult than large organizations according to the prior study of (Brodman & Johnson, 1994). This manuscript finds that larger organizations in general have higher software process maturity levels. One of the possible reasons is that more resources contribute to the improvement of software processes in large organizations. References Brodman, J. G. & Johnson, D. L. (1994) What small business and small organizations say about the CMM, Proceedings of the 16th International Conference on Software Engineering, IEEE Computer Society, pp Chen, Y. S. (1999) The relationships between organizational characteristics, information system maturity, IS department characteristics and software process maturity. Unpublished master thesis, Department of MIS, National Chung-Cheng University, Taiwan. Hsieh, K. D. (1996) Software process maturity and its related factors of large companies in Taiwan. Unpublished master thesis, Department of MIS, National Chung-Cheng University, Taiwan. Li, E. et al. (2002) Software process management of top companies in Taiwan: a comparative study, Total Quality Management, 13(5), pp Li, E. et al. (2003) A longitudinal study of software process management in Taiwan s top companies, Total Quality Management, 14(5), pp Ott, J. S. (1989) The Organizational Culture Perspective (Pacific Grove: Brooks-Cole). Paulk, M. C. (1995) How ISO 9001 compares with the CMM. IEEE Software. Paulk, M. C. et al. (1993) Capability Maturity Model for Software, Version 1.1. Software Engineering Institute, CMU/SEI-93-TR-024, February Tata, J. & Prasad, S. (1998) Cultural and structural constraints on total quality management implementation, Total Quality Management, 9(8), pp Van der Post, W. Z. & de Coning, T. J. (1997) An instrument to measure organizational culture, South African Journal of Business Management, 28(4), pp Wu, M. et al. (2002) Software project process management maturity and project performance: an examination of Taiwan s software companies, Journal of International Technology and Information Management, 11(1), pp Zeitz, G. & Johannesson, R. (1997) An employee survey measuring total quality management practices and culture, Group & Organization Management, 22(4), pp Zubrow, D. et al. (1994) Maturity Questionnaire. Software Engineering Institute, CMU/SEI-94-SR-7, June. Bibliography Dahlgarrd, J. J. et al. (1998) Fundamentals of Total Quality Management (Blackpool, UK: Chapman & Hall). Flood, R. L. (1993) Beyond TQM (Chichester, UK: Wiley). Herbsleb, J. et al. (1997) Software quality and the capability maturity model, Communications of the ACM, 40(6), pp Humphrey, W. S. et al. (1991) Software process improvement at Hughes Aircraft. IEEE Software, 8(4), pp

12 1102 S.-H. Chen et al. Kanji, G. K. & Asher, M. (1993) Total Quality Management: A Systematic Approach (Madras, India: Productivity Press (India) Ltd). Major, J. et al. (1998) Meeting the software challenge: strategy for competitive success. Industrial Research Institute. SEI (2000) Process Maturity Profile of the Software Community 1999 Year End Update. Software Engineering Institute, CMU SEI Report, Mar Wohlwend, H. & Roseenbaum, S. (1994) Schlumberger s software improvement program. IEEE Transactions on Software Engineering, 20(11), pp Wu, F. S. (1992) The empirical study of relationship between manager characters, organization structure, status, role, and performance of IS function. Unpublished master thesis, Department of Business Management, National Sun Yat-sen University, Taiwan.