Software Process Improvements in a Very Small Company ITA RICHARDSON AND KEVIN RYAN University of Limerick, Ireland

Transcription

1 QUALITY MANAGEMENT Following any software process assessment, it is important that an organization implement a process improvement strategy to produce a well-defined software process. In theory, this is simple; however, it is much more difficult in practice. For the large company, it may be possible to devise an improvement strategy by investing in the development of action plans. For the very small company, however, this requires yet another investment from a much smaller purse. The authors of this article have developed a generic method based on quality function deployment that can be used by small software development companies to devise their own software process improvement strategy. The authors implemented the method, the software process matrix (SPM), in two very small software development companies in Limerick, Ireland. This article discusses the implementation in one of these companies. The authors compare the processes in the company at the start and the end of the research period, noting changes that occurred as a result of their intervention. The resulting changes to organization, customer management, and project management processes are emphasized. The authors conclude that while the action plan and pressure for change are integral to software process improvement, the software developers themselves are fundamental to the success of any improvement strategy. Key words: customer management processes, improvement strategy, organizational development, project management processes, quality function deployment, software process matrix Software Process Improvements in a Very Small Company ITA RICHARDSON AND KEVIN RYAN University of Limerick, Ireland INTRODUCTION It is recognized that available software process models do not provide a road map for developing an improvement strategy after a software process assessment (Peterson 1995a; Draper et al. 1995). Although some improvement models have been presented, for example, the IDEAL model for the Software Engineering Institute s Capability Maturity Model (Peterson 1995b), as recently as 1998, Debou and Kuntzmann-Combelles note that due to lack of documentation on the post-assessment phase, assessments are often being performed as a oneshot event without connection to any improvement strategy. For the large company, it may be possible to devise such a strategy by investing in the development of action plans. For the very small company, however, this requires yet another investment from a much smaller purse. The authors have developed a generic method based on quality function deployment (QFD) that can be used by small software development companies in the derivation of their software process improvement strategies. The authors implemented the method, the software process matrix (SPM), in two small software development companies in Limerick, Ireland: Computer Craft and DataNet (both pseudonyms). This article discusses this implementation and consequent changes to the software process in Computer Craft. 23

2 FIGURE 1 Four-phase model for manufacturing Design characteristics Parts characteristics Key process operations Production requirements Customer attributes House of quality Design characteristics Parts deployment Parts characteristics Process planning Key process operations Production planning 2001, ASQ WHAT IS THE SOFTWARE PROCESS MATRIX? The SPM is a method used to establish an improvement strategy based on QFD. QFD has been defined as a way to assure the design quality while the product is still in the design stage (Akao 1990) and as a quality system focused on delivering products and services that satisfy customers (Mazur 1994). Used mainly in manufacturing, its use has spread to services and software development. QFD originated in Japan and is now used in many countries worldwide. In order to collect and maintain the voice of the customer throughout the production life cycle, QFD usually uses a series of matrices that convert the customer s voice into a final product. Different models are available for use, and according to Cohen (1995), the four-phase model adapted by the American Supplier Institute and containing four matrices (Hauser and Clausing 1988) is probably the most widely described and used model in the United States. This research developed the SPM based on the first matrix of this model, the house of quality. The Four-Phase Model In the four-phase QFD model there are four matrices, as shown in Figure 1. These are: 1. Product planning matrix (house of quality) 2. Parts deployment 3. Process planning 4. Production planning Initially, the voice of the customer is collected, and the relative importance of each customer requirement is measured. In the house-of-quality matrix, these FIGURE 2 Outline house of quality Customer requirements Interrelationships Design characteristics Relationships Importance of design characteristics Importance of requirements requirements are used to identify design characteristics that have the greatest impact on customer requirements. Although QFD consists of many matrices, the main focus is often this matrix, as using it alone can have a significant effect on the product development process (Fortuna 1988). The matrix is normally broken down into six rooms, as shown in Figure 2: 1. Customer requirements (WHATs) 2. Design characteristics (HOWs) 3. Overall importance of customer requirements 4. Relationships between customer requirements and design characteristics 5. Importance of design characteristics 6. Interrelationships between design characteristics 2001, ASQ 24 SQP VOL. 3, NO. 2/ 2001, ASQ

3 Overall Importance of Customer Requirements The overall importance of each customer requirement determines its priority level. Generally, the accepted numerical data included in this calculation are: Current status of product in providing the requirements Measurement of competitive analysis Proposed status of future product in providing the requirements Market leverage specific market information that should be considered For the current status measurement analysis and proposed status, the value used in this project ranged from 1 to 5, where 1 represented doing badly and 5 represented doing very well. Market leverage had values of 1.0, when no market leverage existed, 1.2 when there was medium market leverage, and 1.5 when strong market leverage existed. These values are then used to compute: Improvement factor required to get from current to future status Importance of each customer requirement to the product Relationships Between Customer Requirements and Design Characteristics The underlying theory of using QFD matrices is that design characteristics have an effect on one or more of the customer requirements. Therefore, customer requirements vs. design characteristic relationships must be established. These relationships are often stated at four levels of effect: strong, medium, weak, and none. The values given to these are 9, 3, 1, and 0, respectively, and on QFD charts they are normally represented by the symbols: (strong), (medium), (weak), and blank. There is no scientific basis for the values, except that practitioners felt the need to have a wide gap between strong and none. It is now common to use the values as stated. As these values have emerged from more than 30 years of use in Japan and 16 years of use in the United States, Europe, and the rest of the world, they now have a universal currency. Consequently, they were used in the current research. The importance of design characteristics is calculated by using the strong, medium, and weak relationship values combined with the overall importance of customer requirements. QFD for Software Process Using QFD, the software process model is treated as the customer where software processes are the customer requirements. These processes were identified from software process literature. An abstract from the SPM is shown in Figure 3 and described in this section. In the SPM, the design characteristics are the practices that must be followed for processes to be successful. These practices were also identified from the software process literature. Examples of practices are: Test the customer s operation before software implementation Prototype or simulate critical elements of the software Maintain and trace product item histories (Richardson 1999) In developing the SPM, a total of 135 practices were identified. A crucial part of developing the SPM was identifying the relationships between processes and practices. Those that are explicitly mentioned in the literature were easily identified. For example: Practice: To ensure software s traceability to requirements has a strong effect on Process: The systematic development of detailed design Using expert opinion and various statistical techniques, other relationships between processes and practices were identified, resulting in the development and verification of the SPM and validated in industry by the authors. A sample of relationships that were identified as a result of this exercise is shown in Figure 3. These include: Practice: To map requirements to future releases has a medium effect on Process: The development of software requirements and Practice: To specify interfaces between software units has a weak effect on Process: System acceptance testing 25

4 FIGURE 3 Software Process Matrix (SPM) Development of system requirements and design Development of software requirements Development of architectural design Development of detailed design Unit testing and software integration Software testing System integration and testing System acceptance testing Importance of the HOWS 1 Percent importance of the HOWS 2 Max = 14.5 Percent importance of the HOWS Min = Importance of the WHATs 1 Specify/document system requirements 2 Map requirements to future releases 3 User requests determine requirements 4 S/W components based on requirements 5 Define interfaces of the software system 6 Assess existing designs/codes for reuse 7 Transfor S/W component into units 8 Specify interfaces between software units 9 Prototype critical software elements 10 Trace software to requirements 11 Develop and document unit verification 12 Define the order of testing of units 1 Our current product 2 Our future product 3 Improvement factor 4 Market leverage factor 5 Overall performance 6 Percent importance Max = 17.3 Percent importance Min = Standard Strong 9.0 Moderate 3.0 Weak , ASQ The SPM developed by the authors allows organizations to assess the requirements of the software process and evaluate the effect of each practice. From this, the priorities that need to be included in any software process improvement action plan are established. Self-Assessment The basis for the model is that organizations will be able to carry out a self-assessment of their own software development process. Concern about using the self-assessment approach is expressed by Stevenson (1989) after he completed a study of self-assessment by managers in organizations. He states that an individual s cognitive perceptions of the strengths and weaknesses of his organization were strongly influenced by factors associated with the individual and not only by the organization s attributes. Because the questions are focused within one organization and software process, any bias will exist in all answers from that respondent, and therefore, will not compromise the validity of the questionnaire. Furthermore, in the company studied, the authors carried out an independent assessment to validate the results. Even this independent assessment, however, is not without criticism. Research conducted by El Eman, Briand, and Smith (1996) on the reliability of software process improvement and capability determination (SPICE) assessments by independent assessors demonstrates that in the investigation of 21 process instances covering 15 processes, six of the processes did not meet their initial benchmark for interrator agreement, and another eight processes could improve their reliability. The study concludes that while some would like to believe that assessments are sufficiently reliable, our results indicate that this is not always the case. They also note that SPICE has been built upon the experience of previous assessment models, therefore expect that this disagreement would extend to other software process models. If this is the case, then bringing in an independent person may not provide more reliable results than the self-assessment questionnaire. 26 SQP VOL. 3, NO. 2/ 2001, ASQ

5 RESEARCH PROJECT The authors used action research with control groups to investigate the usefulness of the SPM in four companies. Two companies, Computer Craft and DataNet, were involved in action research over one year, and the other two companies, Software Solutions and Ríomhaire, were treated as control companies. Initially, personnel within the action research companies completed a self-assessment questionnaire. Results were entered into the SPM, and an action plan was devised within each company. Implementation of actions and outcome within Computer Craft are discussed in detail in this article, with particular emphasis on organization, customer management, and project management processes, all of which were affected as a result of the project. Computer Craft When the research began, Computer Craft had been in business for 14 years and employed 16 people. The company had two software development groups, one with responsibility for mainframe products and the other for PC products (PC development group). By the time the research was completed, however, only the PC development group remained, employing four software developers. Computer Craft had recently partnered with a U.S. company, giving Computer Craft the rights to modify and install a software package developed by this company. In 30 percent of cases, the product was localized and customized for the needs of the user. Customized features are often retained in the product. During this research, employees were interviewed and observed in their work. The authors were given access to documentation for two projects: one that was developed in the initial stages of the research, and the other that was developed toward the end of the research. STARTING SCENARIO Organization Processes Prior to the authors visiting the company, there was no emphasis on software process improvement. There were some standards within the software development group, but they were not organizationwide. This group included a software quality assurance engineer whose main focus was on testing. Management was willing to give employees time to work on the software process improvement project. When recruitment difficulties arose, however, the project was given lower priority, with software developers reassigned to development projects. As discussed later in the Lessons Learned section, this resulted in some actions being implemented. Customer Management For customized products, after initial customer contact, the customer services project manager wrote up a requirements specification. There was no particular standard in use each customer services project manager identifies the need in his or her own way, and complete information was not always included. This document was signed off by the customer and was passed to the software development group. When the group received the requirements specification, a functional specification was written. This was reviewed and accepted by the customer, with the involvement of the customer services project manager. Software developers realized that they could have quantified what needed to be done earlier in the project if someone from the development group had met with the customer. It was not uncommon for work on functional specifications to be stopped until the customer finalized its requirements. Other difficulties arose with the functional specification, where there were lots of things missing and it was different to what was tendered. There was no formal process for handling changes in customer requirements. One developer stated: There is a huge amount of information from the customer, but it is changing moveable goalposts. This meant that changing features, sometimes called feature creep, continuously impacted the final project schedule. Problems found at later stages in development, such as system test and installation, could have been avoided if requirements had been collected properly. This was frustrating for the software developers, since they were expected to meet deadlines. Normally the developer met with the customer during installation. Waiting until late in the project to meet with the customer caused problems such as the information useful to the design and development was gathered from the customer by the developer during the installation. 27

6 Project Management In Computer Craft, project deadlines were driven by the customer, with no input from the software development group. At the start of a project, the software development manager drew up a schedule with defined phases and milestones, but these were not always adhered to. Critical tasks were decided from the initial customer requirements, as were task assignments and timescales. New features, disrupting the schedule, were sometimes added. Project schedules were usually kept up to date until the project was approximately 60 percent complete. The software development manager was ultimately responsible for completion of the project. Group meetings were important within Computer Craft, mainly to keep people informed of progress on development projects. The software development group met each week, and development tasks were discussed and rescheduled if necessary. Two project meetings were also held each week. INTERVENTION USING THE SOFTWARE PROCESS MATRIX The authors circulated questionnaires on current performance to the software development group. The manager was also questioned on planned future performance and importance to the company. Average current performance was calculated from the results. The overall importance of each process was also calculated (see Figure 4 and Figure 5 for examples). For the process establishment of project teams, current performance of 2.8 is the average score obtained from the questionnaire. Planned future performance of 4.0 means that these would be established by applying an organizational procedure. The improvement factor in Figure 4, a QFD calculation, is: 1 + (planned future performance current performance) * 0.2, giving a value of A value of 4.0 means that this process is of high importance to the company. As shown in Figure 5, the overall importance was calculated by multiplying the importance to the company by the improvement factor. Highest overall importance indicates the processes that are most important to the company. The following processes, listed in order of priority, were the five most important to Computer Craft: 1. Preparation and performance of deliveries/ installations FIGURE 4 Rating customer requirements Planned Current future Improvement PROCESS performance performance factor Software deliveries and installations Establishment of project teams Configuration management FIGURE 5 Calculating overall performance Importance Improvement to the Overall PROCESS factor company importance Software deliveries and installations Establishment of project teams Configuration management Systematic planning of project activities 3. Preparation of the customer for new product release 4. Systematic development and documentation of software code 5. Systematic support of correct and efficient software To implement software process improvement, practices must change. To calculate the importance of each practice, its process importance was multiplied by the strength of the relationship between that process and practice and then totaled. Taking an illustration from the extract shown in Figure 3, define the order of testing units (no. 12), which as one strong correlation, two moderate correlations, and one weak correlation, the calculation is: 9* * * *10.37 = The three most important practices for the company that would be identified if using only this subset of the correlation matrix are: Specify/document system requirements Define interfaces of the software system Specify interfaces between software units 2001, ASQ 2001, ASQ 28 SQP VOL. 3, NO. 2/ 2001, ASQ

7 The top 10 practices identified by using the complete SPM were, in order of priority: 1. Identify this organization s product items 2. Establish product baselines for each product supplied 3. Verify that all changes to requirements are monitored 4. Specify and document system requirements 5. Collect, identify, record, and complete new customer requests 6. Assign a person with software quality assurance responsibilities 7. Identify the initial status of the product 8. Define delivery contents (media, software, documentation, documentation) to customer from subcontractor/software development group 9. Define quality criteria and metrics for the project deliverables 10. Assign responsibility for software development plan, work products, and activities This list of practices was presented at a group meeting. The software development group discussed the practices and decided that some of them should be combined. For various reasons, including cost and implementation time, they decided the company should implement the following (see Figure 6): Action 1: List all the company s products and their dependencies, based on practice 1. This was not documented in Computer Craft, although previously identified as a requirement. Action 2: Establish a procedure to specify and document system requirements (practice 4). This FIGURE 6 Extract from software process matrix Action Practice , 5 3 8, 3 4 9, 6 5 2, , ASQ would be a template rather than a procedure, which was believed would inhibit developers too much. Set up a procedure to collect, identify, record, and complete new customer requests when they come in prior to development, based on practice 5. Action 3: Based on practice 8, set up a procedure to define delivery contents for the customer from the subcontractor/development group. Establish a procedure to verify that all changes to requirements are monitored once the requirements specification has been signed off (practice 3). Action 4: Set up a procedure to define quality criteria and metrics for the project deliverables (practice 9). From practice 6, assign a person with defined software quality assurance responsibilities. Action 5: Based on practice 2, set up a procedure to establish product baselines for each product supplied so developers in Computer Craft would know minimum product to be shipped. Set up a procedure to assign responsibility for the software development plan, work products, and activities, based on practice 10. Action 6: Establish procedure to identify the initial status of the product that could be based on their current handover document (practice 7). RESULTS Organization Processes At the end of the research period, the software development group had been reduced because of natural attrition. It now consisted of the software development manager, two software engineers, and a new quality assurance engineer. The emphasis the company placed on quality assurance was evident the first quality assurance engineer was replaced almost immediately following his resignation. The role of the quality assurance engineer, however, had changed. 29

8 While she had a responsibility for testing and test plans, she also was responsible for writing procedures and improving the company s software process. Customer Management In the main project at this time, Banking Organizer, the software development manager dealt directly with the customer. He spent time on the customer site, met the customer s project manager, and attended meetings about the customer s requirements. The customer project manager became the point of contact for the software development manager, who then passed requirements to the software developers. If the software development manager could not answer the software developers questions, the project manager was available. As the project progressed, the developers had direct contact with the customer and recognized that their customer was the company for which the product was being developed. Using the available template, the software development group wrote program specifications. The customer project manager examined and signed off on the documents. Specifications containing all required information were passed between developers. Document history was updated on some documents, although the software development manager did not normally check this, and the developers updated documents off their own bat. Failure to update specifications with all modifications caused problems during testing. Having guidelines ensured that specifications were consistent, and correspondence indicated that the customer was satisfied with documentation received. One of the developers, however, stated that: The specifications were detailed much more detailed than I have ever seen before. In one way this is a good idea, but sometimes you find problems later because the specification has been taken as gospel. One section of the Banking Organizer project suffered from feature creep. Therefore, the developer found that changes that should be easy but because of this are relatively difficult. Project Management At the start of the Banking Organizer project, the software development manager produced a table of the project phases that included timescales. He considered what personnel were required for the project, taking into account other commitments. Using an automated system, he created a project plan showing planned project tasks, responsibilities, and duration. Developers were expected to enter actual time spent on the project, allowing the software project manager to track project progress. One software engineer stated that project management has improved and consequently, the Banking Organizer project was a tightly controlled project from the start. Project progress was updated regularly, as the manager was now treating the software development group as a business unit. Software development group meetings were held at the start of each week. At these meetings, the group reviewed the work done and action items closed during the previous week, and also discussed tasks proposed for the following week. This provided a formal forum for discussing any existing project-related problems. LESSONS LEARNED Following the intervention by the authors, Computer Craft was to implement six action items based on the top 10 practices prioritized using the SPM. Because a number of key personnel left the organization soon after these were identified, actions 1 and 6 were not implemented. Although action 1, which was given top priority, was not implemented, the implementation of the remaining actions ensured that the overall software process improvement project had a positive effect within the organization. The second action resulted in the development of a specification that included the requirements, functional and technical specification. The development group also improved its method of dealing with customers, meeting with them early in the development process, updating them on the project, and clarifying requests with them. This ultimately improved product testing and implementation. Because this had such a positive effect on the processes within the company, it is important that this procedure is institutionalized within the organization. When implementing action 3, two procedures were written: Installation Procedure and Build Release to Customer Services. Using these procedures, Banking Organizer implementation went smoothly, with few problems. While not formally 30 SQP VOL. 3, NO. 2/ 2001, ASQ

9 reviewed and approved, specifications were updated with modified requirements. This process must be better controlled in the future. Action 4 required Computer Craft to assign a person with software quality assurance responsibilities. This had been done, but her responsibilities had not been clarified. The replacement quality assurance engineer, taking testing as one of her responsibilities, wrote up detailed test plans and based these on the requirements specification. Her experience was lacking, and it was identified that she needed training to fulfill her role within the organization. Nonetheless, the introduction of detailed test plans helped the testing of Banking Organizer to be completed efficiently and effectively. A downside to this was that there was little emphasis placed on software item inspections within the organization, and it would be wise for the company to reconsider this situation. Her other responsibility was writing and approving procedures, a number of which were written during the research period. The other practice to be worked on in action 4 was to define quality criteria and metrics for the project deliverables. This had been partially done but was not accepted as being an organization practice. Action 5 required a procedure to establish product baselines for each product. This was not worked on. It also required that a procedure be set up to assign responsibility for the software development plan, work products, and activities. The software development manager took responsibility for the plan and improved project management within the organization. This is reflected in the improvements evident in many of the project activities. Of the 10 practices identified, three were not completed during the research period. Another three were implemented but not formalized with the organization. It is important that this be done, as the evidence from the Banking Organizer project was that, even without formalization, the software process within Computer Craft was improved. Change in Organization Processes At the end of the research period, a number of procedures, specifications, guidelines, and templates were written that streamlined the software processes. The emphasis on quality assurance and testing of the product remained. There was no interest in formal recognition of the process, as there were no market requirements for the attainment of compliance to, or assessment by, standards such as ISO 9000 or SPICE. Although Computer Craft had not identified any market requirement for such standards, the company was now better placed for proceeding with certification should the need arise. Change in Customer Management Customer management within Computer Craft changed significantly during the research period. Initially, the software development group had little contact with the customer, a customer management philosophy that is contrary to the views of many quality experts. There is the market-in concept of the external customer, described by Shiba, Graham, and Walden (1993) as someone who does not work for the company but receives the company s products or services. Notice, these are not only the immediate customers of your company; they may also be anyone in the customer stream to which your products flow. Juran (1988) describes these as ultimate users: The users are the final destination of the product the ultimate user is a most important category of customer and hence must be identified. In the Banking Organizer project studied at the end of the research, however, the software developers in Computer Craft had direct contact with the customer. The software development group indicated that this contributed positively to downstream development activities. This is because a clean cut-off between requirements engineering and subsequent development activities never exists and the requirements process does not exist in isolation from subsequent software development activities (Sawyer, Sommerville, and Viller 1997). This is indeed what the authors observed in Computer Craft. In the course of the research project, the company had written procedures to specify and document system requirements and to collect, identify, record, and complete new customer requests as part of the actions identified when using the SPM. These actions contributed directly to the manner in which customers were managed. When research started, there were no procedures or guidelines used for collecting and documenting cushttp://sqp.asq.org 31

10 tomer requirements. Documentation and collection methods varied between software engineers, even those working on the same project. By the end of the research period, a template for the program specification had been introduced, containing a section on requirements. This replaced the previous requirements, functional and technical specifications. Software developers were providing and working from consistent information, and the customer was satisfied with the output. Having specifications did not prevent feature creep but helped reduce it. There were no modifications required after implementing the Banking Organizer. Change in Project Management Project management is important in software development companies. In fact, Jones (1997) has established that most successful projects utilize similar patterns of planning, estimating, and quality control technologies and deficiencies of the project management function is a fundamental root cause of software disaster. During the research period in Computer Craft, the project manager implemented, managed, and controlled project plans and schedules, and developers were able to identify the tasks for which they were responsible. Project plans were implemented following use of the SPM. Schedules were set in conjunction with the customer rather than being dictated by the customer services group. Developers were expected to give the manager feedback, and thus, he was able to keep a tighter control on the project. Project management, however, had not been proceduralized within Computer Craft and was personalized to suit the project manager. It was important that, in anticipation of personnel turnover, procedures be written and the process institutionalized, which is achieved when the process becomes embedded in the day-today activities of the organization (Zahran 1998). Analysis of Change in Computer Craft Overall, many changes were made in Computer Craft. Probably the most significant of these was customer contact. The software development manager worked closely with the project manager in the client company, and the group produced specifications containing customer requirements. Changes were discussed with the software developer, modifications were normally made to specifications, and feature creep was not prevalent on the project. This in turn improved both product testing and implementation. The implementation of the practices identified using the SPM was partially responsible for the improvements in Computer Craft s software process. Any quality effort, however, will not succeed without management support (Crosby 1979; Shiba, Graham, and Walden 1993; Batra and Mohnot 1998). This is the case for any software process improvement strategy (Humphrey 1995), a quality program in a specific discipline. As Wiegers (1996) concludes, managers at all levels need to send consistent signals about software process improvement to their constituencies. In this study, management support existed. When using the SPM, identifying the practices alone was not sufficient. It was important that practices were implemented within the organization. The software development manager recognized that changes were required and used the SPM as a basis for identifying the most relevant changes. Management was also willing to provide the authors with employees time and input to the project. Without any guarantee that their software process would be improved, prioritized actions from the SPM were implemented. Computer Craft personnel were interested in having the SPICE assessment carried out on their projects and were willing to give the authors access to documentation to support both the assessment and this research project. Improvement efforts were also practically supported by the software quality assurance engineer, whose role was made more specific following the implementation of the SPM actions, giving the quality assurance engineer responsibility for procedures in the company. COMPARISON WITH OTHER COMPANIES While the authors were intervening within Computer Craft, a similar intervention took place within DataNet. This company had been founded two years previously. It employed nine people, three of whom developed software. At the same time, the software processes within two control companies Ríomhaire and Software Solutions were also investigated. In both Computer Craft and DataNet the successful implementation of actions from the SPM caused posi- 32 SQP VOL. 3, NO. 2/ 2001, ASQ

11 tive improvement to the software process, particularly to customer management and project management. What SPM provided were the actionable first steps and pressure for change that was combined with other factors previously existing in the companies: leadership and vision, capable people, and effective rewards. In DataNet, pressure for change had also risen from the focus to become ISO 9001 certified. Positive improvements were also evident within Software Solutions; in this case, there was also pressure for change because the company wanted to become ISO 9001 certified. In Ríomhaire, there was little evidence of improvement within the organization, mainly because the company was already ISO 9001 certified. Its goal was not improving the current process, but rather ensuring that the process would continue to attain this level during audits. In the future, however, this lack of change in the software process could cause difficulties in Ríomhaire, particularly if customers require assessment against a software-based framework such SPICE, as, according to Paulk (1995) ISO 9001 describes only the minimum criteria for an adequate quality management system, rather than addressing the entire continuum of process improvement. In the context of the small indigenous software development company, the success of the software process is crucially dependent on the quality of the developers. During their study of the four software development companies, the authors noted that it was uncommon for developers to be checked as to whether they carried out a process correctly. Given the resource constraints in a small company this is not surprising, but it implies a high level of trust between the software development manager and the employees. A quality culture must be based on good processes, professionally implemented. Such a small group cannot afford the overhead of checking and rechecking that would otherwise be needed. Capable people are indeed fundamental to success within a small company. Other questions must be answered as to the usefulness of the SPM, and the validity and the generalization of the research: Can SPM be useful to other small indigenous software development companies whose starting process is assessed at a low level? This is considered to be population validity (Gill and Johnson 1997). The actions derived from the SPM had a positive effect on the software processes within both Computer Craft and DataNet, whose starting processes were not documented or controlled. It originally gave a focus to the software process improvement project and provided a list of prioritized action items. There was no evidence to suggest that changes would not happen in similar companies. The next three questions are pertinent to ecological validity (Gill and Johnson 1997), which considers generalization to other contexts and settings. Can SPM be useful to other small indigenous software development companies whose starting process is assessed at a higher level? In companies assessed at a higher maturity level, the starting processes are more controlled and therefore are not similar to the action research companies. While there was no evidence to suggest that the positive effect would not occur, many of the changes that happened as a result of this research project were moving the researched companies out of chaos. This research, therefore, cannot answer this question. Can SPM be useful to small software development companies that are a subsidiary? A subsidiary has the advantage of having access to more funds, particularly for up-front investment, than an indigenous company. Once the processes are assessed at lower levels, then there is no evidence that changes would not happen in subsidiary companies. Can SPM be useful to medium or large software development companies? Large organizations are different from small organizations along several dimensions of bureaucratic structure, including formalization, centralization, complexity, and personnel ratios (Daft 1992). In the cases researched, the software development groups were autonomous within the company. There were no joint projects across cost-center divides, and employees were always located close to each other. This would not be the case for larger companies, and because of this, the current research cannot answer this question. 33

12 SUMMARY The objective of this research was to effect long-term change. In Computer Craft, change occurred because the company used the SPM to identify prioritized action items. Organization processes, customer management, and project management changed significantly. These processes were improved because of the emphasis given to them by using SPM, and procedures were written to encapsulate these improved processes. While it cannot be stated emphatically that the change that has occurred is long term, procedures that had not previously existed within the action research companies were written as a result of this research project. There is also a requirement within the company to use the procedures and evidence that they are being used to good effect. Further research is required to establish whether the initial impact will be sustained. The authors will be carrying out a longitudinal study in the four companies and will then be able to comment on the longer-term results following implementation of the SPM. ACKNOWLEDGMENTS The authors would like to thank their colleagues at the University of Limerick, particularly Professor Eamonn Murphy and all of the participating companies that made the research possible. The reviewers comments have been very helpful. REFERENCES Akao, Yoji QFD: Integrating customer requirements into product design. Portland, Ore.: Productivity Press. Batra, Ajay, and Navyug Mohnot The quality edge: Best practices from high-maturity software units in India. Cutter IT Journal 11, no. 9 (September): Cohen, Lou Quality function deployment: How to make QFD work for you. Reading, Mass.: Addison-Wesley. Crosby, Philip B Quality is free: The art of making quality certain. New York: Mc Graw Hill. Daft, Richard L Organization theory and design, fourth edition St. Paul, MN: West Publishing Company. Debou, Christophe, and Annie Kuntzmann-Combelles Linking software process improvement to business strategies: Experiences from industry. In Proceedings of SPI 98, The European Conference on Software Process Improvement, Monte Carlo. Draper, Lee, Dana Kromer, Judah Moglilensky, George Pandelios, Nate Pettengill, Gary Sigmund, and David Quinn Use of the Software Engineering Institute Capability Maturity Model in software process appraisals. CMM v2 Workshop, Pittsburgh. El Eman, Khaled, Lionel Briand, and Robert Smith Assessor agreement in rating SPICE processes. Software Process Improvement and Practice 2, no. 4 (December): Fortuna, Ronald M Beyond quality: Taking SPC upstream. Quality Progress (June): Gill, John, and Phil Johnson Research methods for managers, second edition. London: Paul Chapman Publishing Ltd. Hauser, John R., and Don Clausing The house of quality. Harvard Business Review (May-June): Humphrey, Watts S A discipline for software engineering. Reading, Mass.: Addison-Wesley. Jones, Capers Patterns of software systems failure and success. Boston: International Thompson Computer Press. Juran, J. M Juran on planning for quality. New York: The Free Press. Mazur, Glenn QFD for small business: A shortcut through the maze of matrices. In Transactions from the Sixth Symposium on Quality Function Deployment, Novi, Mich June: Paulk, Mark C How ISO 9001 compares with the CMM. IEEE Software (January): Peterson, Bill Transitioning the CMM into practice. In Proceedings of SPI 95 - The European Conference on Software Process Improvement, The European Experience in a World Context, Barcelona, Spain: Software Engineering Institute. Software Process Improvement and Practice pilot issue: Richardson, Ita Quality function deployment: A software process tool? In Proceedings of the Third Annual International QFD Symposium. Linkoping University, Linkoping, Sweden, 1-2 October, vol. 2: Richardson, Ita Improving the software process in small indigenous software development companies using a model based on quality function deployment. Ph.D. thesis, University of Limerick. Sawyer, Pete, Ian Sommerville, and Stephen Viller Requirements process improvement through the phased introduction of good practice. Software Process Improvement and Practice 3, no.1: Shiba, S., A. Graham, and D. Walden A new American TQM: Four practical revolutions in management. Portland, Ore.: Productivity Press. Stevenson, Howard H Defining corporate strengths and weaknesses. In Readings in Strategic Management, ed. David Asch and Cliff Bowman, London: Macmillan. Wiegers, Karl Software process improvement: 10 traps to avoid. Software Development (May): Willman, Paul Organizational change in the 1990s. Lecture at University of Limerick, 10 October. Zahran, Sami Software process improvement, practical guidelines for business success. U.K.: Addison-Wesley. BIOGRAPHIES Ita Richardson has a bachelor s degree in applied mathematics from NIHE in Limerick, Ireland, a master s degree in mathematics and computing from the University of Limerick, and a doctorate from the University of Limerick. Richardson worked with Wang Laboratories in Limerick for eight 34 SQP VOL. 3, NO. 2/ 2001, ASQ

13 years until 1991, where she was involved in programming and systems analysis of systems used in the manufacturing plant. In latter years, she also had responsibility for the maintenance of systems standards and training of programmers and analysts. She has been a faculty member of the University of Limerick since Her teaching includes software process improvement and systems analysis and design. Her doctorate research was in the development and application of the software process matrix. Richardson is a founding member of the Small Firms Research Unit at the University of Limerick, whose research is specifically concerned with the growth and development of small firms. Her current research is furthering her investigation into the application of quality tools and techniques to the software development process. Richardson can be reached by at ita.richardson@ul.ie. Kevin Ryan is vice president academic at the University of Limerick. He has a bachelor s degree in engineering and a doctorate in computer science, both from Trinity College in Dublin, Ireland. He lectured in computer science at Trinity College before working as programmer training manager for RCM Ltd. Zambia, where he recruited and trained the first Zambian programmers. In 1979 he was on the faculty of the University of Kansas before rejoining Trinity College in In 1990 he became head of the department of computer science and information systems at the University of Limerick. In 1994 he became the founding dean of the new College of Informatics and Electronics. Ryan has lectured on programming languages, program design methods, business computing, data structures, operating systems, social impacts of computing, software engineering, and artificial intelligence. He can be reached by at kevin.ryan@ul.ie. CALL FOR SUBMISSIONS Software Quality Professional provides readers with significant information intended to contribute to their personal development and success in the field of software quality. By focusing on the practical needs of professionals such as engineers and managers, it seeks to provide an intersection between quality engineering and software engineering. Software Quality Professional would like to feature articles on the following topics in upcoming issues: Networking, including privacy/security, reliability, and performance issues System integration within a business-process context Software/system verification and validation Applications in consumer products, health care, and other service industries Any other topics within the body of knowledge for the ASQ Certified Software Quality Engineer are always welcome. Please inquire before submitting material in areas recently addressed within the journal, such as testing, inspection, project management, or process improvement. Submissions are accepted at any time, but the deadlines for specific issues are: June 15 for December issue September 15 for March issue December 15 for June issue March 15 for September issue Guidelines for authors are printed in the back of this issue. Inquiries to the editor in chief should be directed to SQP_Editor@asqnet.org, phone or fax