The Collaborative Software Process Dissertation Proposal Version 2 Laurie A. Williams lwilliam@cs.utah.edu. None of us is as smart as all of us.

Transcription

1 The Collaborative Software Process Dissertation Proposal Version 2 Laurie A. Williams lwilliam@cs.utah.edu None of us is as smart as all of us. [1] Summary Anecdotal evidence from several sources indicates that two programmers working collaboratively on the same design, algorithm, code, or test perform substantially better than the two working alone. Statistical evidence has shown that programmers perform better when following a defined, repeatable process such as the Personal Software Process (PSP) for individuals or the Team Software Process (TSP) for groups. Bringing these two ideas together, the Collaborative Software Process (CSP) will be formulated. The CSP will be a defined, repeatable process for two programmers working collaboratively. It is hypothesized that software developers using the CSP will have higher productivity rates and will produce products with higher quality than those using the PSP. In order to test the hypothesis, an experiment will be run during the 1999 Fall semester with senior Computer Science students. All students will learn both the CSP and the PSP. Half the students will work in two-person collaborative teams using the CSP to develop their programming assignments. The other half of the students will work independently using the PSP to develop their programming assignments. The productivity and quality of the two groups will be compared to validate the hypothesis. Hypothesis The software industry has particular difficulty producing large software systems on schedule, of high quality and at a pre-established budget. Watts Humphrey, of the Software Engineering Institute (SEI), has approached this problem by defining the Personal Software Process (PSP)[2] and the Team Software Process (TSP)[3] development methodologies. PSP requires each individual software engineer to strictly adhere to a defined, disciplined software development process. PSP has met with great success in industry and is known for consistently and efficiently producing high-quality products. The value of PSP has been documented in several industrial case studies [4] and is also taught at many universities, including the University of Utah. Though not as well documented, accepted, or proven, the Extreme Programming (XP) process, developed primarily by Smalltalk code developer and consultant Kent Beck, attributes great success to 1

2 the use of pair programming. With pair programming, two programmers work side-by-side at one computer, collaborating on the same design, algorithm, code or test. Results [5] demonstrate that the two programmers work together more than twice as fast and think of more than twice as many solutions to a problem as two working alone, while attaining higher defect prevention and defect removal leading to a higher quality product. The Extreme Programming process employs many other techniques counter to currently accepted software engineering practice. Therefore, isolating which factor to attribute its reported success is difficult. The research outlined here will help to dissect the contributing factors. This research will define and validate the Collaborative Software Process (CSP) by incorporating pair programming practices as alternate sub-processes of the PSP. The CSP will give a pair of programmers a framework of techniques and measurement-based feedback to improve their joint performance, and thereby the performance of their team. This research aims to show that software developers using the CSP will have higher productivity rates and will produce products with higher quality than those using the PSP. Importance of Research Crunch mode is not a matter of opportunity it s a matter of survival... The ability to get working software quickly into the hands of users will be characteristic of successful dataprocessing organizations for the foreseeable future. Groups that can produce and install software systems within tight time frames will prosper. Those [that] can t will fail and, in some cases, they will bring the enterprises of which they are a part down with them. Fast response to changing information-processing requirements is a necessity in today s world [6]. Competition demands that software companies be continuously innovative in producing high-quality software in a short time. However, as stated earlier, the software industry already notoriously has trouble meeting its schedule, cost and/or quality commitments. The average software development project overshoots its schedule by half; larger projects generally do worse. And, some three quarters of all large systems are operating failures that either do not function as intended or are not used at all.[7] Additionally, software is increasingly used in safety critical applications. Software defects can cost human life. For example, between 1985 and 1987 two people died and four were seriously injured when they received massive radiation overdoses delivered by the Therac-25 radiation therapy machines. Investigations attributed the accidents to many factors, including poor user interface design, problems with race conditions, and other software defects [8]. 2

3 Most mature engineering disciplines have established procedures and codified, proven solutions, which help yield more predictable, high-quality results. Software engineering, a relatively young discipline, still seeks these procedures and solutions. The software engineering community has realized that it takes a high-quality process to yield high-quality products. Process standards such as ISO 9000 and the Capability Maturity Model have been developed to aid organizations in achieving more predictable results. The PSP allows individual developers to have a high quality process. The PSP provides engineers with the process understanding they need to help improve organizational performance [9]. Watts Humphrey has also developed the Team Software Process [3] for teams of software developers to improve the effectiveness and quality of teams. Common sense leads us to believe that CSP will do likewise for a pair of programmers. Many organizations avoid pair programming, viewing it as an overt redundant use of resources. The results, therefore, will provide valuable insight into the benefits of pair programming as companies strive to be competitive, by reducing cycle time and improving quality. Theory Base of Research Perhaps the oldest engineering process is the Engineering Method, which is typically represented by the following ten steps [10]: 1) Identify the problem 2) Define the problem 3) Search 4) Constraints 5) Criteria 6) Alternative solutions 7) Analysis 8) Decision 9) Specification 10) Communication From an engineering standpoint, a process is defined as step-by-step changes that lead toward a required result... an orderly, systematic approach to a desired end [10]. Historically, engineering fields strive for the development of step-by-step processes to yield the desired ends. A theory behind this research is that following a defined and repeatable process yields high quality software products at a more predictable rate. By conjecturing that transitioning from a Personal Software Process to a Collaborative Software Process would be beneficial, one theorizes that collaboration leads to higher performance than solitary work would. Literature on collaboration overflows with examples of remarkable achievements in many fields that could have only occurred with collaboration. One author contends we are living in a world in which technological complexity increases at an accelerating rate [which] offers fewer and fewer arenas in which individual action suffices. [1]. Software has become the driving force behind most new technologies. But 3

4 the engineering of software is becoming increasingly complicated. A software engineer must balance a variety of competing factors, including functionality, quality, performance, safety, usability, time to market, and cost. Moreover, the size of software systems that are being built is rapidly growing. Conceptual collaboration occurs when people work together to devise concepts, ideas, themes, metaphors, analogies, and so on that frame the overarching goal of the collaboration... Conceptual collaboration yields insights into fundamental notions of the problem, innovation, or discovery that is the focus of the collaboration... Conversely, technical collaboration is not unlike the key that fits into the lock; it's the way people physically represent the conceptual aspects of the task at hand. Technical collaborations are the attempts to solve the problems the conceptual collaborations identify... collaborations that require high levels of both conceptual and technical collaboration [have] a constant reshuffling of ideas and tinkering with models to achieve success [11]. Clearly, the complexities associated with software development require both conceptual and technical collaboration. Simply put, teamwork is needed to create large, complex systems, and creating software is the most complex intellectual tasks human beings have ever attempted. [12] A theory behind this research is that the collaboration that has proven to work so well in other fields will be beneficial to software engineers. In tasks requiring communication among the subtasks, the effort of communication must be added to the amount of work to be done... The added burden of communication is made up of two parts, training and intercommunication... If each part of the [n] task[s] must be separately coordinated with each other part, the [intercommunication] effort increases as n(n+1)/2 [13]. Integrating the partitioned tasks of programmers requires this extra effort of intercommunication. Through pair programming, the number of separate tasks to be integrated can be halved. A theory behind this research is that the intercommunication effort of integration is significantly reduced through the use of pair programming. Related Work The Personal Software Process (PSP) defines a framework that includes defined operations or subprocess and measurement and analysis techniques to help engineers understand their own skills and improve personal performance. Each sub-process has a set of scripts giving specific steps to follow and a set of templates or forms to fill out to ensure completeness and to collect data for measurement-based feedback. This measurement-based feedback allows the programmers to measure their work, analyze their problem areas, and set and make goals. For example, programmers record information about all the defects that they remove from their programs. They can use summarized feedback on their defect removal to become more aware of the types of defects they make to prevent repeating these kinds of defects. Additionally, they can examine trends in their defects per thousand lines of code (KLOC). 4

5 PSP has several strong philosophies. The first is that the longer a software defect remains in a product, the more costly it is to detect and remove the defect. Therefore, thorough design and code reviews are performed for most efficient defect removal. The second philosophy is that defect prevention is more efficient than defect removal. Careful designs are professed, and data is collected to give additional input on where the programmer should adjust their own personal software process to prevent future defects. Lastly, PSP rests on the notion that the best estimates, and therefore the best commitments, for schedule and defect rates can be made with a historical database of information. Data regarding how long previous products took to develop and their defect rates are kept in a database for use with the PROBE estimation sub-process. These processes and philosophies work together to produce excellent results. SEI s data on 104 engineers shows that, on average, PSP training reduces size-estimating errors by 25.8 percent and time-estimating errors by 40 percent. Lines of code written per hour increases on average by 20.8 percent, and the portion of engineers development time spent compiling is reduced by 81.7 percent. Testing time is reduced by 43.4 percent, total defects by 59.8 percent, and test defects by 73.2 percent [4]. The Team Software Process (TSP) requires each individual on a team to follow the PSP for their individual work. The TSPi has been formulated for teams of four to six software developers in a university course; the TSP has been formulated for up to 20 developers in industry. Both add to the PSP additional sub-processes for group activities such as peer inspection or integration and scripts for team roles, such as Team Leader, Development Manager, Planning Manager, Quality/Process Manager, and Support Manager. These sub-process and defined roles guide in allocating work among team members, coordinating each of their tasks, and tracking and reporting on their progress. The TSP is quite new and is currently developing statistical evidence of its efficacy. The first published results were reported on the TaskView project at Hill Air Force Base in Ogden, Utah. It showed a 123% increase in productivity, testing completed in one-eighth the normal time and no defects reported by the customer in acceptance test [14]. The Extreme Programming Process (XP) does not have statistical evidence, as does PSP, to prove its effectiveness. The evidence of XP s success is highly anecdotal, but so impressive that it has aroused the curiosity of many highly-respected software engineering researchers and consultants. The largest example of its accomplishment is the sizable Chrysler Comprehensive Compensation launched in May The payroll system pays some 10,000 monthly-paid employees and has 2,000 classes and 30,000 methods [15]. Additionally, programmers at Ford Motor Company, spent four unsuccessful years trying to build the Vehicle Cost and Profit System (VCAPS) using a traditional waterfall methodology. It duplicated that system, this time successfully, in less than a year using Extreme Programming [5]. XP employs pair programming, as mentioned above. All production code is written with a partner, to the extent that even prototyping done solo is scrapped and re-written with a partner. Working in pairs, they 5

6 perform a continuous code review, noting that it is amazing how many obvious but unnoticed defects become noticed by another person watching over your shoulder. This is, perhaps, the ultimate implementation of PSP s defect prevention and efficient defect removal philosophies. XP s requirements gathering, resource allocation and design practices are a radical departure from most accepted methodologies. Customer requirements are written as fairly informal User Story cards, a rough estimate is assigned to the cards, these are assigned to a programming pair, and coding begins. With no formal design procedures or discussions on overall system planning or architecture, the pair determines which code in the ever-enlarging code base needs to be added or changed and does it. This practice requires the use of Collective Code Ownership whereby any programming pair can modify or add to any code in the code base, regardless of the original programmer. Programming pairs routinely refactor the code base by continuous change and enhancement. They view the code as the self-evolving design they do not spend time on a design document and, therefore, have strict self-documenting code style and comment guidelines. XP has extremely thorough testing procedures. Comprehensive test cases are written prior to actual code changes. The results of running these new tests and previous, regression test cases determine if the change/enhancement to implement a User Story has been done correctly without harming the implementation of other User Stories. While departing significantly from traditional development practices, anecdotally, XP appears to be very effective. Additionally, programmers report that developing with XP practices is much more exciting and enjoyable than with traditional processes. The idea of pair programming did not originate with XP. It was first published as a Developing in Pairs Organizational Pattern in 1995 [16]. The idea behind Organizational Patterns is to make explicit the patterns of organization, process, and introspection that most highly productive organizations exhibit. Using the emerging discipline of generative pattern languages, we can capture the patterns underlying successful projects and use them to establish organizational structures and practices that will improve the prospects for success in a new software development organization [16]. The Developing in Pairs pattern professes that organizations should pair compatible designers to work together that together, they can produce more than the sum of the two individually. Two other studies support the use of collaborative programming. Larry Constantine, a programmer, consultant, and magazine columnist reports on observing Dynamic Duos during a visit to P. J. Plaugher s software company, Whitesmiths, Ltd, providing anecdotal support for collaborative programming. He immediately noticed that at each terminal were two programmers working on the same code. He reports, Having adopted this approach, they were delivering finished and tested code faster than ever... The code that came out the back of the two programmer terminals was nearly 100% bug free... it was better code, tighter and more efficient, having benefited from the thinking of two bright 6

7 minds and the steady dialogue between two trusted terminal-mates... Two programmers in tandem is not redundancy; it s a direct route to greater efficiency and better quality. [17] An experiment studied 15 full-time, experienced programmers working for 45 minutes on a challenging problem, important to their organization, in their own environment, and with their own equipment. Five worked individually, ten worked collaboratively in five pairs. Conditions and materials used were the same for both the experimental (team) and control (individual) groups. This study provided statistically significant results, using a two-sided t-test. To the surprise of the managers and participants, all the teams outperformed the individual programmers, enjoyed the problem-solving process more, and had greater confidence in their solutions. The groups completed the task 40% more quickly and effectively by producing better algorithms and code in less time. The majority of the programmers were skeptical of the value of collaboration in working on the same problem and thought it would not be an enjoyable process. However, results show collaboration improved both their performance and their enjoyment of the problem solving process. [18]. The University of North Carolina-Chapel Hill (UNC-CH) has a Distributed Collaboration research group in their Computer Science department. The University of Hawaii has a Collaborative Software Development Lab. Additionally, many researchers around the world study Computer Supported Cooperative Work (CSCW). Much of this work focuses on the infrastructure and tools for supporting asynchronous and synchronous distributed collaborative software development. Their work has helped established that collaboration is beneficial. Additionally, they have developed concepts pertinent to this work. For example, the notion of collective intelligence (CI) is that a group of human beings can carry out a task as if the group, itself, were a coherent, intelligent organism working with one mind, rather than a collection of independent agents [19]. One professor at UNC-CH, John B. Smith calls for a long-term commitment to building a fully realized theory of collective intelligence and a pedagogy for developing collaborative skills. This research will contribute to collective intelligence theory. Research Methodology Preparatory Chapters. Software engineers are conditioned to performing solitary work. Telling them to work collaboratively without more guidance would be unproductive. Therefore, two preparatory CSP papers will be produced to give this guidance. The intent is that collaborative programmers read these chapters before embarking on CSP to maximize their success as a team. The first, Working as a Collaborative Programmer, will give specifics on mechanics of working collaboratively. For example, one person is at the keyboard driving while the other is doing a continuous code/design review and is thinking more strategically about where the program is going and if the current implementation will do the job. Drivers must switch periodically, etc. The second, Success Strategies for Collaborative Programming, will deal more with the interpersonal 7

8 issues of collaborative programming, such as ego-less programming, negotiation strategies, culture and the like. These chapters will be based on current literature and on experiences of programmers who have worked collaboratively. To provide information for these chapters, data will be gathered on actual experiences with collaborative programming via a web-based questionnaire. Requests for questionnaire respondents will be sent to specific people who are known to have done pair programming, to the pair programming mailing list, and to graduate students at the University of Utah. Additionally, the questionnaire will be publicized on the two main Extreme Programming web sites. Process Document. A major contribution of the research will be a complete process document of the Collaborative Software Process. As stated earlier, the PSP is a thorough, recorded software development process. Each subprocess has a set of scripts, forms and templates to give the programmer specific procedures and to obtain data for measurement-based feedback. Based on theoretical research on collaboration and on actual experiences with collaborative programming, these sub-process descriptions, scripts, forms and templates will be adjusted to give a framework for a pair of programmers to follow the Collaborative Programming Process. The PSP has seven levels (PSP0, PSP0.1, PSP1, PSP1.1, PSP2, PSP2.1, PSP3, consecutively) each with an increasing level of structure and detail. Students and practitioners studying the PSP start at level PSP0 and progress to PSP3. Those studying CSP will do likewise, as outlined in Figure 1. Those familiar with PSP will notice that levels one (Quality Management) and two (Planning Process) have been reversed in the CSP. This leverages earlier the defect prevention and defect removal benefits of collaborative programming. 8

9 The proposed content of each of the levels in the CSP is discussed below. CSP Level 0: Baseline Collaborative Process The main objective of CSP Level 0 is to establish baseline productivity and quality levels for the collaborative programmers. CSP0 involves enhancing their current process by recording these measurements and by following a coding standard. To collect the necessary data, the CSP0 uses four main forms: The Time Recording Log, the Defect Recording Log, the CSP0 Project Plan Summary, and the Process Improvement Proposal (PIP). The Time Recording Log is used to record how much time is spent, by phase, for each of the programmers. The form will allow for time to be recorded by either programmer individually or for both of the programmers together (so the amount of time will be doubled when it is saved). By collecting this data, the pair of programmers develops a history of how long it takes them to accomplish programming tasks. Through time, this information helps them make better estimates, and therefore better commitments, of future work. 9

10 The Defect Recording Log is used to record the defects that are removed by either or both of the programmers, classified by type number, phase injected, phase removed and whether they believed the defect was injected during collaborative or solo work. They also record how much time it took to remove that defect, individually or together. By studying trends in this data, the pair can learn how efficient their defect removal process is and about the types of defects they typically make in order to prevent making these types of defects in the future. During the planning phase, the programmers give rough estimates on the time required to do the task, how large the program will be, and the number of defects they will inject and remove. The Project Plan, as shown in Appendix C, collects and summarizes this information and the actual data that is recorded via the Time and Defect Recording Logs. The Process Improvement Proposal is used to record, in a structured way, process problems, experiences, and improvement suggestions for future work. PSP0 also has Process, Planning, Development, and Postmortem scripts which enumerate steps for accomplishing these tasks. These scripts must be adjusted for work by a pair of programmers, instead of by an individual programmer. For example, procedures and guidelines must be developed for both collaborative and solo development work. Undoubtedly, the pair of programmers will ultimately work independently at times. CSP Level 1: Collaborative Quality Management CSP1 focuses on improving the actual quality of the product produced by the pair of collaborative programmers. Through the dynamics of two programmers working together, defect prevention and defect removal activities are quite different than with solo programmers. Therefore, the largest divergence between PSP and CSP will be in level 1. Design and Code Reviews are performed. PSP procedures for both of these reviews involved a selfcheck by the individual programmer using design and code review checklists. With the PSP, code reviews are performed prior to compilation for most efficient defect removal. Collaborative programming allows many defect to be removed during design and code. As the pair is working together, the programmer that is not actively writing the design or code should be performing a continuous review using checklists, allowing for most efficient defect removal. Because of this, perhaps, no formal design review is performed. Also, code review could potentially be moved after compilation, focusing on logic errors and on ensuring the code actually meets the specification. Specifics of these sub-processes will be determined via the pair programmer questionnaire and experimentation. Additionally, there will be different review procedures for collaborative work and for solo work. 10

11 Finally, the use of the forms and scripts from CSP0 are continued, however the scripts and the Project Plan are enhanced to capture increased focus on quality management. CSP Level 2: Collaborative Planning Process CSP2 enhances the rough planning steps of CSP0. When a pair of programmers becomes familiar with their own performance rates, they can plan their work better, make more realistic commitments, and more consistently meet their commitments. The steps added in CSP2 allow the estimation to be done systematically based on past performance. The use of the forms and scripts from CSP1 are continued, however the scripts and the Project Plan are enhanced to capture increased planning activities. The PROBE size (line of code) estimation method is used. See the template for this estimation method in Appendix C. Additionally, a Test Report template is introduced to plan and track testing results. Task and Schedule Planning Templates are used to plan and track progress on individual tasks and on the overall completion. CSP Level 3: Cyclic Collaborative Development CSP3 structures a cyclic, incremental development process to be used by the collaborative team. A highlevel design is developed and reviewed. Again, this review might be able to be combined with the design process itself as with the lower level design of CSP1. The use of the forms and scripts from CSP2 are continued, however the scripts and the Project Plan are enhanced to capture increased high-level design activities. In summary, the PSP contains 77 forms, templates, and scripts. An initial assessment indicates that 33 of these need a moderate amount of changes, 19 of these need minor changes, and 25 need no change. None require major changes, since the original attempt of the form, template, or script remains unchanged. Details of this breakout are given in Appendix A. Some new scripts related to Quality Management will be added. Supporting Tools. The forms and templates referenced above obtain a substantial amount of data for students to estimate and track quality and productivity data. If the data the students enter is inaccurate, the measurementbased feedback to the students will be misleading and the findings of this research could be incorrect. A recent MS thesis has shown that data inaccuracies can have significant impact on the quality of PSP data [20]. The types of defects fall into seven major categories [21]: 1) Calculation Error: an error in a data field that was derived using any sort of calculation 2) Blank Field: a required field on a form was left blank 11

12 3) Inter-Project Transfer Error: an incorrect data value was copied from a prior project 4) Intra-Project Transfer Error: an incorrect data value was transferred from another form 5) Entry Error: lack of understanding of what was required 6) Impossible Values: indicates that two values were mutually exclusive 7) Sequence Error: improper sequencing through development stages Data inaccuracies are also a concern for the CSP. Therefore, a WindowsNT Active Server Page webbased system for data entry and information retrieval will be developed for the PSP and the CSP. Both systems are required because of the empirical study, which will be discussed below. The system will automate all calculations and data transfers and make it easier for the participants to report accurately. The system will also include methods to check for the kinds of problems outlined in the referenced MS thesis and will be able to display aggregate data for programming pairs. For the student population at the University of Utah, it is important that the tool be web-based. The students often do not do their assignments in the computer lab, but instead work at home and at their place of employment. A webbased tool will allow them to enter their data from wherever they are, increasing the opportunity for accurate data input. Empirical Study. The validation of this new process will be based on an empirical study of students at the University of Utah. The structured experiment to evaluate the effectiveness of the Collaborative Software Process will take place during the 1999 Fall academic semester. The senior classes Software Engineering (CS4510) at the University of Utah will be used for the study. Approximately students will take this class. Some students will develop assignments using the PSP; others will use CSP. Their results will be compared. Details on the specifics of the research design are given in Appendix B. The experiment has been designed to draw statistically significant conclusions on the following: 1) Overall productivity and quality of collaborative work via the CSP vs individual work via the PSP 2) Effects of assignment difficulty on collaboration 3) Skills assimilation of student using CSP vs students using PSP 4) Pre-class and post-class attitudes towards collaborative programming. Attitudes about/voluntary use of collaborative programming in the follow-on class (CS4500 Software Engineering Laboratory). 5) Enjoyment and confidence measures Additionally, the following are exploratory aspects, which will provide anecdotal evidence for potential further study on collaborative teams: 1) Effects of gender makeup 2) Effects of student s past academic performance 3) Effects of friendship among team members 12

13 4) Effects of various personality types as classified by Meyers Briggs testing Limitations and Key Assumptions The findings of the verification experiment are dependent on activities of the students in the class. The students that are supposed to work alone must work alone; the students that are supposed to work in a collaborative team must do that. Several professors at the University of Utah use a script that has been very accurate in detecting instances of plagiarism. This can be used for detecting individual programmers that might not be working individually. The script, however, cannot detect when a collaborative team is being carried by the work of one of the individuals. The students must also follow the various steps of the PSP or CSP process to produce their programs. A quiz will be given the day after each of the three program cycles that will strictly address processes that should have been followed and specifics of program implementation. Poor performance on the quiz would signal problems with participation or with following established process procedures. Additionally, the data entry is a self-reporting procedure. The students must enter data accurately into the PSP/CSP system. Checks will be built into the Active Server Page system to flag inconsistencies. Additionally, it will be critical to have the grading for the class done promptly and to alert the student to additional disparities between data that has been entered and the actual project results. For example, if a student entered very little time, yet produced an excellent program. Inconsistencies will immediately be discussed with him/her. Experience has shown that when the instructor, through immediate feedback, demands good data entry and rejects poor work, meaningful information can be obtained from the system. Research Contributions Mature engineering disciplines have documented, repeatable processes to predictably produce products on-schedule and of high quality. The Personal Software Process has proven a strong contribution in this area to the maturity of software engineering. Through this research, the Collaborative Software Process will add another documented, repeatable process. The empirical study will compare the effectiveness of these two methodologies in terms of programmer productivity, cycle time, and system quality. If CSP outperforms PSP, industry will know that the use of collaborative programming could improve their results. Also, computer science students could also be taught to program collaboratively in the future. Fred Brooks, in his 1975 landmark book The Mythical Man-Month [13] states Brook s Law: Adding manpower to a late software project makes it later. 13

14 The logic behind his law is that when a task is partitioned into subtasks in order to split it between several developers, the extra effort of training and intercommunication must be added to the amount of work done. This extra effort might fully dominate the decrease in individual task time brought about by partitioning. Adding more men then lengthens, not shortens, the schedule. [13] Because of Brook s Law, managers supervising software projects in trouble for the last 24 years have tried in earnest to resist the temptation to add more manpower to projects in order to catch up. If CSP proves to improve productivity, and thereby, cycle time, this research could finally find a way to disprove Brook s Law. It would show managers that they could add manpower to a behind-schedule project to improve cycle time by doubling up developers, if they had not already done so. Intercommunication does not dominate with CSP because the two developers are continually together; benefits of collective intelligence prevail. Steve McConnell of Microsoft has been quoted, Good software engineering ideas will outlast specific technologies by decades [22]. The value of this research transcends the creation of another software development process by providing for the validation of the value a potential good software engineering idea, collaborative software development. Alternately, if CSP yields the similar results to PSP, industry could use PSP or CSP interchangeably, potentially the choice of the individual programmers. Lastly, if PSP outperforms CSP, they will know that collaborative programming is a bad resource decision. Therefore, all possible outcomes of this study will provide valuable information to programming organizations making resource allocation decisions, always striving to produce high quality products on-time. As stated earlier, this research could further the building of a theory of collective intelligence and a pedagogy for developing collaborative skills. Lastly, the research will also provide an Active Server Page tool to support the Collaborative Software Process. Future Study If this academic study proves fruitful, repeating the study in an industrial project would further validate the results. Positive results in academia would give an industrial project more confidence in the process prior to risking its commercial success on what could be viewed an experimental methodology. Traditionally, collaboration in the classroom... has been taboo, condemned as a form of cheating. Yet what we discover... is that collaboration can only make our classrooms happier and more productive [1]. After proving the utility of using the Personal Software Process documented in A Discipline for Software Engineering [2], Watts Humphrey developed a more lightweight Personal Software Process 14

15 appropriate for beginning undergraduates, documented in Introduction to the Personal Software Process [23]. A lightweight Collaborative Software Process should be developed and validated via an empirical study of beginning undergraduates. Larry Constantine, who s observation of P. J. Plaugher s software company were reported above, noted that... for language learning, there seems to be an optimum number of students per terminal. It s not one... one student working alone generally learns the language significantly more slowly than when paired up with a partner [17]. The development of such a process and an empirical study to measure its effectiveness could statistically validate Constantine s observations, perhaps changing how beginning programming classes are taught. Such a study would need to be performed in a university, unlike the University of Utah, where students are generally in fulltime residence, and therefore are able to work collaboratively. The research study also provides a framework and a baseline for further studies on collaborative programming. Technology advances could allow two programmers to work simultaneously and synchronously, but separated geographically. Some synchronous tools, such as audio/video conferencing and meeting-support tools, support verbal and visual interaction. Other commerciallyavailable tools more applicable to collaborative programming, such as shared applications, allow several people to work simultaneously and interactively on the same tangible product, such as a computer program. This mode of work is roughly analogous to several individuals sitting down at the same workstation, where they can see the same display and where they can take turns using the keyboard and mouse[19]. Just as when two programmers working physically together take turns typing and observing, these separated programmers signal when they would like to be the active user, often via a floor control mechanism. Without adding audio/video conferencing, these systems, however, do not generally support the normal conversations that the programmers would carry on if they were physically together. Replicating this Collaborative Software Process experiment using such synchronous tools would be interesting follow-up research. Another version of the Collaborative Software Process could be developed to allow asynchronous shift work. Two programmers work independently on the same problem, perhaps physically together during a transition period between shifts or not at all. Mainstream systems and applications such as , file transfer protocol (ftp), source code control, and distributed files systems can be used for such work. Some specialized asynchronous tools for distributed collaboration have also been developed. 15

16 Proposed Dissertation Chapters Chapter Subject Projected Page Length 1 Overview 10 2 Related Work 5 3 Collaborative Programming 20 Results of Questionnaire Working as a Collaborative Programmer Success Strategies for Collaborative Programming 4 The Collaborative Software Process 15 Discussion of Revisions to PSP CSP Level 0: Baseline Collaborative Process CSP Level 1: Collaborative Quality Management CSP Level 2: Collaborative Planning Process CSP Level 3: Cyclic Collaborative Development 5 Empirical Evidence Experiment Details Data Analysis and Findings Discussion of Findings CSP Tool 5 7 Future Study 3 Bibliography Subtotal Text 82 Appendix Complete Set of CSP Forms, Templates and Scripts 80 (both those revised for CSP and unchanged from PSP) Approximate Page Length 162 Timetable Activity Timeframe Proposal Defense 4/16/99 Develop and Document CSP Present 12/31/99 Develop Active Server Page PSP/CSP System Teach Summer Class 5/18/99-8/25/99 Revise CSP based on class results and feedback Teach Fall Class (CS4510) 8/25/99-12/17/99 Data collection I Data Analysis 1/1/00-3/1/00 Completion of dissertation document 3/1/00-5/1/00 16

17 Bibliography 1. Bennis, W., Biederman, Patricia Ward, Organizing Genius: The Secrets of Creative Collaboration. 1997: Addison-Wesley Publishing Company, Inc. 2. Humphrey, W.S., A Discipline for Software Engineering. SEI Series in Software Engineering, ed. P. Freeman, Musa, John. 1995: Addison Wesley Longman, Inc. 3. Humphrey, W.S., Introduction to the Team Software Process. to be published: Addison Wesley. 4. Ferguson, P., Humphrey, Watts S., Khajenoori, Soheil, Macke, Susan and Matvya, Annette, Results of Applying the Personal Software Process, in Computer p Beck, K., Cunningham, Ward, Extreme Programming Roadmap,. 1999, 6. Abdel-Hamid, T.K., Investigating the Cost/Schedule Trade-Off in Software Development, in IEEE Software p Gibbs, W.W., Software's Chronic Crisis, in Scientific American p Leveson, N.G., Turner, Clark S., An investigation of the Therac-25 accidents, in IEEE Computer p Humphrey, W.S., Using a Defined and Measured Personal Software Process, in IEEE Software p Eide, A.R., Jenison, Roland D., Mashaw, Lane H, Northup, Larry L., Introduction to Engineering Design. 1998: WCB/McGraw-Hill. 11. Schrage, M., No More Teams! Mastering the Dynamics of Creative Collaboration. 1995, New York: Currency Doubleday. 12. Weinberg, G.M., Weinberg on the Essential Team, in Software Testing and Quality Engineering p Brooks, F.P.J., The Mythical Man-Moth. 1975: Addison-Wesley Publishing Company Webb, D., Humphrey, Watts, Using the TSP on the TaskView Project. Crosstalk, February Anderson, A., Beattie, Ralph, Beck, Kent et al., Chrysler Goes to "Extremes", in Distributed Computing p Coplien, J.O., A Development Process Generative Pattern Language., in Pattern Languages of Program Design, James O. Coplien and Douglas C. Schmidt, Editor. 1995, Addison-Wesley: Reading, MA. p Constantine, L.L., Constantine on Peopleware. Yourdon Press Computing Series, ed. E. Yourdon. 1995, Englewood Cliffs, NJ: Yourdon Press. 18. Nosek, J.T., The Case for Collaborative Programming, in Communications of the ACM p Smith, J.B., Collective Intelligence in Computer-Based Collaboration. Computers, Cognition, and Work, ed. G.M. Olson, Olson, Judith S., Curtis, Bill. 1994, Hillsdale, NJ: Lawrence Erlbaum Associates. 20. Disney, A.M., Johnson, Philip M. Investigating Data Quality Problems in the PSP (Experience Paper). in Foundations of Software Engineering Lake Buena Vista, FL: ACM Press. 21. Disney, A.M., Data Quality Problems in the Personal Software Process, in Information and Computer Sciences. 1998, University of Hawaii: Honolulu, HI. 22. Weinberg, G.M., Egoless Programming, in IEEE Software p Humphrey, W.S., Introduction to the Personal Software Process. 1997: Addison-Wesley. 24. Drew, C., Hardman, Michael L. and Hart, Ann Weaver, Designing and Conducting Research: Inquiry in Education and Social Science. 1996, Needham Heights, Massachusetts: Simon and Schuster Company. 25. Meegan, S.P., Group Friendship Composition and Processes of Collaboration, in Department of Psychology. 1997, University of Utah: Salt Lake City. 26. Hargrove, R., Mastering the Art of Creative Collaboration. 1998: McGraw-Hill. 27. Dutoit, A.H., Bruegge, Bernd, Communication Metrics for Software Development. IEEE Transactions on Software Engineering, 1998(August 1998): p

18 Appendix A: CSP Form, Scripts and Templates Heading CSP Change from PSP Form # Process Version CSP 0 CSP0. 1 CSP1 CSP1. 1 CSP2 CSP2. 1 CSP 3 Process Scripts and Summaries Process Script C1 C12 C21 C32 C42 C50 C65 Minor changes Planning Script C2 C13 C22 C33 C43 C51 C66 Moderate changes High-Level C67 No changes Design Script High Level C68 No changes Design Review Script Development C3 C14 C23 C34 C44 C52 C69 Moderate changes Script Postmortem C4 C15 C24 C35 C45 C53 C70 Minor changes Script Project Plan Summary and Instructions PROBE Estimating Script Cyclic Summary and Instructions Forms, Templates, Standards and Instructions Time Recording Log Defect Recording Log Defect Type Standard Process Improvement Proposal C5 C6 C16 C17 C25 C26 C36 C37 C46 C47 C54 C55 C71 C72 Moderate changes Moderate changes C27 X X X No changes C73 C74 No changes No changes C7 C8 X X X X X X X Moderate changes Moderate changes C9 X X X X X X X Minor changes C10 Minor changes C11 X X X X X X X No change C18 C19 X X X X X X Minor changes Moderate Changes Coding Standard C20 X X X X X X No changes Test Report Template Size Estimating Template Task Planning Template Schedule Planning Template C28 C29 C30 C31 C38 C39 C40 C41 X X X X X No changes No changes X X X X X No changes No changes X X Moderate changes Moderate changes X X Minor changes Minor changes Design Review C48 C56 C56 C56 C75 No changes Checklist Code Review C49 X X X X X No changes 18

19 Heading Process Version Checklist Operational Scenario Template Functional Specification Template State Specification Template Logic Specification Template Issue Tracking Log CSP Form # C57 C58 C59 C60 C61, C62 C63 C64 C76 C77 CSP 0 CSP0. 1 CSP1 CSP1. 1 CSP2 CSP2. 1 CSP 3 Change from PSP X X X X No changes No changes X X X X No changes No changes X X X X No changes No changes X X X X No changes No changes X No changes No changes 19

20 Appendix B: Empirical Study of the CSP Experiment Design The validation of this new process will be based on an empirical study of students at the University of Utah. In the summer of 1999, an exploratory course will be taught to pairs of upper-level undergraduates. They will be taught CSP processes in the classroom, which they will apply to programming assignments. The students will keep an electronic journal during the class in which they will record their impressions of using CSP each week. Additionally, these pairs of programmers will be observed to record information about their interactions. Then, at course completion, each student will write a paper critiquing the CSP. These observations and critiques will be used to update and enhance the CSP process prior to a structured experiment. The structured experiment to evaluate the effectiveness of the Collaborative Software Process will take place in the Fall The Software Engineering (CS4510) at the University of Utah will be used for the study. Approximately seniors will take this class. The details of this experiment have been submitted to the Institutional Review Board (IRB) at the University of Utah. The role of the IRB is to determine if they believe the rights of the students will be violated in any way by their participation in an experiment. The IRB deemed that this study was exempt from their surveillance. In order to be declared exempt, an experiment must be conducted in an established educational setting and involve normal educational practices in order to evaluate or compare regular or special educational instructional strategies, curricula or methods. The students will learn of the experiment during the first class. They must know it is an experiment because, as outlined below, some students will work individually and some will work in pairs. Additionally, they will be strongly encouraged to report all data accurately during the semester because of the importance of the outcome. A common research study threat is caused by the Hawthorne effect. The Hawthorne effect refers to a change in sensitivity, performance, or both by the subjects that may occur merely as a function of being in an investigation... The Hawthorne effect becomes a threat to internal validity when one group receives such a special treatment and another does not, thereby introducing a systematic difference between groups in addition to the experimental variable [24]. Since both groups will receive the same information about the study and in lecture materials, the Hawthorne effect should not pose a threat to this study. On the first day class, the students will be given a survey which will probe the students opinion regarding how much they will enjoy working collaboratively, if they believe working collaboratively will improve their confidence in their solutions, and if they believe it will improve their productivity and quality. The survey answer choices will be in the form of a Likert scale (i.e. the degree of agreement to the question will be represented by an ordinal number). The survey will be contrasted with a similar survey given at the end 20

21 of this semester. Attitude changes towards collaborative programming will be examined. Because the survey answers will use an ordinal scale, the results can be analyzed using a Spearman rank-order correlation to make statistically-valid inferences. In order for the students to remain anonymous so they will answer more honestly, they will assign themselves an alias name to be used on both the pre- and the post-class survey. Additionally, the students will take a Meyers Briggs type of personality test the first day of class. The results of these tests will be used for the post-class analysis. The students will keep an electronic journal during the class in which they will record their impressions of using PSP and CSP each week. The majority of the students will be familiar with PSP because they had been instructed on it in their CS1 and CS2 class using the Introduction to the Personal Software Process book [23]. The CS4510 class will proceed using the framework and instructor s guide for the more advanced PSP book, A Discipline for Software Engineering [2]. Many of the PSP sub-process and scripts will be used in the CSP process and will taught to the class. In the cases where a particular sub-process differs between the CSP and the PSP, both sub-processes will be taught to the entire class. For example, there will be a class dedicated to the code review sub-process. The procedures for doing code review alone (PSP) and for doing code review collaboratively (CSP) will both be discussed and contrasted. All aspects of the development cycles for both PSP and CSP will be taught. University of Utah students are fairly non-traditional. By the time the students are seniors, many are employed full time and quite a few are married with family responsibilities. Any student that feels that their obligations prevent them from working on a collaborative team will be asked to discuss their situation with the teaching staff right away. For each of the classes, the students will be randomly divided evenly into Group C (Collaborative) and to Group I (Individual). After the random assignment, the GPA average and mean of the two groups will be compared to ensure the groups are academically equivalent. If they are not, a new random assignment will be performed until the groups are balanced. Each group will perform their assignments differently, as outlined in the Assignment Specifics section below. The students will be assured that grades will be curved, as necessary, independently for each of the groups to ensure that neither group will get an advantage in being academically successful in the class. The PSP and CSP processes require the students to record data about the time they spend and the defects they inject and remove. These data will allow quality and productivity measures to be tracked and analyzed throughout the semester as the students learn to apply more and more sub-processes. Historically, PSP data has shown a productivity improvement and a decrease in defects through a semester as the students apply more of the techniques. This experiment should show the same trend for improved productivity and defects density for all students. However, the results for the CSP projects and for the PSP projects will be analyzed separately and compared. The relationship of the aggregate measurements for all students/groups for each of the two processes will be used to validate the research 21