Exploring the Accuracy of Existing Effort Estimation Methods for Distributed Software Projects - Two Case Studies

Transcription

1 Master Thesis Software Engineering Thesis no: MSE June 2009 Exploring the Accuracy of Existing Effort Estimation Methods for Distributed Software Projects - Two Case Studies Abid Ali Khan Zaka Ullah Muhammad School of Engineering Blekinge Institute of Technology Box 520 SE Ronneby Sweden

2 This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to 20 weeks of full time studies. Contact Information: Authors: Abid Ali Khan Address: Folkparksvägen 19:16, Ronneby, Sweden Zaka Ullah Muhammad Address: Folkparksvägen 22:02, Ronneby, Sweden University advisor: Dr. Darja Šmite Assistant Professor, BTH, School of Computing, SoftCenter, Ronneby School of Computing Blekinge Institute of Technology Box 520 SE Ronneby Sweden Internet : Phone : Fax : ii

3 ABSTRACT The term Globalization brought many challenges with itself in the field of software development. The challenge of accurate effort estimation in GSD is one among them. When talking about effort estimation, the discussion starts for effort estimation methods. There are a number of effort estimation methods available. Existing effort estimation methods used for co-located projects are might not enough capable to estimate effort for distributed projects. This is why; ratio of failure of GSD projects is high. It is important to calibrate existing methods or invent new with respect to GSD environment. This thesis is an attempt to explore the accuracy of effort estimation methods for distributed projects. For this purpose, the authors selected three estimation approaches: COCOMO II, SLIM and ISBSG. COCOMO II and SLIM are two well known effort estimation methods, whereas, ISBSG is used to check the trend of a project depending upon its (ISBSG s) repository. The selection of the methods and approaches was based on their popularity and advantages over other methods/approaches. Two finished projects from two different organizations were selected and analyzed as case studies. The results indicated that effort estimation with COCOMO II deviated % for project A and 9.71% for project B. Whereas, SLIM showed the deviation of 4.17% for project A and % for project B. Thus, the authors concluded that both methods underestimated the effort in the studied cases. Furthermore, factors that might cause deviation are discussed and several solutions are recommended. Particularly, the authors state that existing effort estimation methods can be used for GSD projects but they need calibration by considering GSD factors to achieve accurate results. This calibration will help in process improvement of effort estimation. Keywords: Effort Estimation, Global Software Development (GSD), COCOMO II, SLIM, ISBSG

4 ACKNOWLEDGEMENT First, we are grateful to our creator who blessed us with the abilities to contribute in human knowledge. We would like to express our heartily gratitude for our advisor Dr. Darja Šmite, who always guided and motivated us during this thesis work. Beside an advisor, she used her leadership abilities to encourage and motivate us when we needed it. Whenever we stuck on some issues regarding thesis, she always provided us positive response. She has provided assistance in numerous ways so that we can understand easily. This thesis would not have been possible without her kind support. We are thankful to her for helping us throughout our thesis. We also would like to thankful to Dr. Cidgem Gencel, Peter Vd Stad (QSM) and Christine Green (EDS) for their kind support and providing us valuable information regarding our work. Our sincere thanks also go to our friends and fellows for providing us moral support during our studies. Finally yet importantly, we would like to thank our families for their love, motivation and support that have guided us to achieve this milestone. We dedicate our work to our respective families. ii

5 Contents ABSTRACT...I ACKNOWLEDGEMENT... II 1 INTRODUCTION BACKGROUND PROBLEM DEFINITION AIMS AND OBJECTIVES RESEARCH QUESTIONS RESEARCH OUTCOMES RESEARCH METHODOLOGY Literature Review Empirical Case Studies Data Source Data Analysis VALIDITY THREATS PROJECT MANAGEMENT IN GSD GSD CHALLENGES PROJECT MANAGEMENT IN GSD EFFORT ESTIMATION OBJECTIVES OF EFFORT ESTIMATION EFFORT ESTIMATION METHODS COCOMO II Size estimation Cost Drivers Scale Factors COCOMO II Equation SLIM SLIM Tool CONTINUOUS IMPROVEMENT Reasons for Selecting COCOMO II and SLIM CASE STUDIES PROJECTS OVERVIEW Project A Description Project B Description EFFORT ESTIMATION WITH SELECTED APPROACHES Effort Estimation with COCOMO II Equation for COCOMO II Project A: COCOMO II Project B: COCOMO II Effort Estimation with SLIM Some Facts about SLIM Equations for SLIM Project A: SLIM Project B: SLIM ESTIMATION WITH ISBSG Project A: ISBSG Project B: ISBSG SUMMARY OF CALCULATIONS iii

6 5 ACCURACY OF EFFORT ESTIMATION COMPARISON OF ESTIMATED RESULTS VS ACTUAL EFFORT FACTORS THAT MIGHT CAUSE DEVIATION Project A Project B Comparison of Factors that Might Cause Deviation RELATED WORK RECOMMENDATIONS FOR IMPROVING EFFORT ESTIMATION SUGGESTIONS AND NEW FACTORS INVOLVEMENT FOR GSD Purpose and Process of Calibration DISCUSSION AND CONCLUSIONS RESULTS APPLICABILITY OF DIFFERENT METHODS IN INDUSTRY FUTURE WORK REFERENCES APPENDIX X APPENDIX Y iv

7 LIST OF TABLES Table 1: Cost Drivers COCOMO II Table 2: Analyses of different effort estimation methods [21, 23] Table 3: Selected Effort Estimation Methods [21, 23] Table 4: Actual Effort Project A Table 5: Actual Effort Project B Table 6: Scale Factors Project A Table 7: Cost Drivers (Effort Multiplier) Project A Table 8: Scale Factors Project B Table 9: Cost Drivers (Effort Multiplier) Project B Table 10: Variables and Values for Putnam s Equation Table 11: Analysis and Selection of ISBSG Table 12: SLOC/FP [49] Table 13: Total Functional Size of Studied Projects Table 14: Summer of Calculations with COCOMO II and SLIM Table 15: Summary of Calculations with ISBSG Table 16: Comparison of Estimated Results Vs Actual Effort Table 17: Estimation Results from ISBSG Both Projects Table 18: Factors that might cause deviation All factors Table 19: Factors that might cause deviation Not common in both projects v

8 LIST OF FIGURES Figure 1: Research Outline... 4 Figure 2: Project Management and GSD Factors [50]... 7 Figure 3: Effort Estimation Equation COCOMO II Figure 4: Putnam s Equation for SLIM (Productivity Parameter) [29] Figure 5: Putnam s Equation for SLIM (Effort Estimation) Figure 6: Phase Division Project A Figure 7: Phase Division Project B Figure 8: Putnam s Equation for SLIM with PP showing PI and B value [29] Figure 9: Effort Estimation with ISBSG Project A Figure 10: Effort Estimation with SLIM Project B Figure 11: Comparison of Actual Effort with Estimated from SLIM and COCOMO II Figure 12: Comparison of Actual Calendar Month with Estimated from ISBSG Figure 13: Effort Multipliers Outsourcing Factors for Offshore Outsourcing Software Development [2] Figure 14: Proposed equation for COCOMO II [2] vi

9 Abbreviations GSD Global Software Engineering ISBSG International Software Benchmarking Standards Group RE Requirement Engineering SRS Software Requirements Specification EDS Electronic Data System SLIM Software Life-cycle Model PI Productivity Index MBI Manpower Buildup Index LOC Line Of Code SLOC Source Line of Code PM Person Month EM Effort Multiplier SF Scale Factors QSM Quantitative Software Management COCOMO II Constructive Cost Model SPI Software Process Improvement IEEE Institute of Electrical and Electronic Engineers ACM Association for Computing Machinery HRM Human Resource Management UK United Kingdom PAK Pakistan USA United States of America ISO International Organization for Standardization MS Microsoft vii

10 1 INTRODUCTION 1.1 Background Globalization of the world is on a doorstep, economic growth and rapidly new inventions are forcing the software industry to boost up development speed in order to cope with these challenges. There is a need to globalize software development in order to save the time, cost and resources [1]. Global Software Development (GSD) is a strategy in which the software development is performed beyond the boundaries such as contextual, organizational, cultural, temporal, geographical and political. In GSD the software life cycle activities is distributing among teams across different boundaries [2]. The diverse distribution of the activities among different organizations all over the world causes a number of questions that need to be answered about realization and successful execution of GSD projects. However, there are insufficient tools, techniques and methods available for distributed projects [3]. Software industry is struggling to invent tools, techniques and methods for GSD projects [3]. This is one of the reasons why GSD is still using the same tools, techniques and methods even same practices for effort estimation, which are being used in co-located projects [4]. Effort estimation is a process that estimates necessary effort (cost, time, etc). It is highly important to make accurate/reliable estimates for the project [35, 36] at the beginning [5] to support project planning, and control the project within the budget and schedule [1]. To call a project successful, it has to be on time, in budget and fulfilling the customer s requirements effectively. On time delivery of the project is a burning issue in development organizations [1, 3, 33]. It becomes more difficult when talking about distributed organizations. The complex nature of distributed projects is one of the hurdles in effort estimation due to which organizations fail to estimate accurately. There is a need to overcome these problems. Researchers and software engineering communities are struggling to explore these complexities and their solutions [7, 8, 9]. Researchers have explored the process of software effort estimation extensively in the past couple of decades [4]. Research shows that wrong estimation of effort can lead a project to failure [10]. Wrong estimation of cost reduction tends to fail a project because organizations do not calculate all the risk and cost associated with Outsourcing Development. Statistics show that 60% of GSD projects fail [11]; latest research pointed out this figure as 40% [12]. Research also highlighted that 2 out of 5 international joint-venture project teams show poor performance [13]. That is why, the need of developing tools, techniques and methods, to overcome these failures is a big challenge. Methods like COCOMO II, SLIM, COBRA etc have been developed to facilitate the software development industry [6]. These models have been working and contributing in process improvement [14, 39, 40] and providing help for the planning team. Another research about the regression analysis by Magne Jorgensen [15] explains the accuracy and bias variation of an organizational estimate of software development effort through regression analysis. He collected information about variables that would affect the accuracy or bias of estimates of the performance of the task completed by organization. He concluded, it is possible that the type of formal analysis and regression-based model in some cases, support for human judgment [15, 37, 38]. Estimation of cost and duration is one of the inherent problems and main challenges of the software engineering business. It is difficult for a project manager to give estimate for distributed projects because there are some additional factors involved that should be taken into account when estimating effort. Beside these additional factors, processes of communication and coordination (considerably more effort consuming than in co-located projects) also contribute to the difficulty in estimation of distributed software projects [2]. 1

11 Co-located development practices are frequently being used in the GSD; it is good to use the same practices but the question is, whether the tools, techniques and methods used in co-located development are also efficient for GSD? There are no specific effort estimation methods available for the distributed projects [2]. Distributed projects are using the conventional effort estimation methods. Literature is missing to evaluate and address which effort estimation methods provide accurate results for distributed projects and which lead to faulty results [2]. Literature about the factors affecting accuracy of existing effort estimation methods for distributed projects is also lacking. This thesis is an attempt to explore the accuracy of existing effort estimation methods for GSD projects and investigate the factors affecting their accuracy. 1.2 Problem Definition As stated above, current effort estimation methods used for co-located software projects do not have enough capability to estimate effort and duration accurately for distributed software projects. It results into overly cost and schedule, and most of the projects fail. This is why, it is necessary to find some new methods or calibrate existing methods for distributed software projects. It is required to explore the accuracy of existing effort estimation methods and find the potential improvement alternatives. 1.3 Aims and Objectives The main aim of this thesis project is to explore the accuracy of existing effort estimation methods for distributed software projects. To meet this goal, following objectives are set: Investigation of well known effort estimation methods in software engineering; Collection of empirical data from distributed projects for further analysis; Application of different effort estimation methods; Comparison of results produced by different methods/approaches with the actual effort of investigated projects; Analysis of the results; Comparison of the results from COCOMO II and SLIM with related studies; Improvement suggestions. 1.4 Research Questions Corresponding to the aims and objectives the following are the research questions for this thesis project: Do existing effort estimation methods accurately estimate effort for distributed software projects? Do existing effort estimation methods provide optimistic or pessimistic calculation of the selected studied cases? Which method tends to provide more accurate estimates? In case of deviations, what are the factors that shall be considered when calculating effort for distributed software projects? What could be done for the improvement of effort estimation methods in distributed software projects? 2

12 1.5 Research Outcomes Meeting with above research questions, following outcomes were produced: Analysis of accuracy of effort estimation methods/approaches (COCOMO II, SLIM and ISBSG) for two distributed finished projects; Evaluated trends in deviation of COCOMO II, SLIM and ISBSG effort estimation with actual effort for selected studied cases; List of factors influencing effort estimation in studied cases; Recommendation for Software Process Improvement (SPI) activities in relation to effort estimation for distributed projects; Recommendation with respect to related studies; Hypotheses for future research. 1.6 Research Methodology The research conducted within this thesis was exploratory in nature and based on two case studies. Interviews conducted in order to obtain empirical data considering GSD projects in the studied organizations. A structured questionnaire (See appendix X) was designed and sent to the project manager to let him understand what would be asked during interview. Questionnaire was divided into several parts to collect personal data of the interviewee and, organizational and project information. Following are some main categories defined in questionnaire: Basic Information of Interviewee; General Information about organization and projects/products; Questions regarding finished project and team for that project; Domain related questions; Data related questions. These interviews conducted in a very friendly environment. The authors tried to extract all the related information from interviewee. Project manager helped the authors to fill questionnaire, which was documented later. Afterwards, authors used s and telephonic calls to collect missing data. This collected data was used as an input for analyses, calculations and comparisons later. Actual effort received from organizations, and results from effort estimation methods in this research were compared with each other. These comparisons showed some variation between them. These results were also compared with related scientific studies. From this point, a discussion on deviation of results was started. Factors caused these deviations were also discussed. At last, but not least, suggestions for the involvement of some factors related to distributed environment in existing effort estimation methods were given. This also led towards new hypotheses for future studies. To clear research outline, the authors divided above description in four phases. The following diagram shows the detailed view of research outline. 3

13 Figure 1: Research Outline In phase 1, authors gathered data regarding two finished GSD projects from different organizations. This data consisted of user manuals, SRS documents and size of project in SLOC. Interviews were the main approach for gathering data. The authors performed analyses on gathered data according to the selected effort estimation approaches in phase 2. The estimated results from phase 2 are then compared with actual effort and related studies in phase 3. Phase 4 consisted of analyses of different factors affecting these projects. The authors provided some recommendations according to estimated results and analyses of different affecting factors. At the end, the authors concluded and discussed about outcomes from this research, and provided some hypotheses for future work Literature Review The authors used literature review to have background and current knowledge of effort estimation methods, their pros and cons, and usage in the software industry. This background knowledge helped the authors during empirical case studies. Furthermore, literature review also helped in comparing case studies results with scientific literature. This identified suggestion for the improvement of effort estimation in GSD Empirical Case Studies Data of two finished GSD projects was used in this research project. COCOMO II, SLIM and ISBSG examined with the collected data from the GSD organizations. The authors used collected data as an input for the calculations and found different results. These results compared with different related studies. 4

14 Data Source Scientific literature and the industry were the main sources for this thesis. Online databases were used to gather the scientific research published through IEEE, ACM, and other libraries. Two structured interviews were conducted with project managers of investigated GSD projects. These project managers were highly experienced in the field of software development and of course had vast knowledge for studied cases. These interviews were documented immediately. Both literature review and interviews were used as input for further operations in this thesis Data Analysis The collected data was analyzed according to the objectives of this thesis. The authors analyzed different factors related to COCOMO II, SLIM and ISBSG. These analyses were then described into tabular forms. Investigation was completed on the basis of these analyzed data. Furthermore, these investigations were modeled the possible results and performed further actions such as comparisons, recommendations etc. During analysis, the authors considered all parameters, which helped for the accomplishment of this thesis report. 1.7 Validity Threats As it has mentioned above that the nature of this study was exploratory. The authors investigated two GSD finished projects in order to explore the accuracy of existing effort estimation methods. There were many confidential issues involved for and organization in order to provide the required data for this thesis. Therefore, it was quite difficult to find more case studies in this short period. The authors documented interviewed data before forgetting it. Results gained from these two case studies might not be too strong decision making. However, it showed a trend and authors pointed out some critical issues. It is still needed to validate these results over a number of projects. Furthermore, the case studies organizations were distributed in UK and PAK. It is therefore still required to validate this thesis results in the organizations with different structure and boundaries where the time zone difference is more than eight hours such as Africa, USA, etc. 5

15 2 PROJECT MANAGEMENT IN GSD 2.1 GSD Challenges The phenomenon of Global Software Development (GSD) is taking potential and the importance of GSD boost with the passage of time. GSD offers highly skilled personal with low cost. It is one of the reasons behind shifting software industry from co-located development to GSD. This shifting brought some new challenges in software development. The problems arising from the geographical, temporal and socio-cultural distance are the main challenges for GSD. Lack of formal communication generates misunderstandings in requirements or changes in the requirement specification. Real time contact between the distributed team is more difficult with temporal and geographical difference. Socio-cultural difference leads problems, such as, different opinions about the nature of the software development process. Language problem also discourages employees to online meeting. To avoid this fear he/she prefers to asynchronous communication e.g. etc. However, sometime one cannot express a problem through only words; he needs gesture, body language and pitch [16, 41, 42, 43]. Moreover, the distance and complexity of coordination is directly proportional. As distance increases, the complexity of coordination increases in software development process. This complexity also arise lack of familiarity with remotely located colleagues and increase in communication cost as well. Obviously, trust and teamwork cannot take birth in this situation. GSD environment erg to reduce these complexities also needed to enhance the ability to focus on coordination of resources [16, 17, 44]. There is a need to overcome these challenges and provide useful and common solutions to those challenges. 2.2 Project Management in GSD Because of time zone, geographic and communication differences, the project management is now a great challenge in GSD. Project management seems to be same entity for both co-located and GSD environment and, an experienced project manager can set the project on the success path. Some concealed risks sway project s success. Project managers are advancing and updating themselves according to the new paradigm of GSD. Both (literature and industry) are lacking to provide the standardized approaches, sights of GSD process management and problem solution [18, 45, 46, 47]. Some generic problems and hurdles need to solve and remove. According to [18] conclusion about global project management, it is recommended to overcome problems of communication manner. Employees also hesitate to adopt new means of interactive communications. Switching the personnel between their partners according to task distribution causing problems associated with wrong approach in project process distribution that affects the overall result of the project. There are also some problems of unclear-shared goals in the GSD environment. A project manager has to share this knowledge with the employees and the organizations. Effort estimation is one of important activity of project management. It is closely associated with risk management [50]. For GSD project, risk management requires more intension because there are some additional GSD factors involved. According to [50], GSD factors, e-g multisourcing, geographical distribution, temporal diversity, socio-cultural diversity, linguistic diversity, contextual diversity 6

16 and political & legislative diversity are the main roots for GSD threats that imperial the success of global project. These threats are directly affect the GSD project management activities. These threats reveal the weird nature of GSD project. They also generate a force that develops obstacles in a project. Figure 2 represents the relation of GSD factors, their threats and the effects of these threats on GSD project management. GSD Factors GSD Threats Multisourcing Lack of language skills Geographic Distribution Temporal Diversity Socio-cultural Diversity Terminology difference Lack of joint risk management Linguistic Diversity Contextual Diversity Political and Legislative Diversity Wrong effort estimation Lack of trust Temporal difference Complex project measurement Inconsistency in work practice Figure 2: Project Management and GSD Factors [50] Project managers require significant effort to forecast obvious and hidden risks associated with GSD factors, and perform necessary precautions to overcome challenges associated with these factors to succeed in GSD. All these GSD factors affect project management activities and thus should be taken into account by project managers when estimating project schedule. 7

17 3 EFFORT ESTIMATION 3.1 Objectives of Effort Estimation Effort estimation plays a key role in the field of software engineering. It helps in predicting the effort and duration required for completing a project [9]. Researches show, projects that often exceed, are without planning or using unrealistic approaches to plan their costs and schedules [9, 20, 35]. As the cost and duration of projects continuous to increase, the research attention is now attracting to gain better methods for accurate effort estimation. Accurate software project estimation is one of the most important activities in software development that can solve the problems of delays in projects [20, 35, 48]. It is much difficult to plan and control projects without an effective and strong estimate, in terms of effort and schedule indicating the calendar time required. Therefore, it is necessary to estimate effort for software development projects to set deadline and cost to meet quality. Projects history helps in estimation for new projects and it ensures positive results [20]. It also involves the expertise of estimators that lead towards good results. 3.2 Effort Estimation Methods In this section, two well-known effort estimation methods (COCOMO II and SLIM), used for analyzing data from finished projects in this thesis, are described. Furthermore, the authors also provided some reasons for selecting these methods COCOMO II COCOMO II is an extension to COCOMO 81, originally published in Software Engineering Economics by Dr. Barry Boehm [21] in The word COCOMO derived from COnstructive COst MOdel. This is widely used method for estimating cost and schedule for the projects at early stages. It helps to make software implication decisions. It estimates project cost, derived directly from Person Month (PM) effort. PM is number of hours a person spends on working for software development project in a calendar month. Its nominal value is defined as 152 hours for one person in a calendar month. 160 person-hours are also treated as one PM [20]. Excluding weekends and leaves, allowed in one calendar month, calculate these hours. COCOMO II estimates effort in PM. Keep remembering, PM and duration of project are different from each other. For example, if a project is estimated for 20 PM, it can have 5 months duration [20]. COCOMO II has a well-described structure for estimating necessary effort and duration of projects. It mainly uses size of project i.e. Source Lines Of Codes (SLOC) or Function Points (FP). Many organizations are now using FP as an input for estimating effort in COCOMO II [27]. COCOMO II deals with a large variety of factors, which influence the estimation of project. It has 17 cost drivers (for post architecture model) and 5 scale factors. Scale factors introduced in COCOMO II, were not available in COCOMO 81[20, 22]. There are three sub models for COCOMO II: Application Composition Model; Post Architecture Model; Early Design Model. 8

18 The projects that use Integrated Computer Aided Software Engineering tools are mostly estimated for effort and schedule by using Application Composition Model. Rapid application development uses these tools. Furthermore, these tools also support prototype activities occurring later in spiral model [22]. The Early Design and Post Architecture models are used to estimate effort and schedule for the projects like software infrastructure, embedded systems and large applications [20]. When there is incomplete project analysis, early design model is used for rough estimation. Whereas post-architecture model is used when analysis and top-level design of project is completed and know detailed information about project [20]. Post Architecture and Early Design models use the same approach to find the size of project and scale factors as well. Reusing code and other data from previous projects is also included in product sizing [20] Size estimation Size of software is direct input to calculate effort and schedule estimation in COCOMO II. It becomes very important for reliable estimation. Sizing is a difficult task because it includes new and reused code and modified code. In COCOMO II, aggregate size is used for new and reused code with updates. These adjustments take into account by considering amount of design, code and testing changes. There are two main types of size used in COCOMO II for effort and schedule estimation [20]: Source Lines Of Code (SLOC); Function Points (FP) Cost Drivers COCOMO II enhanced and added more cost drivers as compared with COCOMO 81. These cost drivers are divided into four categories for The Post-Architecture Model, the sub model of COCOMO II. Table 1 shows the cost drivers used in The Post-Architecture Model [20]. Product Factors Platform Factors Personnel Factors Project Factors Required Reusability Execution Time Analyst Capability Use of Software (RELY) Constraint (TIME) (ACAP) Tools (TOOL) Database Size (DATA) Main Storage Constraint (STOR) Programming Capability (PCAP) Multisite Development (SITE) Product Complexity (CPLX) Platform Volatility (PVOL) Application Experience (APEX) Scheduling Factor (SCED) Developed for Reusability (RUSE) Platform Experience (PLEX) Documentation match to life-cyclemodel (DOCU) Personal Continuity (PCON) 9

19 Language & Tools Experience (LTEX) Table 1: Cost Drivers COCOMO II Scale Factors These factors were not available in COCOMO 81 and included later in COCOMO II. These scale factors let the effort estimation team to make better approximation of influencing factors. These factors are related to organizational and team characteristics. Each scale factor has values from range of very low to extra high rating level. Each rating level has a weight/value. The weight of scaling factors for different organizations and projects could be different. Following are the scaling factors in COCOMO II [20]: Precedentedness (PREC); Development Flexibility (FLEX); Architecture/ Risk Resolution (RESL); Team Cohesion (TEAM); Process Maturity (PMAT) COCOMO II Equation The basic equation for COCOMO II is shown as figure 3 [20, 28]; Figure 3: Effort Estimation Equation COCOMO II Where A = 2.94 (for COCOMO II), size is Kilo Source Lines Of Code (KSLOC), EM i is Effort Multiplier, which can be calculated from cost drivers in COCOMO II E is used to calculate Scale Factors (SF) in COCOMO II. Formula for E is; E = B+ 0.01* SF j (j= 1 to 5) Where B = 0.91 for COCOMO II [20] Cost drivers are given in table 1 Equation for duration in COCOMO II is [20, 28] Duration = [C* (PM NS ) (D+0.2*(E-B)) ] Where C = 3.67, D = 0.28, B = 0.91 and PM NS is effort in PM excluding SCED cost driver PM NS = a * Size b * Π EM i (i= 1 to 16) 10

20 3.2.2 SLIM SLIM is another algorithmic method used to estimate effort and schedule of projects. SLIM stands for Software LIfe-cycle Model and is introduced by Putnam [23]. It was developed for measuring the general size of project based on its estimated SLOC. Later, it was modified for effort estimation using Rayleigh curve model [23]. Equation I (Figure 4) is use to find the productivity parameter (PP). PP is use to calculate the effort in man-years. Equation II (Figure 4) is used to calculate effort, using value of PP from equation I. Figure 4: Putnam s Equation for SLIM (Productivity Parameter) [29] Eq. I This implies that: Figure 5: Putnam s Equation for SLIM (Effort Estimation) Eq. II Where B is special skill factor PP is productivity parameter Duration is development schedule length in years Size is Source Lines Of Code SLIM Tool SLIM tool is the product of SLIM (for the proprietary of Putnam s model) which is metrics-based estimation tool, developed by Quantitative Software Management (QSM), using validated data of over 2600 projects. These projects were classified into nine different application categories. This tool helps the management to estimate the effort and time required to build medium and large software projects. The most important thing is that it can be customized for a specific organization [24]. There are two main management indicators, Productivity Index (PI) and Manpower Buildup Index (MBI). PI could be taken as process productivity. PI values were observed from 0.1 to 34, whereas its values were given 0.1 to 40.0 in SLIM tool by QSM to overcome future contingencies. A high PI value means that project s productivity is high and it is low complex. In case of PI having values below average implies 10% more development time and 30 % more cost. Second important indicator MBI is measure of staff buildup rate. Some factors, by which MBI is influenced, are schedule pressure, task concurrency and resource constraints. Its values are observed in the range of -3 to 10. A low MBI value implies [24]: 11

21 Longer time; Fewer people; Less effort; Fewer defects; Fewer LOC/month; Higher LOC/PM. The following are steps involved in effort estimation using SLIM tool [24]: First of all estimate product size; Select/analyze the PI and MBI values for this project; Get minimum time for solution; Determine alternative solution for time-effort-cost; Chose your desired solution; At the end, generate project plan from the chosen solution. 3.3 Continuous Improvement Consideration about calibration is very important when talking about estimation. Calibration is a technique that allows application of a general model to a specific set of data. Some of the case studies have shown that calibration is important for estimation because of the following reasons [22, 25]: Size of the project and its relation with effort, which are used in different methods for estimation, could be different for different environments (skills of coding, tools used, platform understandability etc). There could be high risk involved when using generic algorithms due to its contrast with organization structure and working Calibration is important because, in particular, organizational needs and availability of data varies Historical data about completed project plays integral role in effort estimation. However, it is important to calibrate the historical data accordingly before using it for estimation [34]. There are some factors that influence calibration such as; similar projects executed in past, features of finished project, and the complexity of the software to be developed [25, 26]. It is important for organizations to determine whether their estimates arrived at, for a project are realistic or not. This could be done by validating estimates against completed project data, which would describe the correctness/accuracy of estimates. It is also required to make sure to consider current development environment with respect to previously completed projects [26]. Calibration of co-located project experience would be helpful for GSD projects. Every organization might have their calibrated estimation models. Even in one organization there might be different models used for different kind of projects. Methodology may remains the same but some influencing factors might vary from project to project, thus calibration becomes very important [26, 31]. Organizations have many past-completed projects data. It would be beneficial to use that data to estimate effort and duration for new projects. In general, existing effort estimation methods required to calibrate by the addition/involvement of some new factors related to GSD environment etc. 12

22 There is also a need to introduce new tools and techniques for the calibration of existing effort estimation methods. Dynamic tools and techniques are one in this regard [31, 32] Reasons for Selecting COCOMO II and SLIM There are a number of methods used for effort estimation. In this thesis, two wellknown methods (COCOMO II and SLIM) were used. The authors investigated and analyzed different effort estimation methods. Table 2 shows the analyses of different effort estimation methods for software projects. The authors analyzed these methods by considering different reasons and objectives for selection. Methods Reasons Availability of literature Availability of tools Coverage of parameters Used for Early estimation COCOMO II Many Free tools available More than 17 cost drivers and 5 very important scale factors SLIM Sufficient, some are not free. Yes, but most of the tools are not free. 2 main parameters such as MBI and PP Expert Judgment Available but not sufficient Not specific Depends on expert Analogy Parkinson Price to Win Top Down Bottom- Up satisfactory Few Few Few Few Few - Not specific Depends on the nature of the project, which has to be compared with. - Varies accordingly Not specific Varies accordingly Yes Yes Yes Yes Yes Yes Yes Yes Algorithmic Yes Yes No No No No No No Widely Yes Yes Yes No No No No No Used Possible to calibrate - Yes - - Yes Yes Yes, different factors are available according to situations. Yes, According to QSM Table 2: Analyses of different effort estimation methods [21, 23] After analyses, the authors selected two effort estimation methods (COCOMO II and SLIM) to find their accuracy for selected cases. Both selected methods have many advantages over other methods. Table 3 shows specifically some common objectives of COCOMO II and SLIM. Not specific Varies accordingly 13

23 Methods COCOMO II SLIM Reasons Availability of literature Many Sufficient, some are not free Availability of tools Free tools available Yes, but most of the tools are not free Coverage of parameters More than 17 cost drivers 2 main parameters such as and 5 very important scale MBI and PP factors Used for early estimation Yes Yes Algorithmic Yes Yes Widely used Yes Yes Possible to calibrate Yes, different factors are Yes, According to QSM available according to situations Table 3: Selected Effort Estimation Methods [21, 23] 14

24 4 CASE STUDIES 4.1 Projects Overview After the literature review of both methods, the authors selected two different organizations for data collection. Exchanging s and online interviews ensured the data collection process. Parameters of both methods were considered during the interviews to elicit data according to these parameters. Some other necessary data, such as delay factors and suggestion for improvement from the project manager are also considered in this regard Project A Description Name: Workforce Evaluation Tool The Workforce Evaluation Tool offers an online balance scorecard, helping users to assess changes in workforce practice. It enables organizations to calculate performance in four key perspectives workforce, customer, service, and finance. The tool offers a visual representation of the performance in the aforementioned perspectives, and alerts best and the worst performance indicators. Data collection process was very difficult because of time constraint and unavailability of concerned person. Confidentiality was another issue in this regard. There was also a problem of lack of documentation. Although, the studied organization is ISO 9001:2008 and ISO certified, and also a Microsoft Gold Certified partner, it was observed that the organization is lacking to follow formal steps of software life cycle e.g. documentation and record keeping. An online interview was conducted to discuss the questionnaire and extract some more information related to the project. Interview session was of one and half hours long. A series of s exchanged later to remove the ambiguity from the gathered data. Interview information then documented and arranged in a readable and understandable form. This data was used as input to the both of the methods. Studied project was initially estimated through expert judgment, which estimated three calendar months to finish, but it delayed. Actual completion time of the project was four calendar months. According to the project manager, the development team consisted of 7 persons. The project manager also provided the duration spent on each activity, which helped the authors in finding the actual effort in person months, which was 14.4 person months. There were two locations involved in this project. The project activity distribution was organized as follows. Requirement engineering and deployment performed in the onshore (UK) office of the organization, whereas designing, coding, technical writing and testing performed in the offshore (PAK) office. There was no specific designation for project manager explicitly in this project, and requirement analyzer worked for project management activities. Figure 6 shows the pictorial representation of phases of the project. Table 4 provides detailed distribution of effort spent on each activity. 15

25 Figure 6: Phase Division Project A Phase Number of Person Hours Person Days Persons Requirements Engineering Design Coding Testing Technical Writing Total = person months Table 4: Actual Effort Project A 16

26 4.1.2 Project B Description Name: Computerized account system Computerized account system was an application for accounts system. It facilitated the organization in multi dimensional ways, such as record keeping, inventory, sales, reports etc. This project was built to convert the organizational manual account system into computerize. The client was using a manual record keeping procedure. The client decided to convert their manual record into computerized account system to get benefits from the modern technology. The duration of this project was estimated four calendar months by expert judgment, but it delayed one and half calendar months. Therefore, the total development period of this project was five and half calendar months. The authors used same strategy as for project A for the data collection process. This time, the authors were more prepared for data collection because of previous experience in project A. That is why, data collection time for project B was short as compare to project A. Online interview was arranged with project manager. The same questionnaire was used to gather data. According to the data provided by the organization, it was seventeen person months project. The team of this project consisted on six persons. During the interview, different issues and delay factors were discussed. Like project A, same agenda was strictly followed for data collection. Data was collected according to the requirements and methods parameters. The authors then organized data and discussed all the ambiguities with concerned person through s. This data was used for further process. Figure 7 shows the pictorial representation of phases of the project. Table 5 provides detailed time spent on each activity. Figure 7: Phase Division Project B Phase Number of Person Hours Person Days Persons Project Management Requirement Engineering Design + Technical Writing Coding + Deployment Testing Total = person months Table 5: Actual Effort Project B 17

27 4.2 Effort Estimation with Selected Approaches Effort Estimation with COCOMO II Equation for COCOMO II As already discussed above that COCOMO II model estimates the required effort (in Person-Months) based on size of software project. This size is calculated into thousands source lines of code (KSLOC) of software project. The size collected from organization is supposed to be effected SLOC, excluding blank spaces and comments. Studied organizations ensured that they calculated SLOC using specific tools. Following is the formula to estimate effort in PM for COCOMO II; Effort (PM) = A * Size E * Π EM i (i= 1 to 17)... (1), where A = 2.94 (for COCOMO II) EM i is effort multiplier; one can calculate it by multiplying all cost drivers values related to that project, with each other. E is the exponent derived from five scale factors. Its equation is as follows; E = B+ 0.01* SF j (j= 1 to 5). (2), where B = 0.91 (for COCOMO II) SF j is scale factor introduced in COCOMO II and mostly contain organizational and project team characteristics. It is calculated by adding all five-scale factor values. For both cases, values for scale factors and cost drivers were analyzed using interviews data, studying organizational characteristics, expert judgment etc. For these factors, organizations are contacted several times. Equation to find duration of project is as follows; Duration = [C* (PM NS ) (D+0.2*(E-B)) ]. (3), where C = 3.67, D = 0.28, B = 0.91 and PM NS is effort in PM excluding SCED cost driver. It means that one needs to calculate PM but he should not use SCED cost driver value for multiplication and in this way, result will be called PM NS. i-e PM NS = A * Size E * Π EM i (i= 1 to 16). (4) Project A: COCOMO II Table 6 and 7 show selected scale factors and cost drivers values for project A respectively. Some of the scales were gathered during interviews and remaining was selected by studying the organizational characteristics. 18

28 Scale Factors Value Precedentedness (PREC) Very high: 1.24 Development Flexibility (FLEX) High: 2.03 Architecture / Risk Resolution (RESL) High: 2.83 Team Cohesion (TEAM) Nominal: 3.29 Process Maturity (PMAT) High: 3.14 Table 6: Scale Factors Project A Effort Multiplier Value Product Factors Required Software Reliability (RELY) Nominal: 1.00 Data Size (DATA) N/A Develop for Reuse (RUSE) Nominal: 1.00 Documentation match to life-cycle needs (DOCU) Nominal: 1.00 Product Complexity (CPLX) Low : 0.87 Platform Cost Drivers Execution Time Constraint (TIME) N/A Main Storage Constraint (STOR) N/A Platform Volatility (PVOL) Low : 0.87 Personal Cost Drivers Analyst Capability (ACAP) High : 0.85 Programmer Capability (PCAP) High : 0.88 Applications Experience (APEX) Very High : 0.81 Platform Experience (PLEX) Very High : 0.85 Language and Tool Experience (LTEX) Very High : 0.85 Personnel Continuity (PCON) N/A Project Cost Drivers Use of Software Tools (TOOL) Very High : 0.78 Multisite Development (SITE) Very Low : 1.22 Required Development Schedule (SCED) Low : 1.14 Table 7: Cost Drivers (Effort Multiplier) Project A N/A means that values for this effort multiplier are not applicable in studied project. To solve equation 1 using gathered data for effort estimation, E was calculated first. This value of E was used in effort estimation equation for COCOMO II. E = *12.53 E= Final estimation of effort based on E and the given cost drivers values are as follows; Effort (PM) = 2.94* (10.537) * Effort (PM) = 12.1PM 19

29 Above result shows, effort required for this project is approximately equal to 12.1 Person-Months. 4; Duration of the project is calculated by providing related values to equation 3 and i.e. PM NS = 2.94 * ( ) * PM NS = PM Duration = [3.67* (10.61) ( *( )) ] Duration = 7.54 While executing the gathered data with COCOMO II, it is found that 12.1 PM effort was required to complete this project. Excluding SCED factor from effort, duration is calculated as 7.54 months. This shows that an effort of 12.1 Person Months was required to complete project A with duration of 7.54 months Project B: COCOMO II Table 8 and 9 show scale factors and cost drivers values for project B respectively. Same strategy (as in project A) was used to gather scales/values for cost drivers and scales factors and Scale Factors Value Precedentedness (PREC) Very High : 1.24 Development Flexibility (FLEX) Very High : 1.01 Architecture / Risk Resolution (RESL) Very High : 1.41 Team Cohesion (TEAM) High : 2.19 Process Maturity (PMAT) High: 3.14 Table 8: Scale Factors Project B Effort Multiplier Value Product Factors Required Software Reliability (RELY) Very Low: 0.82 Data Size (DATA) N/A Develop for Reuse (RUSE) Low: 0.95 Documentation match to life-cycle needs (DOCU) Low: 0.91 Product Complexity (CPLX) Low : 0.87 Platform Cost Drivers Execution Time Constraint (TIME) N/A Main Storage Constraint (STOR) N/A Platform Volatility (PVOL) Low : 0.87 Personal Cost Drivers Analyst Capability (ACAP) High : 0.85 Programmer Capability (PCAP) High : 0.88 Applications Experience (APEX) Very High : 0.81 Platform Experience (PLEX) Very High : 0.85 Language and Tool Experience (LTEX) Very High :

30 Personnel Continuity (PCON) N/A Project Cost Drivers Use of Software Tools (TOOL) Very High : 0.78 Multisite Development (SITE) Very Low : 1.22 Required Development Schedule (SCED) Very Low : 1.43 Table 9: Cost Drivers (Effort Multiplier) Project B N/A means that values for this effort multiplier are not applicable in studied project. To solve equation 1 for effort estimation, E is calculated 1 st as below E = * 9.28 E= Final calculation of effort based on E and the given cost drivers values are as follows Effort (PM) = 2.94 * (16.821) * Effort (PM) = 15.8 PM approximately Above result shows, that effort required for this project is approximately equal to 15.8 Person-Months. Duration of the project is calculated by providing related values in equation 3 and 4; i.e. PM NS = 2.94x (16.821) x PM NS = 11 PM Duration = [3.67 * (11) ( *( )) ] Duration = 7.51 While executing gathered data of project B with COCOMO II, it is found that 15.8 Person-Months effort was required to complete this project. Excluding SCED factor from effort, duration is calculated as 7.5 months. This shows that an effort of 15.8 Person-Months was required to complete project B with duration of 7.5 months. 21

31 4.2.2 Effort Estimation with SLIM Some Facts about SLIM Enough scientific literature is available about SLIM. There is a general equation presented by Putnam for SLIM, on which the SLIM tool is working. Moreover, different articles and web sources mentioned different equations calibrated according to the need and situation. That is why; no specific calculation procedure is available. There are some parameters described in the SLIM equation e.g. PI and MBI, but there is not a specific method mentioned to find them. To discuss this issue, QSM was contacted. QSM is an authorized proprietary of the SLIM tool. QSM helped and referred to some useful sources about SLIM. They also referred to an effort estimation department of EDS in Denmark. MS Christine Green contacted by ; she suggested teleconference for detailed discussion. Teleconference was very helpful and she provided valuable suggestions. These suggestions considered valuable when executing SLIM for projects data Equations for SLIM While investigating SLIM, there are many equations found to calculate effort. Each equation consisted of some variables and constant values. Productivity Parameter (PP) calculated from the following equation; Figure 8: Putnam s Equation for SLIM with PP showing PI and B value [29] 22

32 Where table 10 shows the values according to different variables for respective projects. Variables Value for Project A Value for Project B B is special skill factor PP?? (Productivity Parameter) Duration is development schedule length in years Size is Source Lines OF CODE Table 10: Variables and Values for Putnam s Equation Project A: SLIM Using values from table 10, PP is calculated. i-e. PP= 23,676 (Which Is nearly equal to PI for business applications in Putnam s PI table- Figure 8) Calculating effort estimation equation for SLIM; Effort (person months) = 13.8 PM Project B: SLIM Using values from table 10, PP is calculated PP= (Which is nearly equal to PI = 16 for business applications in Putnam s PI table Figure 8) Calculating effort estimation equation for SLIM; Effort (person months) = 15.6 PM 23

33 4.3 Estimation with ISBSG International Software Benchmarking Standards Group (ISBSG) provides more than four thousand projects data. ISBSG contains the data in MS Excel spreadsheet that allows one to perform analyses, benchmarking, estimations and comparisons of different trends. There are different types of parameters; one can easily filter to get the desired information from ISBSG s repository [51]. The main reason of using ISBSG as third effort estimation approach was its popularity and a big repository of more than four thousand projects. The authors wanted to check similar project trends for GSD projects. Table 11 shows some more objectives of ISBSG. Estimate Checker ISBSG Reasons Availability of literature Sufficient Availability of tools Yes, but not free Coverage of parameters More than 15 parameters with sub categories Used for early estimation Yes Algorithmic No Widely used Yes Possible to calibrate Yes Projects History More than 4000 projects Table 11: Analysis and Selection of ISBSG The authors used ISBSG estimate checker to see what ISBSG s trend of related projects for estimation. The studied projects A and B have functional size between 500 and 600. ISBSG estimate checker showed results of 49 projects when data was filtered. The table 12 shows the conversion rate for different languages and table 13 provides total functional size of studied project. Language SLOC/FP C++ 29 C# 21 Java 14 ASP 32 Oracle 29 Table 12: SLOC/FP [49] Language SLOC/ FP Total SLOC Functional Size C# Oracle Table 13: Total Functional Size of Studied Projects Project A: ISBSG Since the language of development of project A was C#, the table 12 shows the conversion figure from SLOC to functional size. According to table 13, functional size of project A was 501. The following parameters were set in ISBSG estimate checker for project A to see the trends. Platform = PC Language Type = 4GL Functional Size = 501 Project Type = New Development 24

34 Figure 9 shows that projects would completed in minimum time of elapsed time (Months) for these projects are 3.32 months, maximum month and an average time of 7.44 months. Figure 9: Effort Estimation with ISBSG Project A Project B: ISBSG Similarly, Oracle is used for the development of project B. According to table 13 the functional size of project was 580. Data treated same as project A, and the parameter set for estimate checker as: Platform = PC Language Type = 4GL Functional Size = 580 Project Type = New Development Figure 10 shows small difference in the results. The checker gave close result as project A. The checker estimated that project B would completed in an average time of 7.92 with minimum time of 3.54 and maximum time of

35 Figure 10: Effort Estimation with SLIM Project B 26

36 4.4 Summary of Calculations Table 14 and 15 shows the summary of above effort estimation calculations. Project A Project B Actual Effort (Person Month) Estimated effort with SLIM (Person Month) Estimated Effort with COCOMO II (Person Month) Table 14: Summer of Calculations with COCOMO II and SLIM Project A Project B Function points ISBSG Output Lower Value Estimate Elapsed Upper Elapsed Lower Value Estimate Elapsed Elapsed time time Elapsed time time (months) (months) time (months) (months) (months) Actual Effort (Month) Upper Elapsed time (months) Table 15: Summary of Calculations with ISBSG 27

37 5 ACCURACY OF EFFORT ESTIMATION Accuracy is defined as the measure of how close a result is to its correct value. There are two ways to compare a result and its correct value: their difference and their ratio. Difference between resultant and actual value is known as deviation. If there is no deviation between these two measures then result will supposed to be accurate. As much closer a result to its actual value, it will be treated as more accurate. It means, accuracy is to measure how close a result is to its actual value [30]. There are two ways to find accuracy of a result; finding their difference or finding the ratio between them. If there are n number of projects available, and a n is the actual value and e n is the estimated value from n number of projects, then difference of accuracy between these two values will be; e n -a n In the same scenario ratio measure of accuracy will be; e n /a n In this thesis, e n - a n strategy is used to find accuracy of the methods. Calculated accuracy of effort estimation methods for studied cases is shown in table 16. Deviations found from studied cases are not very high. However, as project size increase it becomes harder to estimate effort [30]. Projects examined in this thesis are not large. This is why; it is possible that this small deviation between actual and estimated values will increase if there are big projects. With respect to results found, these methods (COCOMO II and SLIM) can be used for distributed projects but need some more calibration by considering distributed environment of projects. 5.1 Comparison of Estimated Results Vs Actual Effort Table 15 shows the effort results, estimated from COCOMO II and SLIM, actual results and their deviations. It also shows what Organizations estimated for respected projects. COCOMO II and SLIM both underestimated effort for both cases. For project A, deviation shows that SLIM estimated more close to the actual effort utilized than COCOMO II estimation. Whereas for project B, estimation from COCOMO II was nearer to the actual effort utilized than SLIM estimation. In first case, deviation between both estimated results is higher than second case. Project A Project B Actual Effort (Person Month) Estimated effort with SLIM (Person Month) Estimated Effort with COCOMO II (Person Month) Deviation with SLIM -0.6= 4.17 % -1.9= % Estimated - Actual (Person Month)= % Deviation with COCOMO II -2.3= = 9.71 % 28

38 Estimated - Actual (Person % Month)= % Table 16: Comparison of Estimated Results Vs Actual Effort Figure 11 is graphical representation of comparison of actual effort with estimated from SLIM and COCOM II. X-axes shows the actual effort in person-months, whereas Y-axes shows the project A and B, and their estimations. Figure 11: Comparison of Actual Effort with Estimated from SLIM and COCOMO II The authors also filtered studied cases with ISBSG s projects repository. ISBSG s results found too different as compare to actual and estimated. It showed the estimation of 7.69 person months with minimum effort of 3.45 and maximum effort of for project A. It does not seem a realistic approach to say that the estimation of a project is 7 person months and it can fall between 3 and 17 months. Similarly, it gave the same trend for project B and one cannot conclude for GSD project based on ISBSG results. Because it is like pointing out the sky and unrealistic, there is a huge gap between the results and cannot be use for decision-making purpose. In addition, there was no parameter related to GSD found in ISBSG. ISBSG was not categorizing whether projects are co-located or distributed. Function points ISBSG Output Lower Value Elapsed time (months) Project A Project B Estimate Elapsed time (months) Upper Elapsed time (months) Lower Value Elapsed time (months) Estimate Elapsed time (months) Upper Elapsed time (months) Table 17: Estimation Results from ISBSG Both Projects Figure 12 shows the graphical representation for actual calendar month spent on both projects, and estimated with SLIM and COCOMO II. X-axes represent the calendar months, whereas Y-axes represent the project A and project B estimations. Detail for deviation has already been discussed above. 29

39 Figure 12: Comparison of Actual Calendar Month with Estimated from ISBSG 5.2 Factors that Might Cause Deviation Results from the table 16 and figure 11, show that both projects were underestimated. During the interviews, some of the factors that might cause deviation were explored. The authors also described these factors separately in the coming section. Authors also observed that most of the factors are common in both projects. These factors are also related to GSD challenges. However, definitely, there were some others as well Project A Following are the factors that which might cause deviation for project A; Time zone difference between on-shore and off-shore offices Delays in confirmations from the client s end. This was the biggest problem observed that might be a cause for deviation. There were different issues related to this, such as; o It was difficult to identify a single source of requirements. Since many people were involved on client s end, it causes problems to gather all of them for a requirement analysis session o Lack of cooperation and hesitation in sharing details from client s end was another issue in this regards. They usually found it difficult to spare sufficient time for requirement analysis phase o From Client side, no availability of alternate person in the absence of concern person o Hesitation and resistance in signing off SRS documents o Client usually did not have clarity about their requirements hence they mostly liked to build the project incrementally, each release/version of the software was followed by the change request from Client. Clients liked to refine their requirements having seen something in operations (i.e. demo of the software) Another factor was shared resource s availability issue in their PK office. Such resources include DBAs, tech writers, graphic designer, test engineer etc Unrealistic project deadlines dictated by clients- Work pressure caused many mistakes, which took lot of time for corrections 30

40 Unavailability of the concern person in a desired time Project B Following are the factors that might cause deviation for project A; Since the RE, design and technical writing activates on the onshore and project management, testing coding and deployment were on the offshore side, so there were lot of communication tradeoff between two team. They consumed much of schedule time and communication cost It was also observed that the cohesion between the offshore team was high but there was lack of co-ordination with onshore team o Time zone difference was one of the reasons behind lack of co-ordination No rich contact with the client Large amount of time wasted on requirement engineering (RE) process because activities were divided in different locations Unavailability of the concern person in a desired time Comparison of Factors that Might Cause Deviation Table 18 summarizes the factors that might cause deviation X= It denotes the existence of the factor -= It denotes that the factor did not exist Factors that might cause deviation Project A Project B Different time zones X X Delay in response X X Unavailability of concerned person X X Trust X - Immature client regarding requirements X X Shared resources - X Unrealistic deadlines X - Communication X X Organization or team structure - X Work pressure X X Table 18: Factors that might cause deviation All factors Table 19 shows the factors that were not common in both projects Factors that might cause Project A Project B deviation Lack of trust X - Shared resources - X Unrealistic deadline X - Organizational structure - X Table 19: Factors that might cause deviation Not common in both projects Rhetorical analysis of factors, in both cases clearly shows that these factors are revolving around the main GSD challenges i.e. geographic, time zone, communication and co-ordination differences. The authors observed that most of the factors that 31

41 probable cause delays in project were raised from GSD new challenges. This is only suggestion and observation at this small level, need more studies for big industrial project. 5.3 Related Work Betz and Mäkiö [2] also concluded that there are GSD factors need to add in COCOMOII. They suggested eleven new GSD factors (Figure 13) to be calibrated with COCOMOII. Figure 13: Effort Multipliers Outsourcing Factors for Offshore Outsourcing Software Development [2] They also proposed new equation for COCOMO II Figure 14: Proposed equation for COCOMO II [2] The new effort multipliers are multiplied with existing effort multiplier in the above equation. The proposed equation is an attempt for improvement, but it does not give a precise figure of accurate estimation and need more calibration and validation. The authors also support the aim of the related studies and suggest for more calibration of the models in order to precise effort estimation for GSD projects. Another study about the efficiency of GSD projects described in [52] that GSD projects take development time 2.5 times more as compare to co-located development. It is erg to reduce this development time gap. This is obvious that there are some obstacles, which affect the GSD speed, and effort estimation might be one of obstacle among them. In addition, there is lack of effort estimation methods for GSD projects. New methods need to invent or calibration required the existing methods to overcome the problem of delay of GSD projects. 32

42 6 RECOMMENDATIONS FOR IMPROVING EFFORT ESTIMATION Considering calculation results from both of the methods and observing them with respect to influencing factors, authors concluded that if there is involvement of influencing factors such as, lack of trust and unrealistic deadlines then it is better to use SLIM as it gave good results in this situation. Whereas, if organizational team structure is big issue and there are limited shared resources available, then COCOMO II would be better choice. Authors concluded this solution only based on two projects. It would be different if projects were large. Therefore, the authors recommend it to analyze these methods for different projects with different organizational structure involving. 6.1 Suggestions and new factors involvement for GSD In order to improve the process of effort estimation in global software development, it is necessary to enhance the performance of effort estimation methods. There are two ways, either introduce new effort estimation methods for GSD or improve the existing methods. Inventing new methods is a good idea but it will take a lot of research time to find new solutions rather to improve existing effort estimation methods in relation to GSD. Empirical studies in this thesis showed that both models underestimated effort. Accurate effort estimation for GSD is still big challenge. Probably, there is a need to calibrate these methods with respect to GSD environment. For this purpose, some necessary factors related to GSD need to be introduced. The question is what are the factors involving that might cause deviation in the results. GSD projects have some new factors that need to be considered in effort estimation in order to get accurate estimation. SLIM and COCOMO II both lacking these factors. The following are some GSD factors that might need to accommodate in the models equations. Time zone difference between onshore offshore sites Cultural diversity and language barriers Problem related to different organizational structure Co-ordination among (onshore/offshore) teams The above factors involved directly or indirectly in the cost of project and the performance of team Purpose and Process of Calibration As it has mentioned above in chapter 3 that COCOMO II and SLIM are two algorithmic and well-organized methods with several parameters. COCOMO II is derived from COCOMO 81. Later, it was calibrated due to numerous complaints from its users. Several new parameters introduced, calibrated and validated over a number of projects [20]. These parameters can be used to calibrate this method according to organization and project needs. One can use or exclude some parameters if not related to that specific project or organization. E-g If a project, where there is less than 50% usage of real memory by software system then the STOR parameter could be ignored in this situation. According to QSM and EDS, SLIM can also be calibrated according to different situations, which is a positive sign for SLIM users. 33

43 An organization can also calibrate such methods by using their projects history. Historical data becomes very useful while calibrating. It plays an important role for calibration. In the context of this thesis, a pilot calibration is shown as under. According to [2], let suppose cultural difference is introduced as a new cost driver in COCOMO II model and set their values between 1 and 2 for different scales. These scale values can be derived by studying the culture difference between onshore and offshore sites. I.e. Same culture would lead to a low scale value, and very different culture would be very high or extra high scale value. Suppose 1.08 is the low scale value for cultural difference cost driver The total effort for project A becomes: PM= Now suppose 1.15 is the nominal scale value for cultural difference, effort becomes: PM= Be remembering that actual effort for project A is; Actual PM = 14.4 Adding only one new cost driver factor to COCOMO II, it affected the total effort. Similarly, if other factors are added as effort multiplier to COCOMO II then it will be a different story. Similarly, in SLIM, if GSD factors are added when calculating PP value for SLIM model, it is obvious that it will affect the figure of PP, which automatically affects the result. Appendix Y provided by EDS organization, they use it for productivity assessment. Although the questionnaire contains the information about the communication, culture and coordination factors but these are not differentiated in context of GSD. The questions under management /leadership and development team categories in appendix Y need more elaboration according to GSD phenomenon. These questions might need sub categorization and add more questions regarding GSD. It is necessary to consider GSD factors when calculating productivity index for GSD organization. SLIM uses very few parameters for estimation as compare to COCOMO II, which makes the process easy, but there is a need to add GSD factors when estimating GSD projects. 34

44 7 DISCUSSION AND CONCLUSIONS 7.1 Results The main concern with this research was to explore the accuracy of existing effort estimation methods for distributed projects. To achieve this milestone, the authors set some research question. Two case studies were used to answers these research questions. Below are the mapping between research question of this thesis and their answers. This mapping verifies the completeness of research questions for this thesis. Some sections are overlapping with respect to questions and answering more than one question. Following are the research questions and their possible answers. Do existing effort estimation methods accurately estimate effort for distributed software projects? The existing effort estimation methods were not accurate for studied cases in this thesis. Sections 4.4 and 5.1 show the comparison of actual effort and estimated with existing methods. Accuracy of existing effort estimation methods is also described in these sections. Do existing effort estimation methods provide optimistic or pessimistic calculation of the selected studied cases? Comparisons in section 5.1 shows that existing effort estimation methods used in this thesis underestimated (pessimistic) effort for both studied cases. Which method tends to provide more accurate estimates? Answering this question, the authors found that each method provided closer result for one studied case than other did. For project A, SLIM was more accurate whereas for project B, COCOMO II was more close to actual. In case of deviations, what are the factors that shall be considered when calculating effort for distributed software projects? There are some GSD factors like cultural diversity, trust, organizational structure and some other factors, which shall be considered when calculating effort for distributed software projects. Section 6.1 summarizes the factors that are necessary to consider when calculating effort for distributed software project. What could be done for the improvement of effort estimation methods in distributed software projects? Suggestions and recommendations for the improvement of effort estimation and its methods are described in section 6 and 6.1. While answering above questions, it is found that existing methods underestimated effort for both cases. Although, the deviations are not too much however, it would be better if they slightly over estimated effort [19]. Underestimated effort is not easy to manage and thus leads to project failures. Results from studied cases show that the existing effort estimation methods lacking factors related to GSD environment. The existing effort estimation methods need improvement so that they estimate accurate effort for GSD projects. It might need to add/remove or merge the existing factors of both the models accordingly. Investigation of new factors might be a difficult challenge for the researchers, not only investigate but also calibrate them to the methods. The current methods e.g. COCOMO II and SLIM require amplification and calibration with respect to GSD requirements. 35

45 When considering average results deviation, SLIM gave closer results to the actual efforts of studied projects. Based on these results, SLIM is more suitable for GSD projects contrary to COCOMO II. However, some constraints and other factors also need to consider for the selection of suitable effort estimation method according to the situation. Applicability of different methods in industry is discussed in following section. 7.2 Applicability of Different Methods in Industry Descriptions of both projects show that organizations used expert judgment for the effort estimation of both projects. For first project, organization estimated it as three calendar months, but it delayed and completed in four calendar months. Similarly, for project B, organization estimated it to complete in four calendar months, but it also delayed and completed in five and half calendar months. While comparing actual and estimated durations, authors found that project A was underestimated 25% than actual, and project B was underestimated 28 % than the actual duration. In both cases, expert judgment gave underestimated results. This is not a good sign and may lead a project toward failure. It would be better if expert judgment gave results closer or at least overly estimated to actual. On the other hand, algorithmic and well-organized methods (such as COCOMO II and SLIM) estimated more close to actual as compare to expert estimation. Comparison from table 16 and graph 11 shows that both (COCOMO II and SLIM) methods underestimated (gave pessimistic) effort for both studied cases. These deviations are not that much and could be ignored if estimated at start of project when enough information about project is not available. Furthermore, there are always some risks involved, which are unknown at start. However, these results were obtained when projects have been completed and maximum information for the projects (such as actual size of project in SLOC) was available. These results show that the deviations might increase if the information of project is not enough when estimating effort e-g at start [19]. The authors expect more deviation indeed in negative (pessimistic) in such situation. Therefore, the used methods should provide accurate results for post estimation. As the results from algorithmic methods were more accurate than expert judgment s results, it shows the importance and need to such organized methods in the industry. Although, usage of such organized methods will lead toward an overly cost on management activities, but it will result into estimation that is more accurate. i.e. More projects will go towards success. It is therefore necessary for organizations to adopt such organized effort estimation methods to reduce their project failures ratio. However, it is not realistic approach / possible to use both methods for estimation. Therefore, it is yet to answer that, which method is more applicable according to cost and other constraints. According to authors, it was not easy to analyze all the parameters of COCOMO II with the low amount of expertise. It took a lot time of authors to find the right scale values for a couple of scale factors. Both of the case studies for this thesis were obtained from Microsoft and ISO certified organizations. Whereas, the selection of process maturity (PMAT), the scale factor of COCOMO II was based on CMMI certifications. Therefore, the authors had to do some extra work for studying and relating the organization characteristics with CMMI levels. Similarly, analyses of effort multipliers (cost drivers) in COCOMO II were another challenge for authors. To make this process fair and realistic, authors had to contact the each organization several times. Other calculations such as solving equations were not difficult; 36

46 COCOMO II team already provided a couple of equations and constants values used in those equations. In addition, section shows that a huge amount of literature is available for COCOMO II. Furthermore, free tools are available for COCOMO II. This shows that COCOMO II does not require too much cost for its adaptation in an organization. However, it requires a lot more expertise to use it because it uses a number of parameters for estimation, and one should be aware of all these. Studying SLIM, two main parameters are found for its calculation i.e. PI and MBI. However, later in equations, authors found only one main parameter PI. There were two ways to find PI value, either from history project or from SLIM database. According to EDS, several other sub parameters are required to consider for calculating PI. It was simply not possible for authors to calculate PI value using those sub parameters. Therefore, authors used PI equation to calculate its value and then used it in main equation of SLIM. In addition, it is not possible for a new organization to determine PI value from projects history. Furthermore, SLIM is a proprietary tool, which is very expensive to buy. This will increase the cost of an organization for projects. Above discussion shows that both methods (COCOMO II and SLIM) have their own pros and cons. An organization should consider above mentioned pros and cons in order to select a method for effort estimation in their organizations. However, SLIM provided more accurate average result than COCOMO II for studied cases. Therefore, authors suggest that if SLIM is freely available, it will be better to use SLIM for effort estimation for GSD projects. 7.3 Future Work Authors suggested a couple of factors that require investigation for the sake of effort estimation methods in GSD. Exploring concealed factors for both of the models and adjusting their values for them would be fascinating future work. The authors suggest some hypotheses for the accuracy of methods that required testing in this manner accuracy of effort estimation methods for distributed and co-located projects is the same. Furthermore, to explore which method provides more accuracy, it can be tested like SLIM provides more accurate estimation than COCOMO II for GSD projects. Testing these hypotheses might be beneficial for the improvement of effort estimation methods. More research work needed in this area in the context of GSD. 37

47 8 REFERENCES [1]. D. Šmite Global Software Development Projects in One of the Biggest Companies in Latvia: Is Geographical Distribution a Problem, In the SPIP journal, Wiley, 2006; Vol.11, pp [2]. Betz.S and Mäkiö.J. Amplification of the COCOMO II regarding Offshore Software Projects. University of Karlsruhe, Institute AIFB, Hertzstrasse Karlsruhe, Germany. Avalible on intenet DOI= abid=58&mid=387 (last visited on feb 09). [3]. D. Šmite, Doctoral Thesis: Global Software Engineering Improvement, University of Latvia, 2007 [4]. Fernando Machado, Luis Joyanes: Effort Estimation in Agile Software Development: A Method and a Case Study. Software Engineering Research and Practice 2005: [5]. Mukhopadhyay, T., Kekre, S Software Effort Models for Early Estimation of Process Control Applications. IEEE Trans. Softw. Eng. 18, 10 (Oct. 1992), DOI= [6]. Boehm, B. W. and Valerdi, R Achievements and Challenges in Cocomo- Based Software Resource Estimation. IEEE Softw. 25, 5 (Sep. 2008), DOI= [7]. Prikladnicki, R., Audy, J. L., Damian, D., and de Oliveira, T. C Distributed Software Development: Practices and challenges in different business strategies of offshoring and onshoring. In Proceedings of the international Conference on Global Software Engineering (August 27-30, 2007). ICGSE. IEEE Computer Society, Washington, DC, DOI= [8]. Shepperd, M., Schofield, C Estimating Software Project Effort Using Analogies. IEEE Trans. Softw. Eng. 23, 11 (Nov. 1997), DOI= [9]. Boehm, B., Valerdi R., "Achievements and Challenges in Software Resource Estimation," IEEE Software, 25(5), 74-83, October 2008 [10]. Kristian Marius Furulund, Kjetil Moløkken-Østvold: Increasing Software Effort Estimation Accuracy Using Experience Data, Estimation Models and Checklists. QSIC 2007: [11]. James F. Kile,Donald Little and Samir Shah., The Importance of Effective Requirements Management in Offshore Software Development Projects, School of Computer Science and Information Systems,Pace University [12]. Betz, S., Makio, J., and Stephan, R Offshoring of Software Development - Methods and Tools for Risk Management. In Proceedings of the international Conference on Global Software Engineering (August 27-30, 2007). ICGSE. 38

48 IEEE Computer Society, Washington, DC, DOI= [13]. P.W. Beamish and A. Delios, Incidence and Propensity of Alliance Formation, in Cooperative Strategies: European Perspectives, New Lexington Press, San Fransisco. [14]. Rahhal, S. and Madhavji, N. H An effort estimation model for implementing ISO In Proceedings of the 2nd IEEE Software Engineering Standards Symposium (August 21-25, 1995). ISESS. IEEE Computer Society, Washington, DC, 278. [15]. Jørgensen, M Regression Models of Software Development Effort Estimation Accuracy and Bias. Empirical Softw. Engg. 9, 4 (Dec. 2004), DOI= [16]. Carmel, E.: Global Software Teams: Collaborating Across Borders and Time Zones, 1st edn. Prentice-Hall, Upper Saddle River (1999) [17]. Ågerfalk, P.J., B. Fitzgerald, H. Holmström Olsson, and E. Ó Conchúir. Benefits of Global Software Development: The Known and Unknown in International Conference on Software Process, ICSP Leipzig, Germany,: Springer Berlin / Heidelberg. [18]. D. Šmite, "Global Software Development Project Management Distance Overcoming", In Proc. of the Int. EuroSPI conference, LNCS, November 2004, Norway, pp [19]. J.M Verner and W.M.Evanco. state of the practice of effort estimation in business environments college of information science and technology Drexel university USA. [20]. Boehm, B. W., Clark, Horowitz, Brown, Reifer, Chulani, Madachy, R., and Steece, B Software Cost Estimation with Cocomo II with Cdrom. 1st. Prentice Hall PTR. [21]. Boehm, B. W Software Engineering Economics. 1st. Prentice Hall PTR. [22]. Clark, B., Devnani-Chulani, S., and Boehm, B Calibrating the COCOMO II post-architecture model. In Proceedings of the 20th international Conference on Software Engineering (Kyoto, Japan, April 19-25, 1998). International Conference on Software Engineering. IEEE Computer Society, Washington, DC, [23]. Kemerer, C. F An empirical validation of software cost estimation models. Commun. ACM 30, 5 (May. 1987), DOI= [24]. Panlilio-Yap, N Software estimation using the SLIM tool. In Proceedings of the 1992 Conference of the Centre For Advanced Studies on Collaborative Research - Volume 1 (Toronto, Ontario, Canada, November 09-12, 1992). J. Botsford, A. Ryman, J. Slonim, and D. Taylor, Eds. IBM Centre for Advanced Studies Conference. IBM Press,

49 [25]. Jeffery, D. R. and Low, G Calibrating estimation tools for software development. Softw. Eng. J. 5, 4 (Jul. 1990), [26]. Vesterinen, P Issues in calibrating effort estimation models. SIGSOFT Softw. Eng. Notes 24, 3 (May. 1999), DOI= [27]. DR.A.LRollo, Functional Size measurement and COCOMO A synergistic approach. Software Measurement Services Ltd, availible on internet, last visited [28]. Overview of COCOMO. Available at last visited [29]. Software Project Management Meets Six Sigma. Available at last visited [30]. Bruce L. and Xiangzhu. G., "ASSESSING SOFTWARE COST ESTIMATION MODELS: CRITERIA FOR ACCURACY, CONSISTENCY AND REGRESSION," School of Multimedia and Information Technology, Southern Cross University, Lismore, NSW Australia [31]. Shaik, T., Khan, Z., and Lehaney, B Improving cost model development process using advanced data generation and data analysis techniques. Int. J. Netw. Virtual Organ. 4, 3 (Sep. 2007), DOI= [32]. Baldassarre, M.T.; Boffoli, N.; Caivano, D.; Visaggio, G., "SPEED: Software Project Effort Evaluator based on Dynamic-calibration," Software Maintenance, ICSM '06. 22nd IEEE International Conference on, vol., no., pp , Sept URL: = [33]. Bashir, H. A., V. Thomson., Models for estimating design effort and time. Design Studies, 22(2) [34]. Chen, Z., Hihn, J., and Lum, K Selecting Best Practices for Effort Estimation. IEEE Trans. Softw. Eng. 32, 11 (Nov. 2006), DOI= [35]. Wittig, G., "Estimating Software Development Effort with Connectionist Models," Monash University Paper 33/ [36]. Vinay Kumar, K., Ravi, V., Carr, M., and Raj Kiran, N Software development cost estimation using wavelet neural networks. J. Syst. Softw. 81, 11 (Nov. 2008), DOI= [37]. Briand, L. C., El Emam, K., Surmann, D., Wieczorek, I., and Maxwell, K. D An assessment and comparison of common software cost estimation modeling techniques. International Conference on Software Engineering, Los Angeles, USA, ACM, New York, pp

50 [38]. Miyazaki, Y., Terakado, K., Ozaki, K., and Nozaki, H Robust regression for developing software estimation models. Journal of Systems and Software 27: [39]. Londeix, B. "Cost Estimation for Software Development" 1987, Addison- Wesley Publishing Company [40]. Matson, J.E. and Barrett, B.C. and Mellichamp, lm. "Software Development Cost Estimation Using Function Points," IEEE Transactions on Software Engineering, Vol 20. No. 4, April 1994, pp.27s-287 [41]. Ilan Oshri, Julia Kotlarsky, Leslie Willcocks, Missing links: building critical social ties for global collaborative teamwork, Communications of the ACM, v.51 n.4, p.76-81, April 2008 [42]. Kenneth E. Nidiffer, Dana Dolan, Evolving Distributed Project Management, IEEE Software, v.22 n.5, p.63-72, September 2005 [43]. Jennifer L. Gibbs, Culture as kaleidoscope: navigating cultural tensions in global collaboration, Proceeding of the 2009 international workshop on Intercultural collaboration, February 20-21, 2009, Palo Alto, California, USA [44]. Ågerfalk, P.J., Fitzgerald, B., Holmström, H., Lings, B., Lundell, B., Conchúir, E.Ó.: A framework for considering opportunities and threats in distributed software development. In: International Workshop on Distributed Software Development, Paris, France, Austrian Computer Society (2005) [45]. Carmel, E., Agarwal, R. "Tactical Approaches for Alleviating Distance in Global Software Development", IEEE Software, Vol. 18, No. 2, 2001, pp [46]. Damian D. and Moitra D., "Global Software Development: How Far Have We Come?", IEEE Software, Vol. 23, No , pp [47]. DeLone, W., et al. "Bridging global boundaries for IS project success" In 38th Hawaii International Conference on System Sciences Big Island, HI, United States: Institute of Electrical and Electronics Engineers Computer Society. [48]. Nguyen, V., Steece, B., and Boehm, B A constrained regression technique for cocomo calibration. In Proceedings of the Second ACM-IEEE international Symposium on Empirical Software Engineering and Measurement (Kaiserslautern, Germany, October 09-10, 2008). ESEM '08. ACM, New York, NY, DOI= [49]. QSM Function Point Programming Langueages Table. availible on internet, last visited on [50]. D. Šmite, J. Borzovs Managing Uncertainty in Globally Distributed Software Development Projects, University of Latvia, Computer Science and Information Technologies, Volume 733, pp Available online: [51]. ISBSG Dataset

51 [52]. Herbsleb, J.D. & Mockus, A. (2003). An empirical study of speed and communication in globally-distributed software development. IEEE Transactions on Software Engineering, 29, 3,

52 APPENDIX X Interview Questions Basic Information of Interviewee Name Position Experience in this field General Information about Organization Overall Introduction of organization and projects/products o What kind of applications you develop? Is your organization certified by ISO and CMMI? o In which level of CMMI you are? o What standard of ISO you have? Questions regarding finished project 1. What type of project it was? Please let us know some general details about this project. 2. Who was the project manager (Name + Experience)? 3. Who was the team leader (Name + Experience)? 4. What was the team size? How many persons were involved in different phases such as, RE, coding, testing? 5. What was overall experience and skill level of team? High Medium Low Effort Estimation 6. What was the actual duration of the project? 7. Do you have the data about the actual effort utilization in the project? In particular, we are interested to know, what was the total effort spent on the project, what was the distribution of effort on every activity such as RE, coding, testing? 8. Do you document all the processes (such as, Software Requirements Specification, Analysis, testing results etc) of the project? 9. When estimating the project size, did you follow any particular procedure or method? Where is the estimation procedure documented? 10. Was estimated effort in-line with actual effort wasted? If not, what were the reasons behind deviation? 11. Which process model did you follow? 12. Which Software tools and techniques you used for this project? What was the experience of team with used tools and techniques? 13. Which programming language and database you used? What is the size of database? (Increasing with time) 14. Was this a completely new project or similar to previous experience? 15. What was the total number of lines of code? 16. How do you track the technical activities in your project? How is the status of the project communicated to the team? 17. How overall communication took place between team members and leaders? Face to face Online meetings (Video Conferencing etc) 43

53 s Online Messenger Others, please explain 18. How communication took place between different stakeholders? Face to face Online meetings (Video Conferencing etc) s Online Messenger Other, please explain 19. What was cohesion among project team? Good Moderate Bad Request for the following documents User Manuals or Software Requirement Specification (SRS) Documents (updated) Actual Effort wasted Updated Gantt chart Detailed effort utilization per activity (statistics from daily or weekly reports) 44

54 APPENDIX Y SLIM Productivity Assessment Summary This section summarizes the Productivity Assessment questions. TOOLS AND METHODS SUMMARY DEVELOPMENT SYSTEM How familiar are you with the development hardware (0-10)? How available is the development system (0-10)? DBMS How extensive is the role of the database management system (0-10)? What is your DBMS tools capability (0-10, 0 = no tools)? SCREEN WRITER How screen intensive is the system (0-10)? What is your screen writer capability (0-10, 0 = no tools)? REPORT WRITER How report intensive is the system (0-10)? What is your report writer capability (0-10, 0=no tools)? FILE HANDLING How intensive is file handling in this system (0-10)? What is the capability of your file handling tools (0-10, 0=no tools)? TOOLS How well do you use diagramming tools (0-10, 0=no tools)? How well do you use testing tools (0-10, 0=no tools)? How well do you use programming tools (0-10, 0=no tools)? How well do you use configuration management tools (0-10, 0=no tools)? How well do you use project management tools (0-10, 0=no tools)? How well do you use documentation tools (0-10, 0=no tools)? How well do you use QA tools (0-10, 0=no tools)? How good are your database conversion utilities (0-10, 0=no tools)? How well are your tools integrated (0-10)? DEVELOPMENT STANDARD How robust are your development standards (0-10, 0=no standard)? How well do you adhere to your development standard (0-10)? What is you level of experience with them(0-10)? How well do your standards tailor to different sized systems (0-10) TECHNICAL DIFFICULTY SUMMARY CONSTRAINTS How memory intensive is this system (0-10)? How data intensive is this system (0-10)? How complex is data manipulation in this system (0-10)? NEW ALGORITHMS What is the volume of new algorithms in this system (0-10)? What is the complexity of developing the new algorithms (0-10)? NEW LOGIC What is the volume of new logic in the system (0-10)? What is the complexity of developing the new logic (0-10)? COMPLEXITY What is the volume of expected requirements changes (0-10)? How complex is the customer interface (0-10)? 45

55 How complex is the external system interface (0-10)? How difficult is existing code integration and testing (0-10)? How intensive are documentation requirements (0-10)? PLATFORM STABILITY How stable is the hardware platform (0-10)? How stable is the system software platform (0-10)? PERSONNEL CAPABILITY SUMMARY MANAGEMENT/LEADERSHIP How good is management and leadership on this project (0-10)? ENVIRONMENT What is the availability of training (0-10)? What is the anticipated level of staff turnover (0-10)? DEVELOPMENT TEAM What is the availability of skilled manpower (0-10)? What is the level of functional knowledge (0-10)? How experienced is the development team with this application type (0-10)? How motivated is the development team (0-10)? How cohesive is the development team (0-10)? What is the level of human communication complexity (0-10)? REUSED; UNMODIFIED SOFTWARE SUMMARY This section summarizes the Reused, Unmodified Software questions. Relative percent of reused software in this system (0 to 10)? Complexity of integrating products with new code (0 to 10)? Experience using the specific products (0 to 10)? Number of functional interfaces present (0 to 10)? Relative percent of functional interfaces that will be used (0 to 10)? Complexity of using the functional interfaces (0 to 10)? Research time required to select products (0 to 10)? Analysis effort required to assess impact to existing code (0 to 10)? Usefulness of system documentation if required (0 to 10)? Effectiveness of external customer support if required (0 to 10)? Additional user documentation required for reused products (0 to 10)? Additional regression testing required for reused products (0 to 10)? 46