3 Software Measurement European Forum Rome (Italy), 11 May 2006 A on the Evaluation of COSMIC-FFP and Use Case Points Çiğdem Gencel, Luigi Buglione, Onur Demirörs, Pınar Efe 1
Agenda Introduction Description of the Project Measurement Results of the Analysis of the Results Effort Estimation Conclusion 2
Introduction Functional Sie Measurement (FSM) methods 3
Introduction ISO/IEC 14143: IT - Software Measurement - Functional Sie In 1996, ISO started a working group on FSM to establish common principles of those methods. ISO/IEC 14143-1 defines the fundamental concepts of FSM promoting the consistent interpretation of FSM principles. 4
Introduction ISO Certified Methods According to the requirements proposed in the ISO/IEC 14143-x series, currently four methods have been accepted as FSMMs: COSMIC FFP 2.1 (ISO/IEC 19761:2003) IFPUG FPA 4.1 (ISO/IEC 20926:2003) Mark II FPA 1.3.1 (ISO/IEC 20968:2002) NESMA FSM 1.0 (ISO/IEC 24570:2005) 5
Introduction Sie Measurement and O-O Software Development Object Points (OP) OP Object Oriented Function Points(OOFP) OOFP Predictive Object Points (POPs) POPs Component Object Points (COPs) COPs Object Oriented Project Sie Estimation (OOPS) OOPS OO-Method Function Points Distance-Based Approach Vector-Based Approach Statistical Object Model (SOM) SOM Shepperd and Cartwright Sie Prediction System Use Case Points (UCP) UCP 6
Methods applied & Objectives Methods applied: COSMIC-FFP Use Case Points Objectives: to explore the applicability of COSMIC-FFP and Use Case Points methods to measure the sie of a data strong system which has also algorithmic components to compare the effort estimates obtained from the sie figures and the actual development effort utilied to evaluate and discuss the results in order to improve both the estimation process as well as strengthen analysis & design practices 7
Description of the Case Project The case project was: a web based, military inventory management project integrated with a document management system developed by Turkish independent supplier, has the ISO 9001:2000 and AQAP 110 certifications and CMMI/SW Maturity Level 3 in internal assessment. started in October 2004 completed in December 2005 a data-strong system, involving typical database operations a number of algorithmic operations 8
Effort by SDLC phases & tools/languages Analysis and Design UML for modeling analysis and design Rational Rose Enterprise Edition Development Java as programming language IBM WebSphere Application Developer Java Development Kit (JDK) Tomcat Application Server Oracle 9i Database Management System Organiational Internal Development Framework 9
Composition of the project team 1 project manager: 1 senior software engineer (development team - full time): 3-years software development experience, expert in object oriented analysis, design and development 2 software engineers (development team - part time): 5-years software development experience, expert in object oriented analysis, design and development 1 software engineer (development team - full time): 15-years experience; 5-years experience as a project manager 2-years software development experience 1 junior software test engineer (development team - part time): 1-year experience 10
Conduction & Data Collection (1/2) The case study was conducted in January 2006 Two people performed the sie measurement; one of them is the developer of this project, and the other is one of the authors of this study experienced in using the COSMIC-FFP method, but not certified by COSMIC the first experience in implementing UCP A UC based software development effort estimation tool named EEstimate was used for the measurement. Effort utilied for COSMIC-FFP measurement: 12.58 person-hours Effort utilied for UCP measurement: 3 person-hours 11
Conduction & Data Collection (2/2) Software Requirements Specification (SRS) document of the case project, which involves 125 Use Cases (UC), was used for measurement. developed according to SRS standard for objectoriented projects of the company itself the use cases, FURs, of the SRS usually corresponds to the logical transactions in COSMICFFP 12
Results Sie (1/3) COSMIC-FFP Functional Processes Entries 125 157 Exits 381 Elements Reads Writes Functional Sie (Cfsu) 336 155 1,029.0 Values Number of Use Cases 125 (all simple) Number of Actors 9 (all complex) Unadjusted Actor Weight (UAW) 27 Unadjusted Use Case Weights (UUCW) 625 Unadjusted Use Case Points (UUCP) 652 TFactor 25 EFactor 11 Technical Complexity Factor (TCF) 0.85 Environmental Factor (EF) 1.07 Use Case Points (UCP) UCP 592.994 13
Results Sie (2/3) UCP Technical Complexity Factors (TCF) TCF UCP Environmental Factors (EF) 14
Results Sie (3/3) SLOC GUI SLOC (XML XML) 11,760 Process Logical SLOC (Java Java) 11,817 Permanent Data Storage SLOC (XML XML) 23,550 15
Analysis of the Results - Commonalities Input documentation both methods use (or can use) some UML diagrams as the main input even if COSMIC-FFP goes more into details for calculating Cfsu and can use UC as a high-level documentation. 16
Analysis of the Results Differences (1/2) Input documentation COSMIC-FFP does not have a limitation to use UC as a FUR Adjustment Factors COSMIC-FFP is defined as a 2nd generation FSM method and has rejected according to the ISO 14143-1 viewpoint the usage of adjustment factors while UCP proposes two adjustment factors (TCF and EF) Layers COSMIC-FFP introduces the concept of layers UCP as the other 1st generation FSM methods consider only a single layer in its measurement process 17
Analysis of the Results Differences (2/2) Viewpoints COSMIC-FFP extends the traditional End-User viewpoint towards a broader view, including several possible viewpoints like Development Viewpoint In UCP, use cases describe the system s behavior under various conditions, responding to the requests from one of the stakeholders, called the primary actor, but considering at the same time different possible scenarios Metrics COSMIC-FFP measures the functional sie by counting the Entry, Exit, Read and Write UCP counts the number actors and UC to derive the sie quite different metrics which makes the sie figures obtained by these methods incomparable directly 18
Productivity Development Effort (person-hours) 6,308.00 Functional Sie Productivity Delivery Rate (PDR) Productivity Rate UCP Cfsu personhours// UCP personhours/ Cfsu UCP / personhours Cfsu/ Cfsu personhours 593 1,029.0 10.7 (1.3375) 6.13 (0.76625) 0.094 (0.752) 0.163 (1.304) 19
Effort Estimation (1/2) for COSMIC-FFP, we considered the data from the ISBSG r9 dataset after similar project data selection from the ISBSG repository (including 3024 projects), 51 projects obtained the linear regression formula derived for such subset in order to verify such values. Starting from the initial dataset of 15 projects, we deleted 3 outliers, improving R2 up to 0.732. simply applying the new linear regression formula (y*=1.2147x+52.788) to the current Cfsu, the effort is estimated as 1302.71 person-hours 20
Effort Estimation (2/2) does not exist a public, consistent project dataset for UCP data as the ISBSG some suggestions in the technical literature for a PDR ; 20hrs/UCP or a range between 15 and 30 hrs/ucp; or an average of 20-28 hrs/ucp. simply applying one of the suggestions by Schneider and Winters, 20 person-hours/ucp, we estimated development effort as 11,859.88 person-hours. 21
Effort Estimation - Relative Errors Actual effort 6,308 person-hours Comparison of the estimated and actual effort The sie of the case study project (1,029 Cfsu) is greater than the maximum value for the MIS projects subset (470 Cfsu) PDR value assumed as-is in a start-up phase without performing some tests and experiments for determining an approximate internal productivity level using UCP The estimation process is a very relevant issue, to be treated with great care 22
Conclusions COSMIC-FFP has a series of advantages such as multi-layer not UML-dependent ISO/IEC standard method COSMIC-FFP needs more time and knowledge of the measurement rules UCP, being UML-based, allow saving time for counting the sie through automatic tools 23
Conclusions (cont.) COSMIC FFP and UCP methods can be seen as complementary and not necessarily alternative It will be possible for an organiation to establish its R2 value between the two methods gathering both measures from projects and building a double-sie historical series which will allow: to verify if UCP have been consistently applied to projects in terms of granularity in the UC and Scenarios definitions a better definition of analysis documentation and UML diffusion inside the organiation 24
Q&A Thank You... Çiğdem Gencel, Luigi Buglione, Onur Demirörs, Pınar Efe cgencel@ii.metu.edu.tr luigi.buglione@computer.org demirors@ii.metu.edu.tr pinar.efe@siemens.com 25