Project Estimation Kostas Kavoussanakis, EPCC 4Aim: To raise awareness of the importance of estimation to project welfare To discuss techniques and methods To link estimation with the other process activities Overview 2 1
Estimation 4Want to start a project? 4It is important that you know in advance: How long it will take How many people it will need How much effort it will require 4You can then discuss cost and shake hands 4Estimation is hard Projects overrun Projects go overbudget 3 Motivation 4Two thirds of all projects substantially overrun their estimates; 4The average large project misses its delivery date by 25 to 50 percent; 4The size of the average schedule slip increases with the size of the project. 4Table from B. Boehm 4 2
Disasters First;Last Estimate Project Cost (M$) Schedule (months) Status at end PROMS (Royalty Collection) 12;21+ 22;46 Cancelled, Month 28 London Ambulance 1.5;6+ 7;17+ Cancelled, Month 17 London Stock Exchange 60-75; 150 19;70 Cancelled, Month 36 Confirm (Travel Reservations) 56;160+ 45;60+ Cancelled, Month 48 Master Net (Banking) 22;80+ 9;48+ Cancelled, Month 48 5 Why so hard? 4Estimates are needed and relied upon early 4The functional requirements do not provide a solid background 4It is not immediately known how long it will take to develop the features Particularly if the desired outcomes are genuinely novel. 4Feature Creep is a killer It is the unpredictable yet near-certain change of the functionality as the project progresses. 6 3
Why so hard? (cont) 4Staff ability Estimators Programmers 4Code reuse Is code reused? Is code to be reused? 4Programming language used 7 Effects of underestimation 4Understaffing 4Underscoping the quality assurance effort 4Setting too short a schedule. 4These in turn could lead to: staff burnout low quality loss of credibility as deadlines are missed. 8 4
Effects of overestimation 4Parkinson's Law: Work expands to fill available time 4The project will take at least as long as estimated even if it was originally overestimated. 9 When to estimate 4The first estimate is necessary before the start of the project. 4Estimation is a process of gradual refinement. 4It does not finish until the project finishes. 10 5
Cone of uncertainty 11 4The effort is concentrated where the money is MIS real-time systems 4 What is wrong with academic projects? Complex computations Challenging algorithms 4Good estimation practices can keep the project on track and even earn some time for the tricky, interesting areas. 4We should standardise the estimation process First inside the individual research centres Then (quite a big step) across the centres in the UK Relevance 12 6
Projects revisited 4Remember the project triangle Cost Quality Time 4Quality for software projects Functionality Correctness of the code 4Cost and time depend on Effort and Schedule 13 Effort and Schedule 4Effort is the total number of time units (e.g. weeks or months) needed to complete the task. 4This may break down to effort from more than one persons, so as to take advantage of certain skills and parallelise the work to gain overall time. 4Caveat: the more people one adds to a project the more one needs to work so as to coordinate them and the more they communicate so as to interact successfully, thus yielding overheads. 4COCOMO: For large projects Effort = a*{size} b For projects that can be achieved with 2-3 people teams Effort = a*{size}+b 14 7
Effort and Schedule (cont) 4Effort division may cause gaps in personnel utilisation. 4Schedule is the breakdown of effort per person at any given time. 4It is sometimes defined as the total time for the project. 4Schedule can be derived from effort. 4Rule of thumb: optimal schedule = 3 * effort 1/3 (McConnell) 15 Estimation Methods 4Expert Opinion 4Estimation by Analogy 4Metrics 16 8
4The classic method Expert Opinion 4Prior experience is the key to estimation This is true in all cases Good practice: only use documented data 4Estimation is left to a senior member of staff, possibly the Project Leader Good practice: involve the developers 4Techniques to mitigate single point of failure Work Break Structure Delphi 17 4Two hierarchies: Software product Process Work Break Structure (WBS) 18 9
Product Hierarchy 19 Process Hierarchy 20 10
4Lay out what you want to do Work Break Structure (cont) Requirements at an early stage Architecture document at a later one Other documents (e.g. priorities list) 4Break the problem into components (workpackages) 4Then break the workpackages in tasks 4Perhaps go another level down Good practice: the more you break and estimate the better The sum of the errors is less than the error of sums 21 Work Break Structure (cont) 4Appreciate the workload of each component Good practice: How many days to the week? Good practice: Check documented experience of similar tasks 22 11
Gantt Charts 4Turn back to the workpackages and seek dependencies E.g. WP1 and WP2 can run in parallel, but can t start WP3 until WP1 is finished 4How to create a Gantt chart Get a biiig piece of paper and loads of post-its Each post-it is a task Try to parallelise tasks as much as dependencies let you horizontal axis represents time, vertical axis resources Length of chart is time Width of chart is effort Gaps represent underutilisation of staff 23 Gantt Chart Example 24 12
4Participants are asked to make assessment individually 4Results are collected, tabulated, and then returned Group discussion optional 4Second round of individual assessments taking into account the previous results. Delphi 25 4Pros: Expert Judgment evaluation Applicable to very original projects. Inherent local calibration. WBS can lead to a well documented process. 4Cons: Big dependency on experts' abilities. Big dependency on experts' presence; how easy does Delphi cope with high staff turnover in a small organisation? There is so much information outside your establishment 26 13
Estimation by Analogy 4The cost of a new project is estimated by analogy to similar completed projects 4Estimates based on historical data from within an organisation are more accurate than estimates based on rules of thumb or educated guesswork. 4International Software Benchmarking Standards Group (ISBSG) maintains and exploits a repository of international software project metrics to help software and IT business customers with project estimation, risk analysis, productivity and benchmarking. 27 4Pros: Estimation by Analogy (cont) Can be accurate. Simple if the organisation repeats similar work. Estimates are immediately available. Encourages detailed documentation. 4Cons: Can be very unreliable. It is very hard to assess the differences between the environments and thus assess the accuracy of the ISBSG data. 28 14
Estimation Metrics 4No definition for project size Something that encompasses all 3 aspects of a project The area of the project triangle 4A metric makes estimation: Transparent: The abilities of the practitioner are irrelevant Repeatable: Exercised by various people at different times yields the same results Reliable: The end results, although never accurate, can be closer to the Truth 4Additionally: The productivity of staff and organisations can be monitored. The results can be shared across the globe. 29 4How many lines in your files? 4Pros: Very natural Easily countable 4Cons: Available too late What is a line? Comments? Compare d = c++; with c++; d = c; Lines of Code 30 15
Function Points 4Function Points provide a more objective and reliable estimate of the size of a project 4 Measure of the size of computer applications and the projects that build them 4With Function Points one can: Measure Productivity (100 FPs produced this month) Estimate development and support (100 FPs required thus XXX man months) Monitor outsourcing agreements (this library requires 100 FPs, says the contract, these are only XXX) Normalise other measures (100 defects in a 100 FPs is far worse than in 10000 FPs) 31 Function Points (cont) 4Started in IBM in the late seventies (Alan Albrecht) 4International Function Point Users Group founded in late eighties 4An active area of research (COSMIC-FFP) 4Measurements are taken from the user's point of view Independent of computer language, development methodology, technology or capability of the project team http://www.ifpug.org/home/docs/ifpughome.html 32 16
4Pros: Function Points Evaluation Prescriptive method for sizing. Can be applied reasonably early in project lifetime. Immune to language and platform idiosyncrasies. Large user base-active effort. 4Cons: Manual, fiddly process. Disagreement about its applicability across the various types of modern projects. Not ideal in the requirements capture period; although the method can be applied, it is too much work for such a volatile description. 33 Estimation Tools 4Approaches depending on estimation method: Expert Judgement Algorithmic Models FPs, for size COCOMO, or SLIM, or tables based on size estimation for effort SLIM or tables based on size of effort estimation for schedule 4Most products are hybrids: Organising expert judgement Exploiting machine learning Refining algorithmic models 34 17
4With estimation tools you can: Estimation Tools (cont) Estimate size using Function Points or other metrics Derive effort and schedule using various algorithms and techniques Perform what if analyses with staffing, duration etc. and appreciate how realistic they are Produce and update Gantt and other charts easily Maintain and exploit a database of historic data Import data from other projects run in organisations with which you have no connection 35 Estimation Tools (cont) 4Tools are important for estimation They make estimation feel like an algorithm They prevent from skipping necessary tasks They help organise, update and archive the results 4Additionally, they provoke you to think about the lifetime of the project and at the same time do the dirty job for you 4Make your own research for tools and their efficiency Compare them against already finished projects Use them in parallel with your current estimation technique 36 18
Estimation is a part of Process 4The most important issue about estimation is frame of mind 4It takes more than a good estimate to keep a project in good shape 4Important things to note: The silver bullet syndrome Choice of development model Track the project progress Estimate in ranges Think before you quote Bring in the right people 37 The silver bullet syndrome 4None of the estimation methods is perfect 4None of the estimation tools is perfect 4Evaluate and calibrate methods and tools 38 19
4Many models are flexible enough to accommodate changes. Development model 4Design to schedule is great for fixed time-fixed cost projects. 4Iterative models are better for projects with many unknowns. Staged delivery Design to schedule Evolutionary delivery Spiral 39 Keep track of your project 4A software project is a live entity with complex behaviour 4Monitor and update estimates Even after the design stage you can expect to be off by 25% 4Complement your estimates with a good tracking policy Various tools available 4Always update the risks list and the priorities list. 40 20
Estimate in ranges 4The tools and methods may give you absolute numbers 4Add some padding, i.e. slack time to make up for errors Do it and the customer does not trust your estimate Don t do it (or don t pad enough) and you may overrun Good practice: estimation range Good practice: buffers for specific risks 4Risk analyses will show things that can go wrong 4 Evaluate their impact to the schedule 4 Provide conditional estimates Start with raw numbers (from tools and methods) Consider risks Consider what could help 41 4Example: Estimate in ranges (cont) The GUI will take 6 months, +2 if the tool-generated code is useless, +1 if Jo goes on holiday, -1 if we can reuse the File menu functionality from project X. 6+3-1". 4Alternatively: [5-9] 4Very different from 8 6+2+1-1=8. 4Incorporation of risks to estimate 42 21
Think before you quote 4When tempted to provide an estimate thinking on your feet... DON'T! 4People remember it, pass it on and hold you accountable. Although it is normal for the estimate at the feasibility stage to be off by 400%. 43 Bring in the right people 42 groups of colleagues one cannot ignore. 4Group 1: those experienced in estimation. They know how to appreciate the workload of components Trust them in conjunction with documented past estimates They know how to do it. 4Group 2: those who will do the work They will know how long it will take them Do not judge the effort from your own standards They can see things from a different angle And foresee the technical challenge They get a feeling of ownership Persistence and application to the project 44 22
References 4 Steve McConnell. Rapid development: taming wild software schedules. Microsoft Press, 1996. 4 B W Boehm, C Abts, A W Brown, S Chulani, B K Clark, E Horowitz, R Madachy, D Reifer, and B Steece. Software Cost Estimation with COCOMO II. Prentice Hall PTR, 2000. 4 Construx Software Inc. http://www.construx.com/estimate. 4 I Sommerville. Software Engineering, Sixth Edition. Addison- Wesley Publishers Limited, 2001. 4 Steve McConnell. Software Project Survival Guide. Microsoft Press, 1998. 4 W.S.Humphrey. Your Date or Mine. In The Watts New Collection. Software Engineering Institute, Carnegie Mellon University, http://interactive.sei.cmu.edu/, 2001. 45 References (cont) 4 B Boehm, C Abts, and S Chulani. Software Development Cost Estimation Approaches - A Survey. Technical Report USC-CSE-2000-505, University of Southern California - Center for Software Engineering, USA, 2000. 4 E Nelson. Management Handbook for the Estimation of Computer Programming Costs. Technical report, Systems Development Corporation, USA, Oct 1966. 4 L Putnam and W Myers. Measures for Excellence. Yourdon Press Computing Series, 1992. 4 C Jones. Applied Software Measurement. McGraw Hill, 1997. 4 R Park. The Central Equations of the PRICE Software Cost Model. In 4th COCOMO Users' Group Meeting, November 1988, 1988. 4 R Jensen. An improved Macrolevel Software Development Resource Estimation Model. In Proceedings 5th ISPA Conference, April, 1983, pages 88-92, 1983. 46 23
References (cont) 4 B Boehm. Software Engineering Economics. Prentice Hall, 1981. 4 L Briand, V Basili, and W Thomas. A Pattern Recognition Approach for Software Engineering Data Analysis. IEEE Transactions on Software Engineering, 18(11), 1992. 4 T Khoshgaftaar, A Pandya, and D Lanning. Application of Neural Networks for predicting program faults. Annals of Software Engineering, 1, 1995. 4 H Zuse. History of Software Measurement, Chapter 5 "Cost Estimation" at http://irb.cs.tu-berlin.de/zuse/metrics/history_05.html. 4 B Boehm, B Clark, E Horowitz, C Westland, R Madachy, and R Selby. Cost Models for Future Software Life Cycle Processes: COCOMO2.0 at http://sunset.usc.edu/publications/techrpts/1995/usccse95-508/usccse95-508.pdf. 47 References (cont) 4 International Software Benchmarking Standards Group (ISBSG). Software estimation, benchmarking, productivity, risk analysis, and cost information for software developers and business at http://www.isbsg.org.au. 4 O Helmer. Social Technology. Basic Books, NY, USA, 1966. 4 B Baird. Managerial Decisions Under Uncertainty. John Wiley & Sons, 1989. 4 R Boehm. Function Point FAQ at http://ourworld.compuserve.com/homepages/softcomp/fpfaq.ht m. 4 Edmond VanDoren. Software Technology Review: Function Point Analysis at http://www.sei.cmu.edu/activities/str/descriptions/fpa_body.htm l. 48 24
References (cont) 4 The Common Software Measurement Metric International Consortium. COSMIC-FFP Measurement Manual. Technical report, The Common Software Measurement Metric International Consortium, available from http://www.lrgl.uqam.ca/publications/private/446.pdf, 2001. 4 A Abran, C Symons, and S Cligny. An overview of COSMIC-FFP field trial results, also available from http://www.lrgl.uqam.ca/publications/pdf/614.pdf. In ESCOM 2001, London, England, April 2-4, 2001. 4 R J Kauffman R D Banker and R Kumar. An empirical test of objectbased output measurement metrics in a computer aided software engineering (CASE) environment. Management Information Systems, 8:127-150, 1991. 4 E Yourdon. Death march: managing "mission impossible" projects. Prentice Hall PTR, 1997. 49 25