Software Project Management Matrics. Complied by Heng Sovannarith

Transcription

1 Software Project Management Matrics Complied by Heng Sovannarith

2 Introduction Hardware is declining while software is increasing. Software Crisis: Schedule and cost estimates -> Inaccurate Software poor quality A productivity rate that is increasing more slowly than the demand for software.

3 Why we need Software Metrics The software crisis must be addressed and, to extent possible, resolved. Accurate schedule and cost estimates, better quality products, and higher productivity. Use software matrices can help better management result, both short time and long term,

4 Software Metrics The terms measure, measurement, and metrics are often used interchangeable. We CANNOT Control what we CANNOT Measure -- Tom DeMarco, American Software Engineer.

5 Software Metrics (cont.) A software metric is a technique or method that applies software measurements to a class of software engineering objects to achieve a predefined goal The goal is obtaining objective, reproducible and quantifiable measurements, which may have numerous valuable applications in schedule and budget planning, cost estimation, quality assurance testing, software debugging, software performance optimization, and optimal personnel task assignments.

6 Software Metrics (cont.) More than 300 metrics have been defined. How can you judge the two images? What will you to judge them?

7 Types of Metrics ( Types of Measured data) Nominal - Categories -- Database programming, operating system... Ordinal - Ranking -- programmer experience level maybe measured as low, medium, or high. Interval - Differences. For example... complexity is 4 unit more complex than a program with complexity f 2. Ratio scale - example. a program which have 2000 line of code is twice as large as a program of 1,000 line...

8 Characteristics of a Metric Object of measurement Products, processes and projects Purpose of measurement (Measurable Characterization, assessment, evaluation, prediction Source of measurement Measured property Context of measurement

9 Who benefit from Measurement? Managers What does each process cost? How productivity is the staff? How good is the code being developed? Will the user be satisfied with the product? How can we improve? Engineers Are the requirement testable? Have we found all the failures? Have we met our product or process goals? What can we predict about our software product in the future?

10 Software Metrics Categories Product metrics Used to measure aspects of artifacts delivered to the customer Information flow metrics, function points, complexity Process metrics Used to measure aspects of the development and maintenance process example: time to correct defect, number of components changed per correction

11 Software Metrics Categories (cont.) Design metrics : Design metrics enable you to decide how much you have deviated from the requirements of the project. Maintenance metrics These measure the progress of a maintenance project.

12 Product Metrics Analysis done using the product metrics can provide you an insight into the quality of work done by the development teams. Product metrics trace and measure the quality of a deliverable in different phases. A set of metrics measures the quality and timeliness of the deliverable.

13 Product Metrics (cont.) To measure the metric of source code, you can use the format displayed

14 Product Metrics (cont.) Effectiveness of a deliverable is measured in terms of the quantity of the deliverable completed and the number of quality goals achieved. You can use the following formula to calculate effectiveness of a deliverable:

15 Product Metrics (cont.) For example, 80 percent of source code is completed and 90 percent of the quality goals are achieved in developing that source code. Effectiveness = 80 * 90/100 Effectiveness = 72

16 Design Metric Complexity is a characteristic measured during this phase. Complexity can be measured for the structure, data components, and the interface design of a software program. Complexity can affect the size, testability, and effort spent on developing and testing the modules of a project.

17 Design Metric (cont.) The measure of the complexity of a software design is called a design metric. Design metrics are seldom used in organizations because organizations find it time-consuming to analyze their results. Moreover, design metrics are difficult to implement.

18 Design Metric (cont.) There can be many varieties of design metric. One variety is architecture design metrics It address the three types of complexities of a software program. Structural complexity Eg. a module invoked by a module parent -> increase the complexity Data complexity Input and output variable System complexity Combine structural complexity and data complexity

19 Project Metrics Project metrics are specific to the actual execution of a project. They enable you to execute a project with a systematic and technical approach. Project metrics can help you avoid project delays. You can also use them to avoid and mitigate project risks.

20 Project Metrics (cont.) Project metrics are also used to assess the quality of a finished product on a periodic basis. Examples of project metrics are the metrics that measure the size, complexity, or the time and effort spent on a project.

21 Project Metrics (cont.) Six important project metrics are commonly used in an organization. Effort the amount of effort required to complete a project Productivity Measure human effort according to the number of hours spend on work. Cost -- measures the planned versus actual expense incurred on a project Size (important) -- calculated for an entire project and not for any particular phase of a project. Defects Defects are errors that occur in a work product The number of defects defines the quality of a project. Testing measure the number of test cases required to test software. Test case is a specification that needs to be executed to test a particular module in a software program.

22 Effort Metrics The effort metric enable you to determine the amount of effort to complete a project. It is calculated in person-month (manmonths). A person-month is a mount of effort required to complete work in a month. Person-month is a measure of work effort. Other similar measures are person-day and person-year.

23 Effort Metrics Effort Metrics Analysis Design Coding Testing Total Planned effort (personmonths) Planned effort (personmonths) Planned effort (personmonths)

24 Effort Metrics (cont.) If a project will take 2 months to finish with 3 people working full time on it, the project requires 2*3 = 6 personmonth effort. If an employee worked 20 days on a project, his contribution to the project is 1*20/30 = personmonth. (Note that month is considered 30 days in such calculations.) Effort percentage question isn't clear but if it means the percentage effort put in by an employee towards a certain project, just calculate his person-months and divide that by total person-months for that project.

25 Productivity Metric What is Productivity? Amount (or value) of output per unit of input (e.g., cars per work hours) Why do we care? To make the right resource allocations & have the right policies Obviously measuring personal productivity is important, and the reasons why are obvious. If you do not know how your staff is doing, then how can you truly know the inner workings of your own company? Staff discontent can put your company in serious jeopardy, while on the other hand, high staff productivity can be your best company asset.

26 Productivity Metric (cont.) The definition of productivity is the output-input ratio within a time period with due consideration for quality. Productivity = outputs/inputs (within a time period, quality considered) The formula indicates that productivity can be improved by (1) by increasing outputs with the same inputs, (2) by decreasing inputs but maintaining the same outputs, or (3) by increasing outputs and decreasing inputs change the ratio favorably. Software Productivity = Function Points / Inputs

27 Productivity Metric The effort in performing an FP (Function Point) of work in an hour is called productivity. To calculate productivity, you first determine the total amount of FP for the SW project. Measurements that help determine how to make more products with fewer resources, such as producing the most parts with the fewest numbers of employees.

28 Productivity Metric Function Points measure software size by quantifying the functionality provided to the user based solely on logical design and functional specifications Has gained acceptance in terms of productivity (for example FP/year) and quality (defects/fp)

29 Productivity Metric (cont.)

30 Productivity Metric (cont.) Measurement parameter Weighting factor Simple Average Complex Number of user inputs Number of user outputs Number of user inquiries Number of files Number of external interfaces

31 Productivity Metric (cont.)

32 Productivity Metric (cont.)

33 Productivity Metric (cont.) To calculate accurate productivity for a particular phase, you need type of data: Actual effort expressed in person hours. Actual size of the project in FP Metric Analysis Design Coding Testing Total Productivity in hours/fp Planned Actual Difference (+/-)

34 Cost metrics A cost metric measure the planned vs actual expense incurred on a project. An important component of the cost metric is the cost of resources (human resource and material resources)

35 Cost Metrics (cont.) Metrics Analysis Design Coding Testing Total Cost in USD Resources Planned 40, , , , ,000 Communi cation Actual 55, , , , ,000 Planned 10,000 7,000 1,000 1,000 19,000 Actual 20,000 4, ,700 Total Actual 75, , , , ,700 Total deviation in percentage (actual planned/planned * 100)

36 Size Metrics Size metric is a calculated for an entire project and not for any particular phase of a project. The size of a project may vary with respect to the changes required by the customer. The size metric directly affects effort, testing, and productivity metrics of a project. There are two main components of the size metric: Change in the required effort due to change in project size Percentage growth in project size

37 Size Metrics (cont.) Line of Code: It is one of the earliest and simpler metrics for calculating the size of computer program. It is generally used in calculating and comparing the productivity of programmers. Function Count: The size of a large software product can be estimated in better way through a larger unit called module. A module can be defined as segment of code which may be compiled independently. For example, let a software product require n modules. It is generally agreed that the size of module should be about line of code. Therefore size estimate of this Software product is about n x 60 line of code.

38 Size Metric (cont.) Token Count: Operands and operator of any computer language is called tokens. Any symbol or keyword in a program that specifies an algorithmic action is considered an operator, while a symbol used to represent data is considered an operand. In terms of the total tokens used, the size of the program can be expressed as N=N1+N2 Here, N1: count of total occurrence of operators N2: count of total occurrence of operands

39 Size Metrics (cont.) Size in FP Metric Total Size Planned 100 Size Actual 120 Change in size +20 Growth of size in % 20

40 Defects Metric The number of defects defines the quality of a project. If the number of defects is large, the quality of the product is inferior. Defects can occur during any phase of the SDLC. Defects not defected in a particular phase continue to be present during the subsequent phases in the SDLC.

41 Defects Metric (cont.) Very useful as measure of organizational capability to produce defect-free outputs Can be compared with other organizations in the same domain Outlier information useful to spot problem projects and problem components Can be used in-process, if comparison is with defect densities of other projects in same phase If much lower, may indicate defects not being found If much higher, may indicate poor quality of work (In other words, need to go behind the numbers to find out what is really happening. Metrics can only provide triggers)

42 Defects Metric (cont.) Defect evaluation is driven by the need for Defect prevention. Understanding defects and metrics surrounding them will help eliminate defect injection and rework. Most metrics won t mean anything at first until we can establish a baseline to compare to. This is digging deeper into the data available and the resources involved finding out why a defect has entered the system. Examples could be Developer Error, Missing Tests/Requirements, Unclear Requirements, or Environmental (Client/Server) or any of the reasons. It s important to identify it correctly to be able to look for patterns or repeats of specific reason types.

43 Defects Metric (cont.) This will help to identify problem areas and propose solutions to resolve.

44 Defects Metric (cont.)

45 Defects Metric (cont.) Errors Defects in the human thought process made while trying to understand given information, to solve problems, or to use methods and tools Faults Concrete manifestations of errors within the software One error may cause several faults Various errors may cause identical faults Failures Departures of the operational software system behavior from user expected requirements A particular failure may be caused by several faults

46 Defects Metric (cont.) Severe defects critically affect the functionality of a product. Major defects logically affect the functionality of a product. A minor defect is a slight defect that may act as an irritant for users but does not disturb the functionality of the application.

47 Defects Metric (cont.) Defect density Number of defects per size of the software project Size of project measured in LOC or Function Points Useful metric in determining if a software release is ready for delivery to the customer Defect density in released code ( defect density at release ) is a good measure of organizational capability Defects found after release / size of released software Can only compare across similar projects

48 Defects Metric (cont.) Pair Programming Two software developers work together at one computer One developer is coding, the other developer is reviewing Potential to find defects earlier because the code is under constant review On average, PP leaves 15% fewer defects than conventional methods

49 Defect Metrics Metric Analysis Design Coding Testing Total Defects per KLOC Severe Planned (less than equal to) Actual Major Planned (less than equal to) Actual Minor Planned (less than equal to) Actual

50 Defect Metrics (cont.) Prevention Methods Introduced by Michael Fagan in the 1970s at IBM Involves reviewing a software work product by a group of trained individuals Reviewers review the software code changes and relay comments back to the author for clarification or further work. Initial costs are in training individuals on the process. Software developers are reviewing instead of coding.

51 Defect Metrics (cont.) Figure is a cost analysis of defect prevention Clearly, defects found earlier saves money Team can be set up dedicated to the defect prevention activities

52 Defect Metrics (cont.) Personal Work Experience Software development experience of 2 years Software test experience of 3 years Use peer reviews to minimize software defects Run unit tests and integration tests with test team witness Been a part of 2 major software releases and several patch releases Tough to anticipate all behaviors of system

53 Testing Metrics (cont.) Test metrics provides the visibility into the readiness of the product, and gives clear measurement of the quality and completeness of the product. Testing metric is used to track actual testing progress against plan and therefore to be able to be proactive upon early indications that testing activity is falling behind. Indicators that show the quality of the software under test and the effectiveness of the testing activities.

54 Testing Metrics (cont.) Test metrics can give us the information about the software quality, and help the PM to make decision. Metrics are the numerical data which will help us to measure the test effectiveness.

55 Testing Metrics (cont.) Test Metrics is a process of measuring the Software Testing Process with respect to the Actual Test Plan in the organizations. Based on the feedback, the organizations will improve the testing process that they are following. Test metrics are essentially data which provide you insight into the status of either 1) your testing itself or 2) the product under test.

56 Testing Metrics (cont.) The first is a metric which provides such insight as how far through your testing are you (how many test cases exist, how many have you executed, how many remain, what's your execution rate, what is your projected end date). The second is a metric providing insight into the state of the product under test. How many open bugs are there? How many resolved bugs? How many bugs are being opened per day? Per test pass? Per test case? Is the rate of bugs being resolved higher than the rate of bugs being opened? How many bugs are 'stale' (older than a certain number of days?), etc.

57 Testing Metrics (cont.) Test Metrics is measuring the Testing Activities. Test Metrics are varying from company. Every company following there own metrics. In some companies, they following metrics are: Number of Test Cases Executed Number of Test Cases Passed Number of Test Cases Failed Total Number of Defects Raised. Number of Defects Raised in SIT (System Integrated Test) Number of Defects Raised in UAT (User Accept Test)

58 Testing Metrics (cont.) The testing metric is also used to measure the number of test case required to test SW. Test case is a specification that needs to be executed to test a particular module in a SW program. There are a separate test cases for integration testing and unit testing: Integration testing refers to the overall black box testing of the entire SW project. In contrast, unit testing refers to testing individual modules of a SW project.. Every testing project should be measured for its schedule and the quality requirement for its release.

59 Why do we need the test metric?

60 Testing Metrics (cont.) Testing Metrics Coding Test cases per FP Planned 5 Actual 4

61 Testing Metrics Test case Defect Density The number of errors found in test cases v/s test cases developed and executed ( Defective Test cases / Total Test cases ) * 100 Example : Total no of test cases developed is 1360, total test cases executed is 1280, total no of test cases passed is 1065, total no of test scripts failed is 215 So Test case Defect Density is : 215 X = 16.8 % 1280 The 16.8 % value can also be called as Test Case Efficiency % which depends upon the total number of Test cases which found defects

62 Process Metrics Process metrics are collected across all projects and over long periods of time They are used for making strategic decisions The intent is to provide a set of process indicators that lead to long-term software process improvement The only way to know how/where to improve any process is to Measure specific attributes of the process Develop a set of meaningful metrics based on these attributes Use the metrics to provide indicators that will lead to a strategy for improvement

63 Process Metrics We measure the effectiveness of a process by deriving a set of metrics based on outcomes of the process such as Errors uncovered before release of the software Defects delivered to and reported by the end users Work products delivered Human effort expended Calendar time expended Conformance to the schedule Time and effort to complete each generic activity

64 Maintenance Metrics Software Maintenance is the general process of changing a system after it has been delivered. The changes made to the software may be simple changes to correct the code errors, more extensive changes to correct design errors, or significant enhancements to correct specification errors or accommodate new requirement.

65 A Typical Maintenance Flow Customer nominal path Help desk Written MR s Proposed M. R. s Approved M. R. s Maintenance engineer Current source & documentation Change control board Modified source & documentation Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

66 Types of Maintenance Fixing Enhancing Corrective defect identification and removal Adaptive changes resulting from operating system, hardware or DBMS changes Perfective changes resulting from user requests Preventative changes made to the software to make it more maintainable

67 Maintenance Metrics (cont.) When development of a software product is complete and it is released to the market, it enters the maintenance phase of its life cycle. During this phase the defect arrivals by time interval and customer problem calls (which may or may not be defects) by time interval are the de facto metrics. However, the number of defect or problem arrivals is largely determined by the development process before the maintenance phase. Not much can be done to alter the quality of the product during this phase.

68 Maintenance Metrics (cont.) What can be done during the maintenance phase is to fix the defects as soon as possible and with excellent fix quality. Such actions, although still not able to improve the defect rate of the product, can improve customer satisfaction to a large extent. The following metrics are therefore very important: Fix backlog and backlog management index Fix response time and fix responsiveness Percent delinquent fixes Fix quality

69 Fix Backlog and Backlog Management Index It is related to both the rate of defect arrivals and the rate at which fixes for reported problems become available. It is a simple count of reported problems that remain at the end of each month or each week. Using it in the format of a trend chart, this metric can provide meaningful information for managing the maintenance process.

70 Fix Backlog and Backlog Management Index (cont.) Another metric to manage the backlog of open, unresolved, problems is the backlog management index (BMI). As a ratio of number of closed, or solved, problems to number of problem arrivals during the month, if BMI is larger than 100, it means the backlog is reduced. If BMI is less than 100, then the backlog increased. More investigation and analysis should be triggered when the value of BMI exceeds the control limits. Of course, the goal is always to strive for a BMI larger than 100.

71

72 Fix response time and fix responsiveness Most organisations have established guidelines on the time limit within the fixes should be available for the reported defects Severe problems are usually being as soon as possible, less severe has more relaxed time The fix response time metric is usually calculated for all problems as follows: Mean time of all problems from open to closed

73 Fix response time and fix responsiveness (cont.) Sometimes there are less severe problems that customers just ignore, so the problem remains open for a long time ->distortion of mean time ->in these cases, medians should be used instead of mean time values Idea: short time in fix process leads to customer satisfaction and determines how good the process is in this field

74 Percent Delinquent Fixes The mean (or median) response time metric is a central tendency measure. A more sensitive metric is the percentage of delinquent fixes. For each fix, if the turnaround time greatly exceeds the required response time, then it is classified as delinquent:

75 Fix Quality Fix quality or the number of defective fixes is another important quality metric for the maintenance phase. From the customer's perspective, it is bad enough to encounter functional defects when running a business on the software. It is even worse if the fixes turn out to be defective. A fix is defective if it did not fix the reported problem, or if it fixed the original problem but injected a new defect.

76 Fix Quality (cont.) The metric of percent defective fixes is simply the percentage of all fixes in a time interval (e.g., 1 month) that are defective. This metric, therefore, should be a straight count of the number of defective fixes. The quality goal for the maintenance process, of course, is zero defective fixes without delinquency.

77 Steps To Create Metrics Define the goal of a metric Identify the requirement of a metric (Human, Data Collection Technique, Methodology) Determine the baseline figure of the metric Review the metric created

78 End

79 References Pok, Leakmony. Software Engineering, Phnom Penh, RUPP, Kan, Stephen H., Metrics and Models in Software Quality Engineering (2nd Edition), Boston: Addison-Wesley, Sommerville, I. Software Engineering (9th edition), Pearson, 2010, Other resources