KPI for Software Development iparadise www.chrisshayan.com
KPI Risks Obtaining a good KPIs may sound easy but I believe there are few challenges in front of implementation of any measurements specially KPI approach. In the following table is tried to depict about perceived challenges which any software company may face to. No Challenge Name Description 1 Lack of Historical Data KPI can be solely useful if there is a range of data in a history of an organization despite of having very clear and preferable metrics but without historical data KPIs are hardly efficient as well as effective. 2 Various Data Sources To make KPIs understandable and more accurate, it needs various data. Usually within organization these data came from different sources, for instance: QA, project planning, IDEs, SCM and etc. 3 Normative benchmarks Normative benchmark is only good as the parameters being used for comparisons. But there are few concerns regarding this approach which are: A) What are the benchmark criteria? B) How these criteria are shown up? KPI for Software Development 1
Risk Mitigation To mitigate the risk of aforementioned items in Risk section, it has been tried in the following table to lessen the severity of each item: Risk Name Severity Mitigation Lack of Historical Data Blocker To diminish the severity of this risk, we have to follow few steps which are: 1. Define the KPI perspectives 2. Narrowing down each perspective to avoid it become subjective. On the other hand everyone ought to understand each element of KPI perspectives by heart. By defining these elements, we must spend some time to capture data according to the elements which have been defined. Various Data Sources Medium There are few approached to address this concern such as: (1) gathering data manually or (2) purchasing some software house KPI tools like Verex or (3) develop some in-house softwares. My recommendation is to follow the first step if your organization is below than 50 person. Then you can decide to go for second or third solution. Normative benchmarks Severe The solely way to overcome with this relativity smart trap is to look for objectives rather than normative metrics. So the question is this how can we achieve this? KPI for Software Development 2
Perspectives There are 3 different perspectives which can be considered within each software house as KPI. These perspectives are: I. Quality II. III. Innovation Effort I think Verex system had a quite relevant implementation that I recommend as well. Their bird-eye view metrics is shown in following figure: Each element of the perspective has been illustrated in following table. KPI Perspective Perspective Name Quality Metric Number of issues per project/ iteration/sprint Description the ratio of new issues appearing vs. those being resolved. These can be taken from Bugzilla or JIRA. The obviously important parameter was the trend, showing whether the number of problems was growing or being reduced. The applied 3D breakdown into the components gives an idea, where the team has the most problems or what is the technology piece causing most issues (in our case it was Hibernate and JavaScript/DHTML). KPI for Software Development 3
Perspective Name Innovation Metric Number of blocker and critical issues known Average time to resolve an issue and effort required to fix all issues Number of test cases per each project/iteration/sprint Documentation Enhancements and improvements Enhancement in specific product/service/process areas Top enhancers Description this gives a very good understanding of which resources are required to maintain a product/project. Critical and blocker issues usually require immediate action knowing how many are in the pipeline allows for much more realistic planning. this is a very good tool for budgeting and forecasting when a desired level of quality can be reached. It is very important to encourage writing tests as you go, or for QA teams to generate test cases when bugs are found. Not only does it indicate the test coverage, but also the trend coming up with a good test plan is a continuous task and new cases should appear en route. Software is continuously documented via SCR (Software Change Request) or user story documents. The SCR is a high-level to low-level document that in theory should be updated as functionality evolves. Typically what happens is that the first SCR version is written and accepted, but rarely updated afterwards and quickly becomes obsolete. The number of SCRs produced is not relevant to the R&D management as they have to be provided in order to kick off implementation anyway. It is the number of changes and updates in the development process that matters as it reflects if a team documents their work. Building an innovative culture into a company and not forgetting good ideas is always a challenge. That is why at any company should be decided to make the enhancements reporting as simple and as least-time consuming as possible this was achieved by just by adding a new category in Bugzilla/JIRA. Enhancements are not rewarded in any way in order to prevent a flood of garbage (pseudo-enhancements), however they are an important aspect of project/sprint team assessments. (e.g. GUI/front end, performance). This metric gives a good indication of which product areas require most work. Some general conclusions can also be drawn, such as general lack of resources to optimize UI response times. The risk here is that most internal users will report usability and GUI related issues, not back-end problems. Who is the most active person in the project? This metric is a good indicator in individual performance reviews which will be made by team mates. KPI for Software Development 4
Perspective Name Efforts Metric Man hours in the project/ iteration/sprint per activity breakdown Velocity Description It is good to see weekly and/or monthly deltas, especially if the effort can be compared against and correlated with other indicators, such as the number of issues in the project. While software development budgets are usually well controlled, the direction of the spending (e.g. new features, product hardening, support & fixes, etc.) is usually a hazy matter. This metric immediately shows where the project effort goes to and the condition of each product. It is applicable if you are using Scrum methodology. KPI for Software Development 5
Action Plan To attain highest values of KPI, I would suggest following procedures: I. Follow the mitigation of Risk-1 [Lack of Historical Data] in a specific time box such as 6 months(which depends on how agile your organization is to deliver different systems). II. III. IV. Then according to facts which have been captured, we put our target objectives. For instance we find out that in last 6 month our percentage of having bug in production is 20%, then we will set objective like this: we want to achieve 5% bug in production in 3 period of 6-month in harmony of: 17%, 12% and 5%. At a end of each time box there shall be a report which covers an assessment like: as-is of a time and the desired to-be of the time. According to the variance if either re-planning or re-prioritizing are needed, it must be done. At the end of the year there must an ROI to be done for each team. In following section an example of ROI has been presented: Return On Investment for 2012 The table below presents the calculation of payback period and 1st year ROI (cost oriented) for a team of 15 developers based on the following assumptions: I. the deployment of KPI Dashboard will require 30 man days of consulting, II. III. the efficiency improvement will amount to 3% (which is a very conservative assumption), no additional benefits are taken into consideration (e.g. better management decisions), only developer efficiency is accounted for. KPI tools licenses 2000 Deployment (30 man days, times consulting rate $450) 13500 Total Cost $15 500,00 Size of development team 15 Annual Salary $75 000,00 Total Annual Salary $1 125 000,00 3% efficiency improvement (per year) $33 750,00 3% efficiency improvement (per month) $2 812,50 Payback Period (months) 6 Total Cost $15 500,00 KPI for Software Development 6
References I. Benchmarking software development projects: The three KPIs http://www.computerworlduk.com/in-depth/ applications/1830/benchmarking-software-development-projects-three-key-kpis/?pn=1 II. Verax Systems http://www.veraxsystems.com/ III. General material from Gartner on KPIs in IT KPI for Software Development 7