Triangle benchmarking in practice IT Confidence 2015 Florence, 19.10.2015 Pekka Forselius 1
Definitions related to benchmarking benchmark: something that is used as a standard by which other things can be judged or measured Source: Longman Dictionary of contemporary English benchmark: reference point against which comparisons can be made Source: ISO/IEC 29155-1 IT project performance benchmarking framework benchmarking: activity of comparing objects of interest to each other or against a benchmark to evaluate charasteristic(s) Source: ISO/IEC 29155-1 2
But before practice a brief look at background and basic elements of Triangle benchmarking 3
Project Management - What? project management consists of the ten knowledge areas (source: Project Management Body of Knowledge, PMI) a project manager must manage all ten areas at same time evaluating success is complex and often (too) challenging 4
A simplified approach Actually, when we try to manage a project to generally agreed success, we are allowed to concentrate on the three most important knowledge areas (source: all articles about Project Management Triangle and Iron Triangle ) 5
Project Management Triangle Everybody KNOWS Project Management triangle, but unfortunately very few ICT decision makers understand its nature correctly! 6
Observations about PM triangle If you can t measure it, you cannot manage it! For management purposes all three dimensions need to be measured, and the measurement principles shall be defined. Examples: Cost = Supplier s development cost from requirements specification to ready to install. Time = Duration of development in months, from the same activities as above. Scope = Quantity or size of the outcomes. Shape and size of the triangle are not constant! 7
A new look at the PM triangle Everybody wants more outcomes with same cost and time, i.e. higher triangle, but how to get it? What are the important elements of a triangle? How to influence the shape of a triangle? 8
Elements of a PM triangle 9
Examples of IT PM triangles 10
Business case, starting point Early requirements from a feasibility study, including business processes Stakeholder analysis, including users Scope statement, system overview picture Will another system be replaced? Investment calculations, including numbers of transactions, volumes, etc. Rough budget and schedule 11
Amount of outcomes Size of the software, estimated and measured in Function Points Methods: all ISO/IEC FSM standards, e.g. FiSMA method (ISO/IEC 29881:2010) at all accuracy levels Tools: FiSMA 1.1 Size Estimator and Experience Service An independent Scope Manager recommended (not necessary to be external) 12
Focus on quality too Quality requirements MUST be connected to functional requirements Method: specify first the requirements for the entire IT system, then for the business processes, and for the lower level FUR if needed. Tools: e.g. FiSMA Quality Requirements Analysis, ISO/IEC 9126 and 25010 (Software product quality standards) 13
Capability of Developer Team The better the capability of developer team, the less re-work needed, and the more competitive price they can propose! Availability of resources and maturity of development process are important! The capability level depends also on skills and experience of the developers, i.e. how well they can: Read, question, communicate, and understand the functional requirements Design the functions Provide the program code Test the outcome units and integrated components Prepare the installation of software. NOTE! The required developer skills are all related to SDLC standard (ISO/IEC 12207) 14
Capability of Product Owner The better the capability of product owner, the faster the delivery of software! Availability of resources and maturity of requirements management process are important! The capability level of product owner also depends on how clearly its team can: Recognize and define all users Write all necessary user stories Specify terms and define ER model Draw business process charts Write and update use cases Define functional requirements Specify quality requirements NOTE! The skills required from the product owner have NOTHING to do with management approach (agile or not). 15
The most important metrics To evaluate first the reality of project plans, and in the end the success of the project we need to measure: Delivery speed = h/b (FPs/month) Unit price = a/h ( /FP) 16
From theory to practice what data is needed, how to draw a triangle from data elements, and what kind of IT projects can be compared by Triangle benchmarking? 17
What data to collect? Project ID (+ connections to program and portfolio if needed) Basic classifiers Software size Total effort and/or cost Start date End date (Duration can be calculated based on the dates) 18
How to draw a standard triangle 1 unit = 200 FP 1 unit = 100 K = 1000 h 1 unit = 3 months 19
Be careful with your presentation! 20
Color coding for Triangle benchmarking Traffic light analogy: Red is bad Yellow is normal Green is good All combinations are possible 21
Scalability issues of Triangle benchmarking The method is applicable for comparing: Input for retrospectives, for evaluating the iterations: Single iterations (e.g. retrospectives) IT projects IT development programs The examples on the next slides show that the triangles work at all levels 22
Example 1: ES40 UI+BL/4SUM Partners 23
Example 1: ES40 UI+BL program 24
Example 2: Valtimo/MoSH subprogram RTK Six iterations: 25
Example 2: Valtimo/MoSH 8 subprograms 26
Example 2: Valtimo/MoSH program 27
Conclusions from Triangle benchmarking results If you see red, don t cry! If you see green, don t laugh! Whatever you see, don t become proud or ashamed! IF you see red, pay all attention to triangle slopes: you can only improve your capability (i.e. your skills, availability and tools). Product owner can improve the way to specify and manage requirements and developer team can improve keeping their focus on the user needs. 28
Thank you! Pekka Forselius, MSc, MBA, Certified Scope Manager, Immediate Past President of ISBSG, Senior Advisor at FiSMA email: pekka.forselius@4sumpartners.com see also www.4sumpartners.com, www.fisma.fi and www.isbsg.org 29