IPMA Research Award 2008 Mario Vanhoucke. Measuring time A project performance simulation study

Transcription

1 IPMA Research Award 2008 Mari Vanhucke Measuring time A prject perfrmance simulatin study Intrductin Earned Value Management systems have been set up t deal with the cmplex task f cntrlling and adjusting the baseline prject schedule during executin, taking int accunt prject scpe, timed delivery and ttal prject budget. It is a well-knwn and generally accepted management system that integrates cst, schedule and technical perfrmance. It is mainly used t calculate cst and schedule variances, perfrmance indices and frecasts f a prject's final cst and duratin. The earned value methd prvides early indicatins f prject perfrmance t highlight the need fr eventual crrective actins. The research prject deals with the prject perfrmance and cntrl phase f the prject life cycle, and the crrespnding feedback lp frm cntrl t planning and scheduling t take crrective actins when necessary. Mre precisely, the fcus is n a reactive scheduling early warning system by means f Earned Value Management (EVM) and Schedule Risk Analysis (SRA). Althugh EVM has been set up t fllw up bth time and cst, the majrity f the research has been fcused n the cst aspect. Recently, different surces in literature shw that the 'classic' earned value metrics fail in predicting the ttal prject duratin in an accurate way (Lipke, 2003). Research scpe The research aims at a detailed investigatin f prject (time and cst) perfrmance measurement methds and risk analysis techniques in rder t validate current and newly develped methds t imprve the crrective actins decisin making prcess during prject tracking. Mre precisely, the target f the research is t measure the prject perfrmance sensitivity and the frecast accuracy (bth in terms f time and cst, but with a fcus n the time dimensin) f the existing and newly develped metrics based n the principles f EVM and SRA. The research questin bils dwn t the determinatin f when and in which cases SRA and EVM culd lead t imprved prject tracking and crrective actins decisin making. Althugh the classic earned value metrics are designed t frecast bth time and cst, the majrity f these metrics is purely cst-based. Recently, a number f new research ideas have been develped (e.g. Lipke (2003) and Vandevrde and Vanhucke (2006)) t increase the accuracy f ttal prject duratin frecasting. This technique, referred t as the Earned Schedule methd, has shwn prmising results n a (small) number f real life prjects, and can be cnsidered as a new alternative fr the mre classic earned value metrics (referred t as the Planned Value methd (Anbari, 2003) and the Earned Duratin methd (Jacb and Kane, 2004)). Hwever, t the best f my knwledge, n research has been dne prir t this research prject t validate these frecasting methds n a wide set f fictive prjects with knwn characteristics, a knwn cntract structure, a predefined risk prfile, etc (see later: research methdlgy). The specific targets and research hyptheses frmulated in the research prject can be summarized as fllws: IPMA Research Award 2008 Page 1

2 - What are the static (befre prject executin) and dynamic (during prject executin) drivers f frecast accuracy? Knwledge abut prject perfrmance drivers and accurate frecast accuracy measures shuld allw the prject manager t critically analyze EVM perfrmance measures and t accurately predict the final cst and duratin f a prject. Static and dynamic drivers that have been investigated int detail are: Static drivers: Prject netwrk structure: Characteristics f the prject can be easily calculated during the cnstructin f the baseline schedule, and affect the accuracy f the perfrmance measurement during prject tracking. Activity criticality: The degree f activity criticality affects the prject tracking prcess and the perfrmance accuracy. Dynamic drivers: Time span f cntrl: The time span and the number f review perids during prject perfrmance measurement clearly affect the accuracy. Schedule adherence: The prject schedule and the adherence t that schedule (in terms f precedence lgic, EVM measurement system, etc.) shuld have an effect n the accuracy f prject perfrmance measurement. - Hw des the prject time sensitivity affect the accuracy f perfrmance measurement? Infrmatin btained during the scheduling step (baseline plan) as well as sensitivity infrmatin and risk analysis btained thrugh SRA shuld allw the prject manager t imprve the prject tracking prcess and the crrective actins decisin making prcess. - Hw des the knwledge n frecast accuracy (tw previus research questins) lead t imprved crrective actins decisin making during prject tracking? Since EVM is a methdlgy t prvide an ften quick sanity check f the prject health n the cst cntrl accunt level r even higher Wrk Breakdwn Structure (WBS) levels, it cannt be cnsidered as an alternative f the ften time-cnsuming activity-based Critical Path Methd (CPM) scheduling apprach. The research aims at detecting when and hw the EVM tracking apprach ffers a full alternative t the detailed CPM prject tracking, and in which cases a need t drill dwn t lwer WBS levels is necessary t take crrective actins. Theretical fundatin and methdlgy The theretical fundatin and methdlgy has been laid in research prjects prir t the current research in the perid between 1995 and 2005 at the Ghent University and Vlerick Leuven Gent Management Schl. In clse cllabratin with the industry, the Technical University f Lisbn and with the help f PhD assistants, prject scheduling algrithms and prject netwrk generatrs have been develped and cmpared with the current state-f-the-art research at varius prject management and scheduling wrkshps. The theretical fundatin and research prcess can be briefly described alng the fllwing three steps (a summary is given in the figure belw): Data cllectin: The data cllectin cnsists f a well-balanced chice between fictive and real life data. The fictive prject data cnsist f mre than 4,000 prjects btained thrugh a well-knwn prject generatr frm literature. T that purpse, the research relies n prjects with a pre-defined netwrk structure and generated by an existing state-f-the-art netwrk generatr (Vanhucke et al. (2008)), rather than relying n nly a smal set f real life prjects with an unknwn netwrk structure r risk prfile. Mrever, a well-defined classificatin f prjects has been defined with respect t structure and risk which ffers a prject classificatin framewrk fr the prject manager. The real life data nly serve as a validatin f the fictive results, and has been btained by friends and clleagues frm the field. Methdlgy: The methdlgy used is Mnte-Carl simulatin in rder t set up a full-factrial simulatin experiment. The simulatin settings t execute all fictive prjects are classified int 9 welldefined and clearly nn-verlapping scenaris, and use the activity criticality as well as the average prject perfrmance as cntrl parameters. As mentined befre, the simulatin runs are als extended t a crrective actins decisin making prcess under a cntrlled design. IPMA Research Award 2008 Page 2

3 Statistical analysis: Based n bth simple rules-f-thumbs and advanced statistical techniques, the best perfrming frecasting technique has been selected, given the characteristics f the prject. Bth the tplgical structure, the risk prfile and the generated schedule play a crucial rle in the selectin f the frecasting technique fr an individual prject. In ding s, the research prvides the prject manager with useful infrmatin abut the technique applicable t his/her prject. The statistical infrmatin is summarized and linked with the crrective actins decisin making prcess, in rder t prvide guidelines and rules-f-thumbs t the prject manager in which cases which tracking and risk analysis technique shuld be used fr an individual prject. Research results The main (static and dynamic) drivers that affect the frecast accuracy during prject tracking have been reveiled, which results in interesting results fr prject managers using in ne way r anther a tracking system similar t EVM. The main results can be briefly summarized as fllws: - The earned schedule methd can be cnsidered as a full alternative t the planned value and earned duratin methds, but nt in all cases. Under nrmal prject behavir, which is defined as prject executin where the EVM based perfrmance measures reprt reliable results, the earned schedule methd utperfrms the tw ther methds n the time dimensin f perfrmance measurement. Hwever, when the EVM based perfrmance measures prvide less reliable results (e.g. when a delay in a nn-critical activity reprts an verall prject delay, althugh the prject is still n track), the planned value and earned duratin methds are mre reliable. - EVM prject tracking prvides reliable results in case the prject netwrk cntains many critical activities. Cnsequently, in these cases, there is n need fr detailed activity based CPM prject tracking, but instead, rugh prject based EVM perfrmance measures can serve as reliable triggers t drill dwn t lwer WBS levels (dwn t the activity level) t lk fr pssible prblems t take crrective actins. - CPM prject tracking cmbined with SRA prvides reliable results in case the prject has many parallel jbs. Mrever, the sensitivity infrmatin btained thrugh risk analysis enables the prject manager t reduce the effrt f CPM prject tracking t thse activities with a high expected effect n the prject perfrmance. Hence, a clear fcus n nly sensitivity activities in rder t reduce the tracking effrt still results in reliable results. - The prject tplgical structure and the prject criticality are the main static drivers f prject perfrmance accuracy. - The lead-time between review perids as well as the schedule adherence are the main dynamic drivers f prject perfrmance accuracy. The research results have been translated int publicatins (sme f them have been published (see e.g. Vanhucke and Vandevrde (2007)) while thers are still under review) and case studies. Mrever, the results have been cmmercialized int a nvel sftware tl PrTrack IPMA Research Award 2008 Page 3

4 ( as a cnsequence f the cnsultancy prjects and results achieved by the research, develped by OR-AS bvba (see figure belw). The research results have als been summarized in a bk titled Measuring Time An Earned Value Simulatin Analysis. The bk is currently under evaluatin, and news abut the prgress can be fund n Practical relevance This research is highly relevant t bth academics and practitiners and cmes twards the need f a mre prfund study f newly develped EVM cncepts fr which nly preliminary results and cnclusins are ften based n arbitrary cases r lse statements with detailed underlying theretical knwledge and/r empirical bservatins based n test runs n large sets f data. Frm an academic pint f view, it is interesting t measure the influence f the prject netwrk characteristics n the behavir f bth scheduling and mnitring tls. In literature, the influence f the structure f a prject netwrk n the risk f prject delay (Tavares et al., 1999), the quality f the cnstructed schedule (Pattersn, 1976), the criticality f a netwrk (Tavares et al., 2004) r the cmputatinal effrt an algrithm needs t schedule a prject (e.g. Elmaghraby and Herrelen, 1980, amngst many thers) have already been investigated in detail. Hwever, t the best f my knwledge, n research attentin has been spent n the influence f netwrk characteristics n the accuracy f prject mnitring tls and techniques. This study can be used by practitiners wh use earned value metrics and EVM principles n a daily basis, but wh are unaware f the merits and the pitfalls f each individual EVM methd. Mrever, this research als prvides a framewrk fr future research and cnsultancy purpses and hence, can be used as a guide fr varius cnsultancy prjects n EVM. The cntributin f the research results t prject managers and practitiners can be summarized alng the fllwing lines: - The research ffers a summary and verview f the verwhelming amunt f metrics, definitins and tls fr prject tracking. - The research ffers static prject based drivers that affect prject perfrmance measurement accuracy. Cnsequently, it ffers the practitiner a framewrk t decide which technique shuld be best used fr his/her specific case and hence, the research prvides mainly general results that can be translated int case-specific settings based n these drivers. - The research ffers the prject manager a clear understanding in prject perfrmance behavir and its dynamic drivers that affect cst and time frecasts based n the current perfrmance. - The research ffers clear guidelines n prject tracking and distinguishes between tp-dwn prject tracking (EVM perfrmance measurement as triggers at high WBS levels t drill dwn int lwer WBS levels) and bttm-up prject tracking (traditinal activity based CPM prject tracking, extended with prject sensitivity infrmatin). - The research als prvides a detailed study f nvel EVM cncepts (e.g. the p-factr (Lipke, 2004)) that have neither been implemented in sftware yet nr widely used in practice, but that ffer prmising tls that can be used in the future. Cnclusin As a brief summary, the main cntributins f the research results are: IPMA Research Award 2008 Page 4

5 - The research ffers a cmparative study t bth academics and practitiners, and prvides an verview f and general results n EVM and SRA used during prject perfrmance measurement and prject tracking. - The research ffers varius prject tracking and crrective actins decisins guidelines based n extensive tests n a wide and diverse set f fictive prjects as well as n a relatively small sample f real life prjects. A detailed bttm-up apprach based n CPM principles and SRA and an EVM based tp-dwn apprach used as a trigger system t warn fr the need fr crrective actins need t be mixed carefully and highly depends n characteristics f the prject. - The research lies its main fcus n the translatin f general results t case-specific guidelines and rules-f-thumbs, directly applicable t individual prjects. Cnsequently, this makes the research a valuable tl fr the prject management discipline with a great ptential and a high practical value! References Anbari, F. (2003). Earned value prject management methds and extensins. Prject Management Jurnal, 34: Elmaghraby, S. and Herrelen, W. (1980). On the measurement f cmplexity in activity netwrks. Eurpean Jurnal f Operatinal Research, 5: Jacb, D. and Kane, M. (2004). Frecasting schedule cmpletin using earned value metrics? Revisited. The Measurable News, Summer:1, Lipke, W. (2003). Schedule is different. The Measurable News, Summer: Lipke, W. (2004). Cnnecting earned value t the schedule. The Measurable News, Winter:1, Pattersn, J. (1976). Prject scheduling: the effects f prblem structure n heuristic scheduling. Naval Research Lgistics, 23: Tavares, L., Ferreira, J., and Celh, J. (1999). The risk f delay f a prject in terms f the mrphlgy f its netwrk. Eurpean Jurnal f Operatinal Research, 119: Tavares, L., Ferreira, J., and Celh, J. (2004). A surrgate indicatr f criticality fr stchastic netwrks. Internatinal Transactins in Operatinal Research, 11: Vandevrde, S. and Vanhucke, M. (2006). A cmparisn f different prject duratin frecasting methds using earned value metrics. Internatinal Jurnal f Prject Management, 24: Vanhucke, M., Celh, J., Debels, D., Maenhut, B., and Tavares, L. (2008). An evaluatin f the adequacy f prject netwrk generatrs with systematically sampled netwrks. Eurpean Jurnal f Operatinal Research, 187: Vanhucke, M. and Vandevrde, S. (2007). A simulatin and evaluatin f earned value metrics t frecast the prject duratin. Jurnal f the Operatinal Research Sciety, 58: Bigraphy is a full prfessr at Ghent University and Vlerick Leuven Gent Management Schl (Belgium). He teaches Prject Management, Business Statistics and Applied Operatins Research. He is prgramme directr f the Cmmercial Engineers and the Advanced Master in Operatins and Technlgy Management at the Ghent University and head f the Operatins Management and Infrmatin Systems department. He is prgramme directr f the Prject Management Prgramme at Vlerick Leuven Gent Management Schl. He is als partner f a cnsultancy cmpany OR-AS ( where he is currently invlved in the develpment f a prject scheduling sftware package with earned schedule tracking capabilities. His main research interest lies in simulatin and ptimizatin mdels in prject scheduling and scheduling in the health-care sectr. He is advisr f varius PhD prjects, in cllabratin with different university hspitals. He has articles published in internatinal jurnals, such as Annals f Operatins Research, Management Science, Operatins Research, The Accunting Review, Internatinal Jurnal f Prductin Research, Jurnal f the Operatinal Research Sciety, Jurnal f Scheduling, Internatinal Jurnal f Prject Management, Prject Management Jurnal, Eurpean Jurnal f Operatinal Research and Lecture Ntes n Cmputer Science. IPMA Research Award 2008 Page 5