The Dynamic f Organiatinal Perfrmance Develpment f a Dynamic Balanced Screcard Keith Linard and Jeph Yn, Schl f Civil Engineering, Univerity f New Suth Wale (Autralian Defence Frce Academy), Autralia E-mail : k-linard@adfa.edu.au and y-yn@adfa.edu.au Abtract: Perfrmance indicatr are key feedback driver f rganiatin dynamic. Hwever, the extenive evaluatin literature give n cientific bai fr the electin r teting f the pprtunitie f uch indicatr. Thi paper dicue the develpment f a dynamic balanced crecard fr a federal gvernment department. The balanced crecard uffer frm the limitatin f all perfrmance indicatr ytem, namely that the interrelatinhip, and epecially delayed feedback, between indicatr are verlked. Therefre it i eential t take uch feedback int accunt. Sytem dynamic i an apprpriate methdlgy t achieve thi. The paper utline the mdelling f caual interactin between rganiatinal repne t wrk verlad, taff mrale and prductivity, rewrk and utput quality. The cmpnent ub-mdel and management implicatin f the dynamic BSC are dicued. Keywrd: ytem dynamic; balanced crecard; perfrmance indicatr PROBLEMS WITH CORPORATE PERFORMANCE INDICATORS Perfrmance indicatr are central intrument fr managing rganiatin. A fundamental purpe f perfrmance indicatr i t give deciin maker feedback n prgram peratin in rder t guide future deciin. Hwever, in rder t be cnfident regarding deciin made cnequent n uch feedback, tw key cnditin mut be atified: the indicatr chen mut be, in me ene, the right indicatr (i.e., we need me ratinal bai r criteria, in additin t cnenu r availability, fr electing indicatr); and there huld be a way f teting deciin rule fr reacting t mvement in thee deciin criteria. A review f 700 perfrmance management dcument in the Defence Manager Tlbx (Department f Defence,1999) failed t find a ingle reference regarding criteria fr electing indicatr. They were imilarly ilent n the iue f develping deciin rule fr guiding actin cnequent n indicatr change. The wider literature n evaluatin and prgram budgeting i imilarly unhelpful in mving beynd the vague platitude. One f the eminal dcument in prmpting gvernment t mve tward prgram management wa the 1965 Manual fr Prgramme and Perfrmance Budgeting prduced the United Natin. Depite a wealth f infrmatin n the pible ue f perfrmance indicatr, it nly cntributin t the prce f defining them wa the tatement Such meaure can be develped and refined after tep have been taken tward... the etablihment f uitable prgramme claificatin... (UNO, 1965). Cntributr t ucceive Autralaian Evaluatin Sciety cnference have al ignred thee iue. Sme prvide hpping lit f plauibly relevant indicatr, but give nly qualitative guideline fr indicatr electin. Other give a valid critique f the apprache t perfrmance indicatr in the Autralian public ectr, quetining in particular wh give guidance n the indicatr, what the indicatr actually meaure and whether valid meaurement can be made. But yet again there i ilence n hw t d better. Neither Autralian nr internatinal evaluatin literature give credible guidance n criteria fr electin r deign f perfrmance indicatr and are ilent n the iue f aigning deciin rule fr actin cnequent n indicatin. Thi failure wa the catalyt fr the ytem dynamic reearch at the Autralian Defence Frce Academy int the develpment f the dynamic balanced crecard. Sytem Dynamic and Perfrmance Indicatr An implicit aumptin behind Autralian public ectr apprache t electing perfrmance indicatr i that the feedback they give t the deciin maker (either directly r via reultant preure r directin frm ther) will caue the deciin maker t make apprpriate adjutment t the input r prcee. Hwever, there i abundant reearch in the field f ytem dynamic (Sterman, 1989), a well a in the field f experimental ecnmic and pychlgy which ugget that manager have great difficulty managing dynamically cmplex tak. Sterman argue frm hi wrk at MIT that there i ytematic miperceptin f feedback epecially when there are delay in the ytem. Mekilde, Laren and Sterman (Mekilde & Sterman,1989) preent the 359

1 t Internatinal Cnference n Sytem Thinking in Management, 2000 reult f 48 imulatin f the Beer Game run with 192 graduate tudent frm MIT and enir executive. The reult hw that their deciin-making n the bai f traight frward perfrmance indicatr, but in the face f delay, reulted n average in ct mre than 10 time the ptimum! Simulatin run at the Autralian Defence Frce Academy with graduate tudent and Defence executive have hwn imilar reult. In ther experiment, where graduate tudent had full infrmatin, training and pprtunitie fr gaining experience, Diehl and Sterman till fund pr managerial perfrmance in the face f variatin in feedback trength and delay (Deihl, 1994). Diehl and Sterman argue that the mental cntruct and heuritic that manager bring t bear n cmplex tak are fundamentally dynamically deficient. Subject were unable t accunt well fr delay and feedback effect becaue (1) peple mental repreentatin f cmplex tak are highly implified, tending t exclude ide effect, feedback prcee, delay, and ther element f dynamic cmplexity; and (2) even when thee element are knwn, peple ability t infer crrectly the behaviur f even imple feedback ytem i pr. The firt deficiency can certainly be addreed thrugh training. The ecnd, hwever,... i a fundamental bund n human ratinality - ur cgnitive capabilitie d nt include the ability t lve ytem f high-rder nn-linear differential equatin intuitively. The implicatin f uch finding i that deciin upprt tl which can addre uch dynamic are eential fr managing cmplex envirnment. It i in thi cntext that the Balanced Screcard (BSC) perfrmance framewrk, prped in 1992 by Kaplan and Nrtn (Kaplan & Nrtn, 1992), ha great ignificance. Tward the Balanced Screcard and a mre ytemic apprach The BSC (Figure 1) i an utcme riented perfrmance management ytem that eek t link the hrt and lng term activitie f an rganizatin with the viin, miin, and trategy f the rganizatin thrugh the etablihment f meaurable, cnenu-driven gal. The BSC ha a number f key characteritic: It i fcued n trategic management; It incrprate indicatr relating t a number f ectr (in the claic verin, Cutmer, Financial, Internal Buine Prcee and Learning & Grwth); It incrprate bth lead and lag indicatr; and The indicatr allw drill-dwn t undertand the caue-and-effect chain in the ytem Kaplan and Nrtn explicitly recgnied the ytemic inter-relatinhip (Kaplan & Nrtn, 1996) within and between the variu ectr, incrprating bth lead and lag indicatr, which impact n rganiatinal perfrmance. In additin, they explicitly argue fr the develpment f deciin upprt ftware which will enable manager t tet the implicatin f deciin n the ytem. Unfrtunately, Kaplan and Nrtn emphai n caue and effect, where acknwledged, i ften expreed in a uni-directinal caue and effect chain baed imply n tracing the data tree ultimately feeding the indicatr. We are nt aware f any implementatin which recgnie the divere interactin, and epecially delayed feedback, between key deciin and perfrmance indicatr. Figure 1: Balanced Screcard (Kaplan and Nrtn) 360

1 t Internatinal Cnference n Sytem Thinking in Management, 2000 TOWARDS A DYNAMIC BALANCED SCORECARD FOR THE PUBLIC SECTOR Our reearch attempt t implement the Kaplan and Nrtn ideal BSC. Wrking with a number f gvernment agencie we are prttyping a dynamic balanced crecard (in eence, a management flight imulatr ) which will it n tp f an agency perfrmance management ytem and enable manager t tet what if cenari. The aim i t prvide a tet bed fr evaluating bth perfrmance criteria and the deciin rule apprpriate t the criteria, and thereby prvide a bai fr true frward lking management. Characteritic f a Public Sectr BSC The primary functin f the Autralian Federal Public Sectr i t prvide a range f ervice t the Gvernment thrugh the repnible Miniter. Department have legal requirement which et up cutmer relatinhip t the Auditr General and t the Parliament in repect f gvernance. Where Department prvide ervice direct t the public, they are required t prepare and implement a ervice charter, prviding a clear cutmer relatinhip. Thu there are three Cutmer wh may need t be addreed in a BSC: fr mt departmental activitie, the Miniter, and thrugh him/her, the Gvernment; in repect f gvernance, the Auditr General and Parliament, a well a the Miniter; fr ervice delivery activitie, the crprate r individual ervice recipient. The prttype dynamic BSC fcue pecifically n the relatinhip between managerial repne t any wrklad- reurcing gap (the Capacity Gap ), and hw thi impact n the firt tw Cutmer abve. The mdel utput relate t an utput quality index; prbability f fraud / prbity incident; and prbability f EEO incident. The fllwing caual lp diagram illutrate the mechanim by which thee are impacted by departmental peratin. The interrelatinhip between the repective wrk area utput and their impact n achievement f utcme indicatr i ubject f a eparate tudy. (a) Interrelatinhip Within the Reurce Sectr Figure 2 illutrate interrelatinhip in the reurce ectr. Cutmer Demand repreent the Gvernment (thrugh the Miniter) expectatin and determine the reurcing level. It i almt aximatic that available reurce will be le than that required fr quality implementatin f all the planned wrklad, let alne the inevitable unplanned demand. The quality f management al impact thrugh errr in wrklad planning (e.g. due t inexperience r pr negtiating kill). Whether the management repne t any capacity gap i innvative r dyfunctinal i a functin f the rganiatinal cmpetencie, which in turn are the reult f leaderhip and invetment in capability, the latter balancing hrt term impact n recurrent reurce. Department Reurce Cutmer Demand Capability Invetment Recurrent Reurce Bae Lad Buine 'New' Buine Ttal Effective Capacity Annual Wrklad Planned CAPACITY GAP Buine Planning Errr Dyfunctinal Repne t Gap Innvative Repne t Gap Management Effectivene Learning Organiatin Cmpetencie Figure 2: Key Reurce Sectr Interdependencie (b) Interrelatinhip Underpinning Excellence in Internal Prcee The Internal Prcee and Prcedure ectr in public ectr rganiatin i cmplex. Very high taff turnver ver the pat decade (typically average 18 t 24 % per annum in key plicy area) ugget that peple and wrklad management i anunderlying prce that what we mut excel at. 361

1 t Internatinal Cnference n Sytem Thinking in Management, 2000 Whilt Gvernment etablihe the plicy and prgram utcme t be achieved in exchange fr the financial reurce, management tranlate the viin and allcate the capabilitie t achieve the agreed utput. The reurce management framewrk thu prvide a backdrp fr an integrated planning prce that link crprate plan, buine plan and individual plan. Once the budget framewrk i et, Federal Departmental manager have limited cpe fr btaining extra reurce. Any increae in wrklad, and any wrklad underetimate, typically will be addreed thrugh: mre intene and lnger hur f wrk (unpaid vertime) reductin in time devted t training and develpment reductin in trategic management activitie (thrugh redirectin f 'management time' t 'tak time') deferring me wrk (which imply ptpne the day f reckning) and reductin in target quality f input (e.g. thrugh cutting backgrund reearch effrt) r utput Ttal Effective Capacity Staff Turnver CAPACITY GAP Prductivity Staff Mrale Wrk Deferred Average Wrklad Target Output Quality Wrk Intenity Re-wrk / Errr Time n Strategic Management Management Effectivene Figure 3: Key Interrelatinhip Impacting n Internal Prcee A illutrated in Figure 3, if uch management repne are prlnged, they tend t prduce dyfunctinal feedback effect which eventually increae the capacity gap, thrugh increaed re-wrk, falling mral, increaed taff turnver. Where there i excellence in leaderhip and a culture characteried by the learning rganiatin ne culd expect innvative repne t any ignificant r prlnged capacity gap. In eence uch innvatin change the rule f the game t achieve the deired reult. Whilt there are lcal example f uch behaviur acr the bureaucracy, the authr wuld be paring in their applicatin f learning rganiatin t Department a a whle. Only dyfunctinal repne have been incrprated in t the mdel at thi tage becaue f lack f data t permit mdelling f the learning rganiatin repne. (c) Interrelatinhip in the Learning & Grwth Sectr Recruitment tandard t the Federal Public Service are high. Almt 50% f all recruit have a Bachelr Degree r higher. Staff receive upprt t upgrade qualificatin. Middle & enir management training in principle ha a high pririty, but take ecnd place t meeting target. Wrk preure n all taff, and unpaid vertime, ha increaed ignificantly ver the pat 15 year, which i an impediment t elf-develpment. In the face f wrk preure, infrmal and frmal taff develpment tend t be amng the firt area t be cntrained. Figure 4 hw the key caual relatinhip impacting n the learning and grwth ectr. The very high taff turnver acr many Department i indicative f prblem in the learning and grwth ectr. The Federal Finance Minitry i the mt triking, where the average turnver at the executive level ha been 25% t 30% per annum, and 20% per annum acr all taff. The Department ha quite evidently lt the reputatin fr excellence it had in the 1980. 362

1 t Internatinal Cnference n Sytem Thinking in Management, 2000 Figure 4:Key Interrelatinhip Impacting n Learning & Grwth Crprate Gvernance & Crprate Health We have been undecided whether crprate gvernance and crprate health huld be regarded a ectr in their wn right. On the ne hand, Crprate Gvernance might be viewed a a key indicatr in the Cutmer Sectr, whilt Crprate Health might be an indicatr fr the Internal Prce Sectr. On the ther hand, failure in thee area tend t have majr plitical implicatin fr the Miniter and the Gvernment a well a ignificant feedback interrelatinhip with the ther ectr. In the current mdel implementatin, they are preented a ditinct ectr. The Autralian Natinal Audit Office (ANAO) define crprate gvernance t encmpa authrity, accuntability, tewardhip, leaderhip, directin and cntrl (ANAO, 1997). The dynamic BSC mdel, at thi tage, ue fraud event a a general urrgate fr the effectivene f gvernance acr the brad range f departmental activitie. The tructure f the Crprate Health Sectr ub mdel i imilar. Crprate Gvernance Sectr Fraudulent and Dubiu Practice Public Awarene f Fraud Public_Awarene_f_Fraud Randm_Num Fraud_Event_MAvg Effect_f_Fraud_Ttal_n_Awarene Increae_in_Fraud_Prb PROBABILITY_OF_FRAUD Fraud_ccuring Fraud_Accumulating!!!! FRAUD_EVENT FRAUD_TOTAL Effect_f_Mrale_n_Fraud MORALE Effect_f_StaffDevelpment_n_Fraud StaffDevelpment_Index Management_Effectivne in_prgre Return T Mdel Sectr Effect_f_Fraud_Ttal_n_Awarene Internal Factr Impacting n Fraud Figure 5. Crprate Gvernance Sectr f Dynamic Balanced Screcard Future Develpment (a) Mdel Validatin & Verificatin Whilt baic validatin ha been carried ut during mdel cntructin, a wide range f parameter value at thi tage are imply baed n the divere management literature r the judgment f experienced public ervice 363

1 t Internatinal Cnference n Sytem Thinking in Management, 2000 fficial. Analye will be undertaken t identify the mdel enitivity t uch parameter and the mre ignificant ne will be ubject t mre detailed reearch. At the ame time a very wide range f management literature i being reviewed t prvide a much upprt a pible fr the key relatinhip. (b) Uer Validatin The real tet f the mdel will be eeing it in ue by crprate manager in their teting f buine cenari. Whilt we have nt prgreed t thi tage, we already have certain frebding. With the dramatic change in financial management (epecially with the intrductin f accrual accunting) and change in crprate gvernance, many enir executive d nt yet have management infrmatin ytem which allw them t meet baic reprting requirement. Their fcu i till n aviding the crcdile rather than n the trategic viin f managing the wamp. Planning the plitic f implementatin i an imprtant apect f the verall reearch. (c) Integratin With Balanced Screcard Sftware The final tage f thi prject will be t link the dynamic imulatr with the Department prped enterprie infrmatin ytem. At thi tage n firm deciin n the ftware ha been made: cntender include SAP, PepleSft and CrVu. The mdel deign i uch that it huld be relatively traight frward t draw bae data directly frm the crprate ytem regardle f the ftware chice. CONCLUSION Management perfrmance indicatr prvide the feedback infrmatin which trigger management deciin an interventin. They are central t any undertanding f rganiatinal dynamic. Thi paper decribed nging reearch at the ADFA campu f UNSW which ha develped a prttype tet led fr evaluating the effectivene f pecific indicatr and their aciated buine rule, and management flight imulatin which enable uer t undertand the implicatin f pible management actin cnequent an perfrmance infrmatin. Key iue which have been identified frm the reearch prject relate mre t rganizatin change factr than t mdeling. Even the mt brilliant mdeling will nt bring abut culture change unle there i a upprtive climate. The prject ha had trng upprt at the level f the Miniter ffice and at junir management level in the Department, but enir management i cncerned mre abut plicy than abut trategic thinking. The fllwing pint are al ignificant. A cmprehenive trategic planning framewrk i a pre-cnditin t implement a BSC ytem. BSC i nt a magic lutin t pr trategy. Withut a tp level champin, the likelihd f implementing any ignificant change in management practice i limited. Sytem dynamic mdeling i mt effective where it i dne a a mutual learning prce between client and cnultant. Where the relatinhip becme ne f expert recipient, the value f any mdel, hwever brilliant, i limited. REFERENCE: ANAO. (1997). Applying the Principle and Practice f Crprate Gvernance in Budget Funded Agencie, Autralian Gvernment Publihing Service, Canberra. Department f Defence (1999). Defence Manager Tlbx (Incrprating the Cmmnwealth Manager Tlbx). Canberra. Diehl, E and J Sterman. (1994). Effect f Feedback Cmplexity n Dynamic Deciin Making. MIT Slan Schl f Management, Reearch Reprt D-4401-1 Kaplan,S and Nrtn, D.(1992). The Balanced Screcard - meaure that drive perfrmance, Harvard Buine Review, January-February, pp.71-9. Kaplan,S and Nrtn, D. (1996). Linking the Balanced Screcard t Strategy, Califrnia Management Review 39(1) Mekilde, E, E Laren and J Sterman. (1990). Cping With Cmplexity: Determinitic Cha in Human Deciin making Behaviur, in J Cati and A Karlqvit (ed.), Beynd Belief: Randmne, Predictin and Explratin in Science. CRC Pre, Btn. Sterman, J. (1989). Miperceptin f Feedback in Dynamic Deciin Making. Organiatinal Behaviuir and Human Deciin Prcee, 43(3), pp. 301-335. UNO Ecnmic and Scial Affair Department (1965). Manual fr Prgramme and Perfrmance Budgeting., New Yrk. 364