West Yrkshire Fire & Rescue Service Data Quality Plicy Ownership: Crprate Services Date Issued: Nvember 2007 Date Last Mdified: August 2012
Cntents Table f Cntents Page N. 1 Intrductin 3 2 Why is data quality imprtant? 3 3 What makes gd quality data? 4 4 Hw can we imprve data quality? 5 5 Data definitins 5 6 Dcumented prcedures 6 6.1 Timescales 6 6.2 Audit trail 6 6.3 Imprvement and develpment 6 6.4 Duplicatin and replicatin 6 7 Respnsibilites 7 7.1 Defined respnsibilities 7 7.2 Respnsibilities f all staff 8 7.2.1 Recgnitin and awareness 8 7.2.2 Data input respnsibilities 8 8 Systems and prcesses 9 9 Mnitring and review 10 10 Presentatin 11 11 Validatin f data frm partners 11 12 External audits 12 13 Actin plan 13 13.1 Staff knwledge and respnsibilities 13 13.2 Prcedures data recrding and reprting 14 13.3 Systems and prcesses 15 13.4 Mnitring and review f data 15 13.5 External audits 16
1 Intrductin All effective rganisatins measure their perfrmance s that they knw hw well they are perfrming and t identify pprtunities fr imprvement. Cnsistent, high-quality, timely and cmprehensive infrmatin is a vital cmpnent t supprt gd decisin-making and imprved service utcmes. It is essential that the perfrmance infrmatin used t infrm, manage and plan activities is accurate, reliable and cmparable, bth currently and ver time. External bdies are increasingly using perfrmance infrmatin t assess ur perfrmance, as either an additin r an alternative t inspectin, and this trend is causing these external bdies t place a grwing emphasis n data quality. Data quality invlves identifying prcesses, applying standards and prviding guidelines t enable and sustain imprvement. In rder t achieve and maintain a high level f data quality, West Yrkshire Fire and Rescue Service (WYFRS) has apprved a Data Quality Plicy. The plicy aims t: Make WYFRS a leading authrity n the cllectin, recrding and reprting f accurate, reliable and cnsistent perfrmance data t infrm the decisin-making prcess Prvide emplyees with a framewrk t ensure sufficient actin is being taken t meet the data quality bjectives set Meet external audit standards and requirements This plicy utlines the steps necessary t maintain high quality standards thrughut the prcesses that result in the prductin f perfrmance infrmatin: frm the establishment f perfrmance indicatrs (PIs) and their definitins, via the input f individual data t a system, t the validatin f these figures (in sme cases, but nt all, by an external auditr). The plicy will be kept under review and will be amended as necessary t reflect any changes in natinal standards r requirements, as well as redefining mnitring arrangements where these are applicable. 2 Why is data quality imprtant? As a public bdy, we must manage resurces t meet the needs f the cmmunities we serve and the perfrmance infrmatin we use t accunt fr ur activities t ur users, partners and external auditrs must be accurate, reliable and timely. Prducing such data shuld nt be an end in itself, but an integral part f effective perfrmance management. Organisatins which place an emphasis n data quality are likely t be actively and effectively managing data in all aspects f their business. Page 3 f 17 Data Quality Plicy
3 What makes gd quality data? There are six key characteristics t describe the quality f data, as detailed in the fllwing table. These characteristics can assist us in assessing the quality f ur data and in taking actin t address any ptential areas f weakness. Accuracy Validity Reliability Timeliness Relevance Cmpleteness Data shuld be sufficiently accurate fr its intended purpses, representing clearly and in sufficient detail the interactin prvided at the pint f activity. Data shuld be captured nce nly, althugh it may have multiple uses. Accuracy is mst likely t be secured if data is captured as clse t the pint f activity as pssible. Reprted infrmatin that is based n accurate data prvides a fair picture f perfrmance and shuld enable infrmed decisin making at all levels. The need fr accuracy must be balanced with the imprtance f the uses fr the data, and the csts and effrt f cllectin. Fr example, it may be apprpriate t accept sme degree f inaccuracy where timeliness is imprtant. Where cmprmises have t be made n accuracy, the resulting limitatins f the data shuld be clear t its users. Data shuld be recrded and used in cmpliance with relevant requirements, including the crrect applicatin f any rules r definitins. This will ensure cnsistency between perids and with similar rganisatins. Where prxy data is used t cmpensate fr an absence f actual data, rganisatins must cnsider hw well this data is able t satisfy the intended purpse. Data shuld reflect stable and cnsistent data cllectin prcesses acrss cllectin pints and ver time, whether using manual r cmputer-based systems, r a cmbinatin. Managers and stakehlders shuld be cnfident that prgress tward perfrmance targets reflects real changes rather than variatins in data cllectin appraches r methds. Data shuld be captured as quickly as pssible after the event r activity and must be available fr the intended use within a reasnable time perid. Data must be available quickly and frequently enugh t supprt infrmatin needs and t influence the apprpriate level f service r management decisins. Data captured shuld be relevant t the purpses fr which it is used. This entails peridic review f requirements t reflect changing needs. It may be necessary t capture data at the pint f activity which is relevant nly fr ther purpses, rather than fr the current interventin. Quality assurance and feedback prcesses are needed t ensure the quality f such data. Data requirements shuld be clearly specified based n the infrmatin needs f the rganisatin and data cllectin prcesses matched t these requirements. Mnitring missing, incmplete, r invalid recrds can prvide an indicatin f data quality and can als pint t prblems in the recrding f certain data items. Surce: Audit Cmmissin Imprving infrmatin t supprt decisin making: standards fr better quality data Page 4 f 17 Data Quality Plicy
4 Hw can we imprve data quality? If we are t achieve cnsistently high standards f data quality in ur perfrmance infrmatin, leadership frm the tp f the rganisatin is essential, underscred by a cmprehensive management and accuntability framewrk supprting this bjective. This framewrk shuld cver: Staff knwledge f data t be prvided The prcedures in place fr data recrding and reprting The respnsibilities f staff t achieve the data quality standards The systems and prcesses in place t ensure data quality The arrangements and cntrls in place fr mnitring and review f data The guidelines fr presenting infrmatin t a variety f audiences These areas are detailed in the fllwing sectins. 5 Data definitins Everyne invlved will be aware which PIs are prduced frm the data they input and hw they are defined (see Respnsibilities). Cmprehensive Perfrmance Assessment (CPA) indicatrs and Best Value Perfrmance Indicatrs (BVPIs) have natinally set definitins. It is imprtant that every detail f the definitin is understd and applied. This ensures that data is recrded cnsistently, allwing fr cmparisn ver time and natinal benchmarking. Althugh n lnger natinally cllated, many Authrities still utilise CPA indicatrs and best value perfrmance indicatrs (BVPIs). Where we are setting lcal perfrmance indicatrs, we need t ensure that we have established a clear definitin and that there are systems r prcesses available t cllect and reprt the data in an agreed frmat. Fr example, we need t be clear abut whether target and utturn figures refer t a snapsht r cumulative psitin. Every PI must have a Nminated Officer (see Sectin 7.1) wh is respnsible fr cllecting and reprting the infrmatin. This ensures that there is cnsistency in the applicatin f definitins and the use f systems fr prviding the data. Every PI must als have a Validating Officer (see Sectin 7.1) wh must keep the Nminated Officer advised f any changes in definitin that may ccur frm time t time. Page 5 f 17 Data Quality Plicy
6 Dcumented prcedures In rder t minimise errrs and achieve gd quality data, apprpriate prcedures and guidance must exist s that staff can be trained and supprted in their wrk. Details f these prcedures and prcesses will be available t all trained staff. 6.1 Timescales Best use can be made f perfrmance infrmatin if it is prduced and cmmunicated n a timetable that allws fr management actin. The timely recrding f data is essential t the efficient and effective prductin f perfrmance infrmatin fr decisin-making and imprved service utcmes. Data needs t be present at the time that prcesses require it, fr bth service delivery and reprting purpses. All perfrmance infrmatin shuld be cllected frequently i.e. mnthly cllectin as ppsed t quarterly cllectin. This may require departments t re-examine their internal prcesses t prvide infrmatin n a shrter timetable withut cmprmising data quality. Staff need t be aware f relevant deadlines and all the key dates must be bserved. 6.2 Audit trail All PIs need t be supprted by a clear audit trail. Examples f dcuments t supprt figures include detailed tables, evidence f calculatins, etc. It is best practice t set up audit files fr all perfrmance infrmatin at the beginning f each year. The check sheet shuld be kept n file and act as a guide as t what infrmatin shuld be included t supprt each PI. A clear and cmplete audit trail must exist fr all PIs. All annual utturns fr PIs must fllw the full sign ff prcedure i.e. the End f Year Self- Assessment Frm and the Annual Outturns Summary. 6.3 Imprvement and develpment Where it is cnsidered necessary, data quality reprts will be fed back t peratinal managers with advice regarding crrective actin t be taken such as imprving prcesses and systems, r advice relating t staff training and develpment. 6.4 Duplicatin and replicatin As stated abve, in the characteristics required fr gd quality data, data shuld be captured nce nly, even thugh it might have multiple uses. T maintain accuracy and reliability, it is essential that the data is held in a single versin r frm a single surce. Page 6 f 17 Data Quality Plicy
Where lcally built sub-sets f data are utilised fr individual prjects r departments (e.g. by taking a cpy f a data set frm the crrect database and then manipulating it t prvide statistical infrmatin), it is mre likely that inaccuracies and incnsistencies will ccur, fr example as a result f using ut f date r unaudited data. Similarly, where data is recrded twice, it is pssible that future references t the data might access the wrng versin r surce and result in further inaccuracies. Such usage and manipulatin f data has becme all t cmmn and can ften result in pr quality data. All data must therefre be btained frm the crrect surce. Duplicatin and replicatin f data in lcally built data sets are frbidden. 7 Respnsibilities Overall strategic respnsibility fr data quality rests with the Directr f Crprate Resurces, but all relevant staff shuld knw hw their day-t-day jb cntributes t the calculatin f perfrmance infrmatin and hw lapses can lead t either errrs r delays in reprting, bth f which limit the ability t manage perfrmance effectively. This means that everyne shuld have an understanding f any PIs affected by the data they cntribute. A basic grasp might be, fr example, knwledge f what the numeratr and denminatr is, and whether there are any imprtant guidelines (fr example, the exclusin f certain cases). This will nrmally be easier t cmmunicate if staff understand the purpse f the indicatr, r the plicy it is meant t mnitr. 7.1 Defined respnsibilities Fr all perfrmance indicatrs there must be a Nminated Officer (with the respnsibility fr cllecting and inputting r recrding the infrmatin) and a Validating Officer (respnsible fr the validatin f the infrmatin). The rle f these fficers is crucial fr the accurate cllectin and reprting f perfrmance infrmatin. The Nminated Officer will: Ensure that the crrect definitins have been applied when calculating the figures fr all PIs, and that any subsequent updates t these definitins have als been cmplied with. Where definitins are nt clear, they must seek advice frm the Validating Officer. Maintain awareness f relevant deadlines and the key dates t be bserved. Maintain an audit file. Ensure that all utturns and future targets are crrectly entered. Ensure that relevant cmments are added t any reprt r prfrma, which explain and relate t the perfrmance utturn psitin, and describe plans fr addressing any majr variatin frm target r previus utturn. Ensure that targets are set fr all PIs and recrded fully n the relevant systems. Page 7 f 17 Data Quality Plicy
The Validating Officer will: Ensure that accurate perfrmance data is reprted this will include a thrugh check f data prduced by the Nminated Officer and pssibly a request fr inclusin f further pieces f evidence t supprt the figures prduced fr audit files. Where definitins are nt clear t the Nminated Officer, the Validating Officer will seek advice frm the relevant defining bdy usually via CLG. Cmmunicate any updates / revised guidance and ensure full cmpliance. Review the additinal cmments prvided, such as any cmmentary r analysis. Crdinate the cllectin f all future years targets and the methdlgy by which they are derived, taking int accunt any gvernment r natinal targets r guidance. Offer general training and supprt t Nminated Officers in their departments. Practively infrm the Nminated Officers f the results f their effrts in ensuring data quality, prviding them with feedback, bth psitive and negative. 7.2 Respnsibilities f all staff 7.2.1 Recgnitin and awareness Everyne must recgnise the need fr gd data quality and hw they can cntribute. Data quality is the respnsibility f every member f staff entering, extracting r analysing data frm any f the Authrity s infrmatin systems. Every relevant persn shuld therefre be aware f his r her respnsibilities with regard t data quality. The cmmitment t data quality will be cmmunicated clearly thrughut the Authrity t reinfrce this message. Respnsibility fr data quality will be reflected in jb descriptins and the appraisal prcess. s are encuraged t ensure that suitable appraisal targets and paragraphs are included in jb descriptins, apprpriate t the level f invlvement staff have in the infrmatin prcess. The imprtance f achieving gd data quality will be addressed with all relevant staff as part f the inductin prcess at the cmmencement f their emplyment. Respnsibility fr the peratinal management f data quality will lie with the managers f all departments affected by this plicy. 7.2.2 Data input respnsibilities There must be adequate cntrls ver the input f data. It is vital that infrmatin input int a system is accurate, t ensure that infrmatin extracted frm it can be relied upn. The aim shuld be 100% accuracy 100% f the time. Page 8 f 17 Data Quality Plicy
It is imprtant that staff have clear guidelines and prcedures fr using systems and are adequately trained t ensure that infrmatin is being entered cnsistently and crrectly. There must be cntrls ver input, ensuring especially that infrmatin is input n an nging basis, rather than being stred up fr input en blc at the end f a perid. This shuld reduce the errr rate and the need fr cmplex verificatin prcedures. All data cllectin and input prcesses will have an audit trail that perates cntinuusly. Any training and develpment issues identified in the curse f auditing will be addressed prmptly. Where an issue relating t data quality cannt easily be reslved thrugh training and develpment at the pint f input, it will be reprted by the Nminated Officer t the Validating Officer and ultimately, if still nt reslved, t the Directr f Crprate Resurces, wh hlds strategic respnsibility fr data quality. 8 Systems and prcesses Systems and prcesses must be fit fr purpse and staff must have the expertise t get the best ut f them. Perfrmance infrmatin will be extracted regularly and therefre the systems and prcesses must allw fr efficient extractin and swift cmmunicatin. Each system r prcess must have a Nminated Officer respnsible fr data quality issues. This Nminated Officer is required t ensure that: Users are adequately trained, where apprpriate by having a frmal training prgramme which is peridically evaluated and adapted t respnd t changing needs. Written prcedures (e.g. user guide) exist fr the purpse f extracting perfrmance infrmatin. Adequate supprt is available t users. The system can prduce apprpriate audit trails. System upgrades can be made where necessary (including t accmmdate amendments t definitins). The system meets the perfrmance infrmatin needs f the Authrity and its managers. Feedback frm users is acted upn. Actins recmmended by system reviews (e.g. by the external auditrs) are implemented. There is security f access / amendment. Page 9 f 17 Data Quality Plicy
Peridic data integrity tests are undertaken. A business cntinuity plan fr the system exists t prtect vital recrds and data. There shuld als be a named substitute fficer wh can deputise in the Nminated Officer s absence by (at least) maintaining the day-t-day functinality f the system. Given the increasingly demanding timescale fr perfrmance reprting, the Authrity cannt affrd t have systems lying drmant during unplanned absences. It is, therefre, als essential that written prcedures are designed s that anther fficer can carry ut the prcedures essential t prviding perfrmance infrmatin if the fficer wh nrmally perfrms these duties is absent. 9 Mnitring and review The Authrity's data quality arrangements will be regularly reviewed by Crprate Services. This Data Quality Plicy will be reviewed n an annual basis and updated as necessary. The Authrity wishes t embed a data quality learning culture s that imprvements can be made where necessary and mistakes avided. As stated in Sectin 7.2.2, the aim shuld be 100% accuracy 100% f the time. T achieve this, data requirements must be designed alng the principle f getting it right first time in rder t avid waste in the frm f time and mney spent n checking data, interfacing between different infrmatin systems, matching and cnslidating data frm multiple databases, etc. Nevertheless, in cmplex prcesses, even where there are strng cntrls ver input, errrs can ccur. Therefre, a verificatin prcedure shuld exist clse t any pint f data input. The frequency f verificatin checks will need t be apprpriate t the frequency f data reprting. Respnsibility fr data verificatin will lie within individual departments, but Crprate Services can ffer advice and guidance regarding verificatin prcedures. Particular attentin needs t be paid t data prvided by external surces (see Sectin 11 belw). It is imprtant that perfrmance infrmatin is subject t scrutiny befre being passed up the line fr management actin. This can be undertaken at several stages in the prcess. The mst basic instance will be a verificatin check n the initial utput. There will als be clse internal scrutiny f external audit submissins by reviewers (see Sectin 12 belw). Page 10 f 17 Data Quality Plicy
10 Presentatin Reprting accurate infrmatin regularly leads t gd decisin-making and imprved perfrmance. Wherever pssible, reprts shuld fllw the crprate traffic light principles, and shuld include diagrammatic summaries and brief cmmentaries. Fr a large number f PIs, that perfrmance will nly be recgnised publicly if it can be substantiated by external bdies. Annual PIs must therefre be presented in such a way as t give an easily understd and accurate picture f ur perfrmance t external auditrs and the public. Perfrmance infrmatin is pr-actively imprved regularly as a result f requests frm, r changes apprved by, the Perfrmance Imprvement Wrking Grup, which includes members f senir management and the Perfrmance Imprvement Member Champin. 11 Validatin f data frm partners When entering int cntracts with service prviders it is essential that, wherever there is a requirement t prvide timely and accurate perfrmance infrmatin, we are clear with the prvider abut their respnsibilities fr data quality and hw we will be checking the infrmatin they prvide. It is the Authrity s respnsibility t ensure the data used is rbust this includes any data we receive frm partners and third parties. The Nminated Officer must be satisfied that any external data is accurate, but must check it in the same way that they wuld internal data. The frmat, presentatin and frequency f cllectin f data must cmply with all Authrity requirements. All key dates fr the submissin f data must be adhered t it is each department s respnsibility t cmmunicate and enfrce this message t any partners. All Nminated Officers wh receive data frm partners r third parties must ensure that these rganisatins have the same shared understanding f data definitins. A clear audit trail t evidence validatin checks is essential and must be kept n file. Wherever pssible, a specific bjective relating t data quality shuld be written int all partner cntracts r service level agreements. Page 11 f 17 Data Quality Plicy
12 External audits When infrmatin is presented fr external audit, at least tw fficers must review wrking papers t cnfirm that the definitin has been fllwed, the calculatins are crrect and the indicatr is supprted by a full audit trail. This full audit trail needs t include the fllwing: calculatin system ntes explanatin f any variance frm the previus year dcumentatin supprting any estimates, sampling, r any apprtinments made supprting infrmatin (e.g. spreadsheet, database, screen dumps) r at least a full descriptin f where the supprting infrmatin is kept. During external audits, there shuld be at least ne ther fficer wh is able t prvide advice and infrmatin n the PI in the absence f the Nminated Officer. This is an imprtant cntrl t ensure that audit wrk prceeds smthly. Page 12 f 17 Data Quality Plicy
13 Actin plan 13.1 Staff knwledge and respnsibilities Actin Number Actin 13.1.1 Every PI t have a Nminated Officer - T be respnsible fr cllecting and reprting the infrmatin. - T ensure cnsistency in the applicatin f definitins. CPA indicatrs and BVPIs: every detail f the definitin must be understd and applied. Lcal perfrmance indicatrs: there must be a clear definitin. - T maintain awareness f deadlines / key dates. - T maintain an audit file. - T ensure cnsistency in the use f systems fr prviding the data. - T ensure that ut-turns and future targets are crrectly entered. - T ensure that relevant cmments are added. Respnsibility Timescale Head f Onging 13.1.2 Every PI t have a Validating Officer - T keep the Nminated Officer advised f any changes in definitin. - Where definitins are unclear, t seek advice frm the defining bdy. - T be respnsible fr validatin (e.g. thrugh check f data; pssible requests fr inclusin f further evidence). - T c-rdinate cllectin f all future targets and the methdlgy used. - T review the additinal cmments prvided. - T ffer general training and supprt t Nminated Officers in their departments. - T practively infrm the Nminated Officers f the results f their effrts in ensuring data quality, prviding them with feedback, bth psitive and negative. Crprate Services Onging 13.1.3 Data input staff t be aware which PIs are prduced frm the data they input, hw they are defined and whether there are any imprtant guidelines r exclusins. Head f Onging Page 13 f 17 Data Quality Plicy
Actin Number Actin 13.1.4 s t reflect respnsibility fr data quality, where apprpriate, by: - the inclusin f suitable paragraphs and appraisal targets in jb descriptins. - addressing as part f the inductin prcess fr new starters. - suitable emphasis in the appraisal prcess. Respnsibility Timescale Head f Onging 13.2 Prcedures data recrding and reprting Actin Number Actin 13.2.1 Apprpriate prcedures and guidance t be dcumented t train and supprt staff. Details f these prcedures and prcesses t be available t all trained staff. Respnsibility Timescale Head f Onging 13.2.2 Staff t be aware f relevant deadlines and all the key dates t be bserved. Head f Onging 13.2.3 All perfrmance infrmatin t be cllected frequently i.e. mnthly cllectin as ppsed t quarterly cllectin (this may require departments t re-examine their internal prcesses t prvide infrmatin n a shrter timetable). Input timescales r cntrls t ensure that data infrmatin is input n an nging basis, rather than en blc at the end f a perid. Head f Onging 13.2.4 Audit files fr all perfrmance infrmatin t be set up at the beginning f each year (check sheet t be kept n file as a guide abut what infrmatin shuld be included t supprt each PI). Crprate Services Annually 13.2.5 All annual utturns fr PIs t fllw the full sign ff prcedure (i.e. the End f Year Self-Assessment Frm and the Annual Outturns Summary). Crprate Services Annually Page 14 f 17 Data Quality Plicy
13.3 Systems and prcesses Actin Number Actin 13.3.1 Each system r prcess t have a Nminated Officer t ensure that: Respnsibility Timescale Head f Onging - Users are adequately trained, where apprpriate by having a frmal training prgramme, which is peridically evaluated and adapted t respnd t changing needs. - Written prcedures (e.g. user guide) exist fr the purpse f extracting perfrmance infrmatin. - Adequate supprt is available t users. - Feedback frm users is acted upn and actins recmmended by system reviews (e.g. by the external auditrs) are implemented. - There is security f access / peridic data integrity tests are undertaken. - A business cntinuity plan fr the system exists t prtect vital data. 13.3.2 There shuld als be a named substitute fficer wh can deputise in the Nminated Officer s absence by (at least) maintaining the day-t-day functinality f the system. Head f Onging 13.4 Mnitring and review f data Actin Number Actin 13.4.1 Crprate Services t regularly review the Authrity's data quality arrangements, including this Data Quality Plicy n an annual basis. Respnsibility Timescale Crprate Services Annually 13.4.2 Verificatin prcedures t exist clse t any pint f data input. The frequency f verificatin checks t be apprpriate t the frequency f data reprting. Head f Onging 13.4.3 All perfrmance infrmatin t be subject t further validatin befre being passed up the line fr management actin. Crprate Services Onging Page 15 f 17 Data Quality Plicy
13.5 External audits Actin Number Actin 13.5.1 When infrmatin is presented fr external audit, at least tw fficers t review wrking papers t cnfirm that the definitin has been fllwed, the calculatins are crrect and the indicatr is supprted by a full audit trail. Respnsibility Timescale Crprate Services Onging 13.5.2 During external audits, at least ne ther fficer t be able t prvide advice and infrmatin n the PI in the absence f the Nminated Officer, t ensure that audit wrk prceeds smthly. Head f Onging Page 16 f 17 Data Quality Plicy
West Yrkshire Fire & Rescue Service Oakryd Hall Birkenshaw BD11 2DY www.westyrksfire.gv.uk