Improving timeliness of industrial short-term statistics using time series analysis

Transcription

1 Improving imeliness of indusrial shor-erm saisics using ime series analysis Discussion paper Frank Aelen The views expressed in his paper are hose of he auhors and do no necessarily reflec he policies of Saisics Neherlands Saisics Neherlands Voorburg/Heerlen, April 2004

2 Explanaion of symbols. = daa no available * = provisional figure x = publicaion prohibied (confidenial figure) = nil or less han half of uni concerned = (beween wo figures) inclusive 0 (0,0) = less han half of uni concerned blank = no applicable = 2003 o 2004 inclusive 2003/2004 = average of 2003 up o and including / 04 = crop year, financial year, school year ec. beginning in 2003 and ending in 2004 Due o rounding, some oals may no correspond wih he sum of he separae figures.

3 Publisher Saisics Neherlands Prinses Bearixlaan XZ Voorburg The Neherlands Prined by Saisics Neherlands - Faciliy Services Cover design WAT onwerpers, Urech Informaion infoservice@cbs.nl Where o order verkoop@cbs.nl Inerne hp:// Saisics Neherlands, Voorburg/Heerlen Quoaion of source is compulsory. Reproducion is permied for own or inernal use. ISSN: Key figure: X-10 Producion code: Saisics Neherlands

4 Conens 1. Inroducion Overview of curren survey process and possible improvemens Curren survey process a Saisics Neherlands Possible improvemens Impuaion of missing daa Inroducion Trend-based mehods using one earlier monh Proposed mehod using ime series analysis Impuaion of missing enerprise daa using aggregae esimaes Implemenaion of he mehods General Currenly used rend-based mehod Improved rend-based mehod Proposed mehod using ime series analysis Simulaions Assessmen of accuracy Sofware Resuls Discussion Conclusions References Appendix A. Overview of SIC divisions and size classes Appendix B. regarima modelling in X-12-ARIMA Appendix C. X-12-ARIMA specificaion file Appendix D. Resuls of simulaions

5 IMPROVING TIMELINESS OF INDUSTRIAL SHORT-TERM STATISTICS USING TIME SERIES ANALYSIS Summary: Naional saisical insiues of he European Union are under growing pressure o improve he imeliness of shor erm economic saisics. The aim of his sudy was o improve he imeliness of he monhly saisics on sales developmen in he Duch manufacuring indusry. The focus was on producing imely indicaors in he presence of missing daa. Currenly, missing daa are impued using, besides already available daa for he monh in quesion, known daa from only one previous monh, whereas we propose o use whole ime series for ha purpose. Furhermore, since here is a radeoff beween imeliness and accuracy, he relaionship beween hese wo qualiy measures was invesigaed in order o be able o make a more balanced decision beween hem. I was concluded ha wih respec o he currenly used mehod, imeliness can be improved from abou 37 days o abou 27 days afer he end of he monh, wihou sacrificing accuracy. Furher improvemen of imeliness is possible a he cos of reduced accuracy. Keywords: shor-erm saisics, manufacuring indusry, imeliness, accuracy, impuaion, ime series analysis, X-12-ARIMA 1. Inroducion The pas few years, naional saisical insiues of he European Union are under growing pressure o improve he imeliness of shor erm economic saisics. Many of he imporan users of hese saisics, such as he European Cenral Bank (ECB), Eurosa, he European Commission (EC), he Inernaional Moneary Fund (IMF), governmens and financial marke analyss, consider i a key issue ha preliminary daa are disseminaed earlier. There is, however, a rade-off beween imeliness and accuracy. Improving imeliness wihou improving he survey process, would almos cerainly reduce he accuracy. The challenge is o improve he mehods so ha when daa are disseminaed earlier he curren level of accuracy can be mainained. On he oher hand, for some saisics, reduced accuracy may be accepable if his resuls in grealy improved imeliness. Therefore, o make a balanced decision in he rade-off beween imeliness and accuracy, i is imporan o know he relaionship beween hem. This relaionship varies among differen saisics, depending on he heerogeneiy of he underlying populaion of enerprises and he specific survey process used. The aim of his sudy was o improve he imeliness of shor-erm saisics, in paricular he monhly saisics on sales developmen in he Duch manufacuring 5

6 indusry. This saisics plays, among oher hings, an imporan role in he forecasing of economic developmen and he seing of moneary policy. Furhermore, for his paricular saisics he relaionship beween imeliness and accuracy was o be invesigaed. The ouline of his paper is as follows. In secion 2 he curren survey process a Saisics Neherlands regarding he monhly saisics on sales developmen in he manufacuring indusry is described briefly, including possible poins for improvemen. This paricular survey is based on an inegral collecion of daa. Secion 3 deals wih one of he possible improvemens, i.e. he impuaion of missing daa. Currenly, missing daa are impued using a rend-based mehod ha uses, besides already available daa for he monh in quesion, known daa from only one previous monh. This mehod is invesigaed and some improvemens are suggesed. Furhermore, we propose an alernaive mehod o impue missing daa using whole ime series of daa. Noe ha already available daa for he monh in quesion are also used here. Simulaions o deermine he accuracy of he curren and improved rend-based mehods, as well as of he proposed ime series mehod, as a funcion of imeliness, are described in secion 4. Resuls of he simulaions are presened in secion 5. The mehods and resuls are discussed in secion 6, and some conclusions are drawn in secion Overview of curren survey process and possible improvemens 2.1 Curren survey process a Saisics Neherlands 2 The saisics on sales developmen in he manufacuring indusry is based on he populaion of enerprises wihin he SIC divisions (Appendix A) ha have more han 20 employees. In he period his concerned on average 6,450 enerprises. Sales daa of hese enerprises are colleced inegrally on a monhly basis, mainly by paper quesionnaires. Mos of he enerprises respond wihin a period of a few monhs afer he end of he monh in quesion. Figure 1 shows, as a funcion of he number of days afer he end of a monh, he percenage of enerprises ha has responded, as well as he percenage of urnover associaed wih hese enerprises. Currenly, preliminary values for he sales developmen in he manufacuring indusry are disseminaed on average 37 days afer he end of he monh. A ha ime 62%±4% (mean ± sandard deviaion; period ) of he enerprises has responded, corresponding o 74%±4% of he urnover. To compare, 14 days afer he 2 During he ime his sudy was done anoher sysem (IMPECT) o compile shor-erm saisics became operaional. In secion 6 i is shown, however, ha resuls of his new sysem, regarding he assessmen of sales developmen in he manufacuring indusry, are expeced o be similar o he mehod described here. 6

7 end of he monh only 26%±6% of he enerprises has responded, corresponding o 23%±8% of he urnover. Evenually, 88%±6% of he enerprises responded, corresponding o 97%±2% of he urnover. respons 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Turnover Enerprises days afer end of monh Figure 1. Percenage of enerprises ha has responded o he quesionnaire and he percenage of urnover associaed wih hese enerprises, boh as a funcion of he number of days afer he end of a monh. Compleed quesionnaires are edied wihin a day afer receip in order o remove eviden errors. This daa ediing is done in par auomaically (e.g. when he urnover is misakenly no given in housands of euros) and in par manually (e.g. when he urnover is no plausible wih respec o previous values). In some cases he responden is conaced. The nex sep in he process is he impuaion of missing daa. A any momen in ime no all enerprises have responded (ye) o he quesionnaire. The missing daa are impued using a rend wih respec o he previous monh. This rend is deermined using an aggregae of more or less similar enerprises ha do have responded. Since daa come in no only for he laes monh bu also for earlier monhs, he sill necessary impuaions of hose earlier monhs are redone wih he mos recen informaion available. More deails on he impuaion mehod are given in secion 3. Finally, he sales developmen wih respec o he same monh of he previous year is calculaed. In doing his, only he enerprises are used ha are in he inersecion of he populaions of hose wo monhs. So, enerprises ha, for one reason or anoher, have moved in or ou of he populaion during he year, are lef ou of he calculaion. Then, he sales developmen of monh wih respec o one year before is assessed sraighforwardly as: ( T T 12 )/ T 12 T =, (1) where T is he sales developmen, T is he oal urnover of all enerprises in he inersecion and subscrips indicae he monh. Since all enerprise daa are known, eiher impued or no, he sales developmen can also be assessed for any desired subdivision of he manufacuring indusry. Noe ha his subdivision is independen of he aggregaes of enerprises ha were used for he impuaion of missing daa. 7

8 Moreover, during a year enerprises can migrae from one subdivision of he populaion o anoher. The sales developmen of a subdivision is based on he enerprises belonging o he inersecion of he subdivisions a monhs and -12. Saisics Neherlands disseminaes (preliminary) sales developmen daa on: The manufacuring indusry as a whole: SIC divisions (MI1); Manufacure of food, beverages and obacco: SIC divisions 15/16 (MI6-1); Manufacure of exiles, clohing, leaher and leaher producs: SIC divisions (MI6-2); Manufacure of wood, producs of wood and building maerials: SIC divisions 20/26 (MI6-3); Manufacure of paper and paper producs, prining and publishing: SIC divisions 21/22 (MI6-4); Manufacure of refined peroleum, chemical, rubber and plasic producs: SIC divisions (MI6-5); Manufacure of meal producs, elecrical and ranspor equipmen, furniure and oher manufacuring: SIC divisions (MI6-6). In parenheses he codes are given ha are used in his paper o indicae he manufacuring indusry as a whole (MI1), as well as he six subdivisions of he manufacuring indusry (MI6). 2.2 Possible improvemens A lo of lieraure is available on he subjec of improving imeliness of shor-erm saisics. Here, only some of he mos imporan possible improvemens are menioned, in relaionship o he curren survey process of he saisics on sales developmen in he Duch manufacuring indusry. Some of hese possible improvemens have only been referred o in lieraure, and have no ye been fully invesigaed. A lo of ineresing ideas are discussed in he final repor of he Exper group on Sampling for Timely European Indicaors (ESTEI - Deville e al, 2002), where also many references can be found. The use of paper quesionnaires is, wih respec o imeliness, no very efficien. More advanced daa collecion mehods, such as inerne/ , ouchone daa enry, fax, or elephone (possibly only for a firs bes guess), will probably improve imeliness. Also adaping quesionnaires more o he accouning pracices of enerprises may be helpful. A furher improvemen for compiling early preliminary daa may be he use of a subsample of enerprises, for which a lo of effor is pu ino geing early figures (Deville e al, 2002). In ha case, he larges enerprises should cerainly be par of he subsample. Compleed quesionnaires are edied afer receip in order o remove eviden errors. Depending on he number and size of quesionnaires, ediing can be quie imeconsuming. Selecive ediing echniques, in which an algorihm is used o auomaically selec he mos imporan (possible) errors, may reduce ediing ime. 8

9 Such algorihms are also in use a Saisics Neherlands, bu hese may be improved by using ime series analysis (Revilla, 2002) insead of using daa from only one or a few previous monhs. Afer daa collecion and ediing, he sill missing daa have o be deal wih. There are a number of approaches o do his. The missing daa can be esimaed for each enerprise separaely or on an aggregaed level. In case he missing daa are esimaed on an aggregaed level, i is sill possible o pass he aggregae esimaes back o he individual enerprises. This laer impuaion approach is he one currenly employed a Saisics Neherlands in he assessmen of sales developmen. The curren mehod, however, uses besides he already received daa for he monh in quesion, daa from only one previous monh. Beer esimaes can probably be made by using all hisorical enerprise daa available, i.e. using whole ime series of enerprise daa. An alernaive approach o deal wih missing daa, is o direcly esimae he populaion oal from he already received daa. Generally spoken, design-based procedures, like he calibraion esimaor or he generalized regression esimaor, and model-based procedures, which use e.g. economeric or ime series models, can be disinguished. This approach faciliaes he use of auxiliary informaion ha is srongly correlaed wih he shor-erm indicaor in quesion in order o improve he accuracy of he esimaes. In ha case, a requiremen is ha he auxiliary informaion is already available a he desired disseminaion ime. Auxiliary informaion ha may be used for insance is daa from business cycle surveys, in which enerprises are asked o assess, in a qualiaive manner, heir presen siuaion and heir expecaions for he near fuure. Resuls from his survey are generally available already a he end of he monh. Summarizing, using more advanced daa collecion and ediing echniques can be very imporan in improving imeliness of shor-erm saisics. In conjuncion, mehods can be developed o deal wih missing daa, since even when daa are colleced faser, here will sill be missing daa a any poin in ime. In his sudy we focussed on improving he impuaion mehod used a Saisics Neherlands in he compilaion of he saisics regarding sales developmen in he manufacuring indusry. 3. Impuaion of missing daa 3.1 Inroducion Suppose a a cerain poin in ime one wans o know he sales developmen for monh for a cerain populaion of enerprises (his can be he enire manufacuring indusry or any par of i). This developmen can be derived from he urnover values of he individual enerprises in he populaion. Then, a ha poin in ime, he problem arises ha he daa for monh are no ye known for all individual 9

10 enerprises. In ha case, he populaion of enerprises can be divided ino wo groups, i.e. a group for which he daa are known (group I) and a group for which he daa are no known (group II). We assume a complee enumeraion of he populaion and hisorical daa for individual enerprises belonging o group I as well as group II are known inegrally, eiher impued or no. These hisorical daa can be used o faciliae he impuaion of missing daa for group II for monh. In order o do he impuaions, firs an esimae YII of he missing daa is made on he aggregae level, and subsequenly his aggregae esimae is passed back o he individual enerprises. The aggregae urnover for group II for monh ( Y II ) ha is needed for he impuaions can be esimaed in several ways. Below, firs a rend-based mehod using daa from only one earlier monh is described (secion 3.2). The mehod currenly used a Saisics Neherlands is based on his principle. Subsequenly, a new mehod based on ime series analysis is inroduced (secion 3.3). In secion 3.4 he mehod of impuing missing enerprise daa, given he aggregae esimaes, is explained. Secion 3.5 concludes wih some pracical aspecs concerning he implemenaion of he mehods. Firs he currenly used rend-based mehod is reaed, second he rend-based mehod wih some improvemens, and hird he mehod based on ime series analysis. 3.2 Trend-based mehods using one earlier monh Ofen employed mehods o esimae he missing daa use hisorical daa from only one earlier monh, e.g. he previous monh ( =1) or he monh one year before ( =12). These mehods assume ha on an aggregae level he rend from he earlier monh o monh is he same for groups I and II. The observed rend for group I can hen be applied o he value of group II for he earlier monh, in order o esimae he aggregae urnover of group II for monh. So, Y II Y = YII, (2) Y I I where Y is he oal urnover, he subscrip indicaes he group and he superscrip indicaes he monh 3. Graphically, his mehod can be inerpreed as follows (figure 2). Firs, a sraigh line is drawn from he origin hrough he poin ( Y I, YII ). Then YII is found as he verical value on he line a he horizonal posiion Y. I 3 Do no confuse he erm rend as i is used here wih he erm rend as i is someimes used in seasonal adjusmen mehods (e.g. in X-12-ARIMA). 10

11 6.000 Aggregae urnover group II Aggregae urnover group I Figure 2. Mehod using daa from only one earlier monh. The do refers o he values for ha earlier monh. The esimae for group II a monh is found as he verical value a he sraigh line a he horizonal posiion defined by he urnover for group I a monh. 3.3 Proposed mehod using ime series analysis Insead of using daa from only one earlier monh in order o esimae Y II, i is possible o use whole ime series of daa for groups I and II (figure 2) for ha purpose. These ime series are consruced by assigning each enerprise in he populaion a monh o eiher group I or group II, based on he availabiliy of daa for monh. Subsequenly, he urnover values of all enerprises are summed, per group, for monh as well as for all previous monhs Aggregae urnover group 1 group ime (monhs) Figure 2. Time series of aggregae urnover for groups I and II. In his example he groups are defined based on he availabiliy of daa for monh 73. So, group I has daa unil monh 73 and group II has daa unil monh 72. The aim is o esimae he urnover value for group II for monh 73. Similarly as was done in figure 1 for one earlier monh, aggregae urnover for group I agains group II can now be ploed for all monhs (excluding monh ) in he ime series (figure 3). The sraigh line shown in figure 3 is he regression line of he aggregae urnover of group II on he aggregae urnover of group I, wih an inercep, based on ordinary leas squares esimaion. I can clearly be seen ha he 11

12 regression line has an inercep ha is differen from zero. This is generally he case for he ime series invesigaed in his sudy. Aggregae urnover group II y = x R 2 = 0, Aggregae urnover group I Figure 3. Regression plo of group I agains group II for all monhs in he ime series shown in figure 2. Noe ha he ime dimension canno be seen in his kind of plo. Because each of he ime series of groups I and II is based on he same enerprises a every monh, he daa poins in he regression plo are no independen, i.e. hey are correlaed in ime. Figure 4 shows a ime series of residuals wih respec o he regression line. I can be seen ha he residuals have a disinc emporal behaviour, including seasonal paerns. Seasonal paerns in he residuals occur when groups I and II each have differen seasonal paerns so ha hey do no cancel each oher ou Aggregae urnover residuals Time (monhs) Figure 4. Time series of he residuals wih respec o he regression line shown in figure 3. Noice he disinc emporal behaviour. The mehod we propose makes use of he ime series of groups I and II in order o make a forecas for group II for monh. Besides regression analysis of groups I and II (cf. figure 3), also he emporal behaviour of he residuals is aken ino accoun (cf. figure 4). This is done by modelling he ime series according o he regarima mehod: Y Y ++ Z, (3) II = 0 1 I 12

13 where 0 and 1 are he regression coefficiens and Z are he residuals. The residuals are modelled using so-called ARIMA models (AuoRegressive Inegraed Moving Average), which are ofen used in he analysis of economic ime series. ARIMA models describe he ime series in erms of seasonal and non-seasonal componens by a number of parameers. These parameers, as well as he regression coefficiens 0 and 1, are esimaed in he regarima mehod by Maximum Likelihood Esimaion (MLE), using he ime series for groups I and II unil monh -1. Subsequenly, he esimaes for 0, 1 and he ARIMA parameers, as well as he known value of Y I, are insered in equaion 3 in order o obain an esimae for Y II. Opionally, he presence of ouliers in he ime series is deal wih (auomaically) by adding more regression coefficiens. For more deails on regarima modelling see Bell (1998). A brief overview is given in Appendix B, where also he mos relevan erminology is explained. Noe ha equaion 2, concerning he rend-based mehod, can be derived from equaion 3 by leing Z =0, =0 and = Y II / Y. 0 1 I 3.4 Impuaion of missing enerprise daa using aggregae esimaes Given he esimae of he aggregae urnover for group II for monh ( Y II ), as well 1 1 as he known aggregae urnover YII for he previous monh, he rend Y II / YII for group II from monh -1 o monh is deermined. The hus found rend for he 1 aggregae level is hen applied o he individual enerprise daa yi for group II for monh -1 in order o deermine he impuaion values for monh : y i Y = Y II, II 1 yi i II. (4) 1 1 Noe ha he rend YII / YII deermined for group II, is generally no he same as 1 he rend YI / YI for group I. Only for he rend-based mehod (equaion 2) wih =1 his would necessarily be he case. 3.5 Implemenaion of he mehods General A pracical complicaion of he daa collecion is ha urnover values of he enerprises were acually colleced by means of observaion unis. In mos cases an enerprise had only one observaion uni, bu i occurred (on average 0.4%) ha an enerprise had more han one. For maers of simpliciy in he res of his paper no explici disincion is made beween he erms enerprise and observaion uni. Noe, however, ha impuaions were done on he level of he observaion uni and ha in case size classes were used o form aggregaes, he size classes of he enerprises were passed on o he underlying observaion unis. Groups I and II were each ime deermined based on he availabiliy of enerprise daa for monh. A complicaing facor here is ha some enerprises repored over 13

14 periods ha were no exacly calendar monhs, wha resuled in siuaions where daa were available for only par of a monh. This complicaion was deal wih by firs deermining he average urnover per working day for he period such an enerprise repored on, and hen muliplying his value wih he oal number of working days in monh. The hus found value was impued in order o be used in he assessmen of sales developmen. To disinguish hese enerprises from hose in groups I and II, hey were assigned o a separae group III, which was no used for furher forecasing Currenly used rend-based mehod The mehod currenly used a Saisics Neherlands is basically he rend-based mehod described in secion 3.2, wih =1. The aggregaion level used for esimaing missing daa is based on a subdivision of he manufacuring indusry ino 124 aggregaes, according o 3- and 4-digi SIC divisions. When for an aggregae no sufficien known daa for monh were available o make reasonable esimaes of he missing daa, known enerprise urnover values of he monh before were impued, aking ino accoun he number of working days in boh monhs. In case no known daa were available for an enerprise he monh before, he average wihin he aggregae for ha monh was used. Each monh, sill necessary impuaions were redone for all monhs of he presen and previous year, using he a ha ime mos recen informaion available Improved rend-based mehod The rend-based mehod using one earlier monh, on which he mehod currenly used a Saisics Neherlands (secion 3.5.2) is based, was also implemened wih a few differences c.q. improvemens wih respec o he curren mehod. Insead of using =1 in he impuaion of missing daa, =12 was used. The impuaion of missing daa for enerprises ha do no belong o he inersecion (cf. secion 2.1) is more problemaic in case =12 is used, since no daa are available for hese enerprises a -12. Therefore, for hese enerprises he impuaions were done using he rend-based mehod (secion 3.2) wih =1, in which only enerprises were used ha also did no belong o he inersecion. In case for a paricular enerprise also no daa were available for -1, because he enerprise did no ye exis or did no ye repor on ha monh, a zero urnover value was impued. Noe ha enerprises ha did no belong o he inersecion were no used in he assessmen of sales developmen. In addiion, he aggregaes used for impuing could be chosen a will. In he simulaions ha were performed (secion 4), much larger aggregaes han he curren ones were used. Therefore, in mos cases enough daa were available o make reasonable esimaes of he missing daa, and he opion of using known or average daa for a monh before (cf. secion 3.5.2) was lef ou. 14

15 Each monh, sill necessary impuaions were redone for he pas 12 monhs, using he a ha ime mos recen informaion available Proposed mehod using ime series analysis The commens made in secion on he impuaion of missing daa for enerprises ha do no no belong o he inersecion and on he use of aggregaes, are also valid for he mehod using ime series analysis. Time series analysis (regarima) was done by he X-12-ARIMA program of he U.S. Census Bureau. The X-12-ARIMA specificaion file ha was used is described in Appendix C. A one monh forecas was made of he ime series of group II. The ime series of group I, as well as a ime series wih a consan value of 1 for each monh, were used as regression variables. This las ime series was added in order o allow for a nonzero inercep in he linear regression. Addiive ouliers (AO) and level shifs (LS) were idenified. Each ime, he bes fiing model was chosen auomaically ou of he 16 ARIMA models (p,d,q)(p,d,q) wih no differencing (d=d=0) and p, q, P and Q zero or one. In case none of he models was acceped by X-12-ARIMA he AUTOMDL spec was lef ou and modelling was resriced o he regression par and oulier deecion. Also here, he sill necessary impuaions were redone each monh for he pas 12 monhs, using he a ha ime mos recen informaion available. However, when he urnover of group II was less han 15% of he oal urnover, no ARIMA modelling was done because of he hen someimes erraic naure of he ime series. So, also in ha siuaion, he AUTOMDL spec was lef ou and he regarima modelling was resriced o he regression par and oulier deecion. 4. Simulaions 4.1 Assessmen of accuracy In order o deermine he accuracy of he developed mehods, and o compare hem wih he mehod currenly used a Saisics Neherlands, a number of simulaions was performed. In hese simulaions accuracy was deermined by comparing sales developmen according o he differen mehods wih sales developmen according o he final daa. Monhly enerprise daa for he manufacuring indusry, from January 1993 unil July 2003, were available for performing he simulaions. Daa colleced up o 23 Sepember 2003 were used. Approximaely 97% of he daa, in erms of urnover, was response, whereas he res was impued using he curren mehod described in secion These daa are referred o as he final daa. Imporanly, he daes when all he enerprise daa were received were known. 15

16 The accuracy of he mehods was deermined for assessmen of sales developmen a a fixed number of days ( d ) afer he end of a monh. To his purpose, sales developmen was assessed for he 60 monhs from January 1998 unil December 2002, each ime using only he daa ha had been received unil d days afer he end of he monh. Noe ha for each monh he sill necessary impuaions were redone for he previous monhs (secion 3.5). The missing (removed) daa were impued using eiher he mehod currenly in use a Saisics Neherlands (secion 3.5.2), he improved rend-based mehod (secion 3.5.3), or he mehod based on ime series analysis (also referred o as he regarima mehod; secion 3.5.4). Regarding he improved rend-based mehod and he mehod based on regarima modelling, simulaions were done using he following aggregaes for he impuaion of missing daa: The manufacuring indusry as a whole (AI1); The manufacuring indusry subdivided ino six aggregaes (AI6) defined as he groups of SIC divisions 15/16 (AI6-1), (AI6-2), 20/26 (AI6-3), 21/22 (AI6-4), (AI6-5) and (AI6-6). Noe ha AI6-6 differs from MI6-6; The manufacuring indusry subdivided ino 30 aggregaes (AI30) defined as he combinaions of size class (SC) and SIC divisions 15/16 (AI30-1-SC), (AI30-2-SC), 20/26 (AI30-3-SC), 21/22 (AI30-4-SC), (AI30-5-SC) and (AI30-6-SC), wih SC 5, 6, 7, 8 or 9 (Appendix A). Evenually, he sales developmen (equaion 1) according o all mehods, a d days afer he end of a monh, was known for 60 monhs. This sales developmen was compared on a monhly basis wih he sales developmen according o he final daa. Minimum, maximum, average and sandard deviaion were deermined for he differences beween he values according o he simulaions and according o he final daa. This was done for he manufacuring indusry as a whole (MI1), as well as for he six subdivisions of he manufacuring indusry (MI6). 4.2 Sofware The enerprise daa were available in a Microsof SQL-server 2000 daabase. Sored procedures were wrien o manipulae daa, consruc ime series, impue daa and sore inermediae resuls. Time series analysis (regarima), including forecasing, was done by he U.S. Census Bureaus X-12-ARIMA, version (U.S. Census Bureau, 2002; Findley e al, 1998), which is a DOS program. Microsof Visual Basic 6.0 was used o run he sored procedures and X-12-ARIMA, as well as o ake care of he communicaion beween he daabase and X-12-ARIMA. The mehod currenly used by Saisics Neherlands, wih which he newly developed mehods were compared, was available as a Microsof Visual Basic 6.0 program using Oracle and Microsof Access daabases. 16

17 5. Resuls The Duch manufacuring indusry consised during he years 1998 unil 2002 on average of 6,450 enerprises wih 20 employees or more. The enerprises were disribued over he aggregaes MI6 (secion 2.1) as follows: MI6-1 15%, MI6-2 4%, MI6-3 8%, MI6-4 13%, MI6-5 10% and MI6-6 49%. In erms of urnover he disribuion is MI6-1 25%, MI6-2 2%, MI6-3 4%, MI6-4 9%, MI6-5 28% and MI6-6 33%. Abou 10% of he enerprises repored over periods ha were no exacly calendar monhs. Furhermore, on average 91% of he enerprises belonged o he inersecion as defined in secion 2.1, and hese enerprises covered on average 96% of he oal urnover of he populaion. Sales developmen in he Duch manufacuring indusry varied during he years 1998 unil 2002 beween -10% and +29%. The relaionship beween imeliness and accuracy of he assessmen of sales developmen was deermined, in a number of simulaions (secion 4.1), for he mehod currenly in use a Saisics Neherlands, he improved rend-based mehod, and he mehod based on regarima modelling. Simulaion resuls are shown in Appendix D. The bes aggregaion level for he impuaion of missing daa appeared o be AI30 for he improved rend-based mehod and AI6 for he mehod using regarima modelling. Resuls shown below are, if no menioned explicily oherwise, based on hese aggregaion levels. The sandard deviaions of he errors are shown in figure 5, as a funcion of he number of days afer he end of he monh, for he manufacuring indusry as a whole (MI1). sandard deviaion of errors 6% 5% 4% 3% 2% 1% 0% days afer end of monh curren rend-based improved rend-based regarima (ime series) Figure 5. The sandard deviaion of he errors wih respec o he final values is shown for he manufacuring indusry as a whole, as a funcion of he number of days afer he end of he monh. This is done for he currenly used rend-based mehod, he improved rend-based mehod and he regarima (ime series) mehod. The sandard deviaion of he errors is a measure of he accuracy of he mehod o assess sales developmen. 17

18 In able 1 a selecion is presened of he mos imporan resuls shown in appendix D, for he manufacuring indusry as a whole (MI1), as well as for he six subdivisions of he manufacuring indusry (MI6). Table 1. The sandard deviaion of he errors wih respec o he final values, for a number of days afer he end of he monh, regarding he currenly used rend-based mehod, he improved rend-based mehod and he regarima (ime series) mehod. Resuls are shown for he manufacuring indusry as a whole (MI1) as well as for he six subdivisions (MI6) of he manufacuring indusry (secion 2.1). Sandard deviaion of errors Number of days Disseminaion level Curren mehod Improved rend-based mehod regarima mehod 14 MI1 5.67% 3.81% 2.45% 14 MI % 4.38% 3.12% 14 MI % 6.48% 3.67% 14 MI % 4.58% 3.24% 14 MI % 4.81% 3.73% 14 MI % 9.90% 5.07% 14 MI % 5.30% 5.13% 21 MI1 2.80% 1.70% 1.57% 21 MI % 2.87% 2.37% 21 MI % 3.24% 2.49% 21 MI % 2.87% 2.45% 21 MI % 2.45% 2.43% 21 MI % 3.67% 2.92% 21 MI % 3.42% 3.42% 28 MI1 1.54% 1.05% 1.14% 28 MI % 1.83% 1.69% 28 MI % 2.51% 1.72% 28 MI % 2.23% 1.84% 28 MI % 1.85% 1.90% 28 MI % 1.94% 1.32% 28 MI % 2.14% 2.47% 37 MI1 1.09% 0.78% 0.81% 37 MI % 1.48% 1.24% 37 MI % 2.19% 1.53% 37 MI % 1.73% 1.48% 37 MI % 1.37% 1.46% 37 MI % 1.23% 1.06% 37 MI % 1.39% 1.66% 100 MI1 0.57% 0.41% 0.55% 100 MI % 0.81% 0.66% 100 MI % 1.41% 1.16% 100 MI % 0.78% 0.72% 100 MI % 0.67% 0.60% 100 MI % 0.51% 0.53% 100 MI % 0.97% 1.37% 18

19 6. Discussion Regarding he accuracy of he mehods, boh he improved rend-based mehod and he regarima ime series mehod perform beer han he mehod currenly used. For disseminaion laer han abou 27 days afer he end of he monh, he improved rend-based mehod performs abou equally well as he regarima mehod. However, for earlier disseminaion, especially earlier han abou 20 days, he loss in accuracy is much less pronounced for he regarima mehod, in comparison wih he improved rend-based mehod (cf. figure 5). Currenly daa are disseminaed a 37±3 days afer he end of he monh. Using he curren mehod he sandard deviaion of he errors a 37 days afer he end of he monh is 1.09%, for he manufacuring indusry as a whole. Concerning he six subdivisions (MI6) of he manufacuring indusry, sandard deviaions are larger. Using he regarima mehod daa can be disseminaed wih he same accuracy a around 27 days afer he end of he monh. Furher improving imeliness resuls in reduced accuracy. A 14 days afer he end of he monh, for insance, sandard deviaions are increased o 5.67%, 3.81% and 2.45%, for he curren mehod, he improved rend-based mehod and he regarima mehod, respecively. sales developmen 30% 25% 20% 15% 10% 5% 0% -5% -10% -15% jan-1998 jan-1999 jan-2000 jan-2001 jan-2002 jan-2003 final 21 days Figure 6. Sales developmen for he Duch manufacuring indusry as a whole, according o he final daa and according o he daa available a 21 days afer he end of each monh. Impuaions were done using he regarima mehod. Besides sandard deviaions of he errors (cf. figure 5) also average errors and he range of errors (minimum and maximum values) are imporan in making he decision regarding he rade-off beween imeliness and accuracy. To illusrae his, figure 6 shows he sales developmen in he manufacuring indusry as a whole (MI1), according o he final daa and according o he daa available a 21 days afer he end of each monh, where impuaions were done using he regarima mehod. Minimum, maximum, average and sandard deviaion of he errors were -4.69% (December 1999), 3.59% (January 1998), -0.08% and 1.57%, respecively. The average error is no significanly differen from zero (assuming normally and independenly disribued errors). In he figure, i can be seen ha he larges error (-4.69%) occurred for a monh were he sales developmen jumped from 14% o 25% (December 1999). A 21 days afer he end of he monh his jump was 19

20 underesimaed, bu he direcion was correc. I may be noed ha he response rae regarding urnover a ha ime was only 31%. The direcion of he monh-o-monh change was in mos cases correc a 21 days afer he end of he monh. The only excepions occur, expecedly, in siuaions where he monh-o-monh changes in sales developmen are small (July 2000 and February 2002). I should be noed ha for inerpreing he change from monh -1 o monh, he value for monh -1 a 21 days afer he end of monh, is no indicaed in he figure. This value is in mos cases, however, close o he final value. In he curren survey process auomaically impued daa are manually correced occasionally. When, a a laer momen in ime, he rue values are received he impued values are overwrien. No hisory of he original values is kep. This may cause differences beween he resuls of he simulaions and he originally disseminaed daa. The las few years, daa were disseminaed a 37±3 days afer he end of he monh by means of press releases. In order o verify he simulaion resuls, he daa in he press releases were compared wih he final daa, and he range, average and sandard deviaion of he errors were deermined. These accuracy measures were compared wih simulaion resuls of he curren mehod, 37 days afer he end of he monh. Differences were generally small and especially he sandard deviaions were very similar. Summarizing, i can be said ha he simulaion resuls for he curren mehod are in agreemen wih he press releases. Regarding he impuaion of missing values from enerprises in group III (daa available for only par of he monh) we impued daa for he missing par of he monh using average urnover per working day during he par of he monh he enerprise did repor on. Anoher way o deal wih hese enerprises could be o ignore he daa received and o rea he enerprises he same as hose of group II. Furhermore, i would be possible o impue group III using he average urnover per working day (as is done now) and hen o add he enerprises of group III o group I. Boh of hese opions were considered bu hey did no improve he resuls. The improved rend-based mehod was implemened using =12. This gave much beer resuls han =1. For insance, 21 days afer he end of he monh he sandard deviaion of he errors increases from 1.70% o 2.38% (AI30) when using =1 insead of =12. This is caused mainly by differen seasonal paerns for groups I and II presen in he ime series, which is no aken ino accoun when using =1. Noe ha he mehod currenly used a Saisics Neherlands uses =1. Furhermore, he bes aggregaion level for he impuaion of missing daa was found o be AI30 for he rend-based mehod. These 30 aggregaes are composed of 6 SIC divisions and 5 size classes. The mehod currenly used a Saisics Neherlands uses 124 aggregaes, based on 3- and 4-digi SIC divisions. Especially for early daa disseminaion i is beer o use larger aggregaes han he curren mehod does. Time series were consruced by summing for each monh he urnover values of he enerprises belonging o a cerain group (I or II) a monh. Since for monhs earlier han monh generally no all of hese enerprises exised ye, he sums for hose monhs were done over less enerprises. For example, of he enerprises in he 20

21 inersecion in June 2000, abou 30% did no ye exis in January A way o deal wih his problem could be by making use of ime series of average enerprise urnover for each monh. Using oal urnover, however, gave beer resuls han using averages. This is caused by he fac ha when enerprises firs come ino he populaion hey generally do no immediaely have he high level of urnover hey have for monhs laer in he ime series. Using averages would in ha case resul in more abrup changes in he ime series. Many differen seings were ried in he X-12-ARIMA specificaion file (Appendix C). Generally he defaul seings (by he U.S. Census Bureau) gave he bes resuls, bu here are a few excepions o his. Firs of all, allowing for an inercep in he regression, improved he accuracy of he mehod and reduced in paricular he number of level shif (LS) ouliers. In conjuncion wih his, i can be assumed ha he ime series of residuals wih respec o he regression line are more or less saionary, so ha no differencing is needed (d=d=0). The auoregressive and moving average parameers p, q, P and Q were allowed o have all possible combinaions of 0 and 1. Sporadically, none of he models was found o be accepable and ARIMA modelling of he regression residuals was lef ou. Allowing values of 2 for p, q, P and Q did no improve resuls. Manual adjusmen of he specificaion file for each specific ime series may furher improve he accuracy of he mehod, alhough in general he specificaion file presened in Appendix C proved o be quie saisfacory. When he urnover of group II was below a cerain percenage of he oal urnover, and he ime series were more erraic, only regression and oulier deecion were done and ARIMA modelling was lef ou. This percenage was se o 15% bu is no very criical. However, seing he percenage o 0% (always ARIMA modelling) or 100% (never ARIMA modelling) gave worse resuls. In mos simulaions a percenage of 15% mean ha only for he las wo or hree monhs he impuaions were (re)done including ARIMA modelling. During he ime his sudy was done Saisics Neherlands changed he curren survey process described in secion The new survey process is par of he sysem for he so-called Implemenaion of he Economic Transformaion Process (IMPECT). For pracical reasons no simulaions could be done ye wih his sysem. The main difference, regarding he impuaion of missing values, beween he IMPECT sysem and he sysem described in secion is ha when impuaion aggregaes are oo small for accurae esimaes, hey are auomaically enlarged. However, he main conclusions from his sudy, regarding rend-based mehods, sill hold. In case of early disseminaion i is beer o ake larger aggregaes (AI30) from he sar, and i is imporan o use =12 insead of =1. In he IMPECT sysem =1 is sill used, and resuls are expeced o be similar o he mehod described in secion

22 7. Conclusions A new mehod, based on ime series analysis, was developed o improve he impuaion of missing daa in he monhly assessmen of sales developmen in he Duch manufacuring indusry. Besides already available daa for he monh in quesion, ime series of hisorical enerprise daa were used. The imeliness of his saisics can be improved from 37±3 days currenly o abou 27 days wih he newly developed mehod, a he same level of accuracy. Furher improving imeliness resuls in reduced accuracy. The final decision abou he momen in ime o disseminae daa has o be made in consulaion wih he mos imporan users of he saisics. To aid making his decision, he relaionship beween imeliness and accuracy was invesigaed (cf. figure 5). Furhermore, i has o be considered in case of early disseminaion, wheher or no o include daa on any subdivisions of he manufacuring indusry. In addiion, i was shown ha he curren rend-based mehod can also be improved considerably by simply using larger aggregaes for he impuaion of missing daa, and deermining rends wih respec o he same monh he year before insead of wih respec o he previous monh. This improved version of he curren mehod performs, for disseminaion laer han abou 27 days afer he end of he monh, similar o he mehod based on ime series analysis. Implemenaion of he ime series mehod, however, will require more subsanial modificaions of he survey process. Therefore, if i will be decided o disseminae daa laer han abou 27 days afer he end of he monh, for pracical reasons i is probably bes o use he improved version of he curren mehod. However, for earlier disseminaion he mehod based on ime series analysis is preferred. Especially for disseminaion earlier han abou 20 days afer he end of he monh, he loss in accuracy is much less pronounced for he mehod based on ime series analysis, in comparison wih he rend-based mehods (cf. figure 5). The ime series mehod developed in his sudy focussed on he monhly saisics of sales developmen in he Duch manufacuring indusry. This saisics is based on an inegral collecion of daa from enerprises wih more han 20 employees. Applying he same mehod o oher saisics ha are based on inegral daa collecion seems sraighforward, alhough differences in populaion characerisics (e.g. heerogeneiy) may have o be aken ino consideraion. The mehod will need o be modified for sample-based saisics for which no ime series are available for all individual enerprises. 22

23 8. References Bell, W.R. (1998). An overview of regarima modelling. Research repor. Saisical Research Division. U.S. Census Bureau. Deville, J.C. e al (2002). Final repor of he exper group on sampling for imely European indicaors. Eurosa. Uni A-4. Findley, D.F. e al (1998). New capabiliies and mehods of he X-12-ARIMA seasonal adjusmen program. Journal of Business and Economic Saisics. 16, Revilla, P. (2002). An E&I mehod based on ime series modelling designed o improve imeliness. Conference of European saisicians. UNECE Work session on saisical daa ediing (27-29 May 2002, Helsinki, Finland). U.S. Census Bureau (2002). X-12-ARIMA Reference Manual version hp:// 23

24 Appendix A. Overview of SIC divisions and size classes Below an overview is given of he SIC divisions wihin he manufacuring indusry, according o he Sandard Indusrial Classificaion of all Economic Aciviies SIC Descripion 15 Manufacure of food producs and beverages 16 Manufacure of obacco producs 17 Manufacure of exiles 18 Manufacure of wearing apparel; dressing and dyeing of fur 19 Tanning and dressing of leaher; manufacure of luggage, handbags, saddlery, harness and foowear 20 Manufacure of wood and of producs of wood and cork, excep furniure; manufacure of aricles of sraw and plaiing maerials 21 Manufacure of pulp, paper and paper producs 22 Publishing, prining and reproducion of recorded media 23 Manufacure of coke, refined peroleum producs and nuclear fuel 24 Manufacure of chemicals and chemical producs 25 Manufacure of rubber and plasic producs 26 Manufacure of oher non-meallic mineral producs 27 Manufacure of basic meals 28 Manufacure of fabricaed meal producs, excep machinery and equipmen 29 Manufacure of machinery and equipmen n.e.c. 30 Manufacure of office machinery and compuers 31 Manufacure of elecrical machinery and apparaus n.e.c. 32 Manufacure of radio, elevision and communicaion equipmen and apparaus 33 Manufacure of medical, precision and opical insrumens, waches and clocks 34 Manufacure of moor vehicles, railers and semi-railers 35 Manufacure of oher ranspor equipmen 36 Manufacure of furniure; manufacuring n.e.c. 37 Preparaion for recycling Furhermore, he enerprises are classified according o he following classes of size. Size class Number of employees or more 24

25 Appendix B. regarima modelling in X-12-ARIMA In his Appendix regarima modelling, as i is done in he X-12-ARIMA compuer program of he U.S. Census Bureau, is inroduced briefly. For more deails on regarima modelling see Bell (1998) or Findley e al (1998). The regarima mehod models discree ime series in he following way: r Y X += Z, (A1) i=1 i i where Y is he ime series in quesion, X i ime series of regression variables wih coefficiens i, and Z he ime series of residuals from he regression. The regression residuals in urn are assumed o follow an ARIMA model. ARIMA models, which are ofen used o describe economic ime series, consis of an AuoRegressive (AR), an Inegraed (I) and a Moving Average (MA) erm. The auoregressive erm relaes Z o previous values of iself, while he moving average erm smoohes buffeing effecs from unpredicable evens ( ). The inegraed erm is added in order o deal wih non-saionary ime series, i.e. ime series whose mean and/or auocorrelaions are no consan over ime. Each of he above-menioned erms has a seasonal as well as a non-seasonal par. Mahemaically, ARIMA models have he following form. Le B denoe he backshif operaor, BZ = Z 1. Then, s d s D s p ( B) P ( B )(1 B) (1 B ) Z = q ( B) Q ( B ), (A2) where s is he lengh of he seasonal period (12 in case of monhly series) and a sequence of independen variables wih mean 0 and consan variance. The erms p (non-seasonal AR), P (seasonal AR), q (non-seasonal MA) and Q (seasonal MA) have he form of polynomials wih degrees p, P, q and Q, respecively. For p example, if p 1, hen p ( B) = 1 1B L p B, and if P 1, hen s s sp P ( B ) = 1 1B L P B. Furhermore, d and D denoe he number of imes differencing (he I in ARIMA refers o inegraed, he inverse of differencing) is done, non-seasonally and seasonally respecively. A paricular ARIMA model is generally referred o as a model wih order (p,d,q)(p,d,q). Ouliers in he ime series are described using he regression variables. Three differen kinds of ouliers are deal wih in X-12-ARIMA: addiive ouliers (AO), level shifs (LS) and emporary changes (TC). Addiive ouliers have an aypical value a one period, level shifs have an effec ha lass for he res of he ime series and emporary changes have an effec ha lass a few periods. X-12-ARIMA esimaes regarima model parameers for a given se of regression variables and an ARIMA model wih a given order (p,d,q)(p,d,q) by Maximum Likelihood Esimaion. The program chooses from a se of user-defined ARIMA models wih differen orders (p,d,q)(p,d,q) which one fis he regression residuals bes. Ouliers can be idenified auomaically by ieraively adding and removing respecive regression variables. 25

26 Appendix C. X-12-ARIMA specificaion file In his sudy he X-12-ARIMA program, release version (U.S. Census Bureau, 2002), was used wih he following specificaion file. Only hose seings are shown ha are differen from defaul. SERIES{ File = "X:\imeseries2.da" Forma = "DaeValue" } OUTLIER{ } REGRESSION{ User = (slope inercep) File = "X:\imeseries1.da" Forma = "DaeValue" } AUTOMDL{ File = "X:\X12A_16.mdl" Mehod = bes Idenify = all } FORECAST{ Maxlead = 1 } The file imeseries2.da conains he ime series of group II and he file imeseries1.da conains he ime series of group I as well as an exra ime series wih a consan value of 1 for each monh. Furhermore, he file X12A_16.mdl consiss of he following models (p,d,q)(p,d,q). (p,d,q)(p,d,q) (0,0,0)(0,0,0) (0,0,0)(0,0,1) (0,0,0)(1,0,0) (0,0,0)(1,0,1) (0,0,1)(0,0,0) (0,0,1)(0,0,1) (0,0,1)(1,0,0) (0,0,1)(1,0,1) (1,0,0)(0,0,0) (1,0,0)(0,0,1) (1,0,0)(1,0,0) (1,0,0)(1,0,1) (1,0,1)(0,0,0) (1,0,1)(0,0,1) (1,0,1)(1,0,0) (1,0,1)(1,0,1) 26