The use of incomplete admin data for monthly and quarterly estimates: findings after two year ESSnet AdminData. Pieter Vlag CBS

Transcription

1 The use of incomplete admin data for monthly and quarterly estimates: findings after two year ESSnet AdminData Pieter Vlag CBS

2 Outline Introduction ESSnet AdminData WP 4 (STS) Work and results 2010 General Observations about use of admin data for STS Results in detail 1. Imputation and weighting (based on admin data only) 2. Regression models (admin data as auxiliary variable) 3. Inter- and extrapolation (use admin data from previous periods or with different periodicity)

3 Scope WP4 : STS AIM WP 4 (SGA-2010) Describe best practices for estimation of STS-variables from adm. sources if due to the timeliness problem - the turnover variable is not yet (completely) available in the VAT registration. - the employment variables (wages, persons employed) are not yet (completely) available in the social security admin sources AIM WP 4 (SGA-2011/2012) - identify common issues (to be resolved), - provide recommendations (practices versus conditions/available data) 2

4 Work SGA st-meeting (May 2010): inventory of practices -> presentations of national practices + challenges 2 nd - meeting (September 2010): more detailed description of practices (DE, FI, LT, IT,NL, UK) -> presentations -> documents -> review (content) + structure 3 rd -meeting (November 2010): Start identification common issues. +bilateral contacts (NL/EE -> FI; FI->NL; NL -> LT/EE; NL-> DE) 3

5 APPROACH adm. Data - Example VAT conclusion ( ): VAT= employment concerning timeliness issues DE (q), IT(OROS-phase 2), NL (q), FI (q) VAT Almost complete coverage Inventory Analyses transf to STS periods Legislation, availability NSI, Stability time-series NL (m), LT (m), UK (m+q), IT (OROS-phase 1), FI(m t+30) VAT no complete coverage VAT only VAT < threshold representative or can be modelled not representative or cannot be modelled without cor. factor macrolevel with cor. factor macrolevel (GREG-type) Est. VAT (t,t-x) (GREG-type) Est. Survey (t,t-x) Historical data no VAT possible cor.factor cor factor for quality 1st res. 4 ESSnet Admin Data WP4: planning for and transf. organisation M -> Q

6 Results SGA-2009 / 2010 Description of practices in: a. Finland, Germany, the Netherlands VAT/turnover (good coverage) b. Italy soc.sec.data/employment (good coverage) + comparison with practices in SE + NL (employ). c. Finland, Lithuania, the Netherlands, United Kingdom VAT/turnover (no good cov.) NOTE: some contribution are more experimental d. Germany Analyses of revisions due to incompleteness 5

7 Steps after describing national practices Producing an overview document summarising the common issues in the national practices. SGA-2010: Working out the common issues to provide recommendations

8 The results. a. general observations b. current pratices b1. imputation (estimation) of missing values (with available VAT) b2. regression models (VAT as auxiliary variable) b3. inter- and extrapolation (VAT from previous periods or with different periodicity) b1. and b2. summary; b3. more detailly 7

9 General conclusions I All countries use a survey for the large(st) enterprises Assumption (implicit): no timeliness problem with survey enterprize size survey admin data But coverage of survey part differs per country DE: turnover > 15 mln or persons employed > 250 NL: persons employed > 100 (now), TOP-1900 (future) IT: persons employed > 500 FI: 2000 turnover; 2000 industrial production 8

10 General conclusions II Periodicity VAT - Year (very small enterprises) neglected - Quarter (small+ medium sixed enterprises) - Month (large enterprises or other) but thresholds differ per country Excecption: UK 3-month periods in stead of quarter Timeliness VAT - VAT-declaration between t+25 (LT) until t+40 (UK) after reporting period - availability VAT for the NSIs between t+ 27 (LT) and t+60 (UK) - transfer of VAT-data from tax office differs per country: (NL daily; DE, UK: once per month) Incompleteness of VAT for estimates differs per period (Q/M), release, country 9

11 General conclusions III: challenges are focussed on monthly estimates quarterly estimates - monthly and quarterly VAT-reporters; generally month and quarter have good coverage - in case of timeliness problems admin data large enterprises (survey) + first two months (admin data) - generally publication at t+45 (or more) monthly estimates - only monthly VAT-reporters (and more countries reduce obligation for monthly reporting) limited number + representativity problem - in case of timeliness problems admin data large enterprises (survey) + NO DATA FOR SMALL ENTERPRISES - early publication dates (t+30 retail trade + manufacturing) risk for no admin data due to timeliness 10

12 General conclusions IV: monthly turnover estimates - NSIs under pressure to work more efficiently, use more admin data and reduce own surveys, but. for monthly (turnover) statistics and therefore for the statistical system as a whole - these aims are under pressure - REASON: tendency in Europe to reduce monthly reporting to admin data holders (i.e. VAT). IN THIS CONTEXT ESSNET ADMIN DATA WP 4 HAS GENERAL INTEREST 11

13 RESULTS CURRENT PRACTICES adm. Data - Example VAT conclusion ( ): VAT= employment concerning timeliness issues DE (q), IT(OROS-phase 2), NL (q), FI (q) VAT Almost complete coverage Inventory Analyses transf to STS periods Legislation, availability NSI, Stability time-series NL (m), LT (m), UK (m+q), IT (OROS-phase 1), FI(m t+30) VAT no complete coverage VAT only VAT < threshold representative or can be modelled not representative or cannot be modelled without cor. factor macrolevel with cor. factor macrolevel (GREG-type) Est. VAT (t,t-x) (GREG-type) Est. Survey (t,t-x) Historical data no VAT possible cor.factor cor factor for quality 1st res. 12 ESSnet Admin Data WP4: planning for and transf. organisation M -> Q

14 Results: methodology MINIMISING SURVEY, Final situation STS Admindata coverage almost complete ESTIMATION ONLY BASED ON ADMIN DATA L.E Admindata L.E Admindata Missing QUALITY STS- ESTIMATES, often measured by: average bias average error Admindata coverage incomplete ADMIN DATA = AUXILIARY INFORMATION B E t t T T 1 T 1 T e e t t L.E Admindata ESTIMATI ON = IMPUTATION WITH AV. ADMIN DATA AS DONOR L.E Ad. survey SURVEY PROPORTION opt Admindata Survey or Predict L.E REGRESSION ESTIMATOR T-1 Ad min dat a T-1 L.E L.E. + model based

15 RESULTS: IMPUTATION / MACRO (I) Estimation missing data: a. O t /O t-1 of available VAT DE b. O t /O t-12 of available VAT NL c. trends of VAT in previous periods (per enterprise) FI Five different imputation rules are used for enterprises with missing data (t = month to be estimated): xˆ t xˆ t d. ( O t-1 final / O t-1 1st release ) * r t IT x x x t 1 t 13 t 1 ˆ 1, x t x t x x xt x 2 t 14 t 1 t 2 xt x x x 2 t 3 t 3 t 15 x (Previous turnover) t 12 (Mean annual change), (Geometric mean of monthly changes) xˆ t avg( xt 1, xt 2, xt 3), (Mean turnover) xˆ t x t 12, (Turnover of comparison month) The methods are tested with data concerning the five latest months. 14

16 RESULTS: IMPUTATION / MACRO (II) Stratification level for estimation: DE: NL: FI: NACE2-digit + 2 size classes NACE3/4-digit + 9 size classes (strata collapsed in case of few donor data ) micro German report The Dutch method has the big advantage that it uses more available auxiliary information which could improve the estimates. The major drawback is its complexity, which could lead to implementation problems. The method has not proven itself yet. At this point the question of the cost benefit ratio arises. As the VAT data is rather complete and the existing imputation method already delivers fairly good results, a further improved imputation method would have in most cases a hardly observable effect on the total turnover. This is even more true in the economic sectors, where a mixed model is used (that is no imputations are needed for the large surveyed enterprises) and only quarterly results are required. 15

17 RESULTS: IMPUTATION / MACRO (III) Use of Business Register DE: IT: VAT matched to Business Register once a year (small ent.) -> estimations for starting/stopping enterprises needed when estimating missing values similar to Germany, -> estimations for overcoverage register NL + UK: Business registers is population frame (also for month / quarter) -> time lag between VAT-register and Business Register 16

18 FUTURE WORK: IMPUTATION / MACRO (III) Provide recommendations under which conditions previous methods can be considered. To be tested: -> below which threshold (VAT-turnover coverage < x %) do these different methods provide different results. -> how to measure desired quality of STS-estimates (and underlying VAT part) Examples: see next slides 17

19 Measuring desired quality STS-estimates and underlying VAT -part Survey: large(r) enterprises S: Ma, q1 N.S.: Ma, q1 Population merges, birth, death, coverage data quality editing S: Ma, q1* N.S.: Ma, q1* S: Ma/Mi,q1* N.S.: Ma, q1* normal / abnormal periods B E t T T 1 T t 1 T e e t t entire population S: Ma, q1 N.S.: Ma, q1 Combining:large-small enterpr. S: Ma/Mi, q2 N.S.: Ma/Mi, q2 S: Ma, q2 N.S.: Ma, q2 Index calculations other Legenda S = systematic error N.S. = non-systematic error q1 = quantitative (error) q2 = qualitative (error description) Ma = macro (level analyses) Mi = micro (level analyses) * Mi analyses cannot be excluded Error 1 a. systematic, b. non-systematic S: Ma, q2 N.S.: Ma, q2 1 quantitive and qualitative. VAT-data: other enterprises Population merges, birth, death, coverage S: Ma, q1* N.S.: Ma, q1* S: Ma, q1 N.S.: Ma, q1 data quality editing S: Ma/Mi,q1* N.S.: Ma, q1*

20 RESULTS REGRESSION (I) VAT is not in time for first monthly turnover estimates (Retail trade) -> Hence, a small survey has to be used. Regression techniques can be used for 2 purposes: 1A.Using VAT of previous month as auxilliary information to decrease the measurement error (LT; turnover estimates) 1 1B.Using VAT of same period as auxilliary information to publish estimates at a more detailed level at a later stage (LT; social statistics) 1 1 Assumption: a good correlation between VATt-1, VAT and survey data at aggregated level, despite the differences in definition: VAT income = STS turnover + income from other not actual for STS survey economical activities ; 19

21 RESULTS REGRESSION (I) 20

22 RESULTS REGRESSION (IV) - using VAT-data t-1 Population size 9395, Sample size 1635 January 2010 February 2010 Turnover (activity 47) Sampling error (CV) Turnover (revised) Sampling error revised (CV) Difference (in %) , , , ,1-1 March , ,5 0,4 April , ,5 0,2 May , ,4 0,2 Monthly Domestic retail trade survey. Estimation results of Turnover in the economic activity G47 (Retail trade, except of motor vehicles and motorcycles). Turnover: VATt-1 as auxialliary variable Turnover (revised): VATt-1 as auxialliary variable 21

23 RESULTS REGRESSION (IV) - for publishing lower aggregates Domain sample size Number of domains HT Ratio GREG Total and > Average of the relative root mean square error (RRMSE) for the number of fulltime employees in domains of a different sample size (the coefficient of correlation equals 0.996) Method also applicable if correlation is (slightly lower): i.e. estimates for part-time employees 22

24 FUTURE WORK: REGRESSION TECHNIQUES Recommendations for other countries should include: Short description of regression models and few main estimators (GREG and Ratio estimators); A description of circumstances in which regression could be applied; Testing Selection of literature for further readings; Description of Software available. 23

25 INTER/EXTRAPOLATION Topic can be divided into two issues: a. Splitting quarterly data into monthly data b. How making estimates in case - VAT is NOT available yet when the estimates have to be made, but becomes available later t+30d: retail trade, t+40d ind. production; GDP-flash - VAT is not available for this period/periodicity 24

26 RESULTS: INTER/EXTRAPOLATION (UK) UNITED KINGDOM M = Q/3 Winters model: seasonal factors+linear trend Not-representative Stagger Oct Nov Dec Jan Feb Mar Apr May Jun 0 M10 M11 M12 M1 M2 M3 M4 M5 A 1 Q4 Q1 B Separate series 2 Q4 Q1 C 3 Q1 Q2 D allmost full data available partial data available Estimation Techniques like NL (DE) 25

27 (Inter and) extrapolation: in case VAT becomes available later <- Trend + auxiliary variable for separate estimate for small enterprises (Finland) using difference in growth rate large enterprises entire population (The Netherlands) -> ESSnet Admin Data WP4: progress 26

28 Inter/Extrapolation: future work Recommendations for other countries should include: - How to split quarterly VAT into monthly VAT - Extrapolation of VAT-data for current period in case of late availability + time-series model with trend+seasonal component + model with trend & auxiliary variable for current period t + model using growth rates of large enterprises and VAT of previous periods - (Estimations in case of no monthly VAT) 27

29 ESSnet AdminData WP 4 - vervolg COST- BENEFIT RATIO, Final situation STS L.E Admindata L.E Admindata Missing kwaliteitscriteria methodes (normal/abnormal circumstances): average bias average error B E t t T T 1 T 1 T e e t t Admindata coverage almost complete ESTIMATION ONLY BASED ON ADMIN DATA Admindata coverage incomplete ADMIN DATA = AUXILIARY INFORMATION L.E Admindata imputation missing data Recommendations best estimation practices starting point: method DE L.E Ad. survey opt Recommendations: Minimum coverage L.E. survey Admindata Survey or Predict L.E T-1 Ad min dat a T-1 recommedation: When appropiate to use regession estimators regressie-schatter L.E recommendation: When appropiate to inter- & extrapolate data

30 Conclusions / most important results a. Description of 8 national practices DE (2 subjects), FI (2 subjects), NL, IT, LT, UK b. Overview: the use of VAT for early turnover estimates - Imputation of missing value in case of good coverage - Regression analyses - Inter-/extrapolation of admin data c. Next period: recommendations practices, taking into account - effect of different admin data (=input) at dif. NSIs - quality output in case of normal & abnormal circumstances - cost/benefit ratio 29