Labour Force Survey s Electronic Collection

Transcription

1 Labour Force Survey s Electronic Collection Collection strategy, test results and transition Anne-Marie Côté and Guy Laflamme, Statistics Canada September 4 th, 2014

2 Outline A. Canadian LFS particularities B. Electronic Questionnaire (EQ) collection strategy C. The pilot test: 1. Objectives 2. Design 3. Collection results 4. Data quality 5. Feedback D. Transition and things to come 2 Statistics Canada Statistique Canada

3 A. Canadian LFS particularities Mandatory Monthly cycle Short collection period (7-10 days) Data released 3 weeks after reference week Sampling unit is the address and all people living in the household are surveyed Once a household is sampled, it has to complete LFS for 6 months (rotate 1/6 of dwellings each month) 3 Statistics Canada Statistique Canada

4 A. Canadian LFS particularities Sample size: households Currently only CATI or CAPI 1st month interview is longer to establish respondents profile Average length of interview 1st month: 13 minutes Following months: 9 minutes Response rates: 90% 1 st month: CATI : 84%, CAPI: 91%, total: 89% Subsequent: 90% 4 Statistics Canada Statistique Canada

5 B. EQ collection strategy Design Use in-house tool to generate electronic questionnaires (EQGS) Same wording of questions as CATI/CAPI Mostly a paging design with basic edits Collection 1st month will still be CATI or CAPI At the end of each CATI/CAPI interview Verify eligibility If eligible, EQ is offered If respondent agrees, address is collected 5 Statistics Canada Statistique Canada

6 B. EQ response process (invitation ) 6 Statistics Canada Statistique Canada

7 B. EQ response process (login) 7 Statistics Canada Statistique Canada

8 B. EQ response process (questionnaire) 8 Statistics Canada Statistique Canada

9 B. EQ response process (questionnaire) 9 Statistics Canada Statistique Canada

10 B. EQ response process (questionnaire) 10 Statistics Canada Statistique Canada

11 B. EQ response process (questionnaire) 11 Statistics Canada Statistique Canada

12 C. Test objectives Primary: Confirm we can do EQ with no changes to release schedule (3 weeks after reference week) Impact on collection operations and systems Impact on processing system Impact on data quality Secondary: Take-up rate, attrition Respondent reaction Mode effect 12 Statistics Canada Statistique Canada

13 C. Test design Collection: March-June 2013 Parallel test (data not used in production) 6,400 households 3,200 births in March + 3,200 in April Sample not representative of entire country, mainly urban areas Guiding principal: simulates real LFS Mandatory During LFS week to avoid/control for week effect Interviewer cannot identify test cases Selected from the LFS frame Same clusters as production increase statistical power 13 Statistics Canada Statistique Canada

14 C. Test results: summary Systems can support EQ No measurable impact on data quality No impact on response rate able to convert EQ non-response to CATI response 20% (+) of respondents used EQ reminders are effective No statistical evidence of mode effect 14 Statistics Canada Statistique Canada

15 C. Eligibility, acceptance, and completion Month Sample size Resp. Eligible (eligible/resp.) Accepted Offer (among eligible) EQ Resp. (among accept.) EQ resp. (among all resp.) March 1/2 2,803 73% 58% n/a n/a April full 5,651 72% 60% 44% 9% May full 5,729 73% 60% 50% 21% June full n/a n/a n/a 59%** n/a 15 Statistics Canada Statistique Canada

16 C. Accepting and responding Accepters Higher education, larger households, younger respondents, employed, geography Online respondents Higher education, larger households, employed, geography Most EQ respondents complete the questionnaire in one session 16 Statistics Canada Statistique Canada

17 17 Statistics Canada Statistique Canada

18 C. Retain early adopters and convert Retain early adopter 75-80% of respondents who accepted offer in previous month will accept it for current month 75% of EQ respondent for previous month will complete the questionnaire online in current month Convert When a non respondent becomes eligible, 57% will accept offer When not eligible becomes eligible, 48% will accept offer When refused offer, only 14% will accept it later 18 Statistics Canada Statistique Canada

19 C. Data quality Estimates (employment, unemployment, distribution by industry and occupation) Edit and imputations Changes in labour force status (gross flows) Question distributions Final variable distributions Respondents behavior Do EQ respondents cheat? Changes in household size Short path More please specify : wording problem? More mark all that apply by design Not more rounding 19 Statistics Canada Statistique Canada

20 C. Mode effect What is it? Impact of mode on answer Variable specific Focussed on those who accepted the offer Compared those who responded by EQ to CATI 3 perspectives: Compared estimates Propensity score adjustment to address selection bias Regression analysis No measurable mode effect 20 Statistics Canada Statistique Canada

21 C. Qualitative feedback From technical support, respondent comments, and interviewer debriefings The comments were related to: Offer/invitation wording (too long, online??) EQ and confidentiality Login process & passwords Short collection period Survey-specific details (misspelled names, question concept ) Interviewer training 21 Statistics Canada Statistique Canada

22 D. Transition to production No embedded experiments Transition Starts in November 2014 For new households (births), verify eligibility, offer EQ not offered to new households in December Ensure that there are no problems One rotation per month from January 2015 on Protocol 4 EQ days: Sunday 2:00 Wednesday 24:00 reminders: Monday 14:00 + Wednesday 9:00 22 Statistics Canada Statistique Canada

23 Future Improve/simplify access (offer, invitation , access key, password ) Interviewer training Monitor transition and on-going Other models for mode effect? 23 Statistics Canada Statistique Canada

24 For more information, Pour plus d information, please contact: veuillez contacter : Guy Laflamme and Anne-Marie Côté [email protected] [email protected] 24 Statistics Canada Statistique Canada