World Bank Development Research Survey Solutions COMPUTER-ASSISTED PERSONAL INTERVIEWING Michael Lokshin, DECRG
DQ = αdc βdm γsm DQ DC DM SM data quality data capturing data management survey management Survey Solutions : α β γ
Background 2011: Comprehensive Assessment of CAPI software products is commissioned to University of Maryland by the WB. no existing software provides exactly the right mix of features necessary for the sort of surveys conducted by World Bank and its clients. 02.2012: LSMS and Computational Tools teams of Research Department of the World Bank joined forces to develop the CAPI system for the complex household and agricultural surveys. 09.2013: First public version of Survey Solutions is released 11.2014: Survey Solutions 3.0 is released
Challenges: Focus on complex, mutli-topic, large scale surveys Data capturing and Survey Management Functionality Simple to master, yet poweful to handle surveys of any complexity Sustainable solution for NSOs with minimal TA
Survey Solutions: technology Survey Solutions is a late entrant into the CAPI market. No legacy products. We build on the latest technology: NoSQL-database storage of unstructured data Event-Sourcing store not data, but events of capturing data. Ability to rewind each interview to any point. Unlimited level of detalization both in terms of timing of interview and the history of responses. Platform independent development*
Survey Solutions: hybrid approach Simple, flexible interface for questionnaire development and testing. Live presentation of questionnaire on a tablet. Tablet interface allows easy navigation through complex questionnaires - multiple questions per screen; grid roster representations, bread crumbs, navigational panel. Standardized survey management protocol based on the best practices of data collection. Intuitive, informative survey status reporting, survey maps. Yet, powerful language (C#) for data validation and control of questionnaire flow.
Survey Solutions: Unique features Out-of-the-box Survey Management System Survey audit: reports, maps, etc. Support of off-line and on-line survey supervision Various modes of HQ hosting (local server, local cloud, international cloud, etc.). No limitations on data security C# as a language for validation and skip conditions
Survey Solutions: data capturing Large questionnaires: several thousand questions All standard types of questions: text, numeric, date; multi-choice; dynamic lists; Linked questions: Whose cow is it? User can select from the list of household members. GPS location; Time; Barcode; Binary files (pictures) HQ/Supervisor-filled questions Rosters: can be generated from: Fixed lists; dynamic lists; numeric; multi-choice questions. Nested rosters with unlimited degree of nesting Interviewer Comments on a question and interview Question Instructions
Survey Solutions: system components Designer: online tool for questionnaire creation and validation @ solutions.worldbank.org Tester: Android app connected to Designer to test questionnaires in real time. @ Google play HQ: online tool for centralized survey management, validation, data aggregation and reporting. Supervisor: online/offline tool to manage process of data collection on a team supervisor level. Tablet CAPI: Android app for data capturing on a tablet.
Researchers design questionnaires using visual tools and upload them to the central server Questionnaires with no errors are uploaded to the HQ central distributes server the sample lists across teams of Internet enumerators Internet Supervisors monitor the submissions Interviewers Supervisors assign synchronize households to their devices individual and interviewers upload completed questionnaires Enumerators WiFi/USB repeat interviews if errors are detected WiFi/USB Interviewers visit households and collect data
Real time status of interviews
Real time access to interviews
Review and validation of an interview
Map of the survey
Monitor the survey by checking the GPS location of where and when the interview took place.
Survey Solutions: Is it for you? Focus on large scale national surveys where data quality is critical. Strict protocols of document flow. Attention to responsibilities of survey team members: HQ Supervisors Teams of interviewers. Rather complex mechanisms of connecting multiple surveys: Listing => Sample Frame => Survey