Designing and developing a questionnaire for children in the Growing up in Scotland study Judith Mabelis, Senior Researcher, ScotCen Social Research
Overview Growing up in Scotland (GUS) Children as respondents in GUS Designing a survey and questions for children Audio-CASI Piloting and testing Informed consent Reflections Next steps on GUS
Growing up in Scotland Large scale longitudinal study following the lives of children from birth into early childhood and beyond, funded by Scottish Government. Explores how experience in the early years impact on later outcomes across range of domains Involves around 10,000 children and families in Scotland Birth cohort 1 (started 2005, age 10 months)- approx 3,600 now aged 9 Birth cohort 2 (started 2011, age 10 months)- approx 6,100 now aged 3 Main source of data: face to face interview with child s main carer Heights and weights, cognitive assessments, linked data www.growingupinscotland.org.uk
Birth Cohort 1 : child questionnaire Sweep 7: 1 st questionnaire launched May 2012 with children aged 8 years Alongside interview with child s carer 10 minute interview in Audio-CASI (Computer Assisted Selfinterviewing) Friends Well-being Parenting School Materialism Funded by MRC-SPHSU and Education Scotland
Children as respondents in GUS Give children a voice Focus of GUS Best placed to answer questions about certain topics Policy interest Raise awareness of GUS amongst children Win trust and goodwill participation in future years-therefore wanted this first experience to be as positive as possible Children as reliable respondents?
Children as survey respondents Children can and do provide reliable responses if questioned in a manner that they can understand and about events that are meaningful to them Kellet and Ding Middle childhood in Fraser et al (eds) (2004) Doing research with children and young people, London, Sage.
GUS: 8 years old Questions Relevant and appropriate Avoid factual information-focus on feelings Easy to understand-take care with vocabulary Short and simple sentences Avoid negative sentences Memory and cognitive processing still developing Care with complexity of questions and number of response categories Retrospective questions might not work
GUS: 8yr olds Design and layout Simple, easy to understand, clear instructions, appealing, font, layout, use of pictures Process Fun, not stressful, meaningful, comfortable setting Social desirability Implications of administering in the home? Thinking about mode Interviewer led interview (CAPI) Self-complete paper questionnaire Audio-CASI
Audio-CASI On a laptop, using headphones, the child listens to prerecorded questions and answers by typing in the number of the response they want to give Interviewer shows child how to use laptop Sound volume checks, practice questions Child to complete on own Questions and answers played in a loop and also shown on screen Possible to skip audio recording if not needed
Why we chose Audio CASI Offers child privacy (completing questionnaire in home) Reinforces feelings of confidentiality-very popular! Overcomes literacy problems (audio version offered as standard) Less intense for child but assistance on hand through audio support and interviewer if necessary Child able to complete on own Fun to use laptop-more engaging than paper self-complete? Constraints: No open questions, no multiple response options, short and simple questions.
Testing on GUS 2 rounds of cognitive testing (one-to-one interviews with 8 yr olds) Preference for questionnaire mode Ease of use of laptop and developing instructions Advice on format, layout on screen, speed of audio recording Exploring understanding and interpretation of questions Understanding of child leaflet 2 full field pilots (approx 120 families) Greater rolling out of process alongside other survey elements Role of interviewer and interviewer administration Interaction between interviewer, parent and child Greater testing of the instrument with more children Timings Ability to complete autonomously? Data checks
Cognitive testing question Do other children in your class make it hard for you to do your work? Never 2) Sometimes 3) Always How many children are in your class? What kind of work do you do in class? What made you choose the box you did? If answers 2 or 3- what do the children do that makes it hard to work? Do you prefer to work in class or at home? Why is that?
Examples of what we learnt from testing/piloting More interest in using computer than paper Difficulties with certain questions Even though had been used on other surveys with children of this age No wide understanding of certain words e.g. praise Fine-tuning instructions to complete questionnaire and practice questions Greater appreciation of impact on respondents Language on survey leaflet and phrases Who are you? child took this to apply to herself (actually referred to Researchers)
Informed consent Participants understand what the research involves and are able to consent (or refuse to consent) to participate Children require careful considerations-there are potential difficulties but vital to seek their fully informed consent. We want to keep them on board for future GUS interviews
Approach to consent Carefully consider approach to parent and child Parents used as gate-keepers Sent letter and leaflets in advance Child leaflet: Explain GUS simply How interview will work Interview topics How to answer questions Not a test Confidentiality Don t have to take part
Gaining consent Interviewers follow protocol to gain consent from both parties Consent from parent Check have read survey leaflet Show blank copy of questions before interview Explain process: child needs to complete on own, answers are confidential Consent from child Talk through leaflet Opportunity to ask questions Explanation and consent question built in at the beginning of the CASI prog as well as right to stop at any point (verbal consent) Aware and sensitive to child s feelings (i.e. even if not verbally expressed) Child has right to refuse No pressure to achieve full interview -fee paid regardless
Follow-up: Child receives headphones as gift Certificate of thanks Newsletter for children with results Developing kids pages on website Children send in drawings/stories
Feedback from the field Positively received by children, parents and interviewers No complaints! Heard from parents and interviewers that children have enjoyed taking part No problems with interviewer administration or any technical issues High response rate-98% amongst achieved cases (with parents) Few cases where child completed and parent didn t 85% of children completed on own without any assistance Took on average 11 minutes to complete-not sure about take up of A-CASI Still to look at data in depth
Reflections Balancing act Being realistic in our expectations -consider child at this age and clear aims Balance need for robust and valid data but also make it fun for child Ethical considerations and keeping child, parent and interviewer on board Importance of piloting and testing Topics/questions that are relevant, meaningful and easy to understand Appreciation of dynamics and sensitivities Even if used on other survey can t assume it will work Other ways to test-children advisory panels, peer researchers, focus groups Think about whole research process Pre- survey, questionnaire process itself (interviewer administration) and follow-up
Coming up on GUS Initial results from child questionnaire due to be published May 2014 How do we present data to children and families? Sweep 8 to launch Sept 2014 includes 20 minute interview with child May cover some sensitive topics Quite basic questionnaire-work needed to make it more appealing Consent Engaging with children as respondents Transition from parent being main respondent to child How to keep children engaged with study as they get older-teenage years and beyond?
Visit our website, sign up to our newsletter, read repots: www.growingupinscotland.org.uk Follow us on twitter: @growingupinscot email : judith.mabelis@scotcen.org.uk For further information