Turning Knowledge into Practice Research Triangle Park, North Carolina
Panel Attrition and its Effects on Results from a Longitudinal Study: An Examination of Changes in Participants and Attitudes in a Web-based Panel Survey of HIV and AIDS Stigma Attitudes Rodney Baxter and Elizabeth Dean Paper presented at the American Association for Public Opinion Research, Phoenix, Arizona, May, 2004
Presentation Overview Introduction Web Panel Background Methodology Results Attitudes/Beliefs Socio-Demographics Respondents Conclusions/Future Directions
Introduction Longitudinal versus cross-sectional Change over time in participation and responses 2003 CDC HIV And AIDS Stigma Study Stigma attitudes Knowledge of AIDS transmission Adding to data obtained by Herek et al in 1991, 1997, and 1999 Measures utilized created by Herek Herek et al findings: C Stigma: 1991-1997, 1997-1999 C Inaccurate Knowledge: 1991-1997, 1997-1999
Importance of Study Topic Importance of study topic Negative attitudes towards people with AIDS, and Inaccurate knowledge of disease transmission can: C Result in social isolation and discrimination C Affect treatment choices and sharing of HIV status Both can contribute to further spread of AIDS
Web Panel Background Knowledge Networks (KN) web-enabled panel Built from probability-based RDD sample all US phone numbers Panel constructed with: Complex stratification design that incorporates known probabilities of selection associated with: C Geographical location C Number of phone lines C Number of people in household C Whether phone number is listed
Web Panel Background (cont d) RDD respondents who agree to participate provided with: Free hardware Free Web access Free e-mail accounts for each resident in hh over age 13 Ongoing technical support Incentive program Hardware: Microsoft s WebTV
Web Panel Background (cont d) Advantages of a web-enabled panel: Rapid data collection Relatively inexpensive Allows access to a known population designed to match the demographic characteristics of the US population Disadvantages: Response rates No prompting for missing answers
Methodology 2000 Survey on Health and Aging Panel T1: 5868 members 2003 HIV Stigma Study Panel T2: 2461 members Notification Email via WebTV (Click button to start survey) Reminder Email sent 1 week later Participation: C 1,978 responded to email invitation (78% contact rate) C 1,553 completed interview (63% completion rate) Field period: May 14 to June 12, 2003
Attitudes and Beliefs
Stigma Attitudes People who got AIDS through sex or drug use have gotten what they deserve. Strongly Agree (scored as stigma) Agree (scored as stigma) Disagree Strongly Disagree
Stigma Attitudes Year Survey Percent (%) n p 1991 Herek 20.3 538 -- 1997 Herek 28.1 1309 0.0001 1999 Herek 24.8 669 ns 2000 CDC 18.3 5232 0.0001 2003 CDC 17.9 1526 ns
Stigma Attitude: Weighted vs Unweighted Year Survey Unweighted Percent (%) Weighted Percent (%) n 2000 CDC T1 all 18.7 18.3 5232 2003 CDC T2 all 20.8 17.9 1526 2000 CDC panel 22.3 17.2 1383 2003 CDC panel 20.8 18.8 1383
Inaccurate Beliefs about HIV Transmission How likely is it that a person could become infected with HIV (the virus that causes AIDS) by being coughed or sneezed upon? Very likely Somewhat Likely Somewhat Unlikely Very Unlikely (Counted as correct) Impossible (Counted as correct)
Inaccurate Knowledge Year Survey Percent (%) n p 1991 Herek 45.7 538 --- 1997 Herek 53.6 1309 0.0001 1999 Herek 50.4 669 ns 2000 CDC 41.4 5232 0.0001 2003 CDC 34.2 1526 0.001
Attrition/Participation Type T1 Panel - Includes: T1 Only - Left panel before T2 survey (Attrition) T2 Respondents T2 Non-Respondents - Answered invitation with decline to participate (Non-response) T2 Panel T2 Respondents T2 Non-Respondents Responses T1 Answers T2 Answers
Stigma: T1 Only Respondents vs T2 Respondents Unweighted T1 Only Panel T2 (ans to T1) Total % n % n % n Agree/Strongly Agree 16.0 564 23.1 395 18.3 959 Disagree/Strongly Disagree 84.0 2961 76.9 1312 81.7 4273 Total 100.0 3525 100.0 1707 100.0 5232 Weighted T1 Only Panel T2 (ans to T1) Total % n % n % n Agree/Strongly Agree 16.1 560 23.8 411 18.7 971 Disagree/Strongly Disagree 83.9 2911 76.2 1312 81.3 4224 Total 100.0 3471 100.0 1723 100.0 5194
Knowledge: T1 Only Respondents vs T2 Respondents Unweighted T1 Only Panel T2 (ans to T1) Total % n % n % n Very/Somehwat Likely 39.7 1554 47.6 911 42.3 2465 Very Unlikely/Impossible 60.3 2363 52.5 1005 57.7 3368 Total 100.0 3917 100.0 1916 100.0 5833 Weighted T1 Only Panel T2 (ans to T1) Total % n % n % n Very/Somehwat Likely 39.8 1543 47.7 924 42.4 2467 Very Unlikely/Impossible 60.2 2334 52.3 1013 57.6 3347 Total 100.0 3877 100.0 1937 100.0 5814
Stigma: Panel T2 respondents T1 and T2 Panel T2 (ans to T1) Panel T2 (ans to T2) % n % n Agree/Strongly Agree 23.8 411 20.8 316 Disagree/Strongly Disagree 76.2 1312 79.2 1206 Total 100.0 1723 100.0 1522
Knowledge: Panel T2 respondents T1 and T2 Panel T2 (ans to T1) Panel T2 (ans to T2) % n % n Very/Somehwat Likely 47.7 924 36.3 560 Very Unlikely/Impossible 52.3 1013 63.7 983 Total 100.0 1937 100.0 1543
Socio-Demographic Comparisons
Panel T1 vs Panel T2 Panel T1 Panel T2 Absolute Difference Gender Male 48.8 47.7 1.1 Female 52.8 52.4 0.4 Age 18-24 10.6 4.1 6.5 *** 25-34 19.4 9.3 10.1 *** 35-44 23.4 19.7 3.7 *** 45-54 20.4 23.4 3.0 ** 55-64 9.9 15.1 5.2 *** 65+ 16.2 28.4 12.2 *** Race White 81.4 86.6 5.2 *** Black 10.6 8.0 2.6 *** American Indian 2.0 1.7 0.3 Asian/Pacific Islander 2.5 2.4 0.1 *p<.05, **p<.01, ***p<.001
Panel T1 vs Panel T2 (cont d) Panel T1 Panel T2 Absolute Difference Education No HS diploma 9.5 9.2 0.3 HS diploma 31.6 36.9 5.3 *** Some college 33.6 30.4 3.2 ** BA+ 25.2 23.6 1.6 Income LT 10K 4.0 4.0 0.0 10K-24K 14.8 17.1 2.3 ** 25K-49K 37.0 39.5 2.5 * 50K-75K 27.4 26.2 1.2 75K+ 16.8 13.2 3.6 *** *p<.05, **p<.01, ***p<.001
Respondent Comparisons
T1 and T2 Panel Respondents Panel T1 Panel T2 R Absolute Difference Gender Male 48.8 47.2 1.6 Female 52.8 50.4 2.4 Age 18-24 10.6 3.7 6.9 *** 25-34 19.4 9.2 10.2 *** 35-44 23.4 20.0 3.4 ** 45-54 20.4 24.3 3.9 *** 55-64 9.9 15.0 5.1 *** 65+ 16.2 27.8 11.6 *** Race White 81.4 87.2 5.8 *** Black 10.6 7.6 3.0 *** American Indian 2.0 1.9 0.1 Asian/Pacific Islander 2.5 2.1 0.4 *p<.05, **p<.01, ***p<.001
T1 and T2 Panel Respondents (cont d) Panel T1 Panel T2 R Absolute Difference Education No HS diploma 9.5 8.4 1.1 HS diploma 31.6 34.3 2.7 * Some college 33.6 32.3 1.3 BA+ 25.2 25.1 0.1 Income LT 10K 4.0 4.0 0.0 10K-24K 14.8 17.4 2.6 * 25K-49K 37.0 39.8 2.8 * 50K-75K 27.4 25.3 2.1 75K+ 16.8 13.5 3.3 *** *p<.05, **p<.01, ***p<.001
Summary Decrease in stigma attitudes and inaccurate knowledge What were possible causes of these differences? Panel lost members with lower stigma attitudes and more accurate knowledge Non-respondents to T2 were more likely to have higher stigma attitudes and more inaccurate knowledge End result may be that the panel attrition from T1 and T2 and the non-response effects may cancel each other out Respondents at both T1 and T2 showed continued decrease in stigma attitudes and increase in accurate knowledge Younger, Black, those with some college, highest income more likely to drop out of panel
Conclusion/Future Directions Results suggest: Longitudinal panels can be used for web surveys With caveats - Panel should be replenished on a regular basis Rolling Panel Improve retention methods Improve invitation methods to get representative results
Contact Information Rodney Baxter rbaxter@rti.org Elizabeth Dean edean@rti.org Full paper available at http://www.rti.org/aapor