Evaluating the Use of an Audience Response Technology System to Collect Research Data among African American Elders

Transcription

1 Ageing Int (2009) 34:60 66 DOI /s Evaluating the Use of an Audience Response Technology System to Collect Research Data among African American Elders Barbara D. Powe & Rachel C. Faulkenberry & Lokie Harmond & Dexter L. Cooper Published online: 31 July 2009 # Springer Science + Business Media, LLC 2009 Abstract Older African Americans are at greater risk of dying from cancer compared to other groups. More research is needed to understand and eliminate these disparities. However, survey data is usually collected using an individual pen / paper approach and surveys are often read to participants individually. This study compared the amount of missing data, time needed to complete the survey, and experiences of using an Audience Response Technology System (n=50) and those completing surveys using a pen / paper method (n=30) among an African American population. Findings suggest that it is possible to collect data from larger numbers of people in a shorter amount of time using the ARTS, to eliminate data entry errors by direct transfer of electronic data to the data base, and to engage older African Americans to use computer technology. A major problem identified was the increased amount of missing data among participants using the ARTS compared to the traditional pen / paper group. Strategies such as smaller group sizes and more sample questions may decrease the amount of missing data. More research is needed to support the benefits of this innovative use of this technology to collect survey data among older African Americans. Keywords African Americans. Technology. Cancer. Aging B. D. Powe (*) Underserved Populations Research, Behavioral Research Center, American Cancer Society, 250 Williams Street, Atlanta, GA 30303, USA Barbara.Powe@cancer.org R. C. Faulkenberry : L. Harmond : D. L. Cooper Behavioral Research Center, American Cancer Society, 250 Williams Street, Atlanta, GA 30303, USA

2 Ageing Int (2009) 34: Introduction Older African Americans are at increased risk for developing and dying from cancer compared to other racial / ethnic groups and more research with this population is needed to understand and eliminate these disparities (American Cancer Society 2009). Most research data from this group is collected using traditional pen / paper surveys through face-to-face methods as opposed to mailed surveys which increases rates of study participation. Although successful in collecting data, the process of individually reading surveys is often labor intensive and time consuming (Yancey et al. 2006). Audience response technology shows promise to help streamline survey data collection but no studies have tested the use of this system with an older African American population. This pilot study aims to compare an Audience Response Technology System (ARTS) to a pen / paper comparison group (PPCG) of African Americans aged 50 and older. The following questions were posed: a) Is there a difference in the amount of missing survey data between the ARTS group and the PPCG based on age (50 70 and 71+)? b) What is the average time per survey question when using the ARTS? c) How do participants rate their experiences using the ARTS? The Audience Response Technology System (ARTS) The ARTS consists of software, a wireless base unit that connects to a computer, and individual wireless remote keypads with numbered buttons (1 9). Questions are loaded into the software and projected on a screen. Participants select their answer by pressing the corresponding button on their remote and receive confirmation by a green light that their answer was captured by the base computer. The data is then exported to statistical software. The ARTS also has the option of assigning a unique personal identification number (PIN) so that longitudinal data can be collected. Methods Sample and Setting ARTS data was collected from participants aged 50 years and older who attended senior citizen centers in a southern coastal region of the United States. The PPCG is a randomly selected, age-matched sample of African Americans selected from the Principal Investigators (PI) database of data obtained from African Americans recruited from federally qualified health centers who completed surveys using the pen / paper method (Powe et al. 2006). Institutional Review Board approval was obtained. Overview of Surveys Data were collected using a CRC knowledge survey, a fatalism inventory, and a demographic data questionnaire. The CRC knowledge survey assessed knowledge of

3 62 Ageing Int (2009) 34:60 66 signs, symptoms, risk factors, and CRC screening and could be answered true (yes) or false (no) ; The Fatalism Inventory assessed attitudes and beliefs about cancer and could be answered yes or no. There were 5 demographic questions (income, marital status, education, gender, possession of a home computer) that were assessed for missing data. Procedures Data were collected between 9:30 am and 1:30 pm at the senior centers. Participants were recruited on site and surveyed in groups of The researchers obtained informed consent and provided a tutorial for the ARTS including an overview of the remote and sample questions describing how to push the buttons to provide answers. The participants were given the remote, a large-sized picture of the remote to help negate poor vision, if present, and a diagram that showed which keys corresponded to each response set (i.e. 1 = true; 2 = false). A diagram listing options for multiple choice questions was also provided. The questions were displayed (one question per slide) on the screen while the PI read each question aloud. A research associate walked through the room to answer questions about how to use the ARTS remote and demonstrated how to press the buttons (older participants needed the most assistance). Participants responded to each question by pressing the appropriate button on the remote. Response levels were monitored by the PI via computer software and the PI prompted the group to select their answers when the computer showed a response rate of less than 100%. The original data for the PPCG was collected from patients at the FQHCs during their routine health care visit (Powe et al. 2006). Results The sample consisted of 80 African American participants. Participants were divided by age groups (50 70 years and older than 71 years). The ARTS group (n=50) included 17 participants in the year age group and 33 in the older than 71 years (Table 1). The PPCG (n=30) include 23 participants in the year group and 7 participants were older than age 71. The majority of the sample (N=80) was female (73%), widowed (40%), with an income range of 0 $20,000 (55%), and an educational range of 9th 12th grade (57%). Forty-six percent of participants reported having a computer in their home. There were no differences across the age groups with regard to whether they had a home computer ð#2 ¼ 1:62; df ¼ 3; p ¼ 0:656Þ. Is there a Difference in the Amount of Missing Data between the ARTS Group and the PPCG? T-tests were used to evaluate the mean level of missing data among the participants based on their age groups (50 70 years and older than 71 years) among the 3 surveys (Fig. 1).

4 Ageing Int (2009) 34: Table 1 Demographics characteristics Age Groups ARTS (%) PPCG (%) Gender Female Marital Status Single 3 Married Divorced/Separated Widowed Income a $0 20, $20,001 30, >$30, Education (years) Some college Graduate/Advanced degree a Percentage does not equal 100 due to missing data Fig. 1 Missing Survey Data. Key: CRC colorectal cancer knowledge survey, CF fatalism survey, Demo demographic questions ARTS PPCG Mean Missing Data CRC CRC 71+ CF CF 71+ DEMO DEMO 71+ Survey by Age Group Key : CRC colorectal cancer knowledge survey CF fatalism survey Demo demographic questions

5 64 Ageing Int (2009) 34:60 66 Comparison of Missing Data for Colorectal Cancer Knowledge Survey Participants in the year age group who used the ARTS had significantly more missing data on the CRC knowledge survey compared to those from the PPCG ðt ¼ 5:71; df ¼ 20; p < 0:0001Þ. Participants in the >70 year age group who used the ARTS also had significantly more missing data compared to the PPCG ðt ¼ 12:9; df ¼ 15; p < 0:0001Þ. Comparison of Missing Data for Fatalism Inventory There was no difference in the amount of missing data from the fatalism inventory between participants in the year age group who used the ARTS compared to those from the PPCG ðt ¼ 0:056; df ¼ 29; p ¼ 0:956Þ. Participants in the >70 year age group who used the ARTS had significantly more missing data on the fatalism inventory compared to the PPCG ðt ¼ 10:6; df ¼ 17:3; p < 0:0001Þ. Comparison of Missing Data for Demographic Data There was no difference in the amount of missing data from the demographic questions between the ARTS group and the PPCG for any age group fðt ¼ 0:938; df ¼ 6; p ¼ 0:384Þ; ðt ¼ 1:72; df ¼ 8; p ¼ 0:124Þg. What was the Time per Question for the ARTS Group? The average time that was spent on each question type was also calculated. The CRC knowledge survey averaged 43 s per question while the fatalism survey and the demographic survey averaged 34 s each. This translates to approximately 20 min for 30 people per ARTS group to complete the surveys compared to min for one person to complete the pen / paper survey. How do Participants Rate their Experiences using the ARTS? The majority (96%) of participants (regardless of age group) reported that the ARTS was very easy or somewhat easy to use. Similarly, the majority (98%) of both age groups were very comfortable or somewhat comfortable using the ARTS. Lastly, all (100%) of both age groups reported the ARTS was very easy to see. Discussion Though literature supports the ARTS as a potentially reliable method of data collection (McCarter and Caza 2009), most studies involving the system have focused on a younger, student population in a predominately academic setting (Cain and Robinson 2008). No research has used the ARTS with an older sample, and reliability issues within this group remain to be explored. While there are limitations

6 Ageing Int (2009) 34: to this study (small sample, retrospective comparison group), there were clearly lessons learned and a need for more research identified. The study showed that it is possible to collect data from larger numbers of people in a shorter amount of time (i.e. 20 min for a group of people compared to 20 min for one person). The study also showed that the ARTS saves time on data entry as the responses can be downloaded directly into the statistical data base which also removes the errors associated with hand-keying the data. Participants also viewed the ARTS as easy to see and use. Given the ability to reduce data collection time, eliminate error due to data entry, and to be viewed as a positive experience by participants, the ARTS shows promise as an innovative way to collect data from this population. An issue, however, for this pilot was the amount of missing data. Factors believed to contribute to the increased amount of missing data for the ARTS groups in the current study include group size, seating position, and age. The size of the largest group was 30 people, which resulted in discussions within the group that had to be redirected by the researcher. It is recommended that the group size when using the ARTS with older adults is limited to 15 people, with chairs in the room arranged so that all participants have full view of the screen. The significant differences in the amount of missing data among older participants suggest that this group may have had a more difficult time using and understanding the remote. It is recommended that the remote buttons are color-coded to correspond with the response set so that those who may have difficulty recognizing numbers may more easily recognize the colors on the buttons. Further, it is believed that the majority of missing data occurred during the beginning of the survey. It is recommended that additional sample questions are added at the beginning which may decrease the amount of missing data. Lastly, the feature of the system to assign a PIN to each remote / participant should be used (not used in the pilot). Then the researcher will know who has not answered a question and can prompt them individually which is similar to checking the pen / paper survey for completeness. Despite these recommendations, the exact reason for the greater amount of missing data for the CRC knowledge survey and fatalism survey remain unknown. This survey was administered first so participants may have needed a longer period to adapt to the new technology. However, it is equally possible that participants had less knowledge of CRC (which is supported by research) and chose not to answer the question or felt strongly about a fatalism question and were more inclined to answer. In future studies using the ARTS, a prospective comparison group will be used and content analysis of each question will be performed to assess whether missing data is associated with the content area of each question. Summary The ARTS is typically used in educational settings but holds promise for data collection among African Americans within community settings. This pilot study identifies key issues that must be addressed in refining the use of this exciting and beneficial technology.

7 66 Ageing Int (2009) 34:60 66 References American Cancer Society. (2009). Cancer facts & figures for African Americans Atlanta. Georgia: American Cancer Society. Cain, J., & Robinson, E. (2008). A primer on audience response systems: current applications and future considerations. American Journal of Pharmaceutical Education, 72, 1 6. McCarter, M. W., & Caza, A. (2009). Audience response systems as a data collection method in organizational research. Journal of Management and Organization, 15, Powe, B. D., Finnie, R., & Ko, J. (2006). Enhancing knowledge of colorectal cancer among African Americans: why are we waiting until age 50? Gastroenterology Nursing, 29, Yancey, A. K., Ortega, A. N., & Kumanyika, S. K. (2006). Effective recruitment and retention of minority research participants. Annual Review of Public Health, 27, Dr. Powe is the Director of Underserved Populations Research. Ms. Faulkenberry is an Intern with Underserved Populations Research. Mr. Cooper and Ms. Harmond are Research Analysts with Underserved Populations Research.