SURVEY RETURN RATES AS A FUNCTION OF PRIORITY VERSUS FIRST-CLASS MAILING '

Transcription

1 Psychological Reports, 2006,99, O Psychological Reports 2006 SURVEY RETURN RATES AS A FUNCTION OF PRIORITY VERSUS FIRST-CLASS MAILING ' CHRISTIANE BREMS, MARK E. JOHNSON University of Alaska Anchorage TEDDY WARNER University of New Mexico School of Medicine LAURA WEISS ROBERTS Medical College of Wisconsin Summa y.-prior research indicates survey procedures that signal significance and individualized mailings have higher response rates. Thus, it was hypothesized that surveys delivered via Priority mail would result in higher return rates than surveys delivered via First-Class. 260 surveys were sent to individuals randomly selected from lists of licensed physical and behavioral healthcare providers in Alaska and New Mexico. Half of the selected individuals were assigned randomly to receive mailings using Priority mail, the other half received First-Class mailings. Return rate was 39% for First-Class and 35% for Priority. Z tests of proportion indicated no statistically significant differences between methods. Given increased costs with no resultant increase in response rate, sending surveys to potential participants via Priority mail does not appear warranted. With response rates to surveys having declined for at least two decades (Tourangeau, 2004), it is imperative that researchers use whatever cost-effective strategies are available to increase their response rates. One strategy empirically investigated for its effect on response rates has been type of mailing. One consistent finding is that using Certified mail has been shown to result in substantially higher response rates (hmm, Stampfer, Colditz, Giovannuci, & Willet, 1990; Del Valle, Morgenstern, Rogstad, Albright, & Vickrey, 1997). The only mailing approach reported to yield higher rates has been the use of a private mail service (i.e., Fed Ex; Kasprzyk, Montano, St. Lawrence, & Phillips, 2001). Less is known about the influence of Priority mail on response rates. One study (Gibson, Koepsell, Diehr, & Hale, 1999), comparing Priority mail and Certified mail, found the latter yielded a higher rate. In the current study, as part of a larger survey that examined differences between urban and rural healthcare providers, the effect of using Priority mail service versus 'Address correspondence to Christiane Brems, Ph.D., ABPP, Behavioral Health Research and Services, University of Alaska Anchorage, 3211 Providence Drive, Anchorage, AK or e- mail (cbrems@uaa.alaska.edu). This research was supported by Grant lrolda13139 from the National Institute on Drug Abuse. Dr. Roberts also acknowledges the su port of a Career Development Award (lko2mh01918) from the National Institute of ~entj~ealth. We gratefully acknowledge the contributions of Marcine Mullen in Alaska and of Dr. Pamela Monaghan and Audrey Solimon in New Mexico.

2 RETURN RATES AND TYPE OF MAILING 497 regular First-Class mail for survey mailings was investigated. It was hypothesized that priority-delivered surveys would engender a higher return rate from potential participants given the higher cost of mailing and implied importance of the mailing. Given the nature of the larger study, it was also possible to examine whether mailing type differentially affected response rates from urban versus rural communities; however, no a priori hypotheses were established for this aspect of the study. Participants To identify potential participants, lists of licensees were obtained from licensing boards in Alaska and New Mexico for physicians, nurse practitioners, registered nurses, licensed mental healthcare providers, social workers, physician assistants, and psychologists (New Mexico only). Each licensee list was stratified into urban and rural addresses, and participants were randomly selected by choosing each nth licensee (determined based on number in a given stratified list). For each of these seven groups, 10 licensees in urban communities and 10 licensees in rural communities were invited to participate. Thus, 40 physicians, 40 nurse practitioners, 40 registered nurses, 40 licensed mental health care providers, 40 social workers, 40 physician assistants, and 20 psychologists were selected. This resulted in a total of 120 individuals (60 urban, 60 rural) solicited in Alaska and a total of 140 (70 urban, 70 rural) in New Mexico. Of these participants, 174 (66.9%) were women and 86 (33.1%) were men. Primary cultural heritage reported by the individuals was 24 (9.4%) Hispanic American, 3 (1.2%) Asian American, 13 (4.9 %) Alaska Native/Native American, 3 (1.2 % ) African American, 209 (80.3 %) White (Anglo or Anglo-American), 5 (1.8%) Other, and 3 (1.2%) did not respond. Their mean age was 48.9 yr. (SD = 9.5). These individuals were targeted for a survey about health care practices. Survey As part of a larger study funded by the National Institute on Drug Abuse, a 21-page survey was developed to obtain healthcare providers' perspectives on a broad range of treatment and ethical issues. Based on extensive qualitative work and the extant literature, this survey included the following sections: ethical challenges, perceptions of illness stigma, training and resource needs, experiences in providing healthcare, barriers faced in providing care, adaptations to barriers, treatment issues related to providing care to minority groups, and providers' practice characteristics. Individual items were rated on an 11-point rating scale, anchored by 0: Not at all or never and 10: Very much or always. The survey required approximately 60 to 90 minutes to complete.

3 498 C. BREMS, ET AL. Procedures Survey procedures recommended by Dillman (2000) were used for all aspects of the survey, which took place between March 1 and April 19, The only deviation from Dlllman's recommendations was that additional time, that is, two weeks instead of one week, was permitted between mailings to accommodate slower mail service to rural communities. A preletter was mailed two weeks prior to the mailing of the survey, outlining the survey's significance and other features. The survey mailing was followed up with a postcard mailing two weeks later to thank respondents and urge nonrespondents to complete the survey if they had not. Two weeks after the postcard, a second copy of the survey was mailed to nonrespondents. Responses to the second wave, i.e., the replacement survey, were not included in this study. Participants were compensated $50 for time spent in completing the survey. During the first wave of the actual survey mailing, that is, at the second contact with potential participants, the samples were randomly divided to receive their survey solicitation either via regular First-Class mail or Priority mail. The randomization procedure involved stratifying the sample by Alaska versus New Mexico and by urban versus rural addresses. Then, within each of these four groups, every other name was assigned to receive Priority mail and the alternating name to receive First Class mail. This sampling procedure resulted in 30 rural and 30 urban dwellers in Alaska receiving Priority mailings and 30 rural and 30 urban dwellers in Alaska receiving surveys via First-Class mail. In New Mexico, mailings went to 35 individuals per group. Thus, a total of 130 individuals received Priority mailings and 130 received First-Class mailings, with half of each group urban and half rural dwellers. First-Class mail was delivered in standard 9x11 envelopes; Priority mail was delivered in priority-labeled envelopes provided by the United States Postal Service (USPS). Cost for the Priority mail was $2.85 and for the First-Class mail was $1.10. If surveys were returned by USPS with new forwarding addresses, the packet was r ed to the new address exactly in the same format as the original packet. If surveys were returned by the USPS due to inability to deliver, that is, wrong addresses or individuals who moved without forwarding addresses, a replacement addressee was chosen, and a new packet was mailed in the same format as that for the original addressee. This procedure assured that the usable sample size remained 120 in Alaska and 140 in New Mexico. Returned Rate Calculations Surveys returned for undeliverable addresses were mailed to a replacement addressee, and the original addressee was not counted. Thus, the sam-

4 RETURN RATES AND TYPE OF MAILING 499 ple size remained constant at 120 in Alaska and 140 in New Mexico. Based on the lack of differential response rates between Priority and First-Class mail during the first mailing, it was decided that it was cost prohibitive to use Priority mail during subsequent mailings. Thus, calculation of return rates was based on the total number of completed surveys received from the first survey mailing. RESULTS Table 1 provides return rates separated by state and by community size, as well as for all data combined. An overall comparison of response rates across the eight groups (Alaska/New Mexico, urban/rural, Priority/First- Class) yielded no significant differences (x,' (N = 13 0) = 1 32, p =.97). Given no overall differences, all responses were pooled across groups to test only Priority versus First-Class mail. The return rate was 39.2% for First-Class mail and 35.4% for Priority mail. The estimated difference between the proportions for Priority and First-Class mail was 3.8% and the 95% confidence interval for this estimate runs from -8.1% to 15.8%. A Z test of binomial proportions indicated no significant effect for method of mailing, p =.52. DISCUSSION Based on this single survey mailing, response rates for First-Class mail and Priority mail did not differ significantly. Thus, in terms of enhancing response rates, the additional expense of sending surveys to potential participants via Priority mail does not appear warranted. This information complements prior research indicating First-Class mail is superior to other forms of mail for mail-out surveys, with the exception of Certified mail (Dillman, 2000). When considering these findings, three limitations must be kept in mind. First, the small sample and resultant low power highlights the exploratory nature of this study and calls for replication. With samples of 130 in each of the two groups, there was only a 13% chance (power) of detecting a real increase from 50% to 55 % at 5 % significance or a 37% chance (power) of detecting an increase from 50% to 60%. Indeed, as a result of the limited power, the study data are still compatible with an approximate 8% increase in response rate using Priority mail as well as a 16% decrease in response rate using Priority mail. Second, the sample used in this study was professional healthcare providers with likely multiple demands on their time. Other samples, such as the general population, could yield different results. Third, given that no data are available to compare respondents and nonrespondents on demographics data or any variables, some unmeasured factor could have served as a mediator or moderator in the response to type of mailing.

5 TABLE 1 RETURN RATES AND 95% CONFIDENCE INTERVALS (CI) BY TYPE OF MAILING, STATE, AND COMMUNITY SIZE State -- First-class Mailing Priority Mailing Response Kate P Mailed Returned Return ~ a g Mailed Returned Return Rate Difference 95% CI Alaska Rural Urban Total New Mexico Rural Urban Total Overall Rural Urban Total (%I (%I (%I

6 RETURN RATES AND TYPE OF MAILING 501 REFERENCES DEL VALLE, M. L., MORGENSTERN, H., ROGSTAD, T. L., ALBRIGHT, C., &VICKREY, B. G. (1997) A randomized trial of the im act of certified mail on response rate to a physician survey, and a cost-effectiveness stuiy. Evaluation & the Health Pmfessions, 20, DILLMAN, D. A. (2000) Mail and internet surveys: the tailored design method. New York: Wiley. GIBSON, P J., KOEPSELL, T. D., DIEHR, P., & HALE, C. (1999) Increasing response rates for mailed surveys of Medicaid clients and other low-income populations. American ]ournal of Epidemiology, 149, KASPRZYK, D., MONTANO, D. E., ST. LAWRENCE, J. S., &PHILLIPS, W. R. (2001) The effects of variations in mode of delivery and monetary incentive on physicians' responses to a mailed survey assessing STD practice patterns. Evaluation & the Health Professions, 24, RIMM, E. B., STAMPFER, M. J., COLDITZ, G. A., GIOVANNUCI, E., & WILLET, W. C. (1990) Effect of various mailing strategies among nonrespondents in a prospective cohort study. American loarnal of Epidemiology, 131, TOU-GEAU, R. (2004) survei-research and societal change. Annual Review of Psychology, 55, Accepted September 5, 2006.