Identifying substance abuse treatment gaps in substate areas

Transcription

1 Journal of Substance Abuse Treatment 23 (2002) Regular article Identifying substance abuse treatment gaps in substate areas William E. McAuliffe, Ph.D. a, *, Ryan Woodworth, B. A. b, Caroline (Hui) Zhang, M. A. c, Ryan P. Dunn, B. A. c a Department of Psychiatry, Harvard Medical School, National Technical Center for Substance Abuse Needs Assessment, North Charles Research and Planning Group, 875 Massachusetts Avenue, 7 th Floor, Cambridge, MA 02139, USA b Pennsylvania State University, State College, PA 16801, USA c National Technical Center for Substance Abuse Needs Assessment, North Charles Research and Planning Group Cambridge, MA 02139, USA Received 24 October 2001; received in revised form 25 April 2002; accepted 13 May 2002 Abstract Investigating concerns about uneven utilization of health services, especially affecting disadvantaged high-risk populations, the authors constructed composite indexes for identifying substance abuse treatment gaps in Rhode Island towns and multi-town planning areas. The Drug, Alcohol, and Substance Abuse Need Indexes combined multiple-year rates of substance-related deaths, hospital discharges, and arrests. These indicators were reliable and possessed convergent validity; the composite indexes were also reliable and had construct validity. Regression of treatment admissions rates on town Substance Abuse Need Index scores revealed that some areas had relative gaps in treatment services. Having an objective and validated method for identifying treatment gaps could help treatment planners ensure equal access to services throughout the state. Reducing travel to treatment facilities can increase treatment utilization and treatment retention. D 2002 Elsevier Science Inc. All rights reserved. Keywords: Treatment service gaps; Needs assessment; Substance abuse indicators; Disparities in access; Small areas 1. Introduction A current social concern is the underutilization of health care services by disadvantaged groups (Allen, 2001; Leshner, 2001). While the discussion of this problem has focused on removing obstacles to access, little attention has been given to the inadvertent consequences of current methods of allocating public health resources. The most commonly used formula for distributing public resources is allocation on a per capita basis (Breer et al., 1996; McAuliffe, Breer, Levine, Rosner, & Williams, 1992). Although they appear to adjust for differential risk, some formulas, such as the federal allocation formula for the Substance Abuse Prevention and Treatment Block Grant, are so highly correlated with population size that they are virtually per capita systems (McAuliffe, LaBrie, Lamuto, Betjemann, & Fournier, 1999). Unfortunately, per capita allocation ignores * Corresponding author x113; fax: address: wmcauliffe@ntc.org (W.E. McAuliffe). differences in population risk. If public resources are allocated to service areas on a per capita basis, high-risk areas will automatically have relatively fewer public services per person in need. Low-risk areas will be relatively well served, and they may even have underutilized capacity. As a result, residents of high-risk areas may have to travel substantial distances to programs with openings, and these programs may not focus sufficiently on cultural barriers that also hinder access to care. Research suggests that transportation issues can affect retention in treatment and presumably treatment outcomes (Friedmann et al., 2001). Development of allocation formulations that accurately reflect differential population needs may therefore be an important component of society s efforts to eliminate uneven access to public services and improve the clinical outcomes of high-risk minority populations. Previously, McAuliffe et al. (1999, 2000) developed an interstate Drug Problem Index and Alcohol Problem Index using data. Designed as measures of quality of life and also suitable as prevention need measures, the indexes included state substance abuse mortality, arrest, and treatment rates. Conscious of the importance of estab /02/$ see front matter D 2002 Elsevier Science Inc. All rights reserved. PII: S (02)

2 200 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) lishing validity, the authors selected the components on the basis of theoretical relevance and the existence of published evidence of their validity. Classical psychometric measurement methods yielded evidence consistent with the convergent and discriminant validity of the components. The resulting composite indexes also had evidence of reliability and construct validity. More recently, the authors created Drug and Alcohol Need Indexes using interstate mortality and arrest data (McAuliffe, LaBrie, Woodworth, Zhang, & Dunn, 2002). The indexes were used to identify state treatment gaps. The present study sought to determine whether the same methodology could be used to identify substate area treatment gaps. The Alcohol Need Index and Drug Need Index employed parallel rates of alcohol and drug mortality, substance-defined arrests, and hospital discharges related to substance abuse complications but not treatment. The study also created a Substance Need Index that combined unduplicated drug and alcohol deaths, arrests, and hospital discharges. The study assessed whether the indexes were reliable and valid at the town level. Finally, the study employed the need indexes to identify gaps in treatment services within the state. The study tested the following hypotheses: 1. Substate areas vary substantially in the risk of alcoholand drug-related problems. 2. Reliable and valid indexes of treatment need can be developed from a small number of carefully selected indicators. 3. There are substantial treatment gaps in some substate areas. 2. Materials and methods The study used town-level data obtained from the Rhode Island Department of Health, the State Police, and the Division of Substance Abuse (see the Appendix). The study team examined each variable carefully and called the relevant agencies to obtain corrected data in the few instances when outliers were evident. Because ten of Rhode Island s 39 cities and towns had populations of less than 10,000, the study combined two or three of the smallest towns with each other or with other relatively small towns into six multi-town planning areas. Analysis showed that mortality rates from areas with fewer than 10,000 residents were unstable. In these planning areas, all of the towns were geographically contiguous and similar in their urban-rural nature (Landis 2000) Indicator selection and definitions To be considered as an index component, an indicator had to be theoretically related to the concept of treatment need, be available for all or virtually all towns, and have evidence of validity in the literature. The study defined the concept of treatment need as requiring professional help or care to recover from an alcohol or controlled drug use disorder. The selected drug indicators included weighted mean rates per 100,000 of drug mortality, drug-defined arrests (possession and sales), and drug-related hospital discharges. The selected alcohol indicators included parallel rates of alcohol mortality, alcohol-defined arrests, and alcohol-related hospital discharges. These indicators reflect elements of substance use disorders and related impairment. Continuing to use alcohol or drugs despite repeated substance-related arrests is a clinical criterion of substance abuse according to the American Psychiatric Association (DSM-IV, 1994). From three quarters to 90% of arrestees for drug sales or possession test positive for drug use (Schlaffer, 1997; Treatment Alternatives for Safer Communities [TASC], 1998). The rationale for drug courts and other diversion programs is that most people who commit drug-related crimes need treatment instead of prison. Repeat drunk drivers are routinely assessed and often required to obtain treatment. Clinical criteria for substance dependence include continued use of substances despite adverse medical effects (DSM-IV, 1994). Existence of severe medical complications of alcohol and drug use such as alcoholic cirrhosis or AIDS associated with injection drug use are primary medical necessity criteria for substance abuse treatment according to the American Society for Addiction Medicine (Hoffmann, Halikas, Mee-Lee, & Weedman, 1991). Death certified as resulting from drug or alcohol dependence, nondependent abuse, withdrawal, overdose, or other disorders medically linked 100% of the time to substance use suggests a strong probability that the decedents had a substance use disorder. It is reasonable to assume that most if not all of the people who were hospitalized for an explicit-mention substance-related disease needed treatment. Arrests, deaths, and hospitalizations caused by alcohol or drug use are thus closely linked to the concept of treatment need. The operational definitions of these indicators drew upon the authors previous research (McAuliffe, Breer, Ahmadifar, & Spino, 1991; McAuliffe et al., 1999, 2000), but included several refinements. For its alcohol-defined arrest measure, the study used liquor law violations (e.g., underage drinking) and disorderly conduct arrests. The study excluded DUI and drunkenness statistics because they lacked empirical validity (see the Appendix). The authors adjusted the alcohol and drug arrest rates for the effects of tourism by using nonseason rates for two towns that had marked increases in arrests during the summer months. The mortality statistics included a slightly refined set of diagnoses from the authors previous studies (McAuliffe et al., 1999, 2000). The hospital discharge diagnoses used the same diagnostic codes, but excluded persons discharged following treatment of substance use disorders, as measured by procedure codes or Diagnostic Related Group (DRG) codes; they were used

3 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) instead to create a measure of hospital-based substance abuse treatment rates Assessing component reliability and validity The study applied Cronbach s (1951) a to multiple years of data in order to assess the stability of the indicators. Alpha is a function of the average intercorrelation among the annual measures and the number of years. The study also measured the reliability of the indicators from the communalities of a factor analysis of the index components (Harman, 1967, p. 19). The factor analysis also included the AIDS rate associated with injection drug use (IDU-AIDS), a composite index of TB, hepatitis B, and syphilis, and the town liquor license rates. In the absence of a gold standard against which to gauge validity, the study employed convergent and discriminant validity as measured in a multitrait-multimethod correlation matrix of the component indicators (Campbell & Fiske, 1959). Convergent validity assumes that several indicators of the same concept should correlate positively with one another. Analysis of the correlation matrix also indicates whether the measures of drug and alcohol need are empirically distinct (discriminant validity): drug indicators correlate more strongly with each other than with other concepts (e.g., alcohol indicators), and vice versa. The outcome of this analysis determined the final composition of the composite need indexes. The construct validation of the resulting composite indexes consisted of assessing their empirical performance in relation to measures of other concepts in a theoretical model of drug and alcohol need (Cronbach & Meehl, 1967). Construct validity does not yield a single quantitative measure of validity. Instead, it depends upon the consistency of the pattern of empirical correlations with theoretical expectations. The present study examined the correlation of the indexes with causes of drug use disorders (poverty, urbanism, and race) and their consequences (e.g., non-drug crimes such as burglary and health consequences such as contagious diseases) Index construction The study constructed three indexes: a Drug Need Index (DNI), an Alcohol Need Index (ANI), and a Substance Need Index (SNI). After converting the populationweighted mean rates for each of the three drug need measures into standard scores (z-scores), the z-scores were summed. The ANI and SNI were constructed similarly, except that the SNI used unduplicated substance abuse deaths and substance abuse hospital discharges. The effective weights of the components (i.e., the proportion of variance that each explains) of the DNI were.33,.35, and.32 for drug mortality, drug hospital discharges, and drug arrests respectively. The effective weights of the three alcohol indicators in the ANI were.35 for alcohol mortality,.34 for alcohol hospitalizations, and.31 for alcohol-defined arrests. The authors scaled the resulting sum of the z-score values from 0 to 100 in order to give each of the three indexes a meaningful metric. The need index value of zero indicated that there were no drug deaths, arrests, or hospital discharges in a town during the years covered by the indicators. The index value of 100 represented a hypothetical z-score sum that combined the highest observed mean values for each of the three component indicators. The actual values of the SNI ranged from 24 to 96, with a median of Comparison of the need index with existing service levels The study used regression analysis to compare the SNI to the state s substance abuse treatment admissions rates. Negative residuals (i.e., observed treatment rate minus rate predicted by the need indexes) were defined as relative gaps in treatment services. 3. Results 3.1. Town variations in risk The maximum and minimum values for all of the need indicators showed that there were substantial variations in risk among the towns (Table 1). The highest drug mortality rate was nearly 3 times higher than the average drug mortality rate, the highest drug hospitalization rate was 7 times the lowest, and the highest drug arrest rate was nearly 6 times greater than the lowest. Similarly, the highest town alcohol mortality rate was 4 times the lowest, the highest alcohol hospital discharge rate was 7.5 times the lowest, and the highest alcohol arrest rate was more than 12 times the lowest. Large variations in town treatment rates were also evident Component stability and reliability Except for the drug and alcohol mortality indicators, the need indicators and treatment admission rates had reasonably high levels of stability (over.90 as measured by Cronbach s a analyses of the average annual rates, Table 1). Drug mortality rates were moderately stable, but the alcohol mortality rates had much less consistency. Although combining the smallest towns into multi-town units reduced the volatility in the alcohol mortality rates, it was still evident in five of the six multi-town planning areas and in five other towns with populations between 10,000 and 15,000. While the reliability estimates from the communalities of a factor analysis of the indicators were generally lower than the a estimates, the reliabilities were moderate to substantial for all components of the composite need indexes (Table 1). The alcohol arrest indicator had the lowest communality, no

4 202 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) Table 1 Characteristics and reliability of alcohol and drug need indicators and treatment admissions Rates per 100,000 Min Max Median Mean Stability of composite (Cronbach s a) Drug indicators Explicit-mention drug mortality, Explicit-mention drug hospital discharges, Drug-defined arrests, Primary drug treatment admissions, , Reliability of mean rate from communality Alcohol indicators Explicit-mention alcohol mortality, Explicit-mention alcohol hospital discharges, Alcohol-defined (liquor law and disorderly conduct) arrests, , Primary alcohol treatment admissions, , Note: The mean and standard deviations are unweighted for town size. The sample size was 32 towns and multi-town planning areas, except for arrests where the sample size was 31. Exeter s arrest rates were estimated and therefore not included in the reliability analyses. doubt reflecting the fact that it is based on two rather than five or six years of information Convergent and discriminant validity of component need indicators Intercorrelations among the three drug need measures and three alcohol need measures were consistent with the hypothesis that the selected variables measured the underlying concepts of drug and alcohol treatment need (Table 2). The average Pearson correlation was.68 among the drug need measures and.55 among the alcohol need measures; all were statistically significant. Lower average correlations among alcohol indicators than drug indicators was consistent with interstate findings (McAuliffe et al., 1999, 2000). The underlined correlations in Table 2 revealed apparent method effects in all four indicators (Campbell & Fiske, 1959). For example, the correlation between drug-related hospital discharges and alcohol-related hospital discharges depends both upon their being based on the same hospital data (e.g., access to the hospital, common errors in coding, etc.) and on the overlap between drug and alcohol use disorders (some people have both problems and therefore are diagnosed for both). Overlap between the disorders is to be expected, but overlap due to the method of data collection is undesirable. The size of these correlations is clear Table 2 Multi-trait, multi-method correlation matrix of drug and alcohol need indicators and admission rates Primary drug treatment admissions Alcohol hospitalization discharge Indicator (rate/100,000) Drug hospital discharges Drug mortality Drug-defined arrests Alcohol mortality Explicit-mention drug hospital discharges, Explicit mention.75* drug mortality, Drug-defined arrests, *.58* Primary drug treatment.86*.79*.62* admissions, Explicit mention alcohol.88*.65*.78*.66* hospitalization discharges, Explicit-mention.59*.60*.68*.51*.69* alcohol mortality, Alcohol-defined arrests.39*.29.56*.43*.45*.50* (dis con/liq law), *.72*.71*.75*.79*.72*.73* Primary alcohol treatment admissions, Alcoholdefined arrests Primary alcohol treatment admissions Results of factor analysis; varimax rotation Drug factor Alcohol factor Note: The entries are Pearson product-moment correlations. * = significant beyond.05, two-tailed. Bolded figures are convergent validities. Underlined figures are measures of method effects.

5 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) evidence of the importance of combining several measures based on different data collection systems in order to balance out the influence of a particular data collection system on the variance of the need indexes. In the factor analysis described above with regard to reliability, a two-factor solution produced some evidence of distinct drug and alcohol factors, although the overlap between the factors was clear (bottom of Table 2). The drug arrest measure, with marijuana arrests included, loaded most strongly on the alcohol factor. The alcohol hospital discharge variable loaded virtually equally on both factors. Fifteen percent of the subjects that had an alcohol diagnosis also had a drug diagnosis, and 28 percent of those with a drug diagnosis had an alcohol diagnosis as well. The same phenomenon is of course present for the mortality variables. The remaining drug variables, including IDU-AIDS and the composite of TB, hepatitis B, and syphilis, loaded much more strongly on the drug factor than on the alcohol factor, and the remaining alcohol variables, including the liquor license rate, loaded more strongly on the alcohol factor than on the drug factor Reliability and construct validation of the need indexes The a reliability of the DNI was.87, and the reliability of the ANI was.78. The first principal component of a factor analysis of the drug need measures explained 79% of the variance of the three items, and the first principal component of the factor analysis of the alcohol need measures explained 70% of the three-item variance. These results suggest that the two composites were substantially reliable. The pattern of correlations between the drug and alcohol need indexes and other variables was generally consistent with the construct validity of the indexes (Table 3). The DNI correlated more highly than the ANI with urbanicity (Landis 2000), the percentage of African American residents, percentage of Hispanic residents, the percentage of residents below the poverty line, the IDU-AIDS rate, the drug-related contagious disease index, property crime rates, and the percentage of residents in prisons. Homeless persons have very high rates of both alcohol and other drug problems (Fischer, 1991), although in most samples the rates of alcoholism are somewhat higher than the rates of drug addiction (Breakey et al., 1989; North & Smith, 1993; Alemagno et al., 1996). Both the ANI and the DNI correlated significantly with the percentage of residents who were living in homeless shelters, and the difference between the two correlations was not significant. As expected, the ANI correlated more highly than the DNI with the 1997 rate of liquor licenses, but the difference was not quite significant. According to national treatment admission statistics (Gustafson et al., 1997), persons 18 to 54 were most likely to receive alcohol and drug treatment. Both the DNI and the ANI correlated significantly with the percentage of a town s population who were in that age cohort. Alcohol treatment Table 3 Construct validity of drug and alcohol need indexes Drug need index (DNI) Alcohol need index (ANI) r DNI -r ANI p-value Indicator Urban core (4), ring (3),.81*.71* ns suburban (2), and rural towns (1) Percent African American, *.56*.01 Percent Hispanic, *.49*.08 Percent below poverty, *.68*.01 Percent living in.73*.70* ns homeless shelters, 1990 Percent aged years, *.41* ns Liquor license rate, *.51* ns Mother drinking during pregnancy rate, Chemically dependent.84* infant rate, IDU-AIDS rate, *.52*.01 Drug-related contagious.78*.51*.01 diseases rate, a Composite of burglary,.81* 72* ns robbery and prostitution rate, Percent of population.50* in prison, 1990 Drug treatment.85*.64*.01 admission rate, Alcohol treatment.81*.90* ns admission rate, Hospital-based drug treatment.80*.55*.05 admission rate, Hospital-based alcohol treatment.63*.69* ns admission rate, Note: The higher correlation in the pair is bolded. a Syphilis, hepatitis B, and tuberculosis. * = p <.05, two-tailed. clients are somewhat less concentrated in this age cohort (Gustafson et al., 1997), and therefore the slightly stronger correlation with the DNI appears to be consistent with the construct validity of the need indexes. The DNI correlated more highly than the ANI with the rate of chemically dependent infants per 100,000 population in , while the Alcohol Need Index correlated more strongly than the DNI with the mean rates of mothers who drank during pregnancy. The DNI and ANI correlated with their respective state-funded treatment admissions rates for The DNI correlated more highly than the ANI with the primary drug treatment admission rate, while the ANI correlated more highly than the DNI with the primary alcohol treatment admissions rate. The same pattern was true for hospital treatment discharge rates, which were counted separately. While the differences between the correlations of the other indicators with the two need indexes were often small and were significant in only 8 of 17 comparisons, the overall pattern of differences was reasonably consistent with the construct validity of the two indexes.

6 204 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) Consistent with the DNI and ANI results, the SNI was reliable (a =.86) and correlated positively with the rate of homelessness (.71; p <.05), the percent below poverty (.80; p <.05), the number of hospital treatment discharges (.68, p <.05), and substance abuse treatment admissions (.88) Identifying treatment gaps There were four relatively distinct areas of Rhode Island with the highest rates of substance abuse treatment needs. They included the urban core and ring around Providence, Woonsocket to the north, suburban Westerly in the southwestern corner of the state, and urban Newport. The lowest rates were in rural and suburban towns. Despite a high correlation between the SNI and total treatment admissions, regressing the total state treatment admissions rate on the SNI (Substance abuse treatment admission mean rate = SNI; R 2 =.77) and mapping the residuals (observed minus predicted rates) revealed where there were noticeable patterns in the supply of state-funded treatment services relative to need (Fig. 1). Two areas of the state (those with darker shading) had a cluster of towns with relatively favorable treatment rates. The larger one included Providence and surrounding urban towns, and the smaller one included towns along the southern edge anchored by Westerly. Providence, with the third highest SNI score in the state (83), had the most favorable difference between the rate predicted by the treatment need score and the rate of residents served. This result paralleled earlier findings by McAuliffe et al. (1987) of a relative concentration of services in the Providence area. While access to the Providence area is only a short drive for most of the Rhode Island population, residents in need who do live in rural towns to the west and south and who lack transportation appear to be at a relative disadvantage compared to those who live in Providence. Woonsocket had a moderately favorable treatment admissions rate when compared to its level of need (43 per 100,000 over the rate predicted by its level of need). In an earlier needs assessment study conducted for the state, McAuliffe et al. (1987) identified Woonsocket as an area in need of additional services, and the state acted upon this recommendation. Newport, which had the state s highest SNI score (96) and the third highest treatment admissions rate, was nevertheless the most relatively underserved in terms of the rate ( 435 admissions per 100,000 residents) and total number of admissions. It would have to increase its residents alcohol and drug treatment admissions by 106 per year to achieve equity with Rhode Island s other cities and towns. Bristol, another of the three areas with the largest treatment gaps, had a moderate SNI of 45 (ranked 15th out of 32 areas in the state), but suburban Bristol had the state s sixth lowest Fig. 1. Adequacy of substance abuse treatment services

7 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) treatment admission rate. Bristol would have to increase its treatment admissions by an average of 87 per year to reach the number predicted by its substance abuse treatment needs. Finally, the rural Richmond-Hopkinton area, which also had a moderate SNI score (46) and a relatively low treatment admission rate (ninth in the state), had the second largest treatment gap. A comprehensive survey of households, homeless shelters and soup kitchens, group homes, and prisons estimated that statewide there were 2,577 residents aged 12 and older who needed treatment (had a past-year diagnosis), had not received treatment in the past year, but would have sought treatment had it been available (McAuliffe, Zhang, & Dunn, 2002). Survey respondents who needed treatment but had not obtained it reported that common obstacles were lack of transportation and programs being too far away. Because the number of interviews in individual towns was too small, the authors used the indicator regression analysis to provide state officials with specific recommendations for increasing the level of services in each city and town. The recommendations allocated the 2,577 admissions to bring each town s annual admission rate up to the rates predicted by the SNI. 4. Discussion The study identified apparent treatment service gaps in Rhode Island. The analysis of the component indicators suggested that the towns varied substantially in their relative need for services. Clearly, a per capita allocation of services would overlook these variations. As a result of previous planning using similar methods (McAuliffe et al., 1991), the correspondence between need and services was greater than the relationship that the authors found in a previous treatment needs assessment of Rhode Island s drug abuse services in 1987 (McAuliffe et al., 1987). In the 1987 study, a disproportionate share of the Rhode Island s treatment services were located in the Providence planning region. While Providence continued to have more treatment services than predicted by the present study s measure of treatment need, other areas that were poorly served in 1987 were well served in the present analysis. Despite the strong overall match between need and services, Newport, a town with a relatively high percentage of disadvantaged minorities and a high treatment service rate, was nevertheless the most underserved town in the state. The analysis of treatment gaps showed that there was no single geographic pattern that identified areas with the greatest amount of unmet need, and the authors found no obvious substitute for measuring need directly using the need indexes. As the state attempts to reduce unmet treatment needs in the future, it can use the recommendations developed from the present methodology to match need and services more closely. Doing so can result in greater access to treatment services, improved retention in treatment, and presumably superior clinical outcomes (Friedmann, Lemon, & Stein, 2001). The substance-specific need indexes can also be used to improve the match between specific clinical needs and services (e.g., determine where specialized facilities such as alcohol detoxification units and methadone maintenance programs should be located). The validity of the measures used in the process is especially important to treatment providers, whose continued funding may depend on the outcome of the treatment allocation methodology. Inadequate documentation of the validity of the measures of treatment need can lead legislators to reject the conclusions of the needs assessment in favor of maintaining the status quo (Breer et al., 1996). In order to validly identify treatment service gaps in Rhode Island, the study team emphasized validity throughout the development of the town-level indicator-based indexes of alcohol and drug abuse treatment need. By design, the measures focused on indicators of substance-use disorders rather than casual drug and alcohol use. The Drug Need Index (DNI) employed a small number of indicators (rates of explicit-mention drug hospitalizations, explicit-mention drug deaths, and drug-defined arrests) that had clear theoretical links to the presence of high rates of drug use disorder treatment needs and for which there was empirical evidence of validity in the literature. Using measures with face validity helps achieve acceptance by providers, legislators, and the lay public. The Alcohol Need Index contained parallel measures of explicit-mention alcohol hospitalizations, deaths, and arrests (disorderly conduct and liquor law violations), which permitted use of the convergent and discriminant validation methodology. The SNI combined unduplicated mortality and arrest rates. There was evidence of convergent validity among the component indicators in the composite alcohol, drug, and substance abuse need indexes. These findings confirmed the results of earlier substance abuse indicator studies that included validation of one or more of these component indicators: Adrian (1983), Ball and Chambers (1970), Beenstock (1995), Beshai, (1984), Breer et al. (1996), Cleary (1979), Frank et al. (1978), Gregoire (2002), Larson and Marsden (1995), McAuliffe et al. (1999, 2000), Person, Retka, & Woodward, (1976), Simeone, Frank, & Aryan, (1993), and Woodward, Retka, & Ng, (1984). The composite DNI, ANI, and SNI were substantially reliable (a of.87,.78, and.86, respectively), and there was evidence of their construct validity. The indexes correlated as expected with causes, correlates, and consequences of substance use disorders. In nearly all of the analyses the DNI correlated more highly than the ANI with drug-related variables, and the ANI correlated more highly than the DNI with alcohol-related variables. The differences reached statistical significance half of the time. The authors observed similar evidence of the construct validity of the ANI and DNI at the interstate level (McAuliffe, LaBrie, et al., 2002). These findings regarding the reliability and validity of the indicators contrasted with the negative findings and

8 206 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) warnings in some of the literature about substance abuse mortality and arrest data (Bennett, 1995; National Institute on Drug Abuse [NIDA], 1998; DeFleur, 1975; Dewit & Rush, 1996; Furst & Beckman, 1981; Gottschalk, McGuire, Dinovo, Birch, & Heiser, 1977; Gruenewald, Treno, & Klitzner, 1997; Hopkins, Grant-Worley, & Bollinger, 1989; Lipscomb & Sulka, 1961; Maltz, 1999; Maxwell, 1986; Pollock, Holmgreen, Lui, & Kirk, 1991; Schmidt & Weisner, 2000; Shai, 1994; United States General Accounting Office [GAO], 1990). These authors raised important issues about the potential limitations of substance abuse indicators that should be kept in mind when using them in research and applied studies. Resolution of the apparently conflicting findings in the literature regarding substance abuse indicators requires additional research on the reliability and validity of substance abuse indicators. The present research suggested, however, that the potential limitations of substance abuse indicators may be avoided by relying on only theoretically direct and empirically validated indicators, carefully cleaning and correcting of the data for seasonal effects, using carefully selected mortality codes and arrest indicators (see the Appendix), using sufficient numbers of years, and aggregating the data over sufficiently large contiguous populations. Use of standard psychometric methods to evaluate both the individual indicators and the final ANI, DNI, and SNI measures is essential to insure that the indexes meet standards of reliability and validity when applied in a specific state. If a theoretically relevant indicator or composite index does not perform adequately in the psychometric assessments, the indicators should not be used to identify treatment gaps. Acknowledgments Research for this article was supported in part by Center for Substance Abuse Treatment contracts with Rhode Island ( ). Dr. McAuliffe s sabbatical leave funded by the North Charles Foundation also supported the study. The authors wish to acknowledge the contributions of Steve Haddad, Richard LaBrie, and Kenneth Nielsen to this study. References Adrian, M. (1983). Mapping the severity of alcohol and drug problems in Ontario. Canadian Journal of Public Health, 76 (5), Alemagno, S. A., Cochran, D. C., Feucht, T. E., Stephens, R. C., Butts, J. M., & Wolfe, S. A. (1996). Assessing substance abuse treatment needs among the homeless: A telephone-based interactive voice response system. American Journal of Public Health, 86 (11), Allen, C. E. (2001) Presidential address: eliminating health disparities. American Journal of Public Health, 91 (7), American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association. Ball, J. C., & Chambers, C. D. (1970). The epidemiology of opiate addiction in the United States. Springfield, IL: Charles C. Thomas. Beenstock, M. (1995). An indicator model of drug use in israel. Addiction, 90 (3), Bennett, A. (1995). Is Tallahassee really as plagued by crime as New York City? Widely used statistics belie key variations in risks facing cities residents. Wall Street Journal, (5), 1. Beshai, N. (1984). Assessing needs of alcohol-related services: a social indicators approach. American Journal of Drug and Alcohol Abuse, 10 (3), Breakey, W. R., Fischer, P. J., Kramer, M., Nestadt, G. R., Romanoski, A. J., Ross, A., Royall, R. M., & Stine, O. C. (1989). Health and mental health problems of homeless men and women in Baltimore. Journal of the American Medical Association, 262 (10), Breer, P., McAuliffe, W. E., & Levine, E. B. (1996). Statewide substance abuse prevention planning. Evaluation Review, 20 (5), Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, Cleary, P. D. (1979). A standardized estimator of the prevalence of alcoholism based on mortality data. Journal of Studies on Alcohol, 40 (5), Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, Cronbach, L. J., & Meehl, P. E. (1967). Construct validity in psychological tests. In: W. A. Mehrens, & R. L. Ebel (Eds.), Principles of educational and psychological measurement. Chicago, IL: Rand McNally. DeFleur, L. B. (1975). Biasing influences on drug arrest records: implications for deviance research. American Sociological Review, 40, DeWit, D. J., & Rush, B. R. (1996). Assessing the need for substance abuse services: a critical review of needs assessment models. Evaluation and Program Planning, 19 (1), Fischer, P. J. (1991). Alcohol, drug abuse and mental health problems among homeless persons: a review of the literature, Rockville, MD: National Institute on Alcohol Abuse and Alcoholism. Frank, B., Schmeidler, J., Johnson, B. D., & Lipton, D. S. (1978). Seeking truth in heroin indicators: the case of New York City. Drug and Alcohol Dependence, 3 (5), Friedmann, P. D., Lemon, S. C., & Stein, M. D. (2001). Transportation and retention in outpatient drug abuse treatment programs. Journal of Substance Abuse Treatment, 21 (2), Furst, C. J., & Beckman, L. J. (1981). Alcohol-related mortality and alcohol consumption statistics: stability of estimates for small areas. Journal of Studies on Alcohol, 42 (1), Gottschalk, L. A., McGuire, F. L., Dinovo, E. C., Birch, H., & Heiser, J. F. (1977). Guide to the investigation and reporting of drug-abuse deaths: problems and methods. Rockville, MD: National Institute on Drug Abuse. Gregoire, T. K. (2002). The validity of a social indicator approach to substance misuse needs assessment. Substance Use and Misuse, 37, Gruenewald, P. J., Treno, A. J., & Klitzner, M. D. (1997). Measuring community indicators: a systems approach to drug and alcohol problems. Thousand Oaks, CA: Sage Publications. Gustafson, J. S., Reda, J. L., McMullen, H., Sheehan, K., McGencey, S., Rugaber, C., Anderson, R., & DiCarlo, M. & National Association of State Alcohol and Drug Abuse Directors. (1997). State resources and services related to alcohol and other drug problems for fiscal year 1995: An analysis of state alcohol and drug abuse profile data. Washington, DC: National Association of State Alcohol and Drug Abuse Directors (NASADAD). Harman, H. H. (1967). Modern factor analysis (2nd ed. rev.). Chicago: University of Chicago Press. Hoffmann, N. G., Halikas, J. A., Mee-Lee, D., & Weedman, R. D. (1991). Patient placement criteria for the treatment of psychoactive substance use disorders. Washington, DC: American Society of Addiction Medicine. Hopkins, D. D., Grant-Worley, J. A., & Bollinger, T. L. (1989). Survey

9 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) of cause-of-death query criteria used by state vital statistics programs in the US and the efficacy of the criteria used by the Oregon vital statistics program. American Journal of Public Health, 79 (5), Landis, B. (2000). What does government cost? It depends on where you live. Providence Journal. ( accessed 5/28/2000). Larson, M. J., & Marsden, M. E. (1995). State-level substance abuse indicators for youth. Waltham, MA: Brandeis University. Leshner, A. I. (2001). Meeting the challenge of reducing health disparities. NIDA Notes, 16 (1), 3 4. Lipscomb, W., & Sulka, E. (1961). Some factors affecting the geographic comparison of alcoholism prevalence rates: implications for the use of the Jellinek estimation formula. Quarterly Journal of Studies of Alcohol, 22 (4), Maltz, M. D. (1999). Bridging gaps in police crime data. Washington, DC: US Department of Justice. Maxwell, J. D. (1986). Accuracy of death certification for alcoholic liver disease. British Journal of Addiction, 81 (2), McAuliffe, W. E., Breer, P., Ahmadifar, N. W., & Spino, C. (1991). Assessment of drug abuser treatment needs in Rhode Island. American Journal of Public Health, 81 (3), McAuliffe, W. E., Breer, P., White, N., Spino, C., Goldsmith, L., Robel, S., & Byam, L. (1987). A drug abuse treatment and intervention plan for Rhode Island, review copy. Providence, RI: Division of Substance Abuse, Rhode Island Department of Mental Health, Retardation and Hospitals. McAuliffe, W. E., Breer, P., Levine, E. B., Rosner, B. A., & Williams, C. N. (1992). A statewide substance abuse prevention plan for Rhode Island. Rhode Island: Rhode Island Office of Substance Abuse. McAuliffe, W. E., LaBrie, R. A., Lomuto, N., Betjemann, R., & Fournier, E. A. (1999). Measuring interstate variations in drug problems. Drug and Alcohol Dependence, 53 (2), McAuliffe, W. E., LaBrie, R. A., Lomuto, N., Pollock, N. E., Betjemann, R., & Fournier, E. A. (2000). Measuring interstate variations in problems related to alcohol use disorders. In: R. A. Wilson, & M. C. Dufour (Eds.), The epidemiology of alcohol problems in small geographic areas, (vol. 36). Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism. McAuliffe, W. E., LaBrie, R., Woodworth, R., Zhang, C., & Dunn, D. P., (2002). State substance abuse treatment gaps. Unpublished manuscript. McAuliffe, W. E., Zhang, C., & Dunn, R. P. (2002). Study of Rhode Island s substance abuse treatment service needs. Cambridge, MA: North Charles Research and Planning Group. Unpublished manuscript. National Institute on Drug Abuse (NIDA). (1998). Assessing drug abuse within and across communities: community epidemiology surveillance networks on drug abuse. Rockville, MD: National Institute on Drug Abuse. North, C. S., & Smith, E. M. A. (1993). Systematic study of mental health services utilization by homeless men and women. Social Psychiatry and Psychiatric Epidemiology, 28, Person Jr., P. H., Retka, R. L., & Woodward, J. A Toward a heroin problem index an analytical model for drug abuse indicators (Technical Paper). Rockville, MD: National Institute on Drug Abuse. Pollock, D. A., Holmgreen, P., Lui, K. J., & Kirk, M. L. (1991). Discrepancies in the reported frequency of cocaine-related deaths, United States, Journal of the American Medical Association, 266 (16), Schlaffer, M. (1997). Fact sheet: drug-related crime. Rockville, MD: White House ONDCP Drug Policy Information Clearinghouse. Schmidt, L. A., & Weisner, C. M. (2000). Estimating alcohol problems in small areas: contrasting data sources, definitions, and measures. In: R. A. Wilson, & M. C. Dufour (Eds.), The epidemiology of alcohol problems in small geographic areas. Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism. Shai, D. (1994). Problems of accuracy in official drug-related statistics. International Journal of the Addictions., 29 (14), Simeone, R. S., Frank, B., & Aryan, Z. (1993). Needs assessment in substance misuse: a comparison of approaches and case study. International Journal of the Addictions, 28 (8), Single, E., Robson, L., Rehm, J., & Xiaodi, X. (1999). Morbidity and mortality attributable to alcohol, tobacco, and illicit drug use in Canada. American Journal of Public Health, 89 (3), Stinson, F. S., & Nephew, T. M. (1996). State trends in alcohol-related mortality, Bethesda, MD: National Institute on Drug Abuse. Stinson, F. S., Proudfit, A. H., Harford, T. C., Dufour, M. C., & Noble, J. A. (1994). County alcohol problem indicators (4th ed). Rockville, MD: National Institute on Alcohol Abuse and Alcoholism (NIH Publication No ). Treatment Alternatives for Safer Communities. (1998). Drug use up slightly among male chicago arrestees. Chicago, Illinois: Illinois Department of Alcoholism and Substance Abuse, Illinois Criminal Justice Information Authority. United States Bureau of the Census. (1993) Census of Population and Housing Summary. Tape File 3c. Washington, DC: US Department of Commerce. United States General Accounting Office (GAO). (1990). Rural drug abuse: prevalence, relation to crime, and programs. Washington, DC: United States General Accounting Office. Woodward, J. A., Retka, R. L., & Ng, L. (1984). Construct validity of heroin abuse estimators. International Journal of the Addictions, 19 (1), Appendix: data definitions and sources of key indicators Mortality rates The Rhode Island Department of Health supplied electronic files of multiple-cause death certificate data for residents during 1993 to Nosologists at the National Center for Health Statistics employed software to code each cause of death according to the International Classification of Diseases, Ninth Edition (ICD-9). The authors extracted all records that included at least one cause-ofdeath code with explicit mention of alcohol or controlled drugs. Deaths due to suicides, assaults, and therapeutic overdoses were excluded, as were accidental overdoses due to antidepressants and other drugs that were unlikely to reflect nontherapeutic abuse. The explicit-mention drug codes included 292.0, 292.1, 292.2, 292.8, 292.9, , , The N poisoning codes included deaths associated with opiates (965.0), other specified analgesics (965.8), surface anesthetics (968.5), sedatives (967.0 to 967.9), other gaseous, intravenous, and surface anesthetics (968.2, 968.3, 968.5), tranquilizers (969.4, 969.5), hallucinogens (969.6), psychostimulants (N969.7), parasympatholytics (971.1), and dietetics (977.0). The relevant accidental or undetermined intent E codes included E850.0-E850.2, E850.8, E851, E852, E853.2, E853.8, E854.1, E854.2, E855.1, E855.2, E855.4, E858.8, E980.0, and E Deaths that lacked an N code but had an accidental E code for a drug of abuse were counted. Deaths due to undetermined external causes involving opiates, cocaine, and dietetics were also counted as cases because the large majority of deaths associated with those

10 208 W.E. McAuliffe et al. / Journal of Substance Abuse Treatment 23 (2002) substances had accidental E codes in national mortality data. Finally, deaths with a poisoning N code, but no E code, were treated the same as a case that had an undetermined E code. The alcohol mortality counts included any case with one or more of the widely used twelve explicit-mention alcohol diagnoses (e.g., McAuliffe et al., 2000; Single, Robson, Rehm, & Xiaodi, 1999; Stinson, Proudfit, Harford, Dufour, & Noble, 1994; Stinson & Nephew, 1996). The codes were 291, 303, 305.0, 357.5, 425.5, 535.3, 571.0, 571.1, 571.2, 571.3, 790.3, E860.0, and E Accidental poisoning cases were those with an N code (N980.0) and an accidental E code (E860.0 or E860.1), an accidental E code (E860.0 or E860.1) but no N code, and an alcohol poisoning N code (N980.0) but no E code or an undetermined E code (E980.9). Excluded were deaths with an E code specifying suicide or assault, and deaths with an undetermined E code (E980.9) but no N code. UCR drug- and alcohol-defined arrest rates The drug abuse arrest statistics (possession and sale/ manufacturing) count only those cases in which the drug offense was the most serious charge (GAO, 1990). Alcohol-defined arrests were not available before The authors imputed the liquor-law violation and drug arrest rates in two resort towns (New Shoreham and Newport) from off-season months only. Review of monthly arrest statistics indicated that there was a much sharper increase during the summer months in those towns than in other towns. In addition, the authors imputed the missing arrest rates for one town from its mortality and hospital discharge variables. Examination of the alcohol-defined arrest rates led to selection of two of them as the alcohol arrest indicators for the present study. Disorderly-conduct and liquor-lawviolation arrest rates correlated significantly positively with each other (.60) and with the rates of alcohol deaths, hospital discharges and treatment admissions. Although DUI and drunkenness arrest rates correlated significantly with each other (.49), they failed to correlate substantially or significantly with rates of liquor-law violations and disorderly-conduct arrests or with other explicit-mention alcohol indicators. Several of the correlations between DUI and drunkenness arrest rates and the other explicitmention alcohol indicators were negative. Public drunkenness is generally considered a symptom of disease rather than a crime, and the offense is reported by only a few towns. The DUI arrests may be especially vulnerable to bias introduced by the UCR s procedure of reporting arrest statistics where the offense occurred rather than where the arrestee lives. Based on this analysis and the empirical evidence of validity of liquor-law violation rates and disorderly-conduct arrest rates in Rhode Island, the study combined them to create the study s alcohol arrest measure. Hospital discharge data To create hospital discharge need indicators, the authors extracted all cases with one or more of the explicit-mention alcohol or drug diagnoses described above for mortality data and for which the town of residence field had a Rhode Island town identified. The study separated all discharge records with procedure codes for alcohol or drug detoxification, rehabilitation, and counseling ( , NIDA, 1998, pp ), as well as cases that lacked procedure codes (e.g., 41% of the discharges in 1993), but had primary diagnoses of alcohol or drug psychoses, dependence or abuse and had Diagnostic Related Group (DRG) codes for detoxification and rehabilitation, with or without complications ( , NIDA, 1998, pp ). These cases were used to create the hospital drug and alcohol treatment variables. State substance abuse treatment admission statistics Rhode Island s substance abuse treatment admission statistics for calendar years came from the Division of Substance Abuse, Rhode Island Department of Mental Health, Retardation, and Hospitals. The data come from all programs that are licensed and funded by the Division. Hospital-based treatment is not included because the State does not fund hospital treatment, and the Division does not license hospital treatment units. The data list the primary, secondary, and tertiary substances for each admission. The records are identified by an anonymous client identification number. Cases were excluded if the town of residence was missing or not a Rhode Island town. Population base of rates In this study, the authors calculated weighted mean rates using the estimated populations for each year obtained from the Rhode Island Economic Development Corporation and from the U.S. Census Bureau s web site at census.gov/population/estimates/metro-city/scful. Other demographic statistics such as % foreign born, homeless, and in prison were from 1990 Census data (U.S. Bureau of the Census, 1993). The study used the entire population as the base of rates instead of the alcohol- and drug-using population because the primary interest was to measure the burden of substance abuse on the state s cities and towns. Other things held constant, towns with large elderly populations have lower rates of abuse than towns with small elderly populations. If the elderly population were removed from the denominator, the resulting rate would overestimate the burden of substance use disorders on the town (e.g., costs for treatment services). Age structure is a relevant cause of the variations in the rates of substance use disorder treatment needs. The goal of the present investigation was to estimate the full magnitude of these variations rather than only the variations that do not stem from town differences in age structure.