Using Washington Circle Measures in State Performance Management Deborah Garnick Andrea Acevedo Margaret Lee Constance Horgan State Systems Development Conference August 20, 2008 Institute for Behavioral Health, Heller School for Social Policy and Management, Brandeis University Supported by the Substance Abuse and Mental Health Services Administration (SAMHSA), the core funds of the Brandeis/Harvard NIDA Center on Managed Care and Drug Abuse Treatment (Grant #3 P50 DA010233), and contributed time from participating states.
OVERVIEW Background of Washington Circle (WC) Conceptual model Measures Data Requirements Definitions Pilot test results Current measure development Relationship to other initiatives Conclusions 08/20/08 2
BACKGROUND Clients entering SA treatment may not receive recommended services Performance measures are key for accountability and quality improvement Process measures are focused on providing the right services at the right time Strong research evidence on association of longer treatment and improved outcomes 08/20/08 3
CONCEPTUAL MODEL WC PERFORMANCE MEASURES (PROCESS) Initiation Engagement Continuity Ongoing treatment? OUTCOMES Substance Use Healthcare Utilization Employment Criminal Activity Family/Social Client Attributes Facility Attributes Community Attributes 08/20/08 4
BACKGROUND Convened in 1998 by SAMHSA s Center for Substance Abuse Treatment (McCorry et al., 2000) Goals: Develop and pilot test performance measures Promote adoption of these measures (McCorry et al., 2000) Began with specification for commercial managed care plans (Garnick et al., 2002) Used by: National Committee on Quality Assurance Veterans Health Administration States 08/20/08 5
BACKGROUND 2004 Formation of WC Public Sector Workgroup Ongoing SAMHSA support through the Brandeis/Harvard National Institute on Drug Abuse (NIDA) Center Goals: Improve delivery of substance abuse treatment services in public sector Adapt WC performance measures for states use in continuous quality monitoring Develop common approach among states 08/20/08 6
WC PUBLIC SECTOR WORKGROUP PARTICIPANT STATES WA VT MA NY CT AZ KS OK TN NC MD Arizona, Connecticut, Kansas, Maryland, Massachusetts, New York, North Carolina, Oklahoma, Tennessee, Vermont, Washington also Veterans Health Administration 08/20/08 7
TRANSLATING WC MEASURES FOR PUBLIC SECTOR APPLICATIONS OPPORTUNITIES AND CHALLENGES Opportunities Importance of public sector Current interest in performance measurement Details available about types of services Measurement Challenges No enrolled population Data completeness influenced by some clients multiple funding sources Variability in states data reporting capabilities 08/20/08 8
APPROACH Expanded original WC measures for initiation and engagement into nine measures Technical specifications publicly available 7 states pilot tested in 2005 1 state, adolescent data only 1 state, used admission/discharge data Iterative process of refining specifications based on states analyses 08/20/08 9
REVISIONS TO SPECIFICTIONS EXPANDED TO NINE MEASURES 1. Initiation after Outpatient 2. Engagement after Outpatient 3. Initiation after Intensive Outpatient 4. Engagement after Intensive Outpatient 5. Continuity of care after Assessment Service 6. Continuity of care after Detoxification 7. Continuity of care after Short-term Residential 8. Continuity of care after Long-term Residential 9. Continuity of care after Inpatient 08/20/08 10
DATA REQUIREMENTS To calculate measures Date of treatment service Type of treatment service (e.g., outpatient) Client ID Provider ID To use for performance management Client characteristics (e.g., gender, race/ethnicity Provider characteristics (e.g., location, ownership) 08/20/08 11
INITIATION MEASURE DEFINITION Initiation = Individuals with an OP index* service who received a second service** within 14 days after the index service Individuals with an OP index service *Index service defined as first service after a 60-day service-free period. Can have assessment or detox during service-free period. **Not detox or crisis care Note: Same formula used for Intensive Outpatient services 08/20/08 12
ENGAGEMENT MEASURE DEFINITION Engagement = Individuals who initiated OP treatment and received two additional services** within 30 days after initiation Individuals with an OP index* service *Index service defined as first service after a 60-day service-free period. Can have assessment or detox during service-free period. **Not detox or crisis care Note: Same formula used for Intensive Outpatient services 08/20/08 13
EXAMPLE 1: NO INITIATION OP OP OP Day 1* Day 30 Day 33 Days *Start of new episode 08/20/08 14
EXAMPLE 2: INITIATION ONLY OP OP OP Day 1* Day 3 Day 35 OP Day 60 Days *Start of new episode 08/20/08 15
EXAMPLE 3: INITIATION AND ENGAGEMENT OP OP OP OP Day 1* Day 3 Day 20 Day 23 Days *Start of new episode 08/20/08 16
OUTPATIENT INITIATION AND ENGAGEMENT, 2005 100% Initiation Engagement 72% 67% 61% 56% 73% 64% 42% 42% 27% 24% 0% 15,795 22,822 33,031 10,115 4,176 CT MA NC OK WA Notes: CT s data include OP and IOP. Numbers below bars are denominators for measures. 08/20/08 17
INTENSIVE OUTPATIENT INITIATION AND ENGAGEMENT, 2005 100% 88% 76% 80% 75% Initiation Engagement 44% 34% 0% 3,163 1,171 3,262 MA NC WA Note: Numbers below bars are denominators for measures. 08/20/08 18
WHAT IS THE SENSITIVITY OF THE 14-DAY SPECIFICATIONS? - OUTPATIENT INITIATION EXAMPLES 40% 30% 20% 10 % 0% 40% 30% 20% 10 % 0% 5% 4% 1% 1% 2% 2% 3% 12% Massachusetts 5% 2% 1% 1% 1% 1% 1% 1% 1% 1% 0% 1% 1% 3% 4% 1% 0 7% 5% 3% 3% 3% 8% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 4% 5% 8% 2% 2% 4% 2% 26% 19% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 0 2 1-2 42 5-2 93 0-3 4 3 5 +No ne x t North Carolina 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 0 2 1-2 42 5-2 93 0-3 4 3 5 +No ne x t se r v i c e 34% se r v i c e 40% 30% 20% 10 % Washington 14% 8% 4% 7% 14% 12% 6% 5% 4% 0% 2% 1% 1% 2% 2% 4% 1% 1% 1% 1% 1% 1% 3% 3% 1% 2% 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21-24 25-29 30-34 35+ No next service 08/20/08 19
CONTINUITY MEASURE DEFINITIONS Detox = Individuals with a detox service who received another service* within 14 days after discharge Individuals with a detox service *Not detox or crisis care 08/20/08 20
MULTIPLE DETOXIFICATIONS IN SHORT TIME FRAME Issue which of multiple detoxifications within a short time frame should be used Solution State selects method Method 1 - Group multiple detox that occur within a short period of a few days as one service. Look for another service within 14 days of the discharge from the last detox in the string of multiple detox. Method 2 Each detox service is viewed as a separate service if there is any gap of days between services. 08/20/08 21
CONTINUITY MEASURE DEFINITIONS (Cont d) Residential= Individuals who had a Residential service that was followed by another service* within 14 days after discharge Individuals discharged from a Residential stay *Not detox or crisis care Note: Calculations made separately for short-term residential, long-term residential, and inpatient services 08/20/08 22
CONTINUITY OF CARE AFTER DETOX, 2005 100% 59% 40% 40% 19% 23% 0% 12,103 52,321 CT MA NC OK WA Note: Numbers below bars are denominators for measures. 7,864 4,828 11,882 08/20/08 23
CONTINUITY OF CARE AFTER RESIDENTIAL, 2005* 100% 9 60% 47% 38% 23% 17% 37% 15% 30% 0% 7,565 3,626 1,363 7,249 6,336 2,618 622 2,655 MA OK WA CT MA NC OK WA Short-term residential Long-term residential Note: Numbers below bars are denominators for measures. *CT and NC cannot differentiate between LTR and STR; all calculated as LTR. 08/20/08 24
ONGOING MEASURE DEVELOPMENT 2008 Retention in treatment through 90 days Ongoing Analyses Distribution of length of treatment by level of care Patterns of treatment (e.g., services without a 30-day gap) Variation across client factors (age, gender, race/ethnicity, reported alcohol and/or drug problems) within level of care 08/20/08 25
RETENTION IN TREATMENT Results from two initial states: Among clients who initiate treatment approximately: 50% have services in 30 days after initiation, 11-30% in next 31-60 days, and 7-20% in next 61-90 days. Among clients who engage approximately: 95% have services in 30 days after engagement, 80% in next 31-60 days, and 70% in next 61-90 days. 08/20/08 26
RELATIONSHIP TO OTHER INITIATIVES: NATIONAL OUTCOME MEASURES (NOMs) Direct Improved engagement rates and retention domain Indirect Research evidence on association of WC measures and outcomes Decreased criminal justice involvement after engagement in outpatient treatment (Garnick et al 2007) Increased abstinence after continuity of care for adolescents in residential treatment (Garner et al 2008) 08/20/08 27
RELATIONSHIP TO OTHER INITIATIVES: PAY-FOR-PERFORMANCE Potential performance measures as states plan P-4-P Engagement concept included in Delaware incentive payment system Other states in planning stages 08/20/08 28
RELATIONSHIP TO OTHER INITIATIVES: QUALITY IMPROVEMENT Similar to NIATx and STAR-SI measures State initiatives Visual displays Aggregation by provider, level of care, client characteristics Timely feedback quarterly data Transparent information on data and methods Provider involvement in interpretation 08/20/08 29
CONCLUSIONS Measure key points when actions can be taken to engage and retain clients Feasible to calculate from routinely available information in some states Useful at centralized (State) and decentralized (County, provider) levels Relate to other initiatives National Outcome Measures (NOMs) Pay-for-performance Quality improvement 08/20/08 30
ADDITIONAL INFORMATION - PUBLICATIONS Garnick, Lee, Horgan, Acevedo, and the Washington Circle Public Sector Workgroup. (forthcoming 2008). Adapting Washington Circle Performance Measures for Public Sector Substance Abuse Treatment Systems. Journal of Substance Abuse Treatment. McCorry, Garnick, Bartlett, Cotter, Chalk (2000). Developing performance measures for alcohol and other drug services in managed care plans. Washington Circle Group. The Joint Committee Journal of Quality Improvement, 26(11): 633-643. Garnick, Lee, Chalk, Gastfriend, Horgan, McCorry, et al. (2002). Establishing the feasibility of performance measures for alcohol and other drugs. Journal of Substance Abuse Treatment, 23(4), 375-385. Garnick, Horgan, Lee, Panas, Ritter, Davis, Leeper, Moore, Reynolds (2007). Are Washington Circle performance measures associated with decreased criminal activity following treatment? Journal of Substance Abuse Treatment, 33(4):341-352. 08/20/08 31
ADDITIONAL INFORMATION - PUBLICATIONS Lee, Garnick, Miller, Horgan (2004). Datapoints: Adolescents with substance abuse: Are health plans missing them? Psychiatric Services, 55(2):116. Garnick, Horgan, & Chalk (2006). Performance measures for alcohol and other drug services. Alcohol Research and Health, 29(1), 19-26. Campbell, K. (forthcoming 2008). Impact of Record Linkage Methodology on Performance Indicators and Multivariate Relationships. Journal of Substance Abuse Treatment. Jones, D. L., Heflinger, C. A., & Saunders, R. C. (2007). The ecology of adolescent substance abuse service utilization. American Journal of Community Psychology, 40(3-4), 345-358. McLellan, A. T., Kemp, J., Brooks, A., & Carise, D. (2008). Improving public addiction treatment through performance contracting: The Delaware Experiment. Health Policy. Harris, A. H., Humphreys, K., & Finney, J. W. (2007). Veterans Affairs facility performance on Washington Circle indicators and casemixadjusted effectiveness. Journal of Substance Abuse Treatment, 33(4), 333-339. 08/20/08 32
ADDITIONAL INFORMATION - WEBSITES The Washington Circle http://www.washingtoncircle.org SAMHSA/National Outcome Measures http://nationaloutcomemeasures.samhsa.gov/ Brandeis/Harvard Center on Managed Care and Drug Abuse Treatment http://nidacenter.brandeis.edu Oklahoma Department of Mental Health and Substance Abuse Services http://www.odmhsas.org North Carolina Department of Health and Human Services http://www.ncdhhs.gov/mhddsas/statspublications/reports National Committee on Quality Assurance http://www.ncqa.org 08/20/08 33