Introducing the National Core Indicators for Aging and Disability Services

Transcription

1 Introducing the National Core Indicators for Aging and Disability Services

2 2 NCI-AD: Why and What

3 Introduction 3 Joint project of the National Association of States United for Aging and Disabilities (NASUAD), Human Services Research Institute (HSRI), and the National Association of State Directors of Developmental Disabilities Services (NASDDDS) Currently exist for the Intellectual/Developmental Disabilities community. (NCI) The Core Indicators are standard measures used across states (and soon programs) to assess the outcomes of services provided to individuals and families. NASUAD is expanding the tool to include populations of older adults and people with physical disabilities. (NCI-AD)

4 NCI-AD Background 4 In 2011, NASUAD s Board voted to begin work to expand the scope of the current NCI to include older adults and adults with physical disabilities receiving services in their state. Grew out of a concern about the limited information currently available to help states assess the quality of LTSS services for seniors, adults with physical disabilities, and their caregivers. NASUAD, with support from their Steering Committee, began working with HSRI and NASDDDS to expand NCI to include this new focus. Began with the revision of the in-person Consumer Survey. Currently piloting in Georgia, Minnesota, and Ohio.

5 Project Timeline NASUAD s Board approved the NCI-AD expansion in May 2013 NCI-AD Steering Committee was formed began work with NASUAD, HSRI, and NASDDDS revise the NCI adult consumer survey to focus on older adults and individuals with physical disabilities June 2013 Meeting to revise the core domains and indicators September 2013 Meeting to discuss the first draft of the survey October 2013 Steering Committee and stakeholder feedback on the second draft of the survey November 2013 Steering Committee, focus group, and stakeholder feedback on the third draft of the survey December/January 2014 Begin pilot in Georgia, Minnesota, and Ohio

6 Purpose of NCI-AD 6 Support state agencies interested in measuring the performance of public long-term service and support systems. Allow for state to state comparisons and service comparisons within states. Provide data at the systems level on outcomes. Assist states to improve the quality of their long-term services and supports. Document the effect of services on the day-to-day lives of the people who receive them. Document the experience of program participants. Track key performance goals and outcomes. Assesse the impact of regulatory activities and financial actions on individual experience. Respond to the demands of consumers and families for information on system responsiveness. Provide rapid access to quality data.

7 What is being Measured? 7 Consumer Outcomes: Community Participation Choice and Decision-making Service and Care Coordination Rights and Respect Health, Safety, and Wellness Relationships Everyday Living and Affordability Work/Employment Self-Direction

8 How does NCI-AD Work? 8 Participating states are asked to compile a sample of at least 400 older adults and individuals with physical disabilities receiving publicly-funded services. States work with HSRI and NASUAD to design the sampling strategy. State or their contractor then conducts in-person NCI- AD interviews with survey participants. State compiles final data and shares data with HSRI. HSRI interprets data and provides state-specific report and state-to-state comparison report.

9 9 National Core Indicators

10 NCI Background 10 NCI is a performance measurement system that enable states to make policy and funding decisions to support practices that work for people. Launched in 1997 in 13 participating states 40 states (including D.C.) and 22 sub-state regions and counties participate Data collected annually on 12,000-20,000 people Unparalleled 17-year database Collaboration between NASDDDS HSRI participating state DD agencies Recent support from AIDD (ACL)

11 11 NCI Background - Participation

12 NCI Unique View 12 Person-centered Individual characteristics of people receiving services The locations where people live The activities they engage in during the day including whether they are working The nature of their experiences with the supports that they receive (e.g., with case managers, ability to make choices, self-direction) The context of their lives friends, community involvement, safety Health and well-being, access to healthcare Family outcomes

13 NCI Data Sources 13 Consumer Survey Random sample of adults In person interviews Family Survey (mail-in) Adult Family Survey (at home, 18+) Family Guardian Survey (out-of-home) Children Family Survey (at home, <18)

14 NCI - Design 14 One overarching framework: Training Sampling Technical assistance Online data entry system (ODESA) Analysis and reporting Base surveys but states can add own questions Valid and reliable: Ongoing face and content validity evaluations Revisions Ongoing inter-rater reliability evaluations Shadowing Internal consistency

15 NCI - Usage 15 Quality Assurance and Service Improvement Meet CMS Waiver Reporting Requirements Compare performance to other states (data are public) Public Accountability and transparency to elected officials, stakeholders and the public

16 NCI - Website 16 Annual Data Reports State Summary Reports Data Briefs Articles National State Data Chart Generator Technical Reports

17 17 NCI AD: Survey Development and Pilot Process

18 Development Process 18 Start: NCI indicators Other tools Brainstorm Result: Approx. 120 indicators 17 domains

19 Development Process 19 June 2013 meeting with Steering Committee Discuss each potential indicator Long day Homework: Rankings Rank each indicator from 0 (not important) to 3 (critical) 12 states - sets of rankings

20 Development Process 20 Delphi method Summary rankings: Weighted score for each indicator: (N of 0s)*0 + (N of 1s)*1 + (N of 2s)*2 + (N of 3s)*3 = Total score Possible range: 0 to 36 Observed range: 10 to 35 Mean: 20.9, Median: 21 Number of indicators cut by half: 1) Above median 2) Different look: exclude total score less than 16, no 3s and fewer than 8 2s, total number of 2s and 3s less than 8 Expert opinion Result: 61 indicators, 17 domains

21 Development Process indicators Draft survey questions Draft background section Risk-adjustment level the playing field Describing the population Focus/drill down into sub-populations Meeting with Steering Committee and Board on 9/7/2013 Discuss first draft

22 Development Process 22 2 Focus groups In-person testing Validity Cognitive testing Inter-rater reliability Total of 7 revisions After in-person testing last revision December 2013 final draft of NCI-AD Consumer Survey, version 1

23 Development Process - Pilot 23 3 pilot states: MN, GA, OH Recipients of aging services through HCBS Waiver Recipients of aging services through OAA Recipients of non-dd disability services through HCBS Waiver Each state to collect at least 400

24 Development Process - Pilot 24 MN: data collection completed GA, OH: data collection almost completed Inter-rater reliability Shadowing in GA Interviewer feedback

25 25 NCI AD: Pilot Experience

26 NCI-AD: Minnesota Pilot 26 Contracted with Vital Research Sample: 800 Waiver serving adults with physical disabilities Waiver serving older adults OAA home delivered meal recipients 355 Surveys Completed HCBS Partners Panel is primary vehicle for stakeholder engagement Lesson learned: need to start the internal communication and coordination efforts earlier

27 NCI-AD: Georgia Pilot 27 Contracted with Delmarva Foundation 1200 people included in the sample: 600 Home and Community Based Services (HCBS - Non-Medicaid) 600 Community Care Services Program (CCSP Medicaid) Approximately 800 will be completed Lessons learned: Notification letters don t always work bring them with Meet with external stakeholders (ADRCs, AAA, providers), notify them Providers are cautious People interviewed enthusiastic

28 NCI-AD: Ohio Pilot 28 Contracted with Vital Research Sample: People receiving one or more OAA services and care coordination People enrolled on a 1915(c) and receiving one or more waiver service and case management Process: Designated project manager to guide the vendor selection, initial data gathering, vendor training Cross agency collaboration (policy, program, and data) Lessons learned: Low refusal rate Interviewers experience: easy to engage consumers in conversation, logical progression Stakeholder engagement: early, often, and tailored to the interest of the audience; not just another survey ; identify shared goals of all stakeholder groups.

29 29 NCI AD: Final Survey Development

30 NCI-AD: Next Steps 30 Data analysis, risk-adjustment methodology by December 2014 Revisions, cognitive testing, reliability studies September 2014-March 2015 Inter-rater, Intra-rater, Test-retest, Construct validity, Internal consistency Final revision of survey by March 2015 Reporting methodology by March 2015 Regular data collection to begin June 2015 in states

31 Project Future NCI-AD and NCI: Will allow for cross-population and cross-service system comparisons Goal: all states Federal buy-in Capacity to expand to other populations Modify and adopt tools Retain framework

32 32 Nationwide Rollout Timeline and Costs to Participate

33 33 Timeline

34 34 Two Rounds of Data Collection

35 Final Process 35 At least 5 new states each year June 1 st May 31 st year long data collection 3-4 months for data cleaning and analysis Reports issued in November and December Aligns with NCI data collection July 1 st June 31 st data collection Reports issued in January

36 36 Data powered by HSRI Project managed by NASUAD Contacts: Julie Bershadsky Kelsey Walter