Using the Ohio Scales for Family Engagement, Treatment Planning, and Outcomes Tracking

Transcription

1 Using the Ohio Scales for Family Engagement, Treatment Planning, and Outcomes Tracking Ben Ogles Ohio University with Charlotte Williams Family Advocate Thanks To Office of Program Evaluation & Research, Ohio Department of Mental Health Southern Consortium for Children Participating agencies Participating families Multiple graduate students If providers don t t measure outcomes, someone else will - M Naditch (1994) 1

2 Overview Outcomes and Accountability ODMH Consumer Outcomes System and The Ohio Scales Using the Ohio Scales OUTCOMES! A Central Focus over the last 15 years Behavioral health, education, healthcare Books, articles, task forces Public and private Administrators, providers, consumers, and payors AN EXAMPLE FROM HEALTH CARE Programs focused on outcomes research, measurement, and management have emerged over the past decade as a result of rapidly rising healthcare costs, questions about the effectiveness of medical intervention, and the need for efficient delivery of care (Mowinski( & Staggers, 1997) -Maloney & Chaiken,,

3 Use Research Based Instruments to Assess Outcomes in Clinical Settings To improve treatment To enhance science To provide accountability To be ethically responsible for quality (Barlow, 1982; Ogles et al, 2002) Ohio s s Public Sector Response Core, common outcome battery for public mental health services Implemented state-wide Move toward outcomes driven management Reduce process measurement Review and Improve the outcomes system ODMH Youth Outcomes Ohio Scales (Parent) Ohio Scales (Youth ages 12-18) 18) Ohio Scales (Agency Worker) 3

4 Desired Characteristics Multiple sources (parent, youth, worker) Multiple content areas (problems, functioning, satisfaction, hopefulness) Practical (cost, time, clinical use) Psychometrically rigorous (reliable, valid, sensitive to change) Final Parent Rated Ohio Scales Problem Severity - 20 Items Functioning - 20 items Hopefulness - 4 items Satisfaction - 4 items Final Youth Rated Ohio Scales Problem Severity - 20 Items Functioning - 20 items Hopefulness - 4 items Satisfaction - 4 items 4

5 Final Agency Worker Rated Ohio Scales Problem Severity - 20 Items Functioning - 20 items Markers (eg, arrests, suspensions) ROLES (Hawkins et al, 1992) Psychometric properties Ogles, B M, Dowell, K, Hatfield, D, Melendez, G, & Carlston,, D (2004) The Ohio Scales In M E Maruish (Ed), The use of psychological testing for treatment planning and outcome assessment (3rd ed, Vol 2) (pp ) 304) Hillsdale, New Jersey: Lawrence Earlbaum Ogles,, B M, Melendez,, G, Davis, D C, & Lunnen,, K M (2001) The Ohio Scales: Practical Outcome Assessment Journal of Child and Family Studies, 10,, Technical Manual - Administration Protocols available at ODMH Web Site General guidelines parents/primary caregivers and youth 12+ administer prior to treatment, periodically, and termination provide help with reading as necessary agency worker following intake and periodically at the same time as parent and youth See also the User s s Manual - 5

6 Administration Family Engagement Give a brief, simple description of the measures Make sure they know it will be used to create the intervention plan and to assess progress Ensure them their point of view on the scales and in-person - will inform treatment Scoring Add up items for each of the 4 scales Missing data? Ask respondent about items that are left blank 4 or fewer blank - use Tables to get the total If more than four blank, no total score No reverse scored items ROLES instructions in the User s s Manual See example case (George J) Sum of completed items Number of Missing Items

7 15 Feeling lonely and having no friends Feeling anxious or fearful Worrying that something bad is going to happen Feeling sad or depressed Nightmares Eating problems (Add ratings together) Total 13 Copyright Benjamin M Ogles & Southern Consortium for Children Initial Assessment Development of Treatment Plan Critical items Identify target problems Identify functional strengths Charting initial scores - severity Comparison of parent and child All can be used to engage the family Critical Items Parent or youth report of - #7 drug or alcohol use #8 breaking the law #12 hurting self #13 talking or thinking about death 7

8 7 Using drugs or alcohol Breaking rules or breaking the law (out past curfew, stealing) Skipping school or classes Lying Can t seem to sit still, having too much energy Hurting self (cutting or scratching self, taking pills) Talking or thinking about death Critical Items - Family Engagement Dialogue with parent about any items that are endorsed Consider how informed the parent is about youth slang for these issues Consider parental stress level Consider amount of time parent is with the youth Be sensitive in your approach to this topic Target Problems Identify 3 or 4 problems rated the highest by the parent and youth Theme or focus Specific target item This may help to focus an interview or reveal nuances of consumer perceptions following an interview Useful for initial treatment planning 8

9 Target Problems Family Engagement Solicit input about issues that are not on the forms Emphasize uniqueness of families Use standardized form, but allow families to rank issues in terms of what is critical to them Functional Strengths Identify functioning items rated as 3 or 4 Identify problem items rated 0 May help to locate strengths that can facilitate change Useful for initial treatment planning Can incorporate resilience research Resilience A A set of qualities that foster a process of successful adaptation and transformation despite risk and adversity (Benard,, 1995) Protective buffers appear to make a more profound impact on the life course of individuals who grow up and overcome adversity than do specific risk factors (Werner, 1996) 9

10 Resilience Qualities of the Child Social Competence responsiveness, flexibility, empathy, communication skills, humor Problem Solving Skills ability to plan, resourceful, creative thinking (reflective) Critical Consciousness reflective awareness of oppression and creative overcoming Autonomy independence, task mastery, control, self-efficacy efficacy Sense of purpose optimism, persistence, hopefulness, goal directed (Benard,, 1995) Some External Factors that Contribute to Resilience Good intellectual functioning Economic resources Caring Relationships High Expectations (authoritative parenting) Opportunities to Participate (school, prosocial organizations) (Masten & Coatsworth,, 1998) Example: Some Ohio Scales Items that provide information about Resilience Social competence -> > Functioning items 1-4, 1 20 Autonomy -> > Functioning items 14, 18, 19 Sense of purpose -> > Hopefulness Scale Clinicians could identify others based on their experience 10

11 Strengths Family Engagement Recognize things have been difficult leading to treatment and the parent may have trouble identifying strengths Help them to find strengths in what they may consider to be routine Lack of opportunity may prevent some abilities to surface Initial Severity Chart scores using figures in the User s Manual (reproduce as needed) Clinical cutoff Problem Severity 20 Functioning 51 (60 for youth) Internalizing/Externalizing Pattern Level of Severity via Calibration Level of Severity CAFAS Ohio Scales Functioning Ohio Scales Problem Severity 0 - none mild moderate severe extreme 140+ <

12 Initial Severity Family Engagement Remind parent the total score comes from their ratings of their child (Validate) Verify that their perception matches the total score level of severity Work with the parent to identify appropriate types and intensities of treatment Compare Sources Compare parent and youth Internalizing youth tend to rate higher Externalizing parents tend to rate higher Compare parent and youth with agency worker Situational issues (court referral) 500 Discrepancy on Problem Severity 400 # of cases Std Dev = 1498 Mean = 26 N = Parent-Youth Difference 12

13 Factors associated with Agreement Gender (better total agreement with daughters than sons; more item agreement with sons) Rater Relationship (parents > others) Ethnicity (Hispanic > Caucasian, multi-ethnic > African American) Area of assessment (internalizing > externalizing & conduct ) Diagnosis (internalizing > externalizing; reverse at item level) (Carlston,, 2003) Discrepancy and Change Bigger discrepancy -> > increased dropout Bigger discrepancy (with parent rating higher) -> > poorer outcome (Carlston,, 2003) Comparing Scores Family Engagement Dialogue with parent to pin point the reasons for large discrepancies This may also be an opportunity to gather information about things the parent has tried in the past (including past treatment) 13

14 Use Initial Data to Inform the In- Person Clinical Assessment Look at critical items, strengths, problems, level of severity, discrepancy, internalizing-externalizing externalizing pattern to inform the initial assessment through: Collateral contacts, Interview with child, Meeting with family, etc Using Initial Data to Empower the Family in Subsequent Contacts Verify parent and youth report in face-to to-face contacts Expand on strengths Clarify goals and focus of services Consider the level of parent and youth hope Describe the use of scales for tracking change and informing the family Discuss differences in parent and youth ratings Rate Worker Scale Once you have sufficient information to begin an individual behavioral plan, then rate the worker version of the scale The worker rating should not be a rote replication of the parent form It should be informed by, but independent of, family ratings 14

15 Review the George J Summary Form Summary form is available to guide the use of the Ohio Scales in the initial treatment planning process See Handout Roberta R Practice Case See Handouts Section Summary Outcome is here to stay Ohio has a core battery that will help for children with mental health needs it is not perfect, but it is a good start The data can be used for the initial assessment to inform treatment planning and for family engagement and empowerment 15

16 Using the Ohio Scales to Improve Quality Individual clinicians develop tracking protocols for use with each client Supervision of the individual clinician with the individual client Aggregate data for management and marketing Tracking Change (Outcome monitoring) Items Total scores Compare scales (problems/functioning) Clinical significance Recovery Improvement Individual Items Can focus on target problems Simulates the Target Complaints measures used in many outcome studies Example using the Treatment for Depression Collaborative Research Program (TDCRP) study 16

17 Item Change Family Engagement Explain that tracking scores on an item is like tracking daily blood pressure or blood sugar levels and that it can be used to see progress Gives parents a frame of reference and helps them to be more attentive to the things that are being tracked Total Scores Primary use of the scales to track individual change using the total scores Plot total problem severity and functioning over time to monitor and evaluate progress Total Score Change Family Engagement Share the change information with the family Provide a simple explanation of the graph Verify parent perspective of change is consistent with the graph 17

18 Comparison of scales Theories of change Howard s s phase model of therapy remoralization (hope) remediation (problems/symptoms) rehabilitation (functioning) Scale Change Family Engagement Be realistic and help the family to be realistic about what can be accomplished At the same time, let the family define their success Clinical Significance Improvement (amount of change) Recovery (end point of change) Both - Clinical Significance (Jacobson & Truax,, 1991) 18

19 Movement into the Functional Distribution Graphic Representation Posttreatment Problem Severity RELIABLE DETERIORATION *D *B *A RELIABLE IMPROVEMENT *C CLINICAL SIGNIFICANCE *E Pre-treatment Problem Severity Clinical Change Family Engagement Very Technical find a way to explain in simple terms Don t t forget that what is meaningful change to the parent may not match with the statistical definition 19

20 Use Outcome Data to Empower the Family in Subsequent Contacts Verify parent and youth outcome report in face- to-face contacts Use progress to leverage additional progress Clarify goals and focus of services when additional progress is needed Consider altering treatment in collaboration with the family when encountering deterioration Help identify when treatment might end Maintain hope Don t t have to wait to re-evaluate evaluate or change the plan Satisfaction- Family Engagement Remember to visit with the family about their satisfaction with treatment and their perception about inclusion in treatment planning Services are not dependent on feedback Watch for changes in satisfaction Review the George J Summary Form Summary form to provide protocol for using the Ohio Scales to monitor change 20

21 Case Roberta R Complete the form using the case Section Summary Outcome implementation provides opportunities to move beyond measuring to monitoring or even managing the quality of mental health services Outcome data can be integrated into routine clinical practice for tracking change client-by by-client Outcome Management Clinical Supervision Aggregate Analyses Satisfaction Client Characteristics, treatment factors Reporting/Marketing/Accountability Provider Profiling 21

22 Clinical Supervision The Missing Piece Client outcome data is rarely considered when studying supervision (usual focus is on the supervisee/trainee) Client outcome data (other than informal testimonials or verbal report) is rarely included in the process of clinical supervision The impact of clinical supervision on client outcome is considered by many to be the acid test of the efficacy of supervision (Ellis & Ladany,, 1997, p 485) there is still virtually no attention given to client change in this line of research (Watkins, 1995, p 668) Using Outcome Data as a Tool in Supervision Case Assignment Treatment Planning/family engagement Periodic review of cases Termination 22

23 Example for Supervision - Prioritizing cases Outcome data can be used to select cases for supervision review Aggregate Example 1 Review the graphs and select the 3 cases you would want to review first in a time- limited supervision session Supervision follow-up More detailed information available for using ODMH consumer outcomes in supervision is available at the ODMH web site as part of the Outcomes Tool kit Aggregate - Satisfaction Can examine the four satisfaction items individually or in aggregate form Does not provide the comprehensive information many agency satisfaction surveys collect (only 4 items) Global, relevant, and from many consumers Aggregate Example 2 23

24 Aggregate - Clinical Average totals across an agency, program, or clinician to report rates of change Compare groups of consumers (eg, by residence, service, referral source) Research oriented or administratively oriented questions Aggregate Example 3 Aggregate - Reporting Outcome reports can be used for marketing or accountability purposes (eg, creating public value Moore, 1995) Aggregate Example 4 Aggregate- Provider Profiling Can also aggregate data by provider/clinician Useful for supervision and training Clinicians may be reluctant 24

25 Example 5 Provider profiling %Recovered Number of Sessions Training Clinic Dr L J (Lambert, Okiishi, Finch, & Johnson, 1998) Example 6 Comparing therapists Whole Center Therapist #16 Therapist # (Okiishi, Lambert, & Ogles, 2003) Section Summary Outcome data may also be useful for clinical supervision Aggregate data may be used to identify client characteristics, programs, or providers that may need focus 25

26 Additional Resources My Web Site oakcatsohiouedu/~ogles/ My Book Ogles, B, Lambert, M, & Fields, S (2002) Essentials of Outcome Assessment New York: John Wiley Ohio Dept of Mental Health Web Site 26