Minnesota Alternative Response Evaluation Second Annual Report. Executive Summary

Transcription

1 Section V. Evaluation Reports A. Minnesota Alternative Response Evaluation Minnesota Alternative Response Evaluation Second Annual Report Executive Summary conducted for the Minnesota Department of Human Services conducted by the Institute of Applied Research St. Louis, Missouri February 2003

2 Copyright 2003 by the Institute of Applied Research 111 N Taylor St. Louis, Missouri (314) iar@iarstl.org website: This document may be copied and transmitted freely. No deletions, additions or alterations of content are permitted without the express, written consent of the Institute of Applied Research. FIRST 5 Santa Clara County Section V Page 2 of 14

3 Alternative Response Evaluation Second Annual Report Executive Summary This is the second annual report of the evaluation being conducted of the Minnesota Alternative Response Project by the Institute of Applied Research. The Alternative Response (AR) Project is a child protection demonstration that provides a differential response to child maltreatment reports. The AR project builds on the work of prior initiatives that have sought to find more effective ways of applying a family-centered and family-friendly approach to resolving issues that bring families to the attention of the child protection system. The project is being generously supported by the McKnight Foundation along with federal, state and county funding. The Minnesota Alternative Response Project began during the latter half of 2000 and will operate as a demonstration program in 20 counties for a period of four years. Participating counties represent the diversity of the state and include large metro counties, fast-growing suburban counties, counties with mid-sized cities that are regional economic centers, and rural counties in different parts of greater Minnesota. A three-part evaluation of the project began in February 2001 and will continue through the end of the demonstration period. It includes impact, process and cost effectiveness studies. This second annual evaluation report provides a summary of process and outcome findings through the end of December Experimental Design, Data Sources and Study Population Monthly extracts of data in the Social Service Information System (SSIS) are provided to IAR and these data allow the tracking of child protection families in the 20 counties participating in the project. This tracking involves the longitudinal monitoring of outcomes associated with families who receive the alternative response. In addition, in 14 counties, a control group of families was selected as part of an experimentally designed impact study. In impact study counties, families with child maltreatment reports appropriate for the alternative response are randomly (although disproportionately) assigned to experimental or control study groups. Families in the experimental group receive the alternative response, while families in the control group receive the traditional response in place prior to the demonstration. The last monthly extract of SSIS data received prior to this report included the end of December, Through that date, a total of 8,318 families with accepted child maltreatment reports in the 20 demonstration counties had been determined to be appropriate for the alternative response. Among these families, FIRST 5 Santa Clara County Section V Page 3 of 14

4 6,158 were in the 14 counties participating in the impact portion of the study and were assigned to either the experimental group (57.4 percent) and received the alternative response or the control group (36.9 percent) and received the traditional response. Other data sources include feedback from families and child protection case workers. Through the end of 2002, primary caregivers in 909 families have provided feedback 270 through interviews and 639 in completed written questionnaires. Detailed case information has been obtained from county CPS workers on 490 families selected in the study sample. Detailed cost information associated with a sample of cases will be collected beginning in the spring of 2003 as part of the cost-effectiveness study. Regular site visits and staff interviews are being conducted throughout the demonstration period as part of the evaluation. Process Findings Screening. The percentage of child maltreatment reports that were screened appropriate for the alternative response rose during the second year of the project. Through the first two years, 36.7 percent of reports were screened for AR. Excluding Hennepin County, the largest county in the state and the one that has been most cautious in its screening, the percent rises to 47.5 percent. There remains considerable variation among counties in the percentage of reports judged appropriate for AR, which ranges from a low of 22 percent in Hennepin to 62 percent in Olmsted and Pope counties. Track Changes. The initial classification of a report as appropriate for the alternative response may be changed to a traditional response at any point while the family is in contact with the child protection system. This is most likely to happen during the assessment phase if safety concerns exceed what was anticipated based on the report or, in some counties, if the family is uncooperative. The frequency of track changes increased during the second year of the project from 3.9 percent at the end of the first year to 8.2 percent through the end of the second year. Changes in Practice: Interim Findings. A basic question in the evaluation is: Has child protection practice changed as a result of the demonstration? The assumption is that a change in practice is a precondition for changes in outcomes. The AR demonstration seeks to change practice in two ways: 1) approaching families as a unit and in a positive manner consistent with sound family-centered practice, focusing on the problems they may be experiencing and on their needs, and involving them in decision making about what to do, and 2) providing services and assistance that fit the needs and situations of families, linking them to other community resources when possible. Through the first two years of the demonstration, findings indicate that FIRST 5 Santa Clara County Section V Page 4 of 14

5 practice is being reshaped in both ways. A number of specific differences have been found in practice with the introduction of the alternative response that rise to the level of statistical significance. Compared with families that received the traditional approach, AR families have been more likely to report: Greater satisfaction with the way they were treated by child protection workers. That they were treated in a friendly manner. That they were more involved in decision making. That CPS workers tried to understand their situation and needs. That they experienced fewer negative feelings following the first visit from a county child protection worker. For example, they were less likely than families who received a traditional assessment to report that they were worried, stressed, confused, afraid or discouraged. That they experienced relief and reassurance following the initial visit from workers. That workers met with them on subsequent occasions in which their children or whole family were present. That workers helped them obtain services. That workers themselves provided direct assistance to families. That workers connected them to other community resources. Correspondingly, compared with the traditional approach to child protection, CPS workers that utilize AR have been more likely to report: That they had more contact with families. That they conducted interim and final assessments. That families were cooperative. That services and support were provided to the families. That the services provided were effective and matched to the needs of families. That services were provided across a broader spectrum of service areas. That families were linked to a broader set of community resources. That extended families were involved in providing support to the families. Reports from families and workers about the provision of services is supported by data extracted from SSIS. Through the first two years of the demonstration, SSIS data indicates that families who received the alternative response were more than twice as likely to have an ongoing case opened than families who received the traditional response. For most families, having a formal case opened is a precondition for receiving services, particularly funded services. FIRST 5 Santa Clara County Section V Page 5 of 14

6 Impact Findings The impact analysis was limited to 1,367 experimental and 961 control families whose cases had closed on or before May 31, This insured that at least six months had elapsed after case closure for each family in the analysis. Measuring the Risk of New Child Abuse and Neglect. Analysis of the Minnesota Structured Decision Making (SDM) Risk Assessment instrument was conducted for families screened as appropriate for AR. Risk assessments are conducted on initial visits to families. Risk assessment scores should be generally predictive of new reports of child maltreatment families with higher risk score should be reported more frequently than families with lower risk scores. This was found to be true of the SDM items associated with child neglect. The abuse items on the SDM risk assessment instrument were not predictive of new reports. Safety and Risk. Child safety and risk are different concepts. Safety refers to immediate dangers from which children must be protected. Risk is a measure of the likelihood of future maltreatment based on the characteristics and past behaviors of family members and on the situations of families. To illustrate the difference, over a third of substantiated reports in the control group were low-risk and about four out of ten unsubstantiated cases were moderate to high-risk. Variation in Risk among Study Counties. The counties in the evaluation varied substantially in their willingness to accept moderate- to high-risk cases for AR. This difference was related to the proportion of total reports each county accepted to AR. The more cases accepted, the more likely a county was accepting moderate- to high-risk cases for AR. Olmsted County, with the longest running AR program, accepted the most moderate- to high-risk families under the AR approach. Safety Problems Found. Excessive discipline was the most frequent type of child abuse and lack of supervision was the most frequent type of child neglect among AR-appropriate families. In a majority (54 percent) of families in the current sample no child safety problems were identified. No further work would have been done with many of the families in this category under traditional CPS because child maltreatment allegations would not have been substantiated. Changes in Child Safety. The primary question concerning changes in child safety was whether children in experimental families (who received the AR approach) were less safe than children in control families who received a traditional CPS investigation. While analyses generally showed greater safety improvement among families who received AR, differences were not statistically significant. At this point in the evaluation, children and families receiving AR are as safe as children in families receiving the traditional response. FIRST 5 Santa Clara County Section V Page 6 of 14

7 Service Orientation. The introduction of AR has brought significant shifts in the service orientation of local offices, including a significant increase in services to families, particularly for low-risk families and families in which no child safety problems are found: Increased Case Management under AR. Case-management workgroups are created when ongoing monitoring and services are thought to be necessary to protect children and to address family and individual needs that have been uncovered. Three times as many case-management workgroups were opened in experimental as in control cases, and nearly all case-management workgroups in control cases were opened only when the investigation had discovered child maltreatment. In addition, experimental and control families on average had virtually identical risk levels; yet, case-management workgroups were opened for the full spectrum of (neglect) risk levels among experimental families while only for the higher-risk families in the control group. Greater Case Management in Non-Impact Study Counties. Among families that received AR, greater proportions had case management initiated in the six counties not participating in the impact study, because as a whole they tended to be higher risk and include more threats to child safety. Casemanagement workgroups are opened more frequently on such families. Preventive Services Emphasized. More services of various types were delivered to experimental families who had received the AR approach. Increased levels of services addressing basic family needs (basic household needs, housing, rent assistance, transportation, training and employment) among a broader range of families. This suggests an increase in preventive services that address both short-term and long-term child protection needs, as well as general child and family welfare. New Reports of Child Maltreatment. No statistically significant differences were found in the level of new child maltreatment reports (after initial cases had closed) for experimental compared to control families. This mirrored earlier findings in the evaluation. No differences were apparent when taking into account case management openings and risk of neglect levels during the initial case. New Case-Management Workgroups Resulting From New Reports. Opening new case-management workgroups indicates that a new report on the family was received, and that after meeting with and assessing the family, workers decided that the safety of the child and/or the needs of the family warranted further monitoring and services. The rate of new case openings was significantly lower for low-risk AR cases compared to control cases. For every 100 low-risk cases provided a traditional response, current results indicate that about 21 could be expected to FIRST 5 Santa Clara County Section V Page 7 of 14

8 return to the system and have case management cases opened over about three years. On the other hand, about 12 of every 100 low-risk AR cases would be expected to return in the same way. This is consistent with the more intensive service response observed for low-risk families under AR. FIRST 5 Santa Clara County Section V Page 8 of 14

9 B. Florida s Infant and Young Children s Mental Health Statewide Pilot Project Program Evaluation Executive Summary 1 June 1, 2000 to June 25, 2003 < Dr. Sandra Adams, Project Director Infant Mental Health Pilot Project Dr. Joy Osofsky, Project Evaluator Louisiana State University Health Sciences Center Dr. Jill Hayes Hammer, Project Evaluator Louisiana State University Health Sciences Center Dr. Mimi Graham, Director Florida Start University Center for Prevention & Early Intervention Policy Overview of the Study This study, conducted by the FSU Center for Prevention and Early Intervention Policy, is an evaluation of a three-year, multi-site, infant mental health pilot, Florida s Infant and Young Children s Mental Health Statewide Pilot Project. The pilot project was initially funded in 2000 by the Florida Legislature and implemented in conjunction with Children's Mental Health in the Department of Children and Families (DCF). Three diverse sites were chosen: Miami (a collaborative project between the Dependency Division of the Juvenile Court, Eleventh Judicial Circuit, and the University of Miami s Linda Ray Intervention Center), Sarasota (Child Development Center) and Pensacola (Lakeview Community Mental Health Center). The purpose of the project was to provide a research-based model of dyadic therapy services with a sample of high-risk infants, toddlers and their families during the critical first few years of life in order to promote bonding and attachment, positive interactions, and secure relationships between the child and mother (or primary caregivers). Because children under the age of three are the fastest growing segment of children entering the foster care system, the population targeted for the pilot project were children under the age of three years who were at risk for out-of-home placement due to abuse or neglect or children who had already been placed in foster care but parental rights had not yet been terminated. Goals and Outcomes Five goals were addressed in the project with the following outcomes: Goal 1: To improve parent/caregiver and child interaction and relationships, reduce occurrence or reoccurrence of abuse and neglect, and enhance the child s developmental functioning. 1 Full report is available online at < FIRST 5 Santa Clara County Section V Page 9 of 14

10 Outcomes. There were four major outcomes related to this goal. First, there was reduction of child abuse and neglect. Reports of abuse/neglect were reduced from 97% of children prior to treatment to 0% of the children completing the pilot project. Secondly, there was reunification with the family or permanent placement for all children completing the pilot who were not in parental custody at the beginning of the project. Thirdly, the health and developmental status of children improved. Following treatment, 58% of children improved in their developmental functioning. And fourthly, the parent-child relationship functioning improved significantly in all domains for both parents and children. Parents showed an increase in behavioral and emotional responsiveness with their children and a decrease in intrusive behaviors. Children showed an increase in positive affect (emotions) and enthusiasm with their parents. The percentage of caregivers reporting depression reduced from 51% pretreatment to 29% following completion of treatment. Seventy percent (70%) of caregivers reported minimal to no depression after treatment. Goal 2: To document the components of quality infant mental health interventions and analyze them for replicability, sustainability, effectiveness, and affordability for potential use in a statewide system. Outcomes. Components of the assessment protocol are recommended for use but will require training and adequate reimbursement. The dyadic therapy model appears to be an effective short-term treatment approach for those dyads who completed their treatment program. Follow-up studies will be needed to determine long-term effectiveness. Engagement/case management/outreach activities provided by the treating therapist within the context of the therapeutic alliance with the parent were considered critical to completion of the treatment program and to the effectiveness of the intervention. Goal 3: To identify barriers and solutions for systemic changes in infant mental health. Outcomes. Several barriers were identified including lack of reimbursement for engagement services, problems with pre-authorization process, inadequate funding for children of working poor or children without insurance, and ineligibility of at-risk children for Part C early intervention services. Potential solutions were suggested. Goal 4: To develop model infant mental health treatment programs which can be replicated statewide. Outcomes. Assessment and treatment strategies used in this pilot project emphasize comprehensive, family-centered, developmentally appropriate intervention that focuses the needs and strengths of the child within home and community settings. Treatment is provided within the context of the child s primary relationships and focuses on the mom/baby dyad as a unit. Replication will require increasing awareness and training, as well as collaboration among agencies. Recommendations: Expansion of Medicaid billing (fee for service) is recommended to include engagement activities as defined by pilot project. FIRST 5 Santa Clara County Section V Page 10 of 14

11 Expand caps on number of individual/family therapy and in home-on site (ITOS) services for 0-5 population. Explore utilization/certification of pilot assessment to replace comprehensive assessments for 0-5 and/or create equitable levels of reimbursement for 0-5 indepth assessments. Create an Infant Mental Health Specialist training, credentialing, and supervision program on a statewide basis to develop a cadre of competent therapists for serving this population. Goal 5: To build capacity in the infant mental health field, especially in the areas of assessment and direct therapeutic interventions. Outcomes. Therapists for the three pilot projects have received training and have gained experience and expertise in assessment and treatment, particularly dyadic therapy. In addition, there has been a substantial increase in training opportunities around the state that have been a spin-off from the pilot project and the community mental health Medicaid policy for children 0-5. FSU Center for Prevention and Early Intervention has been awarded a Harris Foundation grant for Infant Mental Health, joining only 10 other institutes in the country recognized for excellence in training, thereby helping to create an infrastructure for infant mental health training in Florida. Policy Implications and Recommendations Recommendation #1: Intervene early. The more entrenched families are in the system and the more complex risks, the less likely the families were to complete treatment. Recommendation #2: Prevent or reduce multiple placements of infants and toddlers. While placing children in multiple foster homes is detrimental at any age, it can be most damaging during those first few years of life during the window of opportunity for maximizing healthy attachment. Recommendation #3: Fund Florida s EIP system to a level that will enable them to serve infants and toddlers in the foster care system who are at risk for developmental delays, but not yet delayed enough to meet eligibility criteria. Recommendation #4: Accept pilot project s assessment protocol for Comprehensive Assessments. Comprehensive behavioral assessments by community mental health providers are now required for all children in foster care, including children 0-5. Recommendation #5: Expand Medicaid billing to allow reimbursement of engagement activities as defined by pilot project or find other means to compensate for needed engagement services. Recommendation #6: Provide individual psychotherapy and/or psychopharmacological treatment in addition to dyadic treatment for severe maternal mental health and substance abuse problems. Recommendation #7: Reduce transportation barriers. While this is a barrier that crosses many human service programs, it appears to be particularly relevant to effective provision FIRST 5 Santa Clara County Section V Page 11 of 14

12 of intervention efforts with high-risk infants and toddlers whose treatment requires involvement of primary caregivers. Recommendation #8: Promote service integration at the local and state level. Effective mental health services for children 0-5 and their families need to be coordinated with other human service programs. Conclusion The assessment and treatment strategies used in the infant mental health pilot project were found to be effective in improving parent/child interaction and relationships, eliminating abuse and neglect, and increasing reunification or permanent placements of children in foster care for the 43 parent/child dyads that completed treatment. Although the findings are promising, conclusions about program effectiveness cannot be drawn nor can the findings be generalized to other populations due to small sample size. Long term effectiveness of the project will require follow-up study. FIRST 5 Santa Clara County Section V Page 12 of 14

13 Florida s Strategic Plan for Infant Mental Health Establishing a System of Mental Health Services for Young Children and their Families in Florida What is the Array of Infant Mental Health Services? Priority population Description of services/ interventions Professionals responsible for infant mental health services Level 1 Strengthening the Caregiver-Child Relationship, Responsive Caregiving Expectant families and families of all children birth to age five Strengthening the caregiver-child bond by: Helping caregivers to understand and respond appropriately to baby s cues Incorporating brain development research and attachment theory into all aspects of pregnancy, birthing and child s daily care Promoting continuity of care Supporting the child s on-going emotional development within the context and culture of the family Modeling responsive caregiving Providing family support and education Identifying early signs of problems that might impede the parent-child relationship Referring for further screening/ assessment Front-line caregivers including: Parents Childcare providers Healthcare providers Home visitors Parent educators Social workers Child protection case workers Police officers, judges, lawyers Level 2 Developmental, Relationship- Focused Early Intervention Families of children with delays, disabilities, health problems or multiple risk factors Strengthening the caregiver-child bond through: Identifying emotional or attachment concerns Integrating relationship-based practices into the child s existing services (therapies, medical treatment, foster care) Providing direct services based on the context, culture, and needs of the child and family Providing consultation to enhance responsive caregiving Assisting the family in accessing specific infant mental health treatment if needed Developmental professionals such as: Foster care for early interventionists, caseworkers, social workers, service coordinators, psychologists, mental health therapists Child development specialists Therapists (occupational, physical and speech) Public Health nurses Developmental pediatricians Level 3 Infant Mental Health Treatment Families with children or primary caregivers diagnosed with emotional disorders, severe mental health problems, or have experienced abuse, neglect or violence Strengthening the caregiver-child dyad through: Therapeutic interventions for caregivers and young children with specific mental health needs Establishing a nurturing relationship based on trust and respect of family strengths Ongoing, intensive treatment with parent/child dyad Consultation with all other service providers who work with infants and families Licensed mental health professionals with additional training in infant mental health including: Child/adolescent and adult psychopathology Infant/toddler development Quality of parent/infant interaction Assessment and treatment within the parenting relationship An understanding of context, culture, and family systems Dyadic, infant/parent psychotherapy FIRST 5 Santa Clara County Section V Page 13 of 14

14