Running head: PROGRAM EVAULATION OF BUTTERFLY/DRAGONFLY LODGE 1

Transcription

1 Running head: PROGRAM EVAULATION OF BUTTERFLY/DRAGONFLY LODGE 1 Program Evaluation of Butterfly/Dragonfly Lodge Stacy Carrillo, Krysten Belini, and Gissell Reyes College of Saint Elizabeth

2 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 2 The goal of this evaluation is to determine the effectiveness of the Butterfly/Dragonfly Lodge of Carrier Clinic; their goal is to promote a healthy environment for adolescent males and females with a history of psychiatric hospitalizations and/or emotional difficulties as a result of abuse, neglect, and violence. This evaluation will investigative the functioning of the lodge, how the staff treat the residents, the longitudinal outcomes of treatment and the rate of relapse of the residents. Our focus is on how the multiple phases of treatment prepare the residents for a less restrictive future. In addition, this evaluation will help determine the funding needs of the Butterfly/Dragonfly Lodge. CONTEXT EVAULATION As evaluators of the program, we plan to give members of the community unstructured interviews to measure: how they feel about Carrier Clinic being located within their community, their attitudes towards the treatment offered at the clinic and their perception of the need for the services that the clinic provides and their effectiveness. The questions used to evaluate the needs of the community can be found in Appendix B. To assess the functioning of the lodge and the effectiveness of the staff we will administer unstructured interviews to the current residents of the Butterfly/Dragonfly lodge. The questions used can be found in Appendix C. Six months after their discharge, these residents will be contacted via phone and asked to complete an unstructured interview pertaining to relapse and relapse prevention. The questions evaluating rate of relapse and relapse prevention can be found in Appendix D. As part of the evaluation, we will be conducting a cohort study to measure the longitudinal outcomes of treatment provided to the residents of the Butterfly/Dragonfly Lodge at Carrier Clinic. As part of this study, we will follow approximately 10 residents of the clinic from admission, throughout the treatment process, and finally through discharge and 6 months after.

3 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 3 The goal of this longitudinal study is to evaluate the effectiveness of the treatment stages offered to the residents of the Butterfly/Dragonfly Lodge. Since we are looking to evaluate the effectiveness of the numerous treatment stages offered, we will begin by selecting 10 random residents of the Lodge in order to gain a representative sample of the individuals residing at the lodge. During the admission process, we will conduct clinical interviews with each resident and his or her family in order to obtain demographic information on each resident and to gain knowledge on each individual s reason for seeking treatment at Carrier Clinic. We find it effective to establish a good rapport with the residents and families in order to obtain the most valuable and accurate information for our study. It is an important part of our job as evaluators to refrain from showing any bias towards the residents participating in our study and to ensure to our participants that all the information obtained is solely for the purpose of evaluating the effectiveness of the programs and that all personal information obtained shall be confidential and that their identities will remain anonymous. The clinical interview questions can be found in Appendix E. During the clinical interview, we will also be asking the residents and their families to complete the Child Behavior Checklist (Child and Parent Version). We will also be asking that the residents complete a Moods and Feelings Questionnaire (Child Version). During treatment, we will administer the Child Behavior Checklist to the residents (Teacher Version) and will provide a specific assessment depending on what they score on the Mood and Feelings Questionnaire (i.e. KADS for depression, SCARED for anxiety, etc). Before a resident is discharged, we will conduct an exit clinical interview, the Mood and Feelings Questionnaire, Child Behavior Checklist (Child Version) and NICHQ Vanderbilt Assessment Scale (Parent Version) that will be administered to the staff and caregivers of Carrier Clinic.

4 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 4 INPUT EVAULATION Carrier Clinic is the largest private, not-for-profit behavioral healthcare system in the state of New Jersey and specializes in psychiatric and addiction treatment. Carrier Clinic is located on 100+ farm like acres and is easily assessable through route 601. On their acreage a phenomenal, highly beneficial, safe, and sustainable 14-acre solar farm was developed which supplies 50% of Carrier's electrical need. Carrier provides their residents with monitored use of the gymnasium, swimming pool, picnic areas, and tennis courts. Their facility offers special areas for art, dog, and music therapy. They also have an amphitheater, multi-sensory room, library, and chapel. Carrier Clinic s clinical team is made up of several professionals, such as, Board-certified psychiatrists, licensed clinical social workers, registered nurses, crisis intervention specialists, substance abuse counselors, direct care staff and residential counselors. In order to employ the most experienced professional within the field, Carrier Clinic hosts multiple job fairs throughout the year. They also provide a list of current job openings and ask applicants to submit their profile through the application portal on their website. This is vital to the program because it ensures that they hire professionals with the most recent and up-to-date training to better serve their residents and assist in treatment. Within residential treatment programs, like East Mountain Youth Lodge, structure, education, establishing realistic goals, therapeutic intervention and self-reflection are vital. The adolescents residing in the East Mountain Youth Lodge on the Butterfly/Dragonfly Lodge unit will be exposed to therapeutic and coping mechanisms 24/7; these are beneficial and essential for their three stages of treatment. At Carrier Clinic, they provide education for their residents through the East Mountain Day School, which provides quality, individualized special education and strong therapeutic services for grades 7 through 12 with emphasize on community service, service learning and traditional high school activities. The residents within Butterfly/Dragonfly

5 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 5 have access to Adolescent Inpatient Unit for acute crisis stabilization, lab services, and physical therapy when necessary. Other significant resources that are provided and are available for all residents are individual therapy, group therapy, family therapy, crisis intervention, full value contract approach through peer intervention and group accountability, and recreation and adventure-based treatment. Residents are also offered Equine Assisted Psychotherapy, which is similar to pet therapy but utilizes horses that help work through various feelings the residents might have or are feeling.carrier Clinic developed The Caring Solutions Initiative (CSI) to help reduce the use of restrictive methods such as isolation and restraint on their residents. The CSI model has been an active medical tactical advantage at Carrier, which has assisted the resident s satisfaction approach including patient engagement, mutual respect and trauma-informed care. These resources are all intended to help build the residents self-efficiency, provide them with effective coping skills, and prepare them to be successful within their community. There are a specific amount of beds available for each unit at Carrier Clinic, which is updated daily. For Carrier Clinic to provide such adequate services, support and charitable donations from the community and significant partners are essential in maintaining their mission. With the donations from the community, Carrier is able to host well-being seminars and anxiety, stress, depression, and alcohol screenings throughout the year which are provided free of charge. With the assistance and support of partnerships from the community, state and professional affiliations, Carrier Clinic has had help with renovating, and creating impact and specialized programs for the diverse population they serve such as: Clinical Transition Program, East Mountain Hospital, Intensive Outpatient Program for Addiction, Active Seniors Program and The Carrier Diversion Program.

6 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 6 Carrier Clinic offers a biannual publication on community health. The approximately 10- page publication offers information on how Carrier Clinic is impacting the community through charity and volunteer work, as well as the annual Walk of Hope. This is a charity walk whose proceeds go to helping fight the stigma of mental illness and educates the community about the effects of these illnesses. Carrier Clinic advertises their services through commercials, radio announcements, website, and referrals. They also promote their facility by community walks, their blog on the website and their Facebook page, which provides awareness months, community and state events and information about the diverse divisions of mental health, substance abuse, and well-being. Word of mouth from residents, professionals, or families of the residents also broadcast Carrier Clinic services. Carrier Clinic participates with most insurance company s plans such as Medicare and Medicaid; this initially pays for the bed. Perspective residents are told to call their insurance carrier to find what their benefits cover and if Carrier Clinic is on the list as an accepted provider. However, Carrier does not take private insurance and residents would have to self-pay otherwise. A constraint to the Butterfly/Dragonfly Lodge is the limited bed availability for those wishing to reside in this specific residential treatment facility. Another constraint to Carrier is their direct care staff to resident ratio 1:4/5. A barrier of Carrier Clinic is their insurance coverage, especially for those who have private insurance. A final barrier to Carrier Clinic is their harsh and negative reviews from past residents, staff and families on the Internet. As evaluators, we wonder how Carrier Clinic refers perspective residents when there are no available beds. Do they refer them to their specific outpatient program or do they refer them to another inpatient program at a different facility?

7 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 7 PROCESS EVAULATION The flow chart in Appendix A is the process of an adolescent seeking treatment services at Carrier Clinic s Butterfly/Dragonfly Lodge from admission, to the three stages of treatment offered, and finally discharge. Potential residents can be referred by the ER or psychiatric emergency screening center or by calling Carrier Clinic s 24/7 Access Center. PRODUCT EVAULATION In order to evaluate the impact Carrier Clinic is having with its residents, we will evaluate the results of the assessment tools given to our cohorts during their stay. This will include: the multiple Child Behavior Checklists (Child, Parent and Teacher Version); the Moods and Feelings Questionnaires (Child Version) administered during the intake clinical interview; the NICHQ Vanderbilt Assessment Scale (Parent Version) that will be administered to the staff and caregivers of Carrier Clinic during the client s stay; and the specific assessment that is contingent on the Moods and Feelings Questionnaire administered halfway through treatment program; the rate of relapse survey administered six-months after release. With these results, we can determine whether Carrier Clinic is causing the desired change in residents. We will be able to quantify the changes in behavior and thought patterns that can potentially occur as a result of treatment administered at Carrier Clinic, thus determining if the organization is attaining their goals. A positive impact Carrier Clinic has on the community came with the construction of a 14-acre solar farm. It is the largest ground-mount, solar photovoltaic array system in any New Jersey healthcare campus at this time. In order to find out any unintended impacts, (positive or negative) we will also use the Community Unstructured Interview in Appendix B to assess the community s view of Carrier Clinic and the effects the organization has had on the community. This will be administered to a broad range of people in addition to community members (i.e.

8 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 8 Carrier Clinic employees, Butterfly/Dragonfly residents, medical personnel from nearby hospitals, first responders with-in the community, court officials, etc.). The answers given will help determine any unintended effects Carrier Clinic has had on the community. To determine the long-term effects of Carrier Clinic, we will contact individuals who have been released within the years We will re-administer any evaluations these individuals received upon admission to and release from Carrier Clinic and the Rate of Relapse survey in Appendix D. By doing this, we will determine if the individuals current behavior pattern resembles their history of behavior prior to or post any treatment received at Carrier Clinic. As evaluators we are going to conduct a cost-effectiveness analysis. We will determine the outcomes in terms of resident relapse. We will utilize the residents who answered the Rate of Relapse survey in Appendix D. This will allow us to determine whether or not the Butterfly/Dragonfly Lodge is allocating their services and funds appropriately. Expenditures will be evaluated by looking at Carrier Clinic s fiscal records (staff salaries, office space, supplies, services and program offered, etc.).

9 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 9 Appendix A

10 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 10 Appendix B Community Unstructured Interview 1. How do you feel having Carrier Clinic in your community? 2. What do you know about the services being provided at Carrier Clinic? 3. Do you think there is a need for these types of services within the community? 4. Do you feel that the services provided are effective towards the population it serves? 5. Do you know anyone with a history or diagnosis of mental illness? If so, is the person seeking treatment at this time? 6. Would you refer a loved one with behavioral and emotional issues to Carrier Clinic? Why or why not? 7. How do you view people with mental illnesses? 8. Do you think Carrier Clinic has a positively impact in your community? A negative impact? Please explain.

11 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 11 Appendix C The Functioning of the Lodge 1. Do you feel safe? 2. Have you ever felt harassed by another resident? 3. Do you feel group therapy is an important part of your treatment? Why or why not? 4. What part of your treatment do you think is helping you the most? 5. What part do you think is helping the least? 6. Do you think you have enough free time to pursue your own hobbies? 7. Do you feel that the education provided at Carrier Clinic is equal or better than the education you would have received in the community? How is it better? How is it worse? How the Staff Treats the Residents 1. Do you feel that the staff treats everyone effectively and fairly? 2. Do you feel that you can communicate honestly and openly to staff during your therapeutic interventions? 3. Do you feel that there are fair consequences to the rules? 4. Do you think there is enough well trained staff? 5. Have you ever felt harassed by a staff member?

12 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 12 Appendix D The Rate of Relapse 1. Since your release have you been arrested? If so, how soon after your release did it occur? 2. Have you used drugs since you were discharged from Carrier Clinic? If so, how soon after your release did it occur? 3. Have you participated in any self-harmful behaviors since you ve been discharges? If so, how soon after your release did it occur? 4. Have you had a depressive episode? If so, how soon after your release did it occur? 5. Are you in outpatient treatment? If so, where? 6. If you haven t relapse, what are you doing to prevent it? 7. Have you been hospitalized for unsafe behavior, self-harm or for an acute episode of mood/ depressive disorder? If so, how soon after your release did it occur?

13 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 13 Appendix E Clinical Interview Questions 1. Could you tell me about yourself and what brought you here? 2. Can you tell me about your relationship with your mother? What was (is) she like as a person, and what was she like as a parent? Now I'd like you to describe a specific encounter with your mother, something that stands out. It can be an incident that s typical of your relationship, really meaningful, really good, really bad whatever comes to mind. 3. Could you tell me about your relationship with your father? What was (is) he like as a person, and what was he like as a parent? Now I'd like you to describe a specific encounter with your father, something that stands out. It can be an incident that s typical of your relationship, really meaningful, really good, really bad whatever comes to mind. 4. Do you have brothers and sisters? Could you tell me a little bit about them and your relationships with them? Is there anyone else who is really important to you? 5. What is school like for you? 6. Now I'd like you to think of a really difficult, stressful, or upsetting time in the last year or two, and tell me about it. 7. Now I d like to finish up with a few more questions about your mental and physical health, and the ways you feel, think, and see yourself. A) Do you often feel sad? Anxious? Ashamed or embarrassed? Guilty? Angry? B) How about positive feelings, like happiness or pride? C) Do you ever get overwhelmed by your feelings? D) Are there times when you try to shut off your feelings entirely, or when you just feel numb?

14 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 14 E) Have you ever had trouble with alcohol or drugs? G) How do you usually feel about yourself? Do your feelings about yourself change a lot? What do you most like and dislike about yourself? H) Have you ever hurt yourself, tried to kill yourself, or thought seriously about suicide? I) Do you ever feel like you don t know who you are, or like the different sides of you don t fit together? 8. I ve asked you a lot of questions. How has this been? Is there anything we haven t covered that s really important in understanding you as a person? Is there anything else you'd like to add, or anything you'd like to ask? Exit Clinical Interview Questions 1. Overall, how would you rate the quality of care you received while at Butterfly/Dragonfly Lodge? 2. What part of the three stages of treatment benefited you the most? And why? 3. What part of the three stages of treatment was least effective? And why? 4. Throughout your stay at Butterfly/Dragonfly Lodge were you harassed by other residents or staff? 5. Do you feel you were provided with adequate education at the East Mountain Day School? If no, why? 6. Do you feel that you were in a safe and secure environment during your treatment? If no, why? 7. Do you believe that family therapy was beneficial to your treatment? Why or how? 8. Now I d like to finish up with a few more questions about your mental and physical health, and the ways you feel, think, and see yourself.

15 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 15 A) Have you felt sad? Anxious? Ashamed or embarrassed? Guilty? Angry? B) How about positive feelings, like happiness or pride? C) Have you been overwhelmed by your feelings? D) Are there times when you try to shut off your feelings entirely, or when you just feel numb? E) How do you usually feel about yourself? Have your feelings changed throughout your stay? What do you most like and dislike about yourself? F) During your time here, have you felt the need to self-harm? 9. How do you feel about leaving Butterfly/Dragonfly Lodge? 10. How comfortable are you with the aftercare plan that you and your clinical team agreed on? 11. I ve asked you a lot of questions. How has this been? Is there anything we haven t covered that s really important in understanding your time spent at Butterfly/Dragonfly Lodge? Is there anything else you'd like to add, or anything you'd like to ask?

16 PROGRAM EVALUATION OF BUTTERFLY/DRAGONFLY LODGE 16 References Hadley, R. and Mitchell, L. (1995). Counseling Research and Program Evaluation. Brooks/Cole, Cengage Learning. Mental Health Services & Clinic, Mental Health Centers & Psychiatric Hospital of New Jersey. N.p., n.d. Web. 28 Oct < Westen, D. (2007). Clinical Diagnostic Interview, Patient Version. Departments of Psychology and Psychiatry and Behavioral Sciences. Emory University, Atlanta, GA.