Project Lead: Renee Parsons, MSW, RSW, Windsor Regional Children s Centre

Transcription

1 Cover page Title: Windsor Regional Children s Centre Triple P Program Evaluation Project Lead: Renee Parsons, MSW, RSW, Windsor Regional Children s Centre Manager of Service Coordination Authors: Tammy Drazilov, Windsor Regional Children s Centre, Coordinator, Triple P Fedaa Beiti, BA Hounours, University of Windsor Jenna Jones, BA Honours, MA Student, University of Windsor Windsor Regional Children s Centre EPG-1595 Deleted: 3 October 31, P age

2 Table of Contents Executive Summary 3 Introduction and Literature Review Methodology Program Description 5 The Evaluation Purpose 8 Outcomes and Indicators 11 Literature Review 11 Triple P Intervention 13 Ethical Considerations 15 Participants 16 Measures 18 Results, Discussion & Interpretation Data, Assumptions & Findings 23 Discussion and Interpretation 27 Conclusion Recommendations 32 Next Steps 33 Stakeholder Involvement and Knowledge Exchange 34 References and Appendices 37 2 P age

3 Executive Summary Organization name: Windsor Regional Children s Centre Project lead: Renee Parsons, MSW, RSW, Manager of Service Coordination The Triple P (Positive Parenting Program) program is an evidence-based, multi-level parenting training program available worldwide with over 30 years of proven clinical research. At the Windsor Regional Children s Centre (RCC), we provide family-focused interventions, which support enhanced parental capacity as part of an effective way to address the children s mental health capacity. The purpose To evaluate the effectiveness of the RCC s Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program in reducing dysfunctional parenting strategies and improving child behaviour and functioning. To identify if the success in the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program varied based on the child s mental health issues at intake, thus allowing us to determine the best treatment plan for families. To increase the capacity for future and ongoing evaluations of treatment interventions. The program The Windsor Regional Children s Centre is a children s mental health centre that offers a variety of crisis, assessment, treatment, and consultative services to promote healthy functioning of children, youth, and their families. The Centre offers out-patient, residential services, and community based programs in Windsor and Essex County. In some circumstances, parents of children receiving treatment at the RCC were referred to the familyfocused Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program, which enhances parental capacity as part of an effective way to support the children s mental health ability. 3 P age

4 The Triple P Group program evaluated in this project consists of both a Level 4 Group Pre-Adolescent Triple P program and a Level 4 Group Teen Triple P program. These programs were offered by the RCC between 2006 and The plan This evaluation looked at the largely archival data, from a sample of 178 clinic-based Canadian parents, who completed the Triple P program standard parental pre and post self-evaluate measure. They also completed The Brief Child and Family Phone Interview (BCFPI) intake screening measure at RCC. The data analysis program SPSS was used for all analyses. The product The results indicate that families participating in the Triple P Groups Programs (i.e., both Pre-Adolescent and Teen Programs) at the RCC made significant improvements in both parenting abilities and child conduct. It was determined that going forward RCC will continue providing Level 4 Triple P group programs as a family-focused intervention. The opportunity to receive this grant and perform this research project has increased the capacity for the evaluation of: (a) other RCC programs that are offered, and (b) the community-wide Triple P programs provided by the Windsor-Essex Triple P Community Partnership. The results of this study will be shared with internal and external stakeholders via a community presentation on October 18th, 2013 and posted on our social media platforms. Amount awarded: $22, Final report submitted: October 31, 2013 Region: Windsor, Southwestern Ontario 4 P age

5 Introduction and Literature Review Program Description The three P s in Triple P stand for Positive Parenting Program. Triple P was developed in Australia, by Dr. Matt Saunders from The University of Queensland. Triple P is a parent management training program designed to address behavioural, emotional, and developmental problems in children and adolescents up to 16 years of age (Sanders, 2012). The multi-level system (See Table A - Multi-Level Triple P System), has been shown to be effective in reducing the prevalence rates of behaviours and emotional problems in children across a variety of settings and countries (e.g., Fujiwara, Kato & Sanders, 2011; Leung, Sanders, Leung, Mak & Lau, 2003; Sanders). The Triple P system (See Table A - Multi-Level Triple P System) includes levels ranging from community-based knowledge dissemination covering common and minor child behavior issues, to more in-depth group treatment for parents of children with significant problems. Table A - Multi-Level Triple P System Triple P program Level Tripe P Level 1 program Tripe P Level 2 program Tripe P Level 3 program Tripe P Level 4 program Stay Positive Table A - Multi-Level Triple P System Program Name Selected Seminars, Selected Seminars Teen, Selected Seminars Stepping Stones Brief Primary Care, Brief Primary Care Teen Primary Care, Primary Care Teen, Primary Care Stepping Stones Triple P Discussion Groups Group Pre-Adolescent Triple P, Group Teen Triple P program Stepping Stones Standard, Standard Teen, Standard Stepping Stones Self-Directed, Self-Directed Teen, Self- Directed Stepping Stones Online Baby Delivery Format Media Communication 3 seminars, 1 ½ - 2 hours, 20+ parents One brief individual consultation Several (~4) min. individual consultations (telephone or in person) One 2 hour group discussion Five 2 hour group sessions + three 20 min. phone consultations, up to 12 families Ten one-on-one 1 hour sessions Ten self-directed workbook modules Eight interactive online modules In development 5 P age

6 Triple P program Level Tripe P Level 5 program Continued Table A - Multi-Level Triple P System Program Name Delivery Format Enhanced Pathways (risk of maltreatment) Lifestyle (obesity) Note. Adapted (Sanders, 2012). Family Transitions (divorce) Up to eight 1 hour individual sessions Three 1 hour individual sessions, or 2 hour group sessions Ten 1 ½ hour group sessions + four 20 min. phone consultations, up to 10 families Five 2 hour individual or group sessions, in addition to level 4 The present analysis is a sample of clinic-based Canadian parents. Data for this study was largely retrospective in nature and included parental pre- and post-intervention self report measures of those parents who participated in the Level 4 Group Pre-Adolescent Triple P program and Group Teen Triple P program between 2006 and This program has been delivered by RCC staff members that have been Triple P trained and accredited facilitators. The Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program has been delivered internally to RCC clients, as well as on a Windsor Essex community wide basis since being implemented in However, there has been a lack of program auditing for this intervention s effectiveness throughout this time. In the last six years, in our area we also have formed the Windsor-Essex Triple P Community Partnership. This collaborative is 11 eleven local agencies from; children and adult mental health, public heath, advocacy protection service providers and family court system. RCC is part of this collaborative that provides Triple P programs, at no cost to any parent or caregivers in Windsor and Essex County. Primary components of the Triple P program All five levels of the Triple P system are based on the five core principles of the positive parenting: ensuring a safe and engaging environment, creating a positive learning environment, using assertive 6 P age

7 discipline, having realistic expectations, and taking care of yourself as a parent (Sanders, 2012). Specific child management skills are taught to parents in order to address both risk and protective factors that are related to mental health outcomes in children Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program consists of an 8-week program which employs active skills training whereby parents gain new skills and knowledge through practice. Level 4 of Triple P is broad in focus and targets parents with children displaying more severe and detectable behavior problems. The parents receive intensive training in positive parenting skills and learn through observation, discussion, and feedback, both during group sessions and intermittent follow up telephone sessions. Level 4 Group programs allow support and feedback from the facilitator and from other participating parents, thereby normalizing their parenting feelings. Here, parents choose their own goals to monitor and the strategies to implement, thereby allowing parents to take ownership in the process of the intervention. This direct responsibility, by the parents for this positive change builds their confidence. Target audience RCC disseminates the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program as a secondary prevention for children who have been identified as at risk for various mental health concerns. The Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P programs are a broader focused intervention with moderate to high intensity, and it services parents seeking training in positive parenting skills. Parents with children identified as having detectable problems (these may be individual children or whole populations) are the target audience for this treatment intervention (Sanders, 1999). The parents or caregivers of these children participate in this intensive positive parenting training and learn to apply these concepts in multiple contexts. Average number of clients per year The Windsor Regional Children s Centre underwent a major restructuring process in the previous three 7 P age

8 years. The Centre transitioned to a service coordination delivery model, and integrated with a second local children s mental health agency, Glengarda Child and Family Services, to become Windsor/Essex County s primary children s mental health service agency for children aged 6 to 12 years. This restructuring required that some trained and accredited Triple P facilitators move into new positions which did not allow them to deliver this program to parents. This resulted in the Level 4 Group Pre- Adolescent Triple P program and Level 4 Group Teen Triple P program not being offered internally at RCC from 2009 to During this time, some RCC clients were referred to the Level 4 Group Pre- Adolescent Triple P program and Level 4 Group Teen Triple P program as delivered by the Windsor Essex community wide program. However, the data of those parents participating in the community wide program were not included in this study. On average 30 clients participated in the program each year that it was offered (ranges from 4 to 76). The evaluation purpose At the Windsor Regional Children s Centre (RCC), we provide family-focused interventions, which support enhanced parental capacity as part of an effective way to address the children s mental health capacity. The purpose of the present study is to; evaluate the effectiveness of this program in reducing dysfunctional parenting strategies and improving child behaviour and functioning, to identify if the success in the program varied based on the child s mental health issues at intake and to increase our capacity for future and ongoing evaluations of treatment interventions. This evaluation is in keeping with the strategies and directions outlined by Ontario Centre of Excellence for Child and Youth Mental Health s (OCECYMH). Since its implementation at RCC in 2006, the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program has not been audited or analyzed for its effectiveness and overall intervention outcomes. In keeping with our mandate to provide effective intervention programs, the Evidence Based Practice Committee (EBPC) at RCC met with OCECYMH and developed a program logic 8 P age

9 model (see Table B Logic Model), of the RCC, Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program. The EBPC is comprised of both management and non-management staff from several multidisciplinary levels. The process of developing the logic model enhanced the committee members understanding of the fundamentals of the Triple P program, how this intervention is provided at RCC and what the evaluation project would encompass. Input into our logic model was solicited from all of the staff at RCC via s and to the stakeholder / community partners at a stakeholder meeting on November 6, Our current logic model reflects this input, and is a depiction of the program both internally and community wide. The logic model reflects the link between the RCC Triple P programs and the community wide Triple P programs to achieve parent-related short-term outcomes and medium-term outcomes that specifically relate to the community wide program such as; increased knowledge of community supports, increased advocacy, and increased access of community as well as, supporting a joint long-term goal of a healthier Windsor- Essex. These items were not evaluated in this report. We focused our evaluation efforts on assessing if the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program promoted healthy child development by targeting the variables that may be undermining them; peer relationships, self-esteem, self-regulation, mental health functioning, maladaptive behaviour and parental skills and family relationships. If we address parents psychological and emotional issues regarding; coping skills; self-regulation and self-efficacy, we have also addressed, the linkage between the parental and child outcomes in the self-report measures. We anticipated by providing this family-focused intervention that parents completing the Triple P group program would improve their pro-social capacities and their ability to cope and self-regulate. Over time, these enhancing parental competencies and improved mental health capabilities will increase parents self-efficacy and decrease family stress and conflict. The premise is that by addressing parents risk 9 P age

10 factors and enhancing parental capacity, the child outcomes (see Table B Logic Model), for positive peer interaction, self-esteem and self-regulation will improve. In turn, children will be less likely to exhibit maladaptive behaviours and thus increase their potential for improved mental health functioning. By using social learning principles, Triple P programs can teach parents how to create a loving, supportive and predictable family environment. This is assuming that parents are motivated to enhance their parenting knowledge and skills. This family structure is important in raising healthy, well-adjusted children who have the skills and confidence they need to succeed at school and with their relationships. Thus reaching our long term goals of healthy family relationships, in turn, leading to a healthier Windsor-Essex community. Table B Logic Model 10 P age

11 The logic model (see Table B Logic Model), allowed us to develop our research questions with feedback for all the invested participants as stated above. Outcomes and indicators An outcome evaluation allowed RCC the opportunity to review the value of the intervention regarding its effectiveness, so that results can inform decisions about the worth of the intervention, such as whether to continue to support the intervention, or how improvements can be made to it (Gallagher, 2006). The specific indicators being used in this evaluation are parenting strategies, child mental health functioning, and parent mental health functioning. The Evidence Based Practice Committee, internal and external stakeholders identified four specific questions that will be addressed by this study; 1. Are parents increasing positive parenting strategies? (Is the program effective overall in reducing dysfunctional parenting strategies?) 2. How effective is the program in improving children s behavioural issues? 3. What is the relationship between positive child behaviour outcomes and parental stress reduction? 4. Does success in the Triple P program (i.e. parent and child outcomes) vary based on initial ratings of child mental health concerns? These questions will help us evaluate the effectiveness of the RCC Triple P group programs. Literature Review Triple P denotes the Positive Parenting Program (Sanders, 1999), a multi-tiered continuum of service developed in Australia, at the University of Queensland by Dr. Saunders. Triple P is an empirically validated parent training program designed to address behavioural, emotional, and developmental problems in children and adolescents up to 16 years of age (Sanders, 2012). Table A - Multi-Level Triple P System, describes the Triple P system, which includes levels ranging from community-based 11 P age

12 knowledge dissemination covering common/ minor child behavior issues, to more in-depth group treatment for parents of children with significant problems. Behavioural family interventions, such as Triple P, are amongst the most strongly supported interventions for improving parent and child outcomes (Sanders, Markie-Dadds, &Turner, 2003; de Graff, Speetjens, Smit, de Wolfff, & Tavecchio, 2008). These types of interventions support a link between parent practices and child outcomes. Extensive and cross-cultural research has been conducted evaluating the effectiveness of Triple P as a BFI. The overwhelming number of positive research findings has led to its designation as an evidence-based intervention by the National Institute of Clinical Excellence, the World Health Organization and the United Nation s Task force on family based interventions (Sanders, 2012). Level 4 in particular, also known as Group Triple P, encompasses 8 group sessions with practitioner assistance, and has gained credibility from the numerous studies supporting its effectiveness. A Swedish parent management training intervention using practitioner assisted group sessions found that parents in practitioner assisted groups showed greater improvements post intervention on measures of child conduct problems as compared to parents who were in a largely selfadministered style group (Kling, Forster, Sundell, & Melin, 2010). Consistently, studies of Group Triple P have demonstrated improvements in child behaviour, parental skill/ knowledge, stress, self-efficacy, and parental relationship satisfaction post program completion (Cann, Rogers, & Matthews, 2003; Crisante & Ng, 2003; de Graaf et al., 2008; Dean, Myors, & Evans, 2003; Markie-Dadds, & Sanders, 2006; Nowak & Heinrichs, 2008; Ralph & Sanders, 2003). This level of the Triple P program has been found to be superior in positive outcomes compared to waitlist control groups (Fujiwara, Kato & Sanders, 2011; Hahlweg, Heinrichs, Kuschel, Heike & Naumann, 2010; Leung, Sanders, Leung, Mak & Lau, 2003; Matsumoto, Sofronoff, & Sanders, 2007; Matsumoto, Sofronoff, & Sanders, 2010), and care as usual programs (Prinz, Sanders, Shapiro, Whitaker & Lutzker, 2009; Sanders, Ralph, Sofronoff, Gardiner, Thomspon, Dwyer, & Bidwell, 2008). The Group Triple P intervention has 12 P age

13 further shown to be effective in reducing the prevalence rates of dysfunctional behaviour and emotional problems in children across a variety of settings and countries (e.g. Fujiwara et al., 2011; Leung et al., 2003; Sanders et al., 2003). Clinically, findings have also suggested that participation in the Triple P intervention benefits diverse parents equally well by improving knowledge of child development. In a recent clinical study of Group Triple P by Winter, Morawska and Sanders (2011), results indicated that parents significantly increased their knowledge of effective parenting strategies regardless of their education level. Kling et al., (2010) found that only the age of the participating mothers moderated the effect of their parent management intervention, while all other family characteristics had equal intervention effects. This is consistent with previous research (e.g. Nowak et al., 2008; de Graaf et al., 2008) which used randomized control trials of Group Triple P and found that both higher and lower educated parents reduced their dysfunction and reported reductions in their children s externalizing behaviours. The Triple P system boasts effectiveness with various populations, such as Cantonese-speaking Australians (Crisante et al., 2003), parents of preschool children with conduct problems (Markie-Dadds et al., 2006), and parents of teenagers entering high school (Ralph et al., 2003). This body of universal evidence favours the credibility of the Triple P intervention and supports a link between parenting practice/ knowledge and child behaviour outcomes. The present study evaluates the effectiveness of the Positive Parenting Program as a delivered by the Windsor Regional Children s Centre (RCC). Methodology Triple P Intervention The Triple P program evaluated in this project was the Level 4 Group Pre-Adolescent Triple P program, and Level 4 Group Teen Triple P program as conducted at RCC. The delivery format of the program consisted of six 2 hour group sessions and two 15 to 30 minute individual telephone sessions. Group 13 P age

14 sessions included videos, live demonstrations and active group participation, meanwhile the telephone sessions allowed each group member to review their individual goal-setting and to self-evaluate with a program facilitator. All of the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P programs were conducted by facilitators who were employees of RCC and possessed the appropriate post-secondary qualifications in Social Services. These facilitators had previously completed the competency-based training and accreditation processes required by Triple P International s standards (Sanders, Markie- Dadds, & Turner, 2003). RCC staff members who delivered the The Level 4 Group Pre-Adolescent Triple P program received the specified 3 day training and accreditation in The Level 4 Group Pre-Adolescent Triple P program, whereas staff delivering the Level 4 Group Teen Triple P programs received the 3 day training and accreditation in Level 4 Group Teen Triple P program. The Triple P training programs contain a built-in evaluation process. In order to successfully receive their accreditation, practitioners must demonstrate proficiency in the competency and score at least 80% on a multiple-choice quiz. Completion of the accreditation process helps ensure that practitioners deliver the program with fidelity and a reasonable level of proficiency. Only these accredited facilitators implemented the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program with clients and their families. Parents of children receiving treatment at RCC were referred for the Triple P Group program based on informal clinical judgment by caseworkers or service coordinators after intake. The program was then delivered at a designated time and meeting room at RCC. However, due to limitations such as funds, program facilitators and the number of parents participating, many of the Triple P Group programs conducted at RCC between 2006 and 2009 were a combination of the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program. However, this practice was abandoned in P age

15 Ethical considerations At the time of this study Windsor Regional Children s Centre was governed by Windsor Regional Hospital, whose mission is to deliver an outstanding care experience driven by a passionate commitment to excellence. We value research as a fundamental activity for the generation of new knowledge, which thereby improves patient care. Research activities enhance the education and awareness of all medical professionals in our institution. All research conducted in affiliated with Windsor Regional Hospital is done in accordance to the standards of a Research Ethics Board (REB). The mandate of the Windsor Regional Research Ethics Board is to safeguard the rights, safety and well being of all research participants. Accordingly, all research associated with Windsor Regional Hospital must have a staff member or affiliate who will be responsible for conducting research. This project was overseen by Renee Parsons, MSW, RSW, Manager of Service Coordination at Windsor Regional Children s Centre. As well, this project also included three volunteer students from the University of Windsor. Windsor Regional Hospital mandates that all volunteers must receive an orientation conducted through the Department of Volunteer Services. This provides all volunteers sufficient orientation and instruction on the safe and appropriate execution of duties. Signatures of these volunteers were obtained with respect to a volunteer commitment to hospital policies, procedures, standards of service and confidentiality agreement. The volunteers were required to read, understand, and sign the Confidentiality Agreement contained within the Application for Voluntary Service, Form #1229. All three student volunteers completed the necessary steps outlined by Windsor Regional Hospital and were given ethics clearance for this project. In addition to this, two of the students were required to complete the Tri-Council Policy Statement (TCPS) on-line tutorial Course on Research Ethics (CORE). This course is a media-rich learning experience that features interactive exercises and multi-disciplinary examples regarding ethical conduct of research 15 P age

16 involving humans. A certificate of completion was required for both the Windsor Regional Hospital Research Ethic s Board and the University of Windsor s Research Ethic Board. Due too many steps and there expectations this project was delayed until December Questionnaires Participants were required to complete the Triple P recommended self report measures pre and post the intervention program. The participants were mailed the pre-intervention questionnaires and asked to complete them prior to the beginning of the first session. The facilitators were available to provide assistance to parents who were unable or unmotivated to complete the pre-assessments prior to the first session. Most parents were able to complete these questionnaires with-in 30 minutes to 1 hour. The post-assessment questionnaires were distributed 8 weeks later during the final session of the program. The pre and post questionnaires were identical apart from the added Client Satisfaction Survey which was included in the post-questionnaires. The data for parents who did not fully complete both the pre and post questionnaires or who answered them incorrectly were excluded from this study. Once the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program was completed the pre and post parent self report measures were returned to the RCC facilitators. Paper copies of the questionnaires were kept in a locked space at the RCC. To date no formal program evaluation has been conducted using this data. Throughout the duration of the project, measures were taken to ensure that research results could not directly or indirectly identify participants. All client names were removed and new client ID numbers were given. All pre and post measures, client consent forms and Brief Child and Family Phone Interview (BCFPI) information was retrieved from secured Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P programs and client files. Participants The participants for this project included 178 parents who participated in the Level 4 Group Pre- 16 P age

17 Adolescent Triple P program or Level 4 Group Teen Triple P program at the Windsor Regional Children s Centre between 2006 and The Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P programs were provided to families receiving support services as part of their treatment program with RCC. Only participants who completed both the pre and post program measures (N = 89) were included in our program analysis Sample size and participant characteristics Demographic information was collected from all 89 parents (72 children). The target children were mostly males (66 male, 23 female), and ranged from four to 16 years of age, with an average of 9.70 years. 17 parents were married, 15 were single, 8 divorced or separated, 3 were common-law, and 46 did not have this information available. The majority of participants were the target child s mother (64), 13 were fathers, 11 were of other relation (including step-parents), and 1 was missing. Most did not complete post-secondary education (41 mothers, 32 fathers). 22 mothers and 20 fathers completed post-secondary education, and 26 mothers and 37 fathers had missing data for this question. The family structure most commonly reported was sole parent family (6), followed by step-family (5), and original family (4), and 2 reported other (e.g. living with grandparents), and data was unavailable for 72 participants. The number of children in the family ranged from one to six, with the most frequent response as two children (30 families). Four mothers, and seven fathers reported being employed, and nine mothers and four fathers reported being unemployed (76 mothers and 78 fathers were missing this question).the total yearly family income ranged from less than $10,000 to $60,000 or more, and the most frequent response was $60,000 or more (15 families), and the next most frequent response was $10,000 to $14,999 (13 families; 32 families missing this question). The participants children were receiving treatment at RCC for a broad range of emotional, developmental and behavioural issues at the time the parents participated in Triple P. The majority of children (i.e. offspring of the participants) were male (74%) with an average age of 9.7 years (ages 17 P age

18 ranging from four to 16 years). The number of children in the home ranged from one to six, with an average of 2.6 children per household. Measures The following measures were used in the study; the Brief Child and Family Phone Interview (BCFPI; Cunningham, Boyle, Hong, Pettingill, & Bohaychuk, 2008), The Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997), The Parenting Scale (PS; Arnold, O Leary, Wolff & Acker, 1993), The Parenting Scale- Adolescent (PSA; Irvine, Biglan, Smolkowski, & Ary, 1999), The Being a Parent Scale (BPS; Johnston & Mash, 1989), and the Depression-Anxiety-Stress Scale (DASS; Lovibond & Lovibond, 1995a) when available. Although the Family Background Questionnaire (FBQ) is a measure Triple P recommends administering along with the pre-assessment questionnaire, it was not administered to the majority of the families included in this study. The FBQ gathers family social, health, education and economical demographics. This information however was previously obtained from parents at RCC upon intake using the Brief Child and Family Phone Interview (BCFPI; Cunningham, Boyle, Hong, Pettingill, & Bohaychuk, 2008). The Brief Child and Family Phone Interview (BCFPI; Cunningham, Boyle, Hong, Pettingill, & Bohaychuk, 2008). Similar to the FBQ, the intake staff at RCC used the BCFPI as a screening measure to assess the child s symptoms, mental health, family functioning and needs. RCC staff then used this information to recommend the appropriate services at the children s mental health centre. For the purposes of this project the necessary demographic information that coincides with the FBQ was therefore retrieved directly from client files. The BCFPI is a screening measure for childhood disorders based on the Diagnostic Interview Schedule for Children (DISC-IV; Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000). The BCFPI is a 30 minute telephone interview that is conducted to assess the child s (aged 6 to 18) mental health and family functioning. There are 73 required questions, and 59 optional questions in the interview. Scores on these questions are later converted into t-scores. The 18 P age

19 BCFPI provides 14 subscales, and five composite scales pertaining to the child s mental health, functioning, and impact on family functioning. The BCFPI cutoff scores also allowed an exploration of the parent and child outcomes in this intervention using the child s mental functioning as a variable. The Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997) is a 25-item screening measure that assesses parental perceptions of children s (aged 4 to 16 years) positive and negative attributes over the previous six months. Parents rate 25 items regarding their children s behaviour on a 3 point Likert scale of not true, somewhat true and certainly true. The SDQ includes 5 subscales, emotional symptoms, conduct problems, hyperactivity, peer problems, and pro-social behaviour, as well as a total difficulties score. Results are scored as falling within the normal, borderline, or clinical range. There are also supplemental questions assessing the overall impact of the child s behaviour on the family unit. The Parenting Scale (PS; Arnold, O Leary, Wolff & Acker, 1993), measures dysfunctional disciplinary practices and reflects 3 different parenting styles; laxness, over-reactivity and verbosity. Laxness refers to permissive and inconsistent parenting, over-reactivity includes the use of harsh and punitive discipline, and verbosity is the use of long verbal responses to a child s behaviour (Salari, Terreros, & Sarkadi, 2012). Parents rate 30 items on how they typically respond to various parenting situations on a 1 to 7 point scale. These three subscales combine to suggest the level of dysfunctional parenting practices for a total parenting style score. This questionnaire was administered to participants of the Group program only. In 2009, Triple P adopted an alternative factor structure of the Parenting Scale (Rhoades and O Leary (2007). In this structure, laxness and over-reactivity were maintained as subscales, and hostility replaced verbosity as an entirely different subscale measure. The hostility subscale measures forceful physical or verbal aggression toward a child (Rhoades & O Leary, 2007). This new structure of the Parenting Scale maintains the original thirty items of the questionnaire; however different items are 19 P age

20 now retained for each subscale. The old and new questionnaire items are identical; however the items used to calculate the subscales changed. This allowed us to use the new subscales in this evaluation. This questionnaire was given to participants of the Group program only. The Parenting Scale- Adolescent (PSA; Irvine, Biglan, Smolkowski, & Ary, 1999) measures dysfunctional disciplinary practices and parenting styles which may be ineffectual for disciplining their teenage children. This scale is a subset from the original PS questionnaire and consisting of 13 items measuring laxness (6 items) and over-reactivity (6 items). The items on these two factors differ from the factor structure used for the original PS, and an additional item related to parental monitoring is included in the total score, but not the subscales. This questionnaire was administered to participants of the Group-Teen program only. The Being a Parent Scale (BPS; Johnston & Mash, 1989) is a 16 item questionnaire designed to measure parents self-efficacy, competence, and parental satisfaction. Parenting efficacy (7 items) reflects perceived competence, problem-solving ability and capability in the parental role. The parenting satisfaction subscale (9 items) measures parental frustration, anxiety and motivation. Parents are asked to respond to a series of statements about parenting, indicating their agreement or disagreement. Each item is measured on a 7-point Likert scale, ranging from 1 (Strongly Disagree) to 7 (Strongly Agree). The two scales are then aggregated to produce a Total Score. The Depression-Anxiety-Stress Scale (DASS; Lovibond & Lovibond, 1995) is a self-report scale designed to measure the negative emotional symptoms of depression, anxiety and stress being experienced by the parent. Parents read a series of statements and indicate how much the statement applied to them in the past week on a scale from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time). Scores for Depression, Anxiety and Stress are calculated by summing the scores for the relevant items. There is a 42-item and a shortened 21-item version of this questionnaire. These two versions are comparable, and doubling the scores on the 21 item version provides 20 P age

21 approximately equivalent scores as the 42 item version (Henry & Crawford, 2005). Parents in the Group program completed the 42 item questionnaire, whereas parents in the Group-Teen program completed the 21 item questionnaire. The overall reporting by participants who completed the pre and post intervention measures were compared for reductions in dysfunctional parenting practices and improvements in the behaviour and functioning of children. Additionally, it was hypothesized that those parents reporting significantly fewer behaviour and functioning problems in their children at the completion of the program, would also report increased parental competency and decreased negative emotional symptoms such as depression, anxiety and stress. The Family Background Questionnaire (BCFPI) cutoff scores explored if the parent and child outcomes in this intervention vary based on the child s mental functioning. Each research question evaluates specific perceived competency scale outcomes (see Table D Outcome Measures Worksheet) to determine whether the parent reported measures showed decreases in dysfunctional style, increases in parental competency and significantly fewer behaviour problems in their children by the end of the program. Only specific standard Triple P measures that address the evaluation questions were used for this project (see appendices, Table C - Describing Outcomes Measures Worksheet) Description of data analysis plan The data were analyzed by a psychology graduate student from the University of Windsor, under the advisement of a professor in the psychology department. The data analysis program SPSS was used for all analyses. Data were checked to ensure quality and minimize errors. In order to address the first question (how effective is the program in improving parenting skills) a repeated measures MANOVA was conducted. A MANOVA was chosen to reflect the multiple outcome variables that are being used. The outcome variables for this analysis were the subscales for the Parenting Scale, Being a Parent Scale, and Depression Anxiety Stress Scale. 21 P age

22 To address the second evaluation question, a repeated measures MANOVA was conducted. The outcome variables for this analysis were the subscales of the Strengths and Difficulties Questionnaire. For the third evaluation question, a bivariate correlation was run between different scores of the Strengths and Difficulties Questionnaire, and the stress subscale of the Depression Anxiety Stress Scale. Lastly, to respond to the fourth evaluation question, a mixed model repeated measures MANOVA was conducted. Based on the Brief Child and Family Phone Interview, participants were grouped as having externalizing problems only, internalizing problems only, both, or neither. The outcome measures for this analysis were the subscales for the Strengths and Difficulties Questionnaire. Description of evaluation limitations During the data input process several issues resulted in a reduced data pool; We had hoped to include many of the participants from our current RCC clients; however this year many of our parents did not complete the program. More work is underway to further investigate this issue outside of this project. Many of the participants were missing entire pre-intervention questionnaires or postintervention questionnaires. Items on some pre- or post-intervention questionnaires were left incomplete by the reporting parents. If more than 25% of the items on a given questionnaire were left unanswered, responses for that questionnaire were not included in analyses. Some participants were improperly administered the adolescent version of The Parenting Scale, making their data unusable in this study. The state of this archival data reflected both the development of Triple P as a new EIP here at RCC and the lack of previous program analysis. As program delivery was possible without program analysis, the completion of measures was not monitored. Therefore facilitators ensuring that participants completed the provided intervention measures was not always adhered to. This presents the need for more vigilant follow up processes. 22 P age

23 In addition, the assumptions of univariate normality, multivariate normality, and sample size have been violated, and thus the results of the MANOVAs should be interpreted with caution. Results Missing Data Several participants in the sample were missing data on their questionnaires. Participants who did not complete the entire pre-intervention or post-intervention questionnaires were excluded from analyses. Data was imputed for participants who were missing less than 25% of the items on a specific questionnaire. This was done using the expectation maximization method, which is an algorithm in SPSS which uses available information to estimate scores for each missing item. Of the 178 participants of the Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program, 89 were excluded from analyses due to missing data (50% of the original sample).the expectation maximization algorithm was used to impute data for the cases in which less than 25% of the items for a given questionnaire were missing. Assumptions All assumptions for the statistical tests were conducted prior to analysis. Findings Evaluation Question 1: Are parents increasing positive parenting strategies? For the 23 parents who participated in the Level 4 Group Pre-Adolescent Triple P program, the outcome variables for this analysis were the two subscales of the Parenting Scale (Arnold et al., 1993), the two subscales of the Being a Parent Scale (Johnston & Mash, 1989), and the three subscales of the Depression Anxiety Stress Scale (Lovibond & Lovibond, 1995). The results of this one-way, repeated measures MANOVA were significant, p=.042. Improvements were found on all subscales, except for the 23 P age

24 DASS Stress subscale, and the DASS Anxiety subscale (See Table D - Univariate results for Level 4 Group Pre-Adolescent Triple P program, Question 1). Table D - Question 1, Univariate results for Level 4 Group Pre-Adolescent Triple P program Table D - Question 1 - Results for Level 4 Group Pre-Adolescent Triple P program (N=23?) Subscale Mean (SD)- Pre Mean (SD)- Post F P Partial η 2 PS Laxness (1.49) (0.83) PS Over reactivity (1.24) (1.18) PS Hostility 6.28 (0.73) 4.91 (0.49) BPS Satisfaction (1.95) (1.65) BPS Efficacy (1.58) (1.29) DASS Depression 7.17 (1.92) 3.95 (1.21) DASS Anxiety 5.91 (1.72) 3.26 (0.95) DASS Stress (1.97) 7.48 (1.60) PS Laxness (1.49) (0.83) PS Over reactivity (1.24) (1.18) For the 16 parents who participated in the Level 4 Group Teen Triple P program the outcome variables were the two subscales of the PSA (Irvine et al., 1999), and the three subscales of the DASS (Lovibond & Lovibond, 1995). The results of this MANOVA were significant p =.003 and significant improvements were found for all subscales except for the DASS Stress subscale (see table 2 for full results). Therefore, this hypothesis was supported. Table E - Question 1 - Univariate results for Level 4 Group Teen Triple P program Table E - Question 1 - Results for Level 4 Group Teen Triple P program (N=16?) Partial η 2 Mean (SD)- Pre Mean (SD)- Post F P Partial η 2 PSA Laxness (1.15) (1.48) PSA Over reactivity (1.42) (1.13) DASS Depression (3.03) 6.25 (1.91) DASS Anxiety (2.72) 4.44 (1.23) DASS Stress (3.03) (1.91) Evaluation Question 2: How effective is the program in improving children s behavioural issues? Since there were only 8 participants in the group program with complete questionnaires (as opposed to the mixed group and group-teen program), results for both programs have been combined to provided a 72 participant sample group. The outcome variables for this analysis are the five subscales of the 24 P age

25 Strengths and Difficulties Questionnaire (Goodman, 1997). The results of these one-way, repeated measures MANOVA were significant p <.001 and the results suggest that there are improvements on all subscales except for the Peer Problems subscale. See table F Question 2, Univariate Results for the full results. Before the intervention, 90% of the cases were in the clinical or borderline range of functioning on the SDQ, and after the intervention 68% of the cases were in the clinical or borderline range. Table F Question 2, Univariate Results Table F - Question 2 - Results, (N=72) Subscale Mean (SD)- Pre Mean (SD)- Post F p Partial η 2 SDQ- Emotional Symptoms 4.24 (2.49) 3.31 (2.43) < SDQ- Conduct Problems 5.38 (2.19) 4.32 (2.29) < SDQ- Hyperactivity 6.90 (2.41) 5.75 (2.58) < SDQ- Peer Problems 3.80 (2.25) 3.64 (2.34) SDQ- Prosocial Behaviour 5.97 (2.10) 6.58 (1.94) Evaluation Question 3: What is the relationship between positive child behaviour outcomes and parental stress reduction? In order to examine the association between reductions in parental stress and positive child behaviour outcomes, a bivariate correlation was run between difference scores of the DASS stress subscale, and the 88 SDQ total scores. There was a correlation of.121, which was not statistically significant p = 130. This indicates that there is no significant association between changes in parental stress and child behaviour outcomes in this sample. Evaluation Question 4: Does success in the Triple P program vary based on initial ratings of child mental health concerns? Data from 7 participants who completed the Level 4 Group Pre-Adolescent Triple P program and the 45 participants who completed the mixed Level 4 Group Pre-Adolescent Triple P program and Level 4 Group Teen Triple P program were analysed together. Only participants who had BCFPI scores available were included in this analysis. Four groups were created based on clinical cutoff scores from the Internalizing problems and Externalizing problems subscales from the BCFPI (Cunningham et al., 2008). 25 P age

26 There were 19 children classified as having externalizing problems only, 2 children classified as having internalizing problems, 25 children classified as having comorbid internalizing and externalizing problems, and 7 children classified as having neither. Due to sample size limitations and missing data, this analysis was run without the internalizing problems only group, and the SDQ (Goodman, 1997) subscales will be the only outcome variables used. A 2 x 3 mixed model MANOVA was conducted with 51 participants. The two time points (preintervention and post-intervention) are the within-factor, and the three groups based on BCFPI scores are the between-factor (externalizing only, comorbid, or neither). The outcomes for this analysis are the five subscales for the SDQ (Goodman, 1997). There is an overall improvement from pre-intervention to post-intervention, p =.001. Follow-up analyses indicated that significant improvements were seen for all subscales except for the Peer Problems subscale. See (Table G - Question 4, Means for Group, by SDQ Subscales) for means, and (Table H - Question 4) for univariate results. Table G - Question 4, Means for Group, by SDQ Subscales Table G - Question 4 - Means for Group, by SDQ Subscales (N=51) Subscale Group Mean- Pre SD- Pre Mean- Post SD- Post SDQ- Emotional Symptoms Externalizing Comorbid Neither Total SDQ- Conduct Externalizing Comorbid Neither Total SDQ- Hyperactivity Externalizing Comorbid Neither Total SDQ- Peer Problems Externalizing Comorbid Neither Total P age

27 Continued Table G - Question 4 - Means for Group, by SDQ Subscales (N=51) Subscale Group Mean- Pre SD- Pre Mean- Post SD- Post SDQ- Prosocial Behaviour Externalizing Comorbid Neither Total Table H - Question 4, Univariate Results Question 4 Results (N=51)? Time Time * Group Interaction Subscale F P Partial η 2 F p Partial η 2 SDQ- Emotional Symptoms SDQ- Conduct Problems SDQ- Hyperactivity SDQ- Peer Problems SDQ- Prosocial Behaviour There was also a significant main effect of group (F (10, 88) = 2.22, p =.024, Wilk s Lambda =.638, partial η 2 =.201). A follow-up univariate contrast revealed that there was a significant difference between groups on the Emotional Symptoms subscale (F = 10.67, p <.001). The children who had neither internalizing nor externalizing symptoms scored lower on this scale than the children with externalizing symptoms only, or comorbid symptoms only. There was no significant interaction overall between time and group (F (10, 88) = 1.53, p =.143, Wilk s Lambda =.726, partial η 2 =.148), which indicates that there was no differential treatment effect based on groups. Thus, this hypothesis was not fully supported. However, univariate analyses indicated that there is a significant interaction for the Hyperactivity subscale of the SDQ, with the neither group experiencing very little change, and the externalizing only group showing steeper declines. Discussion and interpretation To perform this study, all current and archival parental pre and post parental self-evaluated measures were inputted into SSPS. The results supported the effectiveness of this intervention. Although these findings revealed some positive changes, they must be interpreted with some caution based on the fact that most of this data was archival and much of the data was incomplete. 27 P age