Brandon University Psychiatric Nursing Program Evaluation Report

Transcription

1 Brandon University Psychiatric Nursing Program Evaluation Report Stephanie Knaak, Andrew Szeto, Renee Robinson, Jane Karpa & Scott Patten November 03

2 This project was made possible through funding from the Opening Minds Anti-stigma Anti-discrimination initiative of the Mental Health Commission of Canada. The work of the Mental Health Commission of Canada is supported by a grant from Health Canada. The views expressed in this publication are those of the authors. OPENING MINDS: CHANGING HOW WE SEE MENTAL ILLNESS Stigma is a significant concern for those living with a mental illness. Stigma is a primary vehicle for the entrenchment of discriminatory behaviours, and has been identified as a major barrier to timely and accessible care, recovery, and quality of life for persons living with mental illnesses (-). As such, reducing the stigma and discrimination associated with mental illness is becoming an increasingly important focus. One particular area of focus is that of the healthcare sector. As part of its 0-year mandate, The Mental Health Commission of Canada (MHCC) has embarked on an anti-stigma initiative called Opening Minds (OM) to change the attitudes and behaviours of Canadians towards people with a mental illness. OM is the largest systematic effort undertaken in Canadian history to reduce the stigma and discrimination associated with mental illness. OM is taking a targeted approach, with healthcare providers being one of its main target groups. OM s philosophy is not to reinvent the wheel, but rather to build on the strengths of existing programs from across the county. As such, OM is conducting evaluations of various programs to determine their success at reducing stigma. OM s goal is to replicate effective programs nationally. (3) A key component of these program evaluations is contact-based educational sessions, where target audiences hear personal stories from, and interact with, individuals who have recovered or are successfully managing their mental illness. The success of contact-based anti-stigma interventions is generally supported by international studies as a promising practice to reduce stigma. OM is partnering with programs in Canada who are reaching out to its initial target groups: youth, healthcare providers, the workforce and news media. Over time, OM will add other target groups. For more information, go to: BACKGROUND In the spring of 009, Opening Minds issued a Request for Interest (RFI), seeking existing programs aimed at reducing stigma among its initial target groups of healthcare providers and youth. J. Renée Robinson and Jane Karpa, professors in the department of Psychiatric Nursing, Faculty of Health Studies at Brandon University, responded to this RFI and entered into a partnership with OM.

3 Professors Robinson and Karpa have developed (and currently teach) a course on Psychiatric Rehabilitation and Recovery, which students take in the fourth and final year of degree program in Psychiatric Nursing. The course is offered during the Fall semester at both the Brandon and the Winnipeg campuses. A key component of this course is the Recovery Narrative assignment, whereby students are paired up with an identified client (a person with lived experience of mental illness living in the community), whom they meet over the course of the term and for whom they construct a recovery narrative. The purpose of the assignment is for students to get to know clients on a personal level in order to gain an in-depth understanding of their experiences. Students and clients clients meet a minimum of nine times throughout the term. Meetings take place at a time and location of the client s choosing (other than at the student s home), are approximately one hour in length, and are conducted in person. Students then prepare a to 0 page recovery narrative to describe the client s life story in terms of the following key experiences: trajectory of illness to date and expectations for the future; impact of their mental illness in the areas of education and employment, family and social relationships, finding and maintaining housing, substance use, involvement with the legal system, other health problems, experience with the healthcare system (hospital and community services), income / poverty (including impact on housing, transportation, communication and social/recreational activities), stigma, and recovery. (4) Clients review and provide feedback on the content of the completed narrative. The student s final assignment grade, which is weighted at 30% of their final course grade, is based primarily on the client s evaluation and feedback. Completed recovery stories are held in strict confidence and are only shared (e.g., with case workers, treating physicians, with other students and teachers, etc.) if both the client and student provide written permission. The Recovery Narrative process is facilitated by a preceptor who: discusses the practicum with the client and solicits their participation; introduces the student to the client; serves as a resource to the student; assists the client in evaluating the recovery narrative; monitors clinical practice and client response to student involvement; and provides feedback to instructor. (4) OM teamed up with professors Karpa and Robinson to evaluate the effectiveness of the Recovery Narrative assignment at reducing mental illness-related stigma among student psychiatric nurses. The particular course evaluated was the Fall 0 course offering at both Brandon and Winnipeg sites, involving a total of 43 students. Student enrollment for the course at the Winnipeg site was students. At the Brandon campus, 8 students completed the course. Clients complete and submit an evaluation form. 3

4 3 EVALUATION METHODS The evaluation contained both quantitative and qualitative research components. The quantitative component of the evaluation was undertaken by the research team at OM. The qualitative research was undertaken by Drs. Karpa and Robinson. 3. Quantitative Component In order to assess stigma toward mental illness, students were provided a questionnaire package at four different time points. The first survey was completed by students in early September 0, at the beginning of the course and introduction to the Recovery Narrative assignment (pre-test). The second questionnaire was completed in early December 0, at the end of the Psychiatric Recovery and Rehabilitation course and completion of the Recovery Narrative Assignment (post-test). The first followup survey was administered in mid-february, approximately two months following the completion of the assignment (follow-up ), while the second follow-up was completed in mid-may, approximately five months following completion of the Recovery Narrative assignment (follow-up ). All surveys contained the 0-item Opening Minds Scale for Healthcare Providers (OMS-HC) so that changes over time could be assessed. The OMS-HC is a 0-item scale that measures healthcare providers attitudes and behavioural intentions toward people with a mental illness. () A copy of this scale is provided as Appendix A. To complete the scale, participants are asked the extent to which they agree or disagree with each item. Items are rated on a -point scale: strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. To create a total score for the scale, all 0 items are summed for each participant. Total scores can range from 0 to 00, with lower scores indicating less stigma. For this particular evaluation, Cronbach s alphas for the scale were.73 for the pre-test,.76 for the posttest,.7 for the first follow-up survey and.90 for the final follow-up survey. These results suggest an acceptable level of internal consistency for the scale at all time points. Paired t-tests were used to analyze total score changes on the scale at the four time points. The examination of score changes from baseline to post-test was based on a paired analysis of participants who completed both the pre-test and the post-test surveys (n=3). The examination of scores for the two follow up time periods was also based on paired analyses (n=30 for the analysis of scores at follow-up ; n= for analysis of scores at follow-up ). The direction and magnitude of change from pre to post intervention was also examined. Further, by grouping certain questions from the scale together, the OMS-HC was used to examine three main dimensions of stigma: attitudes toward people with mental illness, healthcare professionals attitudes about disclosure of a mental illness, and desire for social distance. A threshold was also created to While original scale testing suggested two main subscales attitudes towards mental illness and attitudes towards disclosure of a mental illness () subsequent analyses have identified three subscales within the OMS-HC (paper in progress). 4

5 measure success, defined as the proportion of participants who obtained 80% or more correct (nonstigmatizing) answers on the scale. Results are described in Section 4, below. 3. Qualitative Component A total of 0 psychiatric nursing students and 6 clients participated in qualitative interviews regarding their experiences with the Recovery Narrative assignment. The purpose of the qualitative component of this evaluation was to gain a greater understanding from both clients and students about the impacts of the assignment, key learnings, costs and benefits of participating, and ways to improve the assignment. Appendices B and C provide the interview guides used for the student and client interviews. Interviews were transcribed and analyzed for key themes. Preliminary results are described in Section, below. 3 4 QUANTITATIVE RESULTS 4. Participant Demographics A total of 43 students were enrolled in the Psychiatric Rehabilitation and Recovery course at Brandon University for the fall 0 semester. In all, 4 students completed the pre-test survey, 33 completed the post-test survey, 3 completed the first follow-up survey (follow-up ), and six participants completed the second follow-up survey (follow-up ). Table highlights the breakdown of participants by age, gender and ethnicity, as based on responses provided in the pre-test survey (n=4). As shown in the table, approximately six in ten participants were from the Winnipeg campus (6.0%), while approximately four in ten (39.0%) were from the Brandon campus. As well, the vast majority of participants were female (90.%), Caucasian (78.0%), and Canadian-born (9.%). Approximately 4.6% of participants identified as being of First Nations, Métis or Inuit descent. The majority of participants were between the ages of and years (63.4%). The overall mean age of participants was. years. 3 Analysis is not yet complete, and discussion is necessary on a number of elements. A more comprehensive report of this qualitative research will be produced in the coming months, for publication in a peer-reviewed academic journal.

6 Table. Demographic Characteristics of Participants n (=4) % Campus Winnipeg Brandon Gender Female Male Ethnicity Caucasian First Nations / Metis / Inuit Black / African - Canadian Did not respond Birth Country Canada Uganda USA Age (mean age=.) 0 and under Over % 39.0% 90.% 9.8% 78.0% 4.6%.4% 4.9% 9.%.4%.4% 7.3% 63.4%.% 9.8% 4.9%.4% The demographic section of the survey also asked respondents about their personal experience with mental illness. These results are highlighted in Figure. As shown, all participants had previously interacted with persons with a mental illness (00%). As well, the vast majority of participants said they personally knew a close friend or family member with a mental illness (87.%). Just over three in ten indicated that they been treated for a mental illness at some point in their lives (3.7%). Figure. Participant Experience with a Mental Illness (MI) (n=4) 00% 90% Yes No 00.0% 80% 87.% 70% 60% 68.3% 0% 40% 30% 0% 3.7% 0% 0% Ever been treated for a MI.% Know friend or family member with a MI? 0.0% Previous interaction with somone with a MI? 6

7 4. OMS-HC Total Score Change from Pre- to Post-Test As mentioned above, examination of score changes from baseline to post-test was based on a paired analysis of participants who completed both the pre-test and the post-test surveys (n=3). To create total scale scores for the OMS-HC, items were summed across all surveys having complete data from pre-test to post-test. Total scores can range from 0 to 00, with lower scores indicating less stigma. For the pre-test, total scores ranged from 30 to 64, with an average of 40.8 (SD = 6.9). It is worth noting that this baseline score is among the lowest OM has observed on the OMS-HC across the many evaluations it has conducted (6). For the post-test, total scores ranged from to 9, with an average of 37.6 (SD = 6.98). These results are highlighted in Figure. Figure. Average total scores: pre-test and post-test (paired analysis, n=3) 44.0 OMS-HC average total score Pre program Post program This observed difference in score from pre to post program represents a 7.3% relative improvement, and a standardized mean difference (SMD) of.4, which may be considered a moderate effect. Additionally, results of a paired t-test indicate the mean score change from pre to post program to be statistically significant [t(3)=3., p=.004]. Figure 3 shows the percentage of participants who had a total score increase (i.e., more stigma), total score decrease (i.e., less stigma) or a score that had no change. As highlighted in the figure, just under seven in ten participants had scores that improved from pre to post program (68.8%). Just under three in ten had an increase in score from pre to post program (8.%), while 3.% had no change in score from pre to post program. 7

8 Figure 3. Direction of Change from Pre to Post: OMS-HC scale (n=3) Score decrease (less stigma) 68.8% Score increase (more stigma) 8.% No change 3.% The minimum delectable change (MDC) statistic is another method for examining changes in scores from pre- to post-workshop. The calculated MDC for the OMS-HC scale is This suggests that a score increase or decrease of 6. points or more on the OMS-HC scale reflects a true change in attitude one that cannot be attributed to measurement error. As highlighted in Figure 4, when the MDC is applied to participants score changes from pre- to postprogram, results suggest that no participants actually became more stigmatizing (0%). The MDC helps to clarify that the 3% of participants whose score increased (see Figure 3) all had scores within the margin of error measurement and therefore there is no strong evidence that their scores actually deteriorated. When the MDC is applied to the scores that improved, the percent of participants who truly became less stigmatizing as a result of the course is.9%. Figure 4. Pre to Post Score Change using the MDC Statistic (n=3) 90% 80% 70% 60% 0% 40% 30% 0% 0% 0%.9% 0.0% % of 4 The MDC for the OMS-HC scale was calculated based on a standard error of measurement (SEM) of [from test-retest results on the full scale, see()] and a z score of.6 (90% confidence level). The formula for calculating this statistic is as follows: MDC=SEM* *z score associated with confidence level of interest. 78.% Truly became less stigmatizing Truly became more stigmatizing Cannot determine at desired level of confidence 8

9 4.3 Stigma Content Areas The OMS-HC scale contains within it three main content areas, each measuring a specific dimension of stigma. While original scale testing () suggested two main subscales attitudes toward mental illness and attitudes toward disclosure of a mental illness subsequent analyses have identified three subscales within the OMS-HC (paper in progress). The first subscale or dimension is healthcare providers inclinations toward disclosure of a mental illness. This dimension can be used to provide an indication of the stigma healthcare providers believe exists due to having a mental illness and how this would impact help-seeking. The specific scale items used to measure this dimension of stigma are as follows: Q4. If I were under treatment for a mental illness, I would not disclose this to any of my colleagues Q6. I would see myself as weak if I had a mental illness and could not fix it myself Q7 I would be reluctant to seek help if I had a mental illness Q0. If I had a mental illness, I would tell my friends The second dimension is that of attitudes towards people with mental illness and includes the following statements: Q. I am more comfortable helping a person who had a physical illness than I am helping a person who has a mental illness Q. Despite my professional beliefs, I have negative reactions towards people with a mental illness Q3. There is little I can do to help people with mental illness Q4. More than half of people with mental illness don t try hard enough to get better Q8. Healthcare providers do not need to be advocates for people with mental illness Q0. I struggle to feel compassion for a person with a mental illness The third dimension is that of social distance. It includes the following statements: Q3. If a colleague with whom I work told me they had a managed mental illness, I would be as willing to work with him/her Q8. Employers should hire a person with a managed mental illness if he/she is the best person for the job Q9. I would still go to a physician if I knew that the physician had been treated for a mental illness Q7. I would not want a person with a mental illness, even if it were appropriately managed, to work with children Q9.I would not mind if a person with a mental illness lived next door to me 9

10 Scores for these three dimensions were created by calculating the mean score for each of the three content areas. A summary of the mean score changes for each stigma content area from pre to post program is provided in Table. As noted in the table, all of the three content areas showed a reduction (i.e., improvement) in mean score from pre to post program. None of the content area score changes were statistically significant, although for the dimension of disclosure the mean change was approaching statistical significance [t(3)=.9, p=.064]. Also, as shown in the table, both the content areas of attitudes toward people with a mental illness and desire for social distance had very low baseline mean scores, at.49 and.78 respectively. Given these low baseline scores, the possibility exists that the scope for improvement may be less for these dimensions than for other indicators of stigma, like disclosure/help seeking, for example. Table. Stigma Content Areas: Changes in Respondent Mean Scores from Pre to Post Program (n=3) Content Area Pre-test mean (9%CI) Post-test mean (9%CI) Pre to Post % Change (9%CI) Paired t-test Attitude towards people with mental illness.49 (.4-.8).47 (.40-.4) 0.0 ( ).4% t(3)=0.4 p=.687 Disclosure/help-seeking.4 ( ).3 (.3-.4). (-.0-.4) 7.% t(3)=.9 p=.064 Social distance.78 (.7-.8).67 (.9-.7). ( ) 6.% t(3)=.66 p= Individual Item Analysis Using paired data, individual item changes on the scale were measured from pre to post-program. Five of the 0 items on the scale showed a statistically significant improvement from pre to post-program. Those statements are as follows: I would see myself as weak if I had a mental illness and I could not fix it myself. I would be reluctant to seek help if I had a mental illness. There is little I can do to help people with a mental illness. People with mental illness seldom pose a risk to the public. The best treatment for mental illness is medication. No statements showed a statistically significant increase in stigma from pre to post-test. 0

11 4. Threshold of Success Another way to examine the impact of the program on mental illness stigma is to examine how many participants reached a threshold of success on the OMS-HC scale; in other words, how many participants responded to a certain number of items in a non-stigmatizing way. The threshold of success measure is derived by recoding each participant s response on the OMS-HC scale to represent either a stigmatizing or a non-stigmatizing response. For example, the statement I would see myself as weak if I had a mental illness and could not fix it myself was recoded as non-stigmatizing if the respondent selected strongly disagree or disagree, and recoded as stigmatizing if the respondent chose neutral, agree, or strongly agree. Figure shows the cumulative percentages of participants who had non-stigmatizing responses for each possible score out of 0 at pre-test and again at post-test. A threshold of 80% (or at least 6 out of 0 correct - i.e., non-stigmatizing answers) was used as an indication of success on the OMS-HC. As highlighted in the figure, prior to participating in the course, 40.7% of participants were across the threshold of success on the scale. At the completion of the program, the percentage who were across the threshold level of success had increased dramatically, to 7.9% of participants. Figure. Cumulative Percent of Non-stigmatizing Responses on OMS-HC for Pre-test and Post-test 00% 90% 80% 70% 60% 0% 40% 30% 0% 0% 80% Threshold of Success Pre-test Post-test 0% at least 0 at least at least at least 3 at least 4 at least at least 6 at least 7 at least 8 at least 9 all 0

12 4.6 Follow-up Results As noted above, the OMS-HC scale was distributed to program participants approximately two months following the completion of the course, and again at five months post-course completion. Examining scale scores a period of time after program completion allows an examination of the extent to which program impacts may have been sustained over time. Results are based on analyses conducted on paired data. Figure 6 shows the average total scores across time for those participants who completed a survey at each of the first three time points (pre-test, post-test and follow-up ). As highlighted in the figure, the positive score improvement realized from pre to post-program was sustained to the time of the first follow-up survey (pre-test score, 40.7; post-test score, 37.3; follow-up score, 37.). Results of a paired t-test showed the score improvement from baseline to first follow-up to be statistically significant [t(3)=3.00, p=.007]. Figure 6. OMS-HC score for participants at three time points: pre-program, post-program and at followup (paired data; n=4) OMS-HC average total score Pre-test Post-test Follow-up Only six participants completed the second follow-up survey, which was administered approximately five months post-program completion. Paired analysis of data from participants who completed a survey at both baseline and at follow up (n=) shows that scores were improved over these two time points (pretest average total score= 40.60, follow-up average total score=37.3). While these results are consistent with the results reported above, the high level of attrition from followup to follow-up prevents the ability to draw conclusions about the extent to which program impacts may have been sustained to the time of the second follow-up period.

13 QUALITATIVE RESULTS This section provides preliminary findings from the qualitative component of the Evaluation of the Recovery Narrative Assignment at Brandon University, conducted by J. Renée Robinson and Jane Karpa. Final results will be written in article form, for publication in a peer reviewed academic journal. This article is currently in process.. Student Interview Analysis A total of ten psychiatric nursing students, completing their fourth and final year, participated in an interview concerning the Recovery Narrative assignment. The purpose of interviewing the students was to determine the overall effectiveness of this assignment on influencing student attitudes about people who live with mental illness and whether any attitude changes would impact current and future psychiatric nursing practice. Questions were asked about what they learned and the degree to which learning was based on this specific assignment. Questions were also asked about specific aspects of the assignment and ways to improve the assignment (Appendix C). The student interview data was analyzed using a latent content analysis (involving an interpretation of the meaning of the data) as outlined by Graneheim & Lundman (7). The preliminary result is the construction of one major theme titled: They are Us - We are Them followed by four sub-themes: recovery narrative as a process for attitude change through development of broader perspectives; recovery narrative as a process for increasing opportunities to engage with individuals living with a mental illness to promote attitude change; recovery narrative as a process for impacting current and future practice; and recovery narrative as a process of reflecting on feelings of acceptance and vulnerability... They are Us We are Them In this theme, students described how the recovery narrative assignment caused a change in their perceptions of how they viewed and thought about individuals with severe and persistent mental illnesses. Students described being able to understand there was more to the lives of these individuals than just their mental illness; they were indeed human beings with whole lives. It helped to reduce stigma because it just helped me realize that they re a real person, that they have real issues not just issues related to their mental illness, that they have a life. Kind of like having an epiphany, like holy smokes, people can actually live in the community and have schizophrenia and have delusional symptoms and yet still be functioning adults in the community it didn t mean that you were screwed for life, but you could actually have a life. And I had to ask myself, why is this diagnosis important to me, what does the preconceived idea of a diagnosis really mean to me.and I guess it s not actually that important. This section of the Evaluation Report was authored by J. Renée Robinson and Jane Karpa. 3

14 Students discussed reaching a level of recognition that these individuals were experiencing living in the same manner as themselves or their family members. That idea of our clients being us.our mothers, fathers, those around us while another commented; It helps reduce the us and them feeling that we might have it reduces those barriers, you see them as a real person. For the students the recovery narrative assignment became a humanifying process whereby individuals experiencing severe and persistent mental illnesses came to be recognized as humans like us all, and not relegated to the stigmatizing boundaries of a particular mental illness diagnosis. It helped to reduce stigma because it just helped me realize that they re a real person, that they have real issues not just issues related to their mental illness, that they have a life... Recovery Narrative as a Process for Attitude Change through Development of a Broader Perspective This sub-theme highlights the process the students went through to reach the understanding that we are all human beings living a life. The recovery narratives allowed the individuals to tell their life stories and in hearing clients wider life experiences students developed broader perspectives. When you see somebody on a ward you re just seeing them in the moment, and you don t really think about how they were as a child.it (the recovery narrative) makes you see the whole story. importance of recognizing that each person has a story. Yeah, we (psychiatric nurses) like people get very caught up in our own lives and apply our own feelings to others to recognize each person s life is different and their story is different story telling is one of the most important things in humanity special and unique.everybody has a need to tell their story, and recognize their own story not the one you attach to them...3 Recovery Narrative as a Process for Increasing Opportunities to Engage with Individuals Living with a Mental Illness to Promote Attitude Change This sub-theme denotes the overall positive opinions the students had about this assignment and the recommendation that they would appreciate more opportunities for individuals to share their stories with the students, as well as other populations, as a means of helping to promote attitude change about those with significant mental health issues. Students expressed the need for people to have more exposure to individuals recovery processes. Set up a night, like a reading, like a coffee house where people could come and share their stories allowing people to have the opportunity to have positive experiences and seeing that people can actually recover. People need more exposure to positive experiences of seeing people in recovery and when they do their energy level and the environment is different from a place that has a negative attitude. 4

15 ..4 Recovery Narrative as a Process for Impacting Current and Future Practice This sub-theme addresses the students discussions regarding the potential of the recovery narrative to influence their current and future psychiatric nursing practices. For the most part students indicated that the recovery narrative caused them to pay attention to several current psychiatric nursing skill areas and that it will influence their future nursing practices. It (the recovery narrative) taught me how to really listen...and not talk so much and being nonjudgmental and caring as we would care for ourselves. ; Changed my perspective on what nursing is.it is a partnership as opposed to telling them what they are going to do. I think that storytelling peoples experiences and collaborating on that story will really influence me in doing first admittance notes.i want to hear how you came to be here today, which seems like a basic question and it s on the chart but in the way to ask it, I feel it could be storied rather than just a simple answer... Recovery Narrative as a Process on Reflecting on Feelings of Acceptance and Vulnerability This sub-theme is representative of the students reflections on feelings about having the client evaluate the assignment and recommendations to improve the learning associated with the recovery narrative assignment. In general, students saw client evaluation of the recovery narrative as a positive experience. They recognized that this was a partnership and the clients had the power and the right (because this was really their story) to evaluate the narrative. I felt good about it. I think that is the right they should have It would be weird if there wasn't an evaluation. He evaluated my work at a series of points by running lines through things, and I got really upset and pissed off because my initial reaction was to think this is my work but then I had an epiphany that we are a partnership my perspective was changed so in the end it was okay. The students also described feelings of vulnerability related to engaging in the recovery narrative process and the different evaluative experience. They expressed an awareness that the evaluation process was linked to the quality of the relationship they were building with the client, and this awareness caused them to feel vulnerable and scared at times. I thought it was good to have the client evaluate because you got their perspective It may have been different if you did not have a good relationship with the client.. I felt good and like that she had given me positive feedback along the way so I wasn't scared that she was going to give me a negative evaluation, although if she had I would want to know that as well. As a result of these vulnerable feelings, several students expressed wanting further opportunities for dialogue and sharing with the course educator or preceptor as a method of receiving more direct support. More scheduled meetings for debriefing with the professor cause there wasn't really that piece.because you want to be able to keep confidentiality you wanna be able to depack some of

16 the stuff that's coming up.cause I know a lot of the students were having a lot of different things coming up and they didn't know how to handle the situations maybe a little bit more guidance individually.. Client Interviews A total of 6 clients who had participated in the Recovery Narrative assignment at least once before, also participated in this study. The purpose of the client interviews was primarily to obtain client perspectives on the benefits and costs of their participation in the recovery narrative assignment. Questions were also asked about specific aspects of the assignment, ways to improve the assignment, and beliefs about impact on the student (Appendix D). Client participants were asked whether this was their first time participating in the Recovery Narrative assignment, and what factors encouraged participation. This was the first experience with the recovery narrative assignment for half of the client participants but others had participated up to six times. Participants were initially invited to participate by workers, and encouragement by workers was a factor in deciding to participate in the assignment. Comments suggest that workers viewed participation as beneficial for clients. Client participants often reported more than one reason for participating. Clients who participated more than once indicated they continued to participate because they enjoyed it, and some clients chose to participate to help students learn. The majority; however, viewed participating in the recovery narrative as therapy. "It helps me to understand my illness." I wanted to look at my own progress, and to help others. Improve the healthcare system. "I ve been finding that it s very therapeutic to, um, be speaking about the events that have, um, basically changed my life forever I want to get my story out there, and at the same time it helps me as well.".. Benefits of Participating Client participants were asked about their experience in participating in the assignment, as well as any benefits or drawbacks to participation. In general, client participants reported that the experience was pleasant, even fun. It was an opportunity to get out of the house, have coffee, and talk with someone. It was an outing, cause I don't have much of a life other than going to work and going for coffee all the time." "Having your story in front of you is really cool." "It was a thing to do, you know. Is go out, drive around, go through the drive through and come back in and kick somebody at chess, I mean that, that made my morning, you know." Participants consistently reported benefits related to their relationship with the student, therapeutic effects of sharing their story, and satisfaction with helping to educate new health professionals. 6

17 ... Relationship with the student Clients Clients reported looking forward to seeing the student, enjoying consistent contact over the period of the term, feeling comfortable with the student, and valuing the way students listened. A number of clients reported that meeting over the course of the term provided continuity and helped to combat social isolation. Participation also provided opportunities to meet new people. "It was a bit of stability when I didn't have any else." At the time I was, like, super, super isolating, too But, no. I, I always looked forward to meeting her. "One of the things that I really enjoyed about this process was that it was every week for a while. I had some kind of continuity. That was huge for me. Huge, huge, huge." "Well I made a commitment to come in at a certain time and I kept that and that s been very difficult for me. it kind of gave me that extra push to come in. So I think that s been a bit of a positive change." Clients used words such approachable, safe, comfortable, and very human to describe how they felt when speaking with the students. "She had a very pleasant demeanor about her. She was enjoyable to talk to. She made you feel that what you said was important so that was positive." "It was very comfortable. It was, there was, I had no stress. It was really quite an enjoyable time. Just the fact that I could sit down with someone basically that I did not know, and was able to talk fairly comfortably and I think in a fair bit of detail. Clients consistently reported feeling that they were listened to. "She was listening. That was a huge, huge thing. It wasn't like I thought the other professionals weren't listening, but, they also had their own agenda. They had stuff they wanted to tell me." "Get away from having the desk in between you and that client." "It was good just, you know, sitting down with somebody and having them actually listen to you." "Throughout the whole process, she wanted to make sure she got it in my, my voice. And she did a really good job." "She was very careful as we were going through things. I think partly because of the time frame we were both facing, and plus she didn t want me to be totally overwhelmed at the end. Participating in the assignment even led to changes in client attitudes toward service providers. I also learned that there, that, trusting the, the whole area of psychiatry. They re trying really hard to get better. "I think, uh, I've gotten back some old patience that I had lost. I had little patience often with the system. Little patience with the the individuals that were trying to help me." 7

18 "I mean I don t know what kind of selection process they have for the program but having seen these people, I didn t see, I didn t see anyone that I would have problems with dealing with them if I was in the hospital. You know, I think I would have done quite well with all of them." And, and from our side, as, as the clients, it s encouraging to know that there are people that care and understand which is, I mean, I ve had not really bad experiences in the hospital but just talking to these people and seeing how they re happening, it s like I, it s going to be a lot better for people down the road. And that is really encouraging. "So then I figured, well maybe it wasn t that I should have been upset or bitter at the facility. I think I should have probably recognized that the awareness wasn t there. And hopefully through, those of us that are able to and are willing to speak to others, hopefully that awareness will be, you know, be heightened."... Therapeutic Benefits The assignment was not specifically designed to provide therapeutic benefits to the client participants. However, almost every one of the clients reported therapeutic benefits. These benefits included increased confidence and self-esteem. As well, participants reported being more aware of other people, and better able to engage with them. "I think what it actually has done in that sense is that it has given me more confidence and definitely lack of confidence was a big problem for me. So I think I understand myself better and I know I can deal with it. And I can deal with it on my own and I can deal with it with other people. So definitely, I m definitely improving." "And when I talk to women now I tend to be more lighthearted and easy going and, you know, not shy." Many client participants reported increased awareness of personal strengths: "It really taught me to know where my strengths are." "I really have started to be proud about where I am now because I was in such a dark place. And it s really made me see my strengths. Like one, one day she asked me, you know, what are your strengths. And I ve never really thought about that. So it was really good to see, you know, these are my strengths and this is where I ve come from. That was really helpful." growth and recovery: "You know, it's just, I guess, made me open my eyes to see where I'd been and see where I'm at. When you read the recovery story it was like, I can't believe I've been there. It's like I can't believe I've been that down or that depressed or whatever... from then to now I improved a little bit." "When I sat down with the nursing student it became very much about me and that was really the ball of yarn that I needed to go, to untangle." "It allowed me to piece myself back together and to see what, it gave me an idea of where I had come from and it also really gave me an idea of where I was now. Um, by being able to see kind of where I had come from or what I had done once upon 8

19 a time, it, it helped me identify what I had lost and what I had kept, and it also reminded me of what potential was there that might get tapped again. And, before that it had all just been, it was lost and it was very, very, very random as to whether or not I got recovered. "And I remember feeling very woken up by the process. There's so, so, so very, very much that I had forgotten..., there was a lot that I discovered that I don't think I necessarily would have otherwise." "And so then it'd be I became aware of certain aspects of my recovery that I may not have previously been aware of." as well as areas for further growth: It did remind me of that I needed to work on. So that was a good thing. Just making me realize that I still have a lot to work on on myself. And just seeking out counsellors so that I can deal with that stuff that s kind of still tripping me up." Participation in the recovery narrative enabled clients to view their experience in different ways. People who participated more than once reported that they were able to see change with successive stories. "When I read the narrative, I realized that there are different ways of wording what s happened to me. And, and like how it s impacted my life." "I wanna talk about the good things about myself and not the bad things." "And when I talk about things I can figure out problems, and I can think, figure which way to turn and I can figure out, you know, where I can find a job." It makes me look at life in a different perspective."...3 Helping to change the system More than half of the client participants specifically mentioned feeling satisfaction with helping to educate future health professionals and contribute to changing the healthcare system. Clients expressed a belief that reducing stigma is important for society in general and for individuals as well. Participating in the recovery narrative assignment was viewed as an investment in the future. Clients also viewed participation as a means to serve as a role model, and hoped that telling their story would be of benefit to other people who live with mental health problems. Just sort of knowing that I was sitting with someone and having sort of an impact on what their future s going to be is kind of neat." "I think it was a great experience. I just hope that, that this, this can help people who are going into psychiatry to be more aware of what the person is going through. And to maybe find out how to question them more." "Initially I was a little bit hesitant. I mean I wasn t sure what I was getting into. But like I said, the minute that I met her, it was like another door had opened for me. And it was another opportunity to be able to hopefully help someone. I mean going into the, healthcare field where someone would be dealing with individuals with mental illness, I think it s important that that some of our 9

20 personal stories do come out and, to me it was all about helping her for her narrative. And I think for her, it was about helping me get through another step, another door, another hurdle. And it was, to me it was beneficial for both of us. And, um, I really appreciated the opportunity. It was, it really helped me open up about some things that I haven t opened up about before." Client participants were asked whether they believe that student learning would have a lasting impact on their future practice. Some clients did not feel well positioned to respond to the question, but were hopeful that student participation would influence their practice. Other client participants reported specific areas where practice would be affected. "She'll be a little bit more open about some of the comments made by clients she'll keep that in mind." "I think they saw an aspect of mental health that they need and may not have expected to find." "I think they... could be inspired. "I don't know... how many people she knows that are mentally ill, but I think she found out that people with mental illness do move on She can understand it a bit better... when she goes into the work force. She'll have an idea of how to deal with people with mental illness... hand on hand experience, that's what you need to make yourself a good psychiatric nurse and go into that field." "Actually I think she had a pretty good attitude to begin with. So I don t think that I m going to change anything. Hopefully I will reinforce where she s at." "She actually said that she enjoyed being able to understand from another point of view. Because now this helps her, she said, when she goes out and works with people, this will help her in how to deal with people. And a lot more than before because, you know, you can go to school and learn anything. But if you don t know what somebody experienced." I think she was able to be a bit more compassionate just by realizing that, you know, we all have a story that we re healing from." So I think in that way, it, it, opened her eyes to the fact that you can t always tell what s really happening. You have to ask the questions. "We even talked about it that listening is one of the major components that needs to be stressed to psych nurses. And she agreed. Like she said, that s one thing she s learnt is to listen to the story. Not to make judgment on it.".. Costs of Participation While client participants universally reported that there were no drawbacks to participation, three of the clients reported that bringing up old emotions was stressful even though the discussion became an opportunity for growth. "When I would meet with her, I would tell her all the stuff and then I d go home and think, OK, what else do I need to say? And it would bring up a lot of stuff that I wasn t prepared to, to really look at at the time. You know there was one thing that I hadn t thought about in 0 years. And I 0

21 never put words to it. She handled it really well. She got me to write it down instead of tell her what it was because I just wasn t at a place where I could say it. So that was really a hard, just remembering a lot of different things. And not feeling like I had someone to debrief with. But I just wasn t prepared to have all these memories come in and, yea, that was, that was quite hard." "At the time everything was going fine. But I think what I didn t realize is that after I brought up all that stuff, I kinda went through a little bit of issues by talking about it. Like bringing up things that maybe I hadn t quite resolved 00% and just bringing old feelings up, you know. But accepting that that happened was what I had to do. And then just being OK with not feeling so great for a little while. I ve got, think some balance now and I m looking at the future and what I can and can t do. And I think that really has helped me now feel a little bit more at ease with everything." The people who are quoted above both stressed that the student handled it well, and that the experience led to growth. For instance, in the first circumstance, the client spoke to the student about it and they built a debriefing into the session. The second client reported that the disclosure led to growth and acceptance of feelings, and gave him permission for self-care. "She, she did a terrific job. I m really impressed with, with what she did and how she handled, you know, having these feelings come up and respecting me with that and giving me the minutes to debrief."..3 Course Structure..3. Efforts at role reversal Having clients evaluate student performance provides an opportunity for reversal of traditional roles. Client participants were asked how they felt about evaluating the recovery narrative in order to explore client perspectives on the shift in traditional roles. None of the clients answered in a way that suggest awareness of the goal, and instead reported on how they felt personally in reviewing the story, how they felt in participating in this research, or how the student performed. There were two examples though where responses to other questions suggest client participants valued having control. The first client reported liking that he had control over the content and distribution of the story. He reported that it was easier to be really honest when he knew that the story was not going to be shared with a particular person. The second client had participated in the narrative a number of times. He required that the student play chess with him and, if the student didn t know how to play, he would teach them. In this circumstance the client participant clearly assumed the role of teacher...3. Areas for improvement in the assignment Participants had several suggestions for improving the assignment, many of which involved greater attention to client preferences. For instance, proceeding at a pace that was not overwhelming, providing a place to meet that was quiet instead of distracting, or using a recorder instead of taking notes. Students are instructed to tailor the meetings to the needs / preferences of each client, but greater attention to this aspect would be of benefit.

22 Client perspectives differed on the ideal length of the assignment, and the point at which students share their work. Some clients thought that the assignment should require fewer visits and others would have preferred additional visits. Some clients wanted to see pieces of the story each week so that they could clarify or elaborate on the work, while others wanted to make revisions to the largely complete narrative. The most important recommendation on improving the assignment was to ensure that clients had ongoing support in the event that discussion brought up distressing issues for the client. "Maybe just having a little bit more support for the, for the participant if something does come up and they re having a hard time dealing with it. Just, you know, someone you can call or, you know, just that emergency person. someone to debrief with, um, right afterwards or if you need it or. I m not sure just, just a little bit more support during the process." In all, the qualitative analysis to date suggests that the Recovery Narrative Assignment is useful for both students and clients. Analysis is not yet complete, and discussion is necessary on a number of elements. A more comprehensive final report on the qualitative component of this evaluation will be produced in the coming months. 6 SUMMARY AND CONCLUSIONS The evaluation of the Brandon University Psychiatric Nursing Program s Psychiatric Rehabilitation and Recovery course with Recovery Narrative assignment showed positive, favourable results in terms of its effectiveness for reducing mental illness related stigma among student healthcare providers. Evaluation results included the following major findings: Statistically significant lower scores on the OMS-HC stigma scale at post-test as compared to baseline, with sustained positive improvements at follow-up; The determination (at 90% confidence) that.9% of the sample truly became less stigmatizing towards mental illness from pre to post program. Using this same measure, no participants became more stigmatizing; A notable increase in the percentage of students who gave non-stigmatizing responses to at least 80% of the questions at post-test as compared to baseline, from less than half at pre-test (4.%), to a strong majority at post-test (7.9%); Improvements in score on the three dimensions of stigma captured in the OMS-HC. Although these improvements were not statistically significant, the dimension of disclosure / help seeking was approaching significance, and the dimensions of attitudes toward mental illness and social distance had very low baseline mean scores. In addition, there were statistically significant improvements on a number of individual scale items, with no scale items showing statistically significant increases in stigma; Qualitative findings from student interviews suggest that the Recovery Narrative assignment is not only valued by students, it also functions to decrease social distance; improve perspectives,