Evaluation of a Substance Abuse Training Program for Healthcare Supervisors 1. Donald Truxillo, PhD Portland State University

Transcription

1 Evaluation of a Substance Abuse Training Program for Healthcare Supervisors 1 Donald Truxillo, PhD Portland State University David Cadiz, MBA, PhD Oregon Nurses Foundation Chris O Neill, RN, DMin Oregon Nurses Foundation 1 Funding for the development of the Fit to Perform training was provided by the Oregon Department of Human Services, the Oregon Nurses Foundation, and Serenity Lane. The views expressed in this presentation do not reflect the official policies of the funders; nor does mention of trade names, commercial practices, or organizations imply endorsement by them.

2 TABLE OF CONTENTS Introduction..1 Method..3 Results...5 Discussion and Recommendations..13 References 15 Appendix A: Survey Measures 17

3 Introduction Background This report describes the findings of the evaluation of a healthcare supervisor training entitled Fit to Perform. The purpose of the training is to fulfill a provision of a law passed by the Oregon Legislature (HB 2345) to incorporate supervisor training into a newly created alternative-to-discipline program. The objective of the program is to protect patient safety by monitoring health professionals diagnosed with a substance use or mental health disorder to insure that healthcare professional is fit to practice. The goals of the training are to increase supervisor s knowledge, ability, and confidence, and to reduce stigma toward health professionals recovering from substance abuse issues. Training Approach The theoretical framework for the training includes constructive confrontation, stigma, and social learning. First, the training content is grounded in constructive confrontation, which utilizes a series of steps from informal conversations to firmer actions to build pressure for behavior change by an employee and to create readiness in the worker to seek or accept help, rather than to terminate the employee (Trice & Sonnenstuhl, 1988). Prior research provides evidence that the constructive confrontation approach by supervisors can lead to relevant and desirable outcomes including improved workplace performance (Trice & Beyer, 1984a), and enhanced supervisor ability to detect employee impairment on the job (Gerstein, Eichenhofer, Bayor, & Valutis, 1989). However, to our knowledge, this technique has not been applied by supervisors monitoring subordinates in a healthcare setting. The training content also integrates theory about stigma to reduce stigmatization of employees returning to the workforce that have sought treatment for substance abuse and/or mental health reasons. The content focuses on decreasing the prejudice against those who have sought treatment and are returning to work. For example, some of the training material focuses on improving understanding of the effectiveness of treatment and an individual s potential for recovery (Corrigan et al., 2001). Indeed, stigma reduction education interventions programs have observed some success for changing stigmatizing beliefs and attitudes toward mental health and substance abuse (Corrigan et al., 2001). Finally, the presentation of the training material takes a multi-faceted approach by using multiple training methods including lecture, discussion, and behavioral role modeling. Lecture is the most popular training method and has been found have a positive effect on training performance (Callahan et al., 2003). Moreover, behavioral role modeling is rooted in social learning theory and focuses on a trainee s ability to acquire knowledge through observing someone else performing the task (Bandura, 1977). The effectiveness of behavioral role modeling as an effective method for learning skills is supported meta-analytically (Taylor et al., 2005). In a quantitative review assessing the effectiveness of training programs in organizations, Arthur et al. (2003) observed that multi-method training approaches are effective in producing 1 P a g e

4 changes in knowledge, skill-based, and affective outcomes. Thus, we feel confident in our training approach. Training Goals The basic goals of the training include the following: Introduction to the new alternative-to-discipline law in Oregon and program called the Health Professionals Services Program. Prevalence of substance abuse in the health care profession Legal and ethical responsibilities of a nurse supervisor. What is meant by "objective observations of employee performance. How to prepare for a meeting with an employee about performance problems, what to focus on, and what reactions to expect. How to observe and objectively document employee behavior. The concept of "fitness for duty." The elements of effective communication with employees. How to deal with an employee suspected of being under the influence of substances Overcoming tolerance and/or fear of intervening in situations where substance is suspected Emphasizing the importance of following through after developing an action plan Training Evaluation Strategy The evaluation methodology was based on Kraiger et al. s (1994) framework for training outcomes: Knowledge outcomes, skill-based outcomes, and affective outcomes. Knowledge outcomes were measured with multiple-choice knowledge tests; skill-based outcomes were measured with a scenario-based, multiple-choice situational judgment test item (SJT); and affective outcomes were measured via self-ratings of self-efficacy (e.g., Bandura & Locke, 2003), a key predictor of learning and transfer (e.g., Colquitt et al., 2000) and substance abuse stigma. In addition, we measured participants perceptions of the utility of the training material, as this has been found to be a predictor of learning and application of the training material (Alliger et al., 1997). Data were collected at 2 time-points: Time 1 (pre-test) and Time 2 (post test). 2 P a g e

5 Data Collection Overview Method Data collection for pre- and post-tests (Time 1 and Time 2) took place in November 2010 through February of Trainees (the majority being nurse supervisors) provided their date of birth on the surveys so that individual surveys could be matched for statistical analysis. Participants completed Time 1 surveys just before training and Time 2 surveys just after training. The second author was responsible for distributing and collecting the Time 1 and Time 2 surveys, which were completed anonymously. Participants There were a total of 136 training participants. Matched data were collected for 97 trainees which is combined response rate of 71%. A significant majority of the participants were women (82%) and Caucasian (96%). The other ethnicities represented are Asian (2%) and African American (2%). The mean age of the trainees is 50 years old (SD = 9.30). Eighty-two percent of the participants indicated that they were healthcare supervisors. Measures All of the measures discussed below are located in Appendix A which is located at the end of the report. Knowledge items. The knowledge items were developed to reflect training content and were adapted from previous evaluations of a similar training program (Truxillo & Weathers, 2006). There were two types of questions items that asked explicit questions about the training content and situational judgment test (SJT) items. In order to adapt the original items, the trainers and two training evaluators examined the updated and evaluated training material (overheads and handouts). All knowledge and SJT items were developed using a multiple-choice format with 3 or 4 response alternatives each. Initially, ten knowledge items were developed to sample training content and three SJT items were created to assess trainees ability to apply this knowledge. However, after the first training event, we examined the items and decided to add three additional knowledge items in order to better sample the training content. Moreover, we removed one of the SJT items because no one missed the item, and therefore, it was determined that it was too easy. Before the final training event, in an effort to reduce the amount of time participants were spending on the pre-training survey, we reduced three knowledge items (3 knowledge items and 1 SJT item) because these items were observed to be too easy in that participants were did not incorrectly answer these items in the before the training (i.e., Time 1 survey). Due to the removal of several knowledge and skill (e.g., situational judgment) questions between training events, we ended up with 8 total questions to assess participant s knowledge. In fact, we decided to combine the knowledge and skill items into one measure labeled knowledge 3 P a g e

6 because we only had on remaining situational judgment item which did not allow us to make any assessment of the ability to apply the skills taught in the training. Self-rated knowledge. Self-rated knowledge items (11 items) were developed to measure trainees self-assessment of their level of knowledge, understanding of the training goals and skills taught in the training. As with the knowledge test items, the self-rated knowledge items were adapted from previous training evaluations (Truxillo & Weathers, 2006). After the training (Time 2), trainees were asked to retrospectively assess their pre-training level of knowledge and understanding and their post-training level of knowledge and understanding. Trainees responded to these items using a 5-point Likert scale. Self-efficacy. We measured self-efficacy because it has been shown to be a consistent predictor of training success (e.g., Baldwin & Ford, 1988; Colquitt et al., 2000;) Self-efficacy was assessed using 3 items based on Truxillo, Bauer, Campion, and Paronto s (2002) selfefficacy scale. These items assessed trainees evaluation of their ability to master the training material. Trainees responded to these items using a 5-point Likert scale. Self-efficacy was collected at both Time 1 and Time 2. For the final training, four additional self-efficacy items were created that focused on people s confidence in the skills that were taught and practiced in the training. Substance abuse stigma. Substance abuse stigma was assessed with 7 items from Luoma et al. (2007). These items assessed the trainee s perceptions about the amount of stigma that they felt a recovering person would face in the workplace. Trainees responded to these items using a 5-point Likert scale. Stigma was collected at both Time 1 and Time 2. Utility of the training. We assessed perceived training utility because past research has shown that this factor is an important predictor of transfer (e.g., Alliger et al., 1997). Training utility was assessed with 6 items based on Ford and Noe (1987). These items assessed the trainee s evaluation of whether the training was relevant to their job including whether the skills and information would be used. Trainees responded to these items using a 5-point Likert scale. Utility of training was only collected in the Time 2 survey. 4 P a g e

7 Results Data were entered manually into SPSS statistical software version 17. We report means and statistical tests and also provide graphical representations of the results for each measure. The expectation is that there would be significant increases for the knowledge, self-rated knowledge, and self-efficacy. A significant decrease was expected for substance abuse stigma. Table 1 displays a tabulated representation of the results. Individual descriptions of the results are also provided. Table 1. Mean Comparison Table Time 1 Time 2 N t Partial Eta 2 Cohen's D Mean SD Mean SD Knowledge ** Self-Rated Knowledge ** Self-Efficacy ** Stigma * Table Notes. N = the number of matched cases used in the analysis. Partial Eta-squared reflects the percent of variance accounted for by the training for that variable. Cohen s D reflects the standardized difference between two means, i.e, the difference between two means in terms of standard deviation units. A Cohen s D value of.20 is considered small,.50 is considered moderate, and.80 is considered large (Cohen, 1977). * p <.05 and ** p <.01. Knowledge Knowledge has dimensions in this study: knowledge of the course material, and self-rated understanding of the material. In this section we report on the results of the multiple-choice knowledge test that was taken before and after the training. Results indicated that participant s knowledge of the training material was higher after the training than it was just before the training. The average percent correct on the knowledge test increased from 82% before the training to 88% after the training. Figure 1 is a graphical display of the percent change results. The mean change between Time 1 and Time 2 was statistically significant (t = 4.49, p <.01). Partial Eta-squared associated with the change is.18 which indicates that training accounts for 18% of the variance in knowledge between Time 1 and Time 2. The Cohen s D associated with the change before and after the training is.52 which is considered a moderate effect. 5 P a g e

8 Figure 1. Mean percent knowledge scores for Time 1 and Time 2. Self-rated Knowledge In this section we describe the results from our investigation of the change in the trainee s assessment of the change in their knowledge before and after the training. This differs from the knowledge test described above in that the trainee evaluates their own level of knowledge wheras on the knowledge test the number of correct answers indicates the trainee s level of knowledge. Results indicated that participants self-assessed knowledge of the training goals and skills taught in the training were higher after the training compared to before the training. The participants mean knowledge of the training material prior to the training is 3.40, which is between moderate and high on the rating scale, and the mean knowledge after the training is 4.35, which is between high and very high on the rating scale. Figure 2 is a graphical display of the mean change. The mean difference between before and after the training is statistically significant (t = 4.41, p <.01). Partial Eta-squared associated with the change is.74 which indicates that training accounts for 74% of the variance in self-rated knowledge before and after the training. Moreover, the Cohen s D associated with the change before and after the training is 1.62 which is considered a large effect. These results are displayed in Table 1. Table 2 provides the average ratings for the 11 training dimensions before and after the training. As indicated in the overall mean score for self-rated knowledge, trainees felt that that their knowledge and understanding of the material improved across each training dimension. 6 P a g e

9 Figure 2. Mean self-rated knowledge scores before and after the training. 7 P a g e

10 Table 2. Mean self-rated understanding of each training dimension (all ratings made posttraining.) Before training After training Dimension My legal responsibilities as a nurse supervisor regarding nurses enrolled in monitoring What is meant by "objective observations" of nurse performance How to prepare for a meeting with a subordinate about his/her impairment problems What issues to focus on during a meeting with a subordinate about his/her impairment problems The steps to take during a meeting with a subordinate about his/her impairment problems What reactions to prepare for when I discuss performance problems with a licensee in monitoring How to observe and document licensee performance problems How to give feedback to nurses who exhibit signs of substance abuse The concept of "fitness for practice." How to prepare for and respond to employee resistance during a performance review meeting How to best deal with a subordinate that I suspect of having a substance abuse problem Average across the items P a g e

11 Self-Efficacy Three-item self-efficacy measure. Results indicated that participants confidence to manage behavioral problems were higher at Time 2 compared to Time 1. The participants mean rating at Time 1 is 4.01and the mean rating after the training is Figure 3 is a graphical display of the mean change. The mean difference between Time 1 and Time 2 is statistically significant (t = 17.04, p <.01). Partial Eta-squared associated with the change is.17 which indicates that training accounts for 17% of the variance in self-rated knowledge before and after the training. Moreover, the Cohen s D associated with the change before and after the training is.48 which is considered a relatively moderate effect. These results are displayed in Table 1. Pilot of new four-item self-efficacy measure. In addition to the original self-efficacy measure, we created four additional self-efficacy items that we piloted in the final training event. These new items specifically focused on the skills taught in the training whereas the original items broadly focus on confidence to manage behavioral problems at work. The ratings on the four-item self-efficacy scale increased between Time 1 (M = 3.84) and Time 2 (M = 4.24). The difference between self-efficacy at Time 1 and Time 2 is significant (t = 3.16, p <.01). Although the results are very promising, they should be considered preliminary because they are based on only 23 matched pairs from the final training event. Figure 4 is a graphical display of the mean change. Figure 3. Mean self-efficacy ratings at Time 1 and Time 2 9 P a g e

12 Figure 4. Mean self-efficacy ratings on the new scale at Time 1 and Time 2 10 P a g e

13 Workplace Substance Abuse Stigma Results indicated that participants workplace substance abuse stigma were higher at Time 1 compared to Time 2. In other words, stigma decreased from before the training to after the training. The participants mean rating at Time 1 is 2.87 and the mean rating after the training is Figure 4 is a graphical display of the mean change. The mean difference between Time 1 and Time 2 is statistically significant (t = -2.43, p <.05). Partial Eta-squared associated with the change is.07 which indicates that training accounts for 7% of the variance in workplace substance abuse sigma before and after the training. Moreover, the Cohen s D associated with the change before and after the training is.20 which is considered a relatively small effect. These results are displayed in Table 1. Figure 4. Mean stigma ratings at Time 1 and Time 2 11 P a g e

14 Utility of the training Since utility of the training was only collected at Time 2, we are unable to make a comparison across the data collection points. However, we report the mean rating across the 6 items and the mean score as a single measure. Table 3 contains the mean item and measure ratings for the utility measure. The mean rating across the items is The highest item mean rating (M = 4.46) was for the item that stated that the training was useful for the trainee s development. The lowest item mean rating (M = 4.19) was associated with the item stating that the material was relevant to the skills that the trainee hoped to develop. Overall, trainee s agreed that the training was useful and that they would be able to apply the skills taught in the training in their job. Table 3. Mean item and measure ratings for utility Item Mean I believe the training will be useful to my job as supervisor The training was useful for my development Most of the material in the training was relevant to skills I had hoped to develop The time spent in the training program was worthwhile I expect to be able to apply to the job what I learned in training I expect to have opportunities to practice the skills that I learned in training on my job Average of the utility items P a g e

15 Discussion and Recommendations Based on the results of this study, the Fit to Perform supervisor training appears to positively affect training outcomes such as knowledge of the course material, self-rated understanding of the material, and self-efficacy (e.g., Alliger et al., 1997; Kraiger, et al., 1996). Moreover, the training also had a positive effect on significantly reducing manager s substance abuse stigma. The change in an attitude like stigma is quite impressive because this was a relatively short training event and it is difficult to change value-related prejudices like stigma (Corrigan & Penn, 1999). Overall, the results described in this report support the efficacy of the training to improve knowledge, confidence, and attitudes in the context of managing and monitoring employees with identified substance abuse and mental health issues enrolled in an alternative-to-discipline program. Future Research As with any research study, this study is not without its limitations. Moreover, these results suggest areas for future research regarding this training. Design issues: Longitudinal designs and control groups. From a methodological standpoint, this study used a research design that may be susceptible to a number of threats to experimental validity. In other words, there may be other factors that could have caused the results found in this study. Although we think it is unlikely that certain of these threats (e.g., maturation) could have occurred because of the short time between the pre-test and post-test, future studies should utilize a control group who did not receive the training but whose performance would be tracked over time to address many of these threats. More importantly, longitudinal follow-up of a large sample of trainees is necessary to see whether trainees were able to apply their skills on the job. Effects of organizational and group culture and climate. Participating organizations should consider the transfer climate in their organizations in terms of support provided by supervisors, coworkers, policies, and workload. In addition, extending the training to the subordinate level could also improve the constructive confrontation climate within a team which has been shown to positively affect team decision making (Kellermanns et al., 2007). Effects on behavioral outcomes. Future research should attempt to measure effects on other outcomes, such as actual supervisor and employee behaviors to determine whether trainees are actually applying the training material. These might include tracking the performance of the monitored employee, the employee s compliance in the monitoring program, successful completion of the program, and the number of self-referrals into the program. Moderators of training effectiveness. Future research should examine the effects of other factors that may affect the effectiveness of training. Such factors include job type and trainee personality. 13 P a g e

16 Measures. Several of the measures used in this study were adapted from previous evaluations of this type of training in a different setting (Truxillo & Weathers, 2006). However, one concern is the knowledge items since we decided to remove some of the original items because their difficulty level appeared to be too low for this population. Indeed, we had to reduce the situational judgment (SJT) items from 3 down to 1, which meant that we were not able to assess the trainee s ability to apply the skills taught in the training. Future research should develop additional SJT items so the training s impact on skills can be evaluated as an outcome. An additional measurement concern is that we had difficulty finding an appropriate measure of stigma because of the challenge of finding items that accurately measure stigma without being concerned that they are being answered in a social desirable way. A third measurement concern is our newly created four-item self-efficacy measure that we piloted at the final training event. The measure showed promising results, but the sample size for the pilot was too small to perform a psychometric evaluation and to feel comfortable interpreting the results of our statistical analysis. Future research should include these self-efficacy items in the evaluation of this training to further investigate the training s impact on trainee s change in self-efficacy to perform the skills taught in the training. Finally, we recommend that future evaluations include assessments of transfer climate and group norms in order to investigate the effect that organizational context plays in the application of the training to people s daily work. 14 P a g e

17 References Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., & Traver, H. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50, Arthur, W., Bennett, W., Edens, P.S., & Bell, S.T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall. Bandura, A., & Locke, E. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88, Callahan, J.S., Kiker, D.S., & Cross, T.(2003). Does method matter? A meta-analysis of the effects of training method on older learner training performance. Journal of Management, 29, Colquitt, J. A, LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, Corrigan, P. W., & Penn, D. L. (1999). Lessons from social psychology on discrediting psychiatric stigma. American Psychologist, 54, Corrigan, P. W., River, L. P., Lundin, R. K., Penn, D. L., Uphoff-Wasowski, K., Campion, J., Mathisen, J., Gagnon, C., Bergman, M., Goldstein, H., & Kubiak, M.A. (2001). Three strategies for changing attributions about severe mental illness. Schizophrenia Bulletin, 27, Ford, J.K., & Noe, R.A. (1987). Self-assessed training needs: The effects of attitudes toward training, managerial level, and function. Personnel Psychology, 40, Gerstein, L.H., Eichenhofer, D.J., Bayor, G.A., & Valutis, W. (1989). EAP referral training and supervisor s beliefs about troubled workers. Employee Assistance Quarterly, 4, Golembiewski, R. T., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science, 12, P a g e

18 Kellermanns, F.W., Floyd, S.W., Pearson, A.W., & Spencer, B. (2007). The contingent effect of constructive confrontation on the relationship between shared mental models and decision quality. Journal of Organizational Behavior, 29, Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, Luoma, J.B., Twohig, M.P., Waltz, T., Hayes, S.C., Roget, N., Padilla, M., & Fisher, G. (2007). An investigation of stigma in individuals receiving treatment for substance abuse. Addictive Behaviors, 32, Taylor, P.J., Russ-Eft, D.F., & Chan, D.W.L. (2005). A meta-analytic review of behavioral modeling. Journal of Applied Psychology, 90, Trice, H.M. & Beyer, J.M. (1984a). Work-related outcomes of the constructive-confrontation strategy in a job-based alcoholism program. Journal of Studies on Alcohol, 45, Trice, H.M., & Sonnenstuhl, W.J. (1988). On the construction of drinking norms in work organizations, Journal on Studies in Alcohol, 51, Truxillo, D. M., Bauer, T. N., Campion, M. A., & Paronto, M. E. (2002). Selection fairness information and applicant reactions: A longitudinal field study. Journal of Applied Psychology, 87, Truxillo, D. M., & Weathers, V. M. (2006). Evaluation of a substance abuse training program for supervisors: A longitudinal study. Technical Report for Workdrugfree, Portland, OR. 16 P a g e

19 Appendix A. Survey Measures Knowledge Test 1) The prevalence of substance use among nurses and pharmacists is: a. Less than the general public b. About the same as the general public c. Double (200%) that of the general public d. Triple (300%) that of the general public 2) Which one of the following statements does not reflect the guidance provided in the American Nurses Association Code of Ethics for Nurses? a. The nurse promotes, advocates for, and strives to protect the health, safety, and rights of the patient. b. The nurse must be alert to and take appropriate action regarding any instances of incompetent, unethical, illegal, or impaired practice by a member of the healthcare team. c. In a situation where a nurse suspects another s practice may be impaired, the nurses duty is to take action designed both to protect patients and to assure that the impaired individual receives assistance in regaining optimum function. d. When a nurse states her concern to another nurse about her unsafe or unprofessional practice, it is not necessary for the nurse to report it up the chain of command if the performance improves. 3) Which of the following is not a service of the new Oregon comprehensive health professional monitoring program? a. Confidentially enrolling self-referring licensees without notifying the appropriate licensing board b. Conducting a clinical evaluation of each licensee to determine a diagnosis c. Assessing the capacity of employers to supervise a licensee who is returning to the practice setting d. Reporting substantial non-compliance of a licensee with terms of their monitoring agreement 4) In dealing with an employee you suspect of having a substance abuse problem, your responsibility as a supervisor is to: a. Focus on the employee s job performance. b. Explain to the employee how to get help for the substance abuse problem. c. Convince the employee to change his/her ways. 5) What should you do if you have reasonable suspicion of employee impairment? a. Take action immediately. b. Wait until you're sure before confronting the employee. c. Check with a supervisor, then take action. 6) In discussing performance problems with employees, which of the following is a sidetracking behavior on the part of the employee? a. Giving assurances that they'll do better. b. Crying or falling apart during a conversation. c. Both a and b. 17 P a g e

20 7) What does "fitness to perform" mean? a. An employee is not under the influence of alcohol or drugs while on the job and thus is able to perform the job safely. b. An employee is physically, emotionally, and cognitively able to perform the job safely. c. An employee is physically able to perform the job. 8) Reasonable suspicion of substance abuse can be based on which of the following? a. The employee's coworkers have complained about his/her conduct. b. You've observed the employee behaving unsafely on the job. c. The employee has a reputation for unsafe and/or unproductive conduct on the job. 9) Which of the following is a way to reduce your risk of being considered negligent in handling an employee substance abuse problem? a. Ask yourself if the employee is fit to perform. b. Talk to another supervisor about your concerns. c. Both a & b 10) What is a common life problem that persists after a person experiences even after successful treatment for a substance use disorder? a. Financial problems b. Depression c. Managing chronic pain d. All of the above 11) Which of the following should NOT be discussed in a meeting with an employee with behavior problems? a. Your concerns about their behavior/conduct. b. Your perceptions of the cause of their behavior/conduct. c. Your expectations for their behavior/conduct. 12) Rachel is one of your best employees. Recently, you've noticed what smells like alcohol on her breath, and her eyes are red. When you approach her with your concerns, she tells you that she is taking a new medication and has not been getting enough sleep. What should you do next? a. Collect more evidence before taking further action. b. Discuss the situation with your supervisor, because you're not sure what to do. c. Take some immediate action such as requesting a drug test. 18 P a g e

21 Self-efficacy Please indicate how much YOU agree with the following statements using a scale of 1 to 5, with 1 being "strongly disagree" and 5 being "strongly agree." either Agree nor Disagree Strongly Disagree Disagree Strongly Agree Agree 1. I am confident in my ability to manage employee behavior problems. 2. I can effectively deal with employee behavior problems. 3. I can master the skills needed to handle employee behavior problems. Pilot Self-efficacy Please indicate how much YOU agree with the following statements using a scale of 1 to 5, with 1 being "strongly disagree" and 5 being "strongly agree." either Agree nor Disagree Strongly Disagree 1. I am confident in my ability to prepare for a meeting with a subordinate about his/her performance problems. 2. I am confident in my ability to communicate with a subordinate about their performance problems. 3. I am confident in my ability to give feedback to a nurse enrolled in the monitoring program. 4. I am confident in my ability to respond to employee resistance when confronting a subordinate about their fitness to perform. Disagree Strongly Agree Agree 19 P a g e

22 Self-rated Knowledge Please rate YOUR level of knowledge, awareness, or understanding on each of the following dimensions before and after completing the Fit to Perform training. Please circle the appropriate rating using the following scale: Very Low Low Moderate High Very High Dimension 1. My legal responsibilities as a nurse supervisor regarding nurses enrolled in monitoring. 2. What is meant by "objective observations" of nurse performance. 3. How to prepare for a meeting with a subordinate about his/her impairment problems. 4. What issues to focus on during a meeting with a subordinate about his/her impairment problems. 5. The steps to take during a meeting with a subordinate about his/her impairment problems. 6. What reactions to prepare for when I discuss performance problems with a licensee in monitoring. 7. How to observe and document licensee performance problems. 8. How to give feedback to nurses who exhibit signs of substance abuse. My knowledge, awareness, or understanding before completing this training My knowledge, awareness, or understanding after completing this training 9. The concept of "fitness for practice." 10. How to prepare for and respond to employee resistance during a performance review meeting. 11. How to best deal with a subordinate that I suspect of having a substance abuse problem. 20 P a g e

23 Substance Abuse Stigma Using the following scale, please respond to the following statements circling the appropriate number to the right of each item. either Agree nor Disagree Disagree Strongly Disagree Strongly Agree Agree 1. Most people would be unwilling to accept a person in recovery as a coworker. 2. Most people believe that nurses in recovery are less trustworthy than their other coworkers. 3. Most people feel that entering substance abuse treatment is a sign of personal failure. 4. Most people think less of a person who has been in substance abuse treatment. 5. Most healthcare organizations would not hire a person in recovery even if he or she is qualified for the job. 6. Most people at my work would treat a person in recovery differently. 7. Once coworkers know a person was in substance abuse treatment, they expect less from him/her. Utility of the training Using the following scale, please respond to the following statement circling the appropriate number to the right of each item. either Agree nor Disagree Disagree Strongly Disagree Strongly Agree Agree 16. I believe the training will be useful to my job as supervisor. 17. The training was useful for my development. 18. Most of the material in the training was relevant to skills I had hoped to develop. 19. The time spent in the training program was worthwhile. 20. I expect to be able to apply to the job what I learned in training. 21. I expect to have opportunities to practice the skills that I learned in training on my job. 21 P a g e