Evaluation of the Effectiveness of a Program to Orient Social Work Students toward Their Fieldwork

Transcription

1 Evaluation of the Effectiveness of a Program to Orient Social Work Students toward Their Fieldwork Chau-kiu Cheung Department of Applied Social Studies City University of Hong Kong Hong Kong SAR, China Tel: , Fax: , ssjacky@cityu.edu.hk Tak-yan Lee, City University of Hong Kong Wai-man Kwong, City University of Hong Kong Abstract : In response to the pressing need of the social work profession for highly qualified and competent social work graduates to join the work, the social work program of the City University offered a 14-week orientation module, consisting of a two-hour lecture each two weeks, to pre-fieldwork students in the first semester of The captioned study is to evaluate the effectiveness of the orientation module for helping students to achieve good performance in fieldwork. Its core hypothesis is that the attendance at the orientation module promotes the student s use of appropriate learning styles and adjustment to fieldwork placement which raise fieldwork performance as a result. The hypothesis receives support from the results. Introduction Social work students need to attend some kind of orientation program before they undergo their fieldwork placement. Despite the need, it is not at all clear about what form of orientation program can better serve its purpose. Specifically, it is not certain whether a semester-long orientation program is more effective than a weekend orientation program. With the shift from a weekend orientation program to a semester-long orientation module in 1997/1998 for social work degree training in the City University of Hong Kong, the present study is then possible to compare the two forms of orientation programs. The new orientation module or fieldwork preparation program is different from the old module prior to 1997 in the following aspects. First, the new module employed 14 weeks of a semester whereas the old module concentrated in one day or one-and-a-half day. Students attended the new modules every two weeks rather than a day or so. The new module obviously allows students to have more time to learn and reflect upon the orientation. Second, the new module is one semester ahead of the fieldwork placement. This gives more time for students to prepare for the placement. Third, attendance at the new module is compulsory and contributes to fieldwork hours. In the old module, attendance was compulsory but did not contribute to fieldwork hours. Fourth, the new module employs a variety of teaching methods, including lectures, presentation, video shows, case discussion, self-understanding through responding to questionnaires, and assessment exercise. Fifth, the assessment exercise contains 116 true-and-false questions. Every student should take the test once or twice until he or she answered 80% or more correctly in one test. This requirement is to assure that the quality of students meets the standard. Sixth, the new module has the following aims: (1) to promote understanding of the unique way of learning in fieldwork, (2) to foster self-understanding by means of an aptitude test, (3) to help students understand their own learning styles, (4) to help them understand assessment criteria for fieldwork, and (5) to help them understand legal, ethical, and professional requirements of social work practice and departmental expectations of fieldwork placement. The new module includes the following 14 topics: (1) understanding fieldwork commonly asked questions, (2) learning new roles, (3) understanding learning in fieldwork, (4) understanding your own learning style, (5) integration of theory with practice in fieldwork, (6) code of ethics, (7) workload and structured tasks, (8) personal data (privacy) ordinance, (9) recording practice, (10) working with field instructor and agency staff, (11) administrative briefing, (12) assessment and moderation, (13) revision, and (14) assessment exercise. 1

2 Theory A longer, formal, well-organized semester-long orientation module would be more effective than a program that clusters all materials into one or two days in a weekend. The reason behind is that the former program lengthens the time for students to learn and prepare for fieldwork, offers more opportunity for students to learn and reflect, and discusses pertinent topics that are helpful to students. Through substantial learning, the student would adapt more favorably to fieldwork so as to have more motivation to learn in the fieldwork placement and benefit more from the support of the fieldwork instructor and agency. As a result, the student would attain a good grade from the fieldwork placement. Put it simply, the hypotheses simply reflect a learning process dictated by social learning or social cognitive theory (Bandura 1986). Moreover, the effect also reflects the operation of selfregulation theory (Zimmerman 1989) because of the opportunity for the student s self-understanding and execution of learning effort. Alternative Studies The present study is important because it addresses issues untapped before. Past local studies at best examined students satisfaction with fieldwork, without examining factors due to the approach of teaching (Choy et al. 1998; Yeung et al. 1998). They demonstrated some relationships between the student s satisfaction and the instructor s role and support. However, they did not clarify causal relationships between teaching and learning with regard to fieldwork placement. In this connection, the present study would be a credible examination of the causal relationship because (1) attendance at the orientation module has come before the fieldwork placement, as well as students experiences and performance, (2) attendance at the module, either old or new, is compulsory rather than a result of the student s choice, and (3) there are genuine and substantial differences between the new and old modules. Hence, the study employed a quasi-experimental design in which different students attended different orientation programs to examine the causal relationship. The new module has demonstrated its effectiveness most remarkably through the significant predictive effect of the student s grade obtained in the new orientation module on the student s grade obtained from field placement (Lee & Cheung 1999). Moreover, the orientation grade shows significant relationships to the student s social work values, notably humanitarianism and valuation of caring, social responsibility at the interpersonal level. However, the encouraging results stemmed from a study of only 78 students who had attended the new orientation module. Hence, that study did not compare results of 78 students from those students in the past who had not attended the new module. To verify the effectiveness of the new module, the present study employed data not only from those having attended the new module, but also from those having attended the old module instead. In essence, this study aims at examining effects of having attended the new module on the student s performance and experiences encountered in the fieldwork placement. The following hypotheses give more details. Hypotheses Hypothesis 1: A student who has attended the new orientation module will have a higher grade in the fieldwork placement than will another student who has attended the old module instead. Hypothesis 2: A student who has attended the new orientation module will have more favorable evaluation of the orientation module than will another student who has attended the old module instead. Hypothesis 3: A student who has attended the new orientation module will have more favorable evaluation of the agency which provides the fieldwork placement than will another student who has attended the old module instead. Hypothesis 4: A student who has attended the new orientation module will have more favorable evaluation of the student s fieldwork instructor than will another student who has attended the old module instead. Hypothesis 5: A student who has attended the new orientation module will show more learning effort in fieldwork than will another student who has attended the old module instead. 2

3 Methods Data on students evaluation and experiences with fieldwork placement came from responses to a structured questionnaire whereas data on grades on the fieldwork placement came from documentary sources. The questionnaire was administered both at the middle and end of each fieldwork placement to all students going through fieldwork from 1995 to Among these students, those full-time Year 2 students in the year attended the 14-week orientation module whereas others only attended a one-day or 1.5 day orientation program. Students attending the two programs would learn the same core concepts but under different organizations and time-spans. Altogether, the data set contained 1,080 cases involving student responses in the middle and end of two placements for each student. Among these cases, 198 (18.3%) came from students placed in the year 1995/96, 511 (47.3%) from students placed in 1996/97, 371 (34.4%) from students placed in 1997/98. They were different stages of studying and placement (see Table 1). These cases involved more than 435 different students and 106 different field instructors, given that in a few cases students did not write down their IDs on the questionnaires. Table 1: Distribution of cases of Years 2 and 3 and first and second placement Year 2 Year 3 Total Middle of placement End of placement Total Measurement Except the term-end grade for the fieldwork placement, variables measured by the questionnaire included evaluations of the orientation program, agency, fieldwork instructor, and learning effort. The measurement relied on a number of items coded on some rating scales (3-point, 4-point, or 5 point). Resultant scores were all logically between 0 (lowest rating) and 100 (highest rating). Composites of items measuring the same variable proved to be reliable in terms of internal consistency (see Table 2). Table 2: Reliability of composite scales Composite scale Number Alpha of items Evaluation of the agency Evaluation of the orientation program Instructor helpfulness Learning effort Adaptive learning Reflective learning Observational learning Accordingly, the overall evaluation of the orientation program, which combined eight items, yielded a reliability ( ) of.799. The eight items were: (1) understanding the place of field training in social work education, (2) understand how to learn from field training, (3) understand and reflect on your own learning style, (4) evaluate how your learning style may enhance or hinder your learning in field training, (5) have some initial ideas about the teaching style of your fieldwork instructor and his/her expectation toward you as a student social worker in field training, (6) prepare how you will make use of fieldwork supervision, (7) Acquire an overall idea of the structure and organization of field training in this fieldwork placement, and (8) understand the Department s expectations on field instruction and its demand on quality. Evaluation of the fieldwork instructor included measure of the instructor s helpfulness The composite score of 12 items was reliable ( =.827) in terms of internal consistency. These items tapped the instructor s help in the student s (1) developing practice skills, (2) acquiring practice knowledge, (3) 3

4 seeing the connection between knowledge and practice, (4) appreciating the nature of social work practice, (5) appreciate the place of ethics and values in practice, (6) developing critical ability, (7) developing self-knowledge, (8) critically examining knowledge, (9) critically examining competence, (10) developing confidence, (11) developing motivation, and (12) learning in the placement. Evaluation of the agency involved eight items. The composite score was reliable ( =.747). The evaluation focused on: (1) The agency has been responsive to my learning needs; (2) The agency is able to provide direct practice experience for me to learn what practitioners; (3) I have good access to agency staff in getting assistance and in learning about their work; (4) The agency supervisor is generally supportive and feels responsible too in helping me to learn; (5) The social environment in the agency conveys a spirit of professionalism; (6) The social environment in the agency conveys a spirit of commitment to professional values and ethical practice; (7) I experienced difficulties in recruiting participants for my programs, and/or in having access to clients; (8) Overall, I think my placement agency is suitable for placing students in their first/second field placement. Learning effort had three dimensions, (1) adaptive learning, (2) reflective learning, and (3) observational learning. Composite scores of the three dimensions all achieved satisfactory reliability ( =.764,.559, &.573). Adaptive learning included (1) seeking corrective feedback from my fieldwork supervisor on what I have done (2) seeking guidance from my fieldwork supervisor to plan my action, (3) implementing changes suggested by my fieldwork supervisor and evaluating the result, and (4) rehearsing what I am going to do with my fieldwork supervisor and getting coaching from him/her. Reflective learning included: (1) doing and reflecting on my action at the same time, (2) doing and reflecting on my action afterward, (3)using knowledge to plan my action and evaluating the result afterward, and (4) getting feedback from my clients. Observational learning included: (1) observing other practitioners at work and (2) observing the practice of my fieldwork supervisor. Analytic Strategy Because of the design of quasi-experiment, students attending the new module might be different from those attending the old module. Moreover, cases also differed in the time of evaluation, mid-term or term-end, first placement or second placement. They were also different in the field instructor responsible for the fieldwork supervision. Hence, statistical control for all significant differences was necessary to isolate the causal effect of the module. An efficient means was the use of stepwise regression analysis procedure to screen significant (p <.05) predictors, including dummy variables to indicate students and instructors, of the hypothesized outcomes. This procedure could engender a parsimonious set of predictors. It was reasonable because there was no strong theory arguing for the difference among students and instructors. To ensure the control for important timing variables concerning the arrangement of fieldwork, the next step of analysis therefore included those variables into the regression model. In all, the approach of analysis would control for individual differences due to the student per se, the instructor, timing, and other background and fieldwork characteristics. Results The regression analysis could substantially explain large portions of variance in the outcome variables (see the column of R 2 in Table 3). This finding, that there was little unexplained variance, guaranteed the reliability of estimated effects from regression analysis. The effects of the new orientation module were credible also because the regression analysis already controlled significant effects of confounding variables. Table 3: Effects of the new orientation module on fieldwork experiences b β R 2 Grade *.950 Evaluation of the orientation module *.520 Understand the place of field training in social work education *.334 Understand and reflect on your own learning style *.407 4

5 Evaluate how your learning style may enhance or hinder your learning *.371 in field training Evaluation of the agency I have good access to agency staff in getting assistance and in learning *.450 about their work Instructor helpfulness develop critical ability in evaluating the quality of service and agency *.441 orientation supportive and caring such that I feel motivated and positive toward *.452 whatever feedback he/she gives me Learning effort Adaptive learning Reflective learning *.529 Observational learning *: p <.05 (2-tailed) Notes: Controlling for background variables, including variables indicating if the placement was the first or the second, the year of placement, years of study, and full-time or part-time study. Other predictors included dummy variables for students and instructors. Hypothesis 1 regarding the impact of the new module on the grade obtained at the end of the fieldwork placement receives support. The new module had a significant standardized effect of.087 on the grade. It led to an advantage of 1.66 point in the grade scored from 0 to 100. The difference was small because of the limited variation of the grade across cases (SD = 6.72). Hypothesis 2 receives support from findings that the composite score and three items for evaluation of the orientation module exhibited significant positive effects as expected. Accordingly, the new module had a standardized effect of.162 on the overall evaluation. Furthermore, it had stronger standardized effects on the effectiveness of the orientation module for the student s (1) understanding and reflecting on learning style (.343), (2) evaluating the use of learning style (.176), and (3) understanding the place of field training (.165). At best, the new module would contribute points higher than the old module in its ability to make students understand and reflect on their learning style. Hypothesis 3 attains partial support in terms of the significant effect of new module on an item of evaluation of the agency. Accordingly, the new module had a significant standardized effect of.126 on the student s access to agency staff. In other words, a student who had attended the new module would have points higher on the access than would another student who had attended the old module instead. Hypothesis 4 obtains partial support from findings that the new module exhibited significant effects on two of the items. Accordingly, the standardized effects on evaluations of the instructor s helpfulness for developing the student s critical ability in evaluating the quality of service and agency orientation and for motivating the student were.103 and.108 respectively. In other words, the new module made a student to rate the instructor s motivation points higher. This difference would reflect the ability of the new module to promote the student s sensitivity to the instructor s assistance. Hypothesis 5 receives partial but important support from the finding that the new module had a significant effect (.223) on the student s reflective learning. Accordingly, the new module made the student to spend 9.98 points more often to learn by reflection. Although the new module had no significant effect on the student s adaptive learning and observational learning, its significant effect on reflective learning was crucial. Its importance would stem from the notion that reflective learning is essential for the construction of knowledge central in self-regulated learning theory (Paris & Byrnes 1989) Discussion All the five hypotheses attain important support from the results. The support is mostly clear in terms of effects of the new module on the student s grade obtained for the fieldwork placement and 5

6 evaluation of the orientation module. Support is also evident in effects of the new module on the student s reflective learning and some items of evaluations of the agency and fieldwork instructor. Effects contributing to the support were rather substantial except that on the fieldwork grade. The relatively weak effect on the grade may be a reasonable finding in that the new orientation program alone should not largely determine fieldwork performance evaluated by the fieldwork instructor several months later. Otherwise, there was no need for the student s learning effort and the fieldwork instructor s supervision on the fieldwork. Hence, a reasonable view would find the new orientation module to enhance the student s motivation and understanding of fieldwork, which in turn stimulate the student to spend more effort in the fieldwork, and eventually give an edge for the fieldwork performance over the old module. Even though the significant edge was small, the student could benefit from those more substantial increases in understanding of fieldwork and his or her own learning style, adaptability to the agency, sensitivity to the instructor s assistance. Some of the benefit might not be conducive to a higher grade in the fieldwork placement. Nevertheless, the benefit would contribute to the student s overall learning. A notable limitation inherent in the research design is that the design is not a true experiment, which includes randomization of students into different modules. The limitation stems from the practical and ethical unfeasibility to assign students randomly to different modules in an educational setting. Because of the limitation, there might be some unmeasured differences between the two modules and led to findings of the present study. Related to the limitation was the possibility that a time or cohort factor might confound the effect of the new orientation module. Accordingly, there was a clear watershed in time between the old and new modules. Hence, the time or cohort factor just coincides with the factor of the orientation module. As a result, the study might wrongly take effects of time or cohort as those of the new module. However, insofar as the absence of strong theory to justify the time and cohort variation, the presence of such a mistake may not be reasonably likely. Nevertheless, the study seems to be immune from other threats to its internal validity, including maturation, testing, instrumentation, statistical regression, experimental mortality, causal time ordering, diffusion, compensation, rivalry, and demoralization. It is likely because the study employed the same instruments and procedures for collecting data from students having attending different programs in different years. Students of different years were not aware of the comparison and would be unlikely to learn or compete for favorable assessment. Unlike other experimental studies, experimental morality was not a problem because all students had to undergo the required fieldwork placement. The study has no worry about threats of testing and statistical regression because it did not rely on comparison with the student s scores prior to attending the orientation program. Because of sample characteristics of the present study, generalization of its findings may not be legitimate. The study examined fieldwork learning and teaching only in one institution in Hong Kong. More data collected from a wider spectrum of students from various institutions are decidedly necessary to uphold the generality of the findings. Conclusion Despite the fact that learning in field instruction is crucial to social work students, very few researchers have systematically framed and studied the issue (Raskin, 1989). For the most part, knowledge and data on what and how students learn in their field placement are not available. Instead, some theories and speculations appear to arise from an outsider s perspective without understanding what students experience in the field. On the other hand, the present study collected data from students and their academic records so as to contribute to theory and research on social work learning in the field. It reveals that an improved orientation program can help social work students achieve better performance in fieldwork through a promotion of the use of appropriate learning styles and better adjustment to fieldwork placement. References Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall. 6

7 Choy, B.K., Leung, A.Y.L., Tam, T.S.K., & Chu, C.H. (1998). Roles and task of field instructors as perceived by Chinese social work students. Journal of Teaching in Social Work, 16(1/2), Lee, T.Y., & Cheung, C.K. (1998). Improving the quality of teaching and learning in the first fieldwork placement of social work students: A fieldwork preparation model. Paper presented in the First Conference to Promote Teaching and Learning, Dec 11, Hong Kong. Paris, S.G., & Byrnes, G.J. (1989). The constructivist approach to self-regulation and learning in the classroom. In B.J. Zimmerman & D.H. Schunk (Eds.), Self-Regulated Learning and Academic Achievement: Theory, Research, and Practice (pp ). New York: Springer-Verlag. Raskin, M.S. (1989). Empirical Studies in Field Instruction. New York: Haworth. Yeung, V., Lee, T.K., & Chung, K.W. (1998). "Student Satisfaction in Social Work Placement in Hong Kong. Asia Pacific Journal of Social Work, 8(1), Zimmerman, B.J. (1989). Models of self-regulated learning and academic achievement. In B.J. Zimmerman & D.H. Schunk (Eds.), Self-Regulated Learning and Academic Achievement: Theory, Research, and Practice (pp.1-25). New York: Springer-Verlag. 7