Student Information Resource Utilization in Problem-Based Learning Louise F. Deretchin, Lynn C. Yeoman, Charles L. Seidel Baylor College of Medicine

Transcription

1 Student Information Resource Utilization in Problem-Based Learning Louise F. Deretchin, Lynn C. Yeoman, Charles L. Seidel Baylor College of Medicine Abstract Purpose: To examine the profile of medical students resource use in a longitudinal problem-based learning course and to examine patterns of change. Method: Over a six-month period, 116 students indicated on resource checklists which resources they had used for independent research of learning issues identified in their problem-based learning sessions. On the checklist, resources were categorized as printed, electronic, human, or physical evidence (i.e., models, demonstrations). Results: Over the six-month period, the percentage of use (number of times a resource category was used / number of weekly reports submitted) declined from 64.0% printed, 81.7% electronic, and 4.3% physical to 44.0%, 69.8%, and 3.9%, respectively. Use of human resources increased from 29.1% to 36.6%. Use of a variety of resources (resources from 2 categories of resources) also declined. In medicine, the need to enhance information access skills has been intensified by the volume of new information being generated information that modifies, reverses, or adds to the existing collective body of knowledge. A growth in the variety of resource formats and methods of dispersion of information has had a compounding effect on skill level demands. It has become increasingly important for students, who are entering a profession with a lifelong learning ethic, to develop an ability to seek out sources on their own and gain facility in exploring new resources. 1 In order for medical schools to nurture skills in information access and use, explicit attention needs to be paid to the students patterns of resource use and habits in resource selection. Research has supported the contention that use of a variety of resources is associated with the use of deep learning strategies (i.e., learning strategies associated with seeking an understanding of information through critical self-directed inquiry). 2-3 Problembased learning (PBL), which has self-directed learning and independent investigation of learning issues as important curricular components, encourages deep approaches to learning. 4-6 Other research has shown that students in a PBL curriculum use a greater number and variety of resources than students in a traditional lecture-based curriculum. 7-9 The purpose of the present study was to examine resource use to see whether or not changes occurred in the types, quantity, or variety of resources used by students as they progressed through a longitudinal problem-based learning course. The resources examined were categorized as printed materials, electronic resources, human resources, or physical evidence (i.e., models, displays, and demonstrations). The use of a variety of resources i.e., resources from more than one category of resources has been encouraged in the PBL curriculum so that students may gain experience and skill in assessing the perspective taken by an information source, the nature and quality of the information, and the audience for which the material is appropriate. The focus of interest of the research was to observe whether the patterns of resource use changed over a 6-month period. The 6-month semester (Fall Semester) was divided into three academic blocks. Blocks one and two were each 8-weeks long. Block 3 was 5 weeks long. In order to determine whether or not the use of multiple resources was sustained over this period of time, the following were examined: 1. The number of resources used by students to research their learning issues. 2. The variety of resources used. Subjects The subjects were first-year medical students enrolled in a hybrid curriculum which had approximately 20 hours of traditional lectures and three hours of problem-based learning per week. The students were randomly assigned to groups of six to eight students with a trained facilitator that was either a faculty member, professional staff person, or an advanced medical student. Students were randomly assigned to groups and remained in the same group throughout the first three academic blocks (August through January). 1

2 Method Beginning in the first academic block (8 weeks) of the first year of medical school and continuing throughout the six-month period covered in this research, students participated in a problem-based learning course. They met in small groups with a facilitator once a week for three hours to discuss medical cases related to basic science information being presented in their morning lectures. In the PBL course, there was no specific content for which the students were held responsible. Process and skills in clinical reasoning, collaboration, and research were the foci of the course. During each session, students identified gaps in their knowledge which became learning issues that were investigated independently prior to the next week s session. At the beginning of each session, students were handed their personal itemized resource checklist. They placed a checkmark next to the resources they used to investigate their learning issue(s) from the previous week. Listed resources were categorized as printed, electronic, human, or physical evidence resources. Space was provided to write in resources not itemized. The checklist was returned to the facilitator who redistributed the resource record to the students at the beginning of the next session. At the end of each block, the resource list was turned in and the data entered into SPSS for analysis. This process was repeated throughout each block. Blocks 1 and 2, each 8-weeks long, had five sessions during which learning issues were generated and researched and four cases that were discussed (learning issue weeks 2-6 and 8-12). Block 3, differed from the other blocks in that it was 5-weeks long, had two sessions in which learning issues were generated, and had only one case discussed instead of several (learning issue weeks 14 and 15). Students use of resources was tracked through the three academic blocks to observe changes in the quantity and variety of types of resources used. Quantity of resources used was determined by summing the number of items checked by each student for each week, then finding the mean number of resource items used by each student each week. To determine the variety of resources used (i.e., resources drawn from different categories -- printed, electronic, human, physical evidence) first, a score of 1 was given to a category of resources if one or more resources within that category was used by a student researching a learning issue. Then, a count of the number of categories of resources used was obtained by summing the scores. Thus, for any one week, a student could receive a maximum summed score of 4 for the variety of resource types (a score of one for each category of resource used) but could have used several resources within each category. In a similar fashion, when determining the pattern of use of a resource category over time, a score of 1 was given to a category if any or all of the resources within the category were reported used by a student. The sum of these scores constituted the total number of uses for the category of resources. To examine changes in the amount of use for the four resource categories over time, a percentage of use was calculated for each academic block using the following formula: Total number of times a resource category was used Number of weekly reports submitted If students did not have a checklist submitted for all three academic blocks, their information was excluded from analyses. Reasons for not having a complete set of resource lists included groups not understanding that the checklist submission was mandatory (8 of the 31 groups), students returning mid-year from a leave of absence, and missing individual checklists. The remaining students, 116 out of 165 (70% of the class) submitted a total of 1392 weekly reports (116 students x 12 weeks of learning issues). The investigators used pattern and distribution of checkmarks to assess the validity of the data collected. No consistent patterns were observed, suggesting the data was valid (i.e., students were not indiscriminately checking resources). Distribution of checkmarks varied among students and within each student s submission. Descriptive statistics were used to examine the number of resources used. Analysis of variance (ANOVA) was used to examine differences by academic block of quantity of resources reported. Kruskal-Wallis one-way analysis of variance 10 was used to examine the change in variety of resources used as the course progressed from one block to the next. Variety was defined as a week s resources that included items from two or more of the four resource categories. The alpha level was set at The nature of the case and learning issue selected by the student was recognized as possibly influencing the resources used. Due to the possibility that a student may have had more than one learning issue from more than one case to research in a week, no attempt was made to attribute the resources used to a particular learning issue or case. It is believed that over the course of several cases, the case/issue effects would level out. 2

3 4 Mean Number of Resources Total Resources Electronic Printed Human Physical Block 1 Block 2 Block 3 Week Number Within the Course Figure 1. Mean number of reported uses of resources for each problem-based learning session. The legend indicates the resource categories. No research was done for weeks 1, 7 and 13, hence no values corresponding to these weeks are included. Weeks 1, 7, and 13 mark the beginning of academic blocks 1, 2, and 3 respectively. Results A graph of total resources use by week indicated that the total number of resources appeared to gradually decline from block to block (Figure 1). Electronic resources appeared to drive the shape of the curve. When comparing the percent of use each category of resources received over time (number of times a category of resources was used number of weekly reports submitted) the results indicated that the use of printed resources and physical evidence, as well as electronic resources, appeared to decline from block to block: printed resources %, 57.2%, and 44.0% for blocks 1, 2, and 3 respectively; electronic resources 81.7%, 74.8%, 69.8%; physical evidence 4.3%, 4.5%, 3.9%. Human resource use increased: 29.1%, 32.2%, and 36.6%. Significant decreases were indicated in the total resources used between blocks 1 and 2 (p<.001), and blocks 1 and 3 (p<.001). Significant decreases also occurred in the use of electronic resources between blocks 1 & 2 (p<.001) and blocks 1 & 3 (p<.001). Patterns of use for specific resources were examined by looking at the percentage of weekly reports within each block that indicated a resource had been used (Table 1). The results revealed that each of the printed resources declined from block to block. Five of the nine electronic resources decreased, four fluctuated. In the human resources category, the use of faculty and clinicians increased while consulting other students decreased. Use of physical evidence as a resource was minimal and fluctuated. Overall, as is evidenced by the graph in Figure 1 and Table 1, the use of electronic resources was strong. The use of printed textbooks was high, but journal article use was low. The use of Medline declined drastically from the first block to the second. 3

4 Table 1: Reported Use of Resources for Academic Blocks 1 3. N=116 Students Block 1 Block 2 Block 3 All 3 Blocks # of weeks with learning issues # of weekly reports submitted % a (count b ) % (count) % (count) % (count) Printed Resources Journal articles 7.8 (45) 6.6 (38) 5.6 (13) 6.9 (96) Textbooks 53.4 (310) 48.6 (282) 35.8 (83) 48.5 (675) Other 13.3 (77) 10.7 (62) 9.5 (22) 11.2 (156) Electronic Resource World Wide Web using: - a medical Web site 53.6 (311) 39.0 (226) 32.8 (76) 44.0 (613) - a general search engine 36.6 (212) 27.4 (159) 43.1 (100) 33.8 (471) - a medical search engine 40.9 (237) 30.0 (174) 20.7 (48) 33.0 (459) Medline 20.2 (117) 7.1 (41) 1.7 (4) 11.6 (162) Harrison's 31.0 (180) 38.6 (224) 19.4 (45) 32.3 (449) Stedman's 5.5 (32) 5.0 (29) 1.3 (3) 4.6 (64) Goodman & Gilman's 2.9 (17) 6.0 (35) 3.4 (8) 4.3 (60) Radiologic Anatomy 1.4 (8) 1.2 (7) 1.1 (15) Other: 3.8 (22) 2.8 (16) 5.2 (12) 3.6 (50) Human Resources Faculty 6.4 (37) 9.5 (55) 11.2 (26) 8.5 (118) Other students 15.3 (89) 16.0 (93) 11.6 (27) 15.0 (209) Clinicians 10.5 (61) 10.5 (61) 13.8 (32) 11.1 (154) Other 3.1 (18) 1.9 (11) 9.1 (21) 3.6 (50) Physical Evidence 3D Models.7 (4).3 (2).4 (1).5 (7) Displays 1.7 (10) 2.9 (17).9 (2) 2.1 (29) Demonstrations (acting out) 1.9 (11).9 (5) 1.7 (4) 1.4 (20) Other.9 (5).7 (4).9 (2).8 (11) a Percentage of weekly reports in which the resource category was reported as having been used. b Number of weekly reports in which the resource category was reported as having been used. 4

5 The mean variety of resource types (i.e., mean number of categories from which resources were selected) decreased from 1.83 to The differences were not statistically significant between blocks. In order to examine the use of resources from a variety of categories further, a variable was created to indicate whether or not a student s weekly report of resources had included resources from two or more categories. The ratio of reports indicating 2 resource categories to those with <2 was 65 to 35 in block 1; 58 to 42 in block 2; and 53 to 47 in block 3. Kruskal-Wallis one-way ANOVAs revealed statistically significant differences among blocks in the proportion of weekly reports that indicated 2 resource categories (p<.01). Post hoc tests on pairs of blocks of data indicated significant differences between blocks 1 and 2 (p<.05) and blocks 1 and 3 (p<.01). However, blocks 2 and 3 were not significantly different. Discussion This study examined changes in resources used by medical students in a longitudinal problem-based learning course. The data indicated a significantly lower number of resources being used by students to research self-selected learning issues as the course progressed. The trend in evidence during the three academic blocks comprising the fall semester may indicate a greater efficiency in finding the information sought. On the other hand, it may indicate a decline in student effort in seeking information. A study of the quality of the students research may help to interpret these findings. A decline in the mean variety of resources, although not statistically significant, is of concern particularly since it indicates that, on average, fewer than two different categories of resources are being consulted weekly. A progressively decreasing proportion of students consulting resources from more than one resource category for a learning issue (65% in block 1 down to 53% in block 3) supports this concern. If having the students use a variety of resource types for the purpose of gaining a perspective on the nature, value, and limitations of resources remains a goal of the PBL course, then an intervention may be necessary to re-stimulate the use a variety of resource types as the course progresses. Such interventions may include additional group facilitator training on the use of resources or a revision of course guidelines. Before considering an intervention, it would be important to investigate the cause(s) of the decline. This may require some qualitative data collection. The upswing in human resource use in the third block is interesting from the standpoint that it may indicate that students have become more comfortable with consulting with faculty and clinicians as they mature into the PBL process. Continued tracking of students resource use would be needed to see if the trend continues. On the other hand, the increase in use of human resources may be due to the nature of the PBL case that was presented during the third academic block. The case had strong psychosocial content that encouraged finding out about social support for the elderly. Non-human resources may not have been adequate for investigating these particular learning issues. If the latter reason for the increase were to prove correct, it would be evidence of cases influencing resources used. To test the hypothesis, it would be necessary to relate case and learning issues to resources used. Due to the overlap of cases within sessions and the possibility of a student s having more than one learning issue for a week, our current system does not allow for capturing such data. A system for recording case, issue, and resources for each issue would have to be designed that would capture the information without overburdening the student with recording information. The heavy use of electronic resources, particularly the World Wide Web, supports speculation that resource use habits of students appears to be moving away from a predominant reliance on printed materials. The predominance of use of resources from the electronic category over other categories of resources was sustained across the three academic blocks. It is reasonable to expect this pattern to continue. Although this study followed students as they progressed in their assigned groups through the first three academic blocks, it may be of value to follow the students progress through the remaining four blocks of the pre-clinical curriculum. In the fourth block, students are re-assigned to new groups and new facilitators. For the sixth block (beginning of the second year of medical school), the students are reassigned to new groups once again and the format for case presentation is changed. Further tracking of resource use would allow a determination of whether the trends identified initially continue to persist and whether resource use can be re-stimulated by changes in group assignment and/or case presentation format. 5

6 References 1. Mann MD, Clark FJ: A poster exercise to teach skills of information retrieval and presentation to first-year medical students. Teaching and Learning in Medicine 1995; 7: Hilliard RI: How do medical students learn: Medical student learning styles and factors that affect these learning styles. Teaching and Learning in Medicine 1995; 7: Newble DI, Entwistle NJ: Learning styles and approaches: Implications for medical education. Medical Education 1986; 20: Barrows HS, Tamblyn RM: Problem-based learning: An approach to medical education. New York: Springer Publishing Company, Mitchell R: The development of the Cognitive Behavior Survey to assess medical student learning. Teaching and Learning in Medicine 1994; 6: Schilling K, Ginn DS, Mickelson P, Roth LH: Integration of information-seeking skills and activities into a problem-based curriculum. Bull Med Libr Assoc 1995; 83: Lawrence SL, Grosenick DJ, Simpson DE, Van Susteren TJ: A comparison of problem-based and didactic approaches to learning on an ambulatory medicine clerkship. Teaching and Learning in Medicine 1992; 4: Marshall JG, Fitzgerald D, Busby L, Heaton G: A study of library use in problem-based and traditional medical curricula. Bull Med Libr Assoc 1993; 81: Saunders K, Northup DE, Mennin SP: The Library in a Problem-Based Curriculum. In: Kaufman A., ed. Implementing problem-based medical education. New York: Springer, 1985: Huck SW, Cormier WH: Reading statistics and research. New York: HarperCollinsCollegePublishers, Dr. Deretchin is Education Director, Academic Informatics Services, Baylor College of Medicine. She can be contacted via at: louised@watson.bcm.tmc.edu. 6