Traditional Lecture Format Compared to Computer-Assisted Instruction in Pharmacy Calculations

Transcription

1 Traditional Lecture Format Compared to Computer-Assisted Instruction in Pharmacy Calculations Jeffrey C. Delafuente, Oscar E. Araujo and Sue M. Legg College of Pharmacy, Health Science Center, PO Box , University of Florida, Gainesville, Florida Learning outcomes as measured by exam scores in a pharmacy calculations course were compared between two groups of students: one group in a traditional lecture format and one group in a computer-assisted instruction (CAI) format. One hundred ten students were divided into two groups seven weeks after both groups were in a lecture format. Exam scores for the two groups were similar before the groups split into their respective learning formats as well as after six weeks of learning by either lecture or CAI format. The CAI program was easy to use and students evaluation of the program was favorable. Students in the CAI group spent less time learning the material than did their counter parts. We conclude that the evaluated CAI program resulted in similar learning outcomes compared to those of a traditional lecture format. INTRODUCTION Computer-assisted instruction (CAI) has been available for about two decades but it has not been used in pharmacy education to any great extent. One reason for its limited use has been the lack of available commercial software. Although some pharmacy faculty have created their own software, the process is time consuming, software development can be expensive, and many faculty lack the expertise to write computer programs. CAI is an attractive learning method as it engages the student in an active learning process(1,2). CAI requires students to be responsible for their own learning, a quality necessary for professional practice. Interactive software gives students immediate feedback and allows students to learn at their own pace. Our faculty decided to incorporate the use of CAI into our revised curriculum. We thought that our required pharmacy calculations course would be a good choice for CAI because the students have divergent abilities to solve mathematical problems and therefore have varied needs for instruction. CAI would allow students to work at their own pace and review the instructional content as often as needed. Before replacing faculty with computers, it must be known how the software to be used would affect students learning. Learning from a CAI program must be at least the same or better than learning from a traditional classroom setting to be a useful substitute for lecturing. We tested a commercially available pharmacy calculations computer program to determine if student learning was different between CAI and a traditional lecture format. METHODS One hundred ten students in a required course in pharmacy calculations were randomized into two groups of 55 students each; one was a control group assigned to a traditional lecture format, the other group was assigned to a CAI course. During the first seven weeks of the course both groups were assigned to the traditional lecture format. The professor teaching the traditional lecture format has been teaching this course for more than 30 years. The lecture format and course content were not altered for this study. At week eight of the course the two groups were split into their respective learning formats. All students in the CAI group were offered copies of the calculations software program to use on their own personal computers. For students who did not have access to a computer at home, the software program was installed in a computer laboratory which was accessible to students for approximately 40 hours/week, Monday through Friday during normal business hours. Students in both test groups were encouraged to purchase an extensive set of practice calculation problems that were similar to the examples used in the classroom format. The professor teaching the course was available to provide learning assistance to students in both groups during scheduled office hours. The calculations software program evaluated was Basic Calculations in Pharmacy (version 3.0), provided by PCCAL International, Bath, UK. The program contains 25 modules, of which 11 were assigned to the CAI group. The modules used in our course were: percentages, percent weight in volume, percent weight in weight, percent volume in volume, parts, components and ratios, solubility expressions, normality and equivalence, conversion between concentrations, increasing/decreasing formula, number of doses, and alligation. The four modules on percentages were analyzed as a single module. These modules corresponded to the topics being taught in the traditional lecture format class. Most of the activities within each module follow a similar structure. The student is given a brief introduction to the topic that might include definitions and where the calculation is used in pharmacy practice. Next an equation is shown that can be used for the calculation along with its derivation. Example calculations are then given, and students are offered an opportunity to practice using these equations. When a student responds incorrectly to the example problems, the computer program notifies the student and allows the student to try again. Solutions to the example problems are given when asked by the student. Sample questions for the material covered in that module are then presented, with questions getting more difficult as the lesson progresses. Finally, each module ends with a summary of the materials presented. The first exam of the course occurred following the first 62 American Journal of Pharmaceutical Education Vol. 62, Spring 1998

2 Fig. 1. Evaluation of module components. Data are the combined responses for the ratings of good and excellent for each of the four evaluated module components in all eight modules (n=16). Table I Comparison of test grades (mean ± SD) Computer assisted instruction Traditional lecture format P value a Exam ± ± Exam ± ± Exam ± ± a ANOVA for differences between paired means. seven weeks of the course, when both groups were in the traditional lecture format. The second exam, which was not cumulative, was given after the remaining six weeks in the course. A cumulative final exam was given two weeks after the second exam. Exam grades were compared between the two groups using a one-way ANOVA for differences between paired means (True Epistat, ver. 4.00, Epistat Services, Richardson, TX). The week prior to final exams a questionnaire (Appendix) was distributed to all students in the CAI component of the course. The questionnaire had eight major topic headings listed and asked students to rate the quality of the software for each of the modules. For each module students were asked to assess the learning objectives, concept explanation, examples, and sample questions used in the modules. A scale was used for rating the software with 1 = poor, 2 = fair, 3 = adequate, 4 = good, 5 = excellent. The questionnaire also allowed students to write specific comments about each of the eight modules. In addition, students were asked to respond to five open-ended questions regarding the CAI. These questions were designed to identify what the students liked and disliked about the CAI program, how they perceived it affected their learning, and how much time they spent working on the CAI program. One hundred seven students were used for analysis. Two students, one in each test group, were not included in the analysis because they were repeating the course. Another student assigned to the CAI group dropped out of the course because of poor grades and was not included in the results. RESULTS Test Grade Results Table I shows the exam grades for the two test groups. There was no difference between the two groups for exam Fig. 2. Evaluation of modules. Data are the combined responses for the ratings of good and excellent for each of the evaluated modules (n=16). one, when both groups were in the traditional lecture format. Exam two covered material that was taught by the either CAI or lecture format. Again, there was no significant difference in test grades, although the traditional lecture format group did score higher, which was close to being statistically significant. Grades for the two groups were comparable for exam three that was cumulative for the entire course. An analysis of exam three test questions pertaining to material covered by the CAI showed that there was no difference (P=0.99) in the percentage of correct answers between the two groups. There was no significant difference for final course grades between the two groups (data not shown). Assessment of the CAI Software Only 16 (29 percent) students returned the evaluation questionnaire. The means for the ratings of the modules are shown in Table II. The assessed module components all had mean scores that corresponded to ratings between adequate and good. Module component data for the eight assessed modules were combined and the percentages of excellent and good responses for the four module components are shown in Figure 1. Fifty percent or more of the students evaluating these four components rated them as excellent or good. The percentages of responses for students rating the eight modules taught by the CAI as either excellent or good are shown in Figure 2. These percentages were derived by averaging the percentage of excellent and good responses for each of the four components of each module. There is an obvious difference in how the students perceived the quality of these modules. Reducing/increasing formulas, parts, components, and ratios, and calculating the number of doses and percent error were the three modules that received the highest ratings. Only 35 percent of the students rated the module on equivalence, milliequivalence, and normality as good or excellent. Students Written Comments Students liked the time flexibility of the CAI. They were able to learn using the module when it was convenient to their schedules, particularly for those students who were able to use a home computer. We also received favorable comments regarding the ability of students to work at their American Journal of Pharmaceutical Education Vol. 62, Spring

3 Table II. Evaluation of the CAI module components Objectives Concept explanation Examples Sample questions Percentages 3.75 a Solubility Conversion between concentrations Pharmacy practice: alligation Pharmacy practice: reducing/increasing formula Pharmacy practice:number of doses/percent of error Parts, components, and ratios Equivalence, milliequivalence, normality a Data shown are means for the scale: l=poor, 2=fair, 3=adequate, 4=good, 5=excellent. n=16. own pace and review concepts multiple times if necessary. Students who had to rely on the computer lab for CAI access complained that the lab hours were inconvenient. There were also some technical problems which took a couple of weeks to resolve which also caused inconvenience and some anguish. Noise in the computer lab also frustrated some students. Some students perceived that the exams were geared toward the students taking the traditional lecture format, placing the CAI group at a disadvantage. The students felt this way because the exam questions were similar to the example questions used in class which were different in structure from those of the CAI format. Some students thought that the sample calculation questions in the CAI were not as difficult as those on the exam and suggested that the difficulty of the questions increase in the CAI program. Some students used the program on a regular weekly basis, approximately two to four hours per week. Other students wrote that they waited until exam time, and then spent a few hours on the CAI course prior to the exam. DISCUSSION Although CAI has been used in pharmacy education, the extent of such use apparently lags behind other health professions. A Medline search using the search headings of computer assisted instruction plus one of the following terms, pharmacy, nursing, medicine, dentistry, revealed interesting results. From February 1997 only 10 records appeared for CAI plus pharmacy. Nursing had 487 records, medicine had 351 records, and dentistry had 29 records. These diverse results may be explained in part because the nursing and medicine professions are larger in terms of the numbers of practitioners, colleges and schools, and faculty than in pharmacy. Perhaps these other professions have had more resources or formed consortia to develop CAI. Also, Medline may not index journals where computer applications in pharmacy might appear. None the less, it would appear that CAI in pharmacy education is not prevalent. CAI has built in advantages that are in sync with contemporary thinking about pharmacy curricula and learning outcomes. First, they have the ability to enhance computer skills, particularly in those students who are novices. CAI can contribute to enhancing problem solving skills and develop a student s sense of responsibility for their own learning. In a setting such as that which we used, students must learn time management skills to schedule appropriate amounts of time for learning, a skill which one must have as a health care professional. CAI is an active learning process and a well-written CAI program will give students immediate feedback on their learning. For the reasons just listed, and others, our faculty has agreed to use CAI as part of our new curriculum. Our goal was to evaluate a computerized course in pharmacy calculations to determine if it would be a suitable substitute for our traditional lecture format course. Before embarking on a CAI program that has not been reviewed in the literature we wished to determine if learning outcomes, as measured by exam scores, were similar between the CAI program chosen for study and a traditional lecture format. During the first seven weeks of the course both experimental groups learned in a traditional lecture format. The material taught during this time period did not have corresponding modules in the CAI software. As shown by exam scores following this period, there was no difference between the two groups ability to perform pharmacy calculations. There was also no difference in exam scores when the two groups were divided into their respective learning formats. Others have shown comparable test scores between CAI and traditional classroom teaching(2-7). We evaluated the computer program by asking questions about the major components of each module. Overall, students rated the learning objectives, explanation of the calculation, use of example calculations, and sample questions as adequate to good. However, more than 50 percent of the respondents rated each category as good or excellent. This suggests that the program as written is performing well. There was a difference in how the students perceived the quality of the individual modules. Fifty percent or more rated the modules on reducing/increasing formula, parts, components, and ratios, doses and percent error, and conversions as being good to excellent. The other four evaluated modules did not rate as high, but still had mean scores between adequate and good. These lower ratings may be because the concepts and applications of these topics are often difficult for students in our traditional classroom setting, and the same may be true in this CAI program. We learned several key management points regarding a computer laboratory. Normal business hours for a CAI laboratory may be inconvenient for students. Many students prefer evening and weekend access. It is important to have all the technical problems worked out prior to assigning students to the CAI. Although most of the students were understanding about some minor technical problems we encountered, it was a cause of frustration which may have negatively affected their attitudes about CAI. A computer lab operator needs to be available during times when the laboratory is open to assist students who have difficulty with the software or hardware. Some students thought that the CAI group was at a disadvantage because they perceived that the exam was written based on classroom discussions and example calculations. The CAI used example calculations that did not always follow those used in the traditional setting. Prior to 64 American Journal of Pharmaceutical Education Vol. 62, Spring 1998

4 beginning this study we tried to avoid teaching to the test to prevent a bias in favor of the traditional lecture format. Despite our attempts, this may have happened as indicated by the higher mean score on the second exam in the lecture group. Our interpretation of some students comments suggest that there may have been a problem with their ability to transfer knowledge obtained from the CAI practice problems to the examination problems. A supplemental set of sample problems was available for students to purchase that contained problems similar to those on the exam and discussed in the lecture format class. The available study guide should have prepared CAI students for the examination, however, we do not know how many students used it. If questions used in a CAI format are of a different nature than those that are to be used for assessment of learning, then an appropriate study guide or other form of help needs to be available. Although the CAI group of students had access to the professor to receive tutoring or clarify concepts, no students took advantage of this opportunity. Answers to the open-ended questions on the evaluation form revealed that some students only spent several hours on the modules. Some students commented that they were able to skip portions of the module on concepts that they already knew. In general, it appears that the students in the CAI group spent less time working at the computer then their counter parts spent working in the traditional lecture format. Similar results have been shown by other investigators(2). One complaint we frequently hear from students as we have moved towards a problem-based learning format is the increased time they spend preparing for classes and working in small groups outside of class. CAI may help with this problem by decreasing studying time without compromising learning. The use of CAI would appear to be a time efficient means of delivering learning materials as the student can use the computer at times that are convenient to his or her schedule. Some students commented that the CAI should be used as a supplement to a lecture format, rather than a standalone course. This attitude by students has been reported previously(8). From reading students comments it is apparent that some students prefer to be passive learners wanting to go to class and take notes. A few students felt cheated in that they had paid tuition, but did not have a professor lecturing to them, as that was their expectation of a college course. As more students enter college with advanced computer skills perhaps this attitude will change. Our findings should be interpreted carefully. We have no way of knowing if any of the students assigned to the CAI format received copies of class notes from their classmates assigned to the lecture format. Likewise, students in the classroom setting could have accessed the computer lab or used a classmate s computer loaded with the CAI. Such cross-contamination would invalidate the findings. The low response rate (29 percent) to the questionnaire used to assess students attitudes about the CAI course was unexpected. However, we did not send out follow up surveys to those students not returning the questionnaire. There was no penalty for those students who chose not to participate in the evaluation process. It is not clear why there was such a poor response. One possibility is that we asked the students to fill out the evaluation instrument just prior to final exams. Perhaps students felt that they were too busy to evaluate the CAI format. Apathy might be another explanation. CONCLUSIONS Exam scores for pharmacy calculations taught in a traditional lecture format are similar to exam scores for those students learning the same material by use of CAI. The particular CAI program we evaluated was easy to use and was found to be acceptable by the students. Use of CAI was time efficient, as students in the CAI format did as well on exams as did the lecture format students, but generally spent less time learning. The CAI program does not contain modules that parallel all of the topics that have traditionally been taught in our long-standing pharmacy calculations course. This would limit the usefulness of such a CAI program as there would still be a need for additional teaching, either via lecture, self-study guide, or an additional CAI program to cover the additional topics and help students transfer their learning to the clinical setting. Am. J. Pharm. Educ., 62, 62-66(1998); received 4/7/97, accepted 1/5/98. References (1) Brimberry, W. M. and Riffee, W. H., Computers in the classroom: a new form of active learning, Am. J. Pharm. Educ., 59, 1-7(1995). (2) Anderson-Harper, H. M., Mason, H. L., Popovich, N. G., Developing, implementing and evaluating microcomputer instruction in a nonprescription drug course, ibid., 52, (1988). (3) Hurst, A. K., Noguchi, J. K. and Besinque, K. M., Use of computerassisted case presentations in A clinical therapeutics course, ibid., 52, 56-58(1988). (4) Jim, L. K., Gee J. P., Hyneck, M. L., Shannon, M. C, Fox, J. L. and Filibeck, D. J., A Computer-assisted instructional approach to teaching applied therapeutics, ibid., 48, 20-25(1984). (5) Clem, J. R., Murry, D. J., Perry P. J. and Alexander, B. Performance in a clinical pharmacy clerkship: computer-aided instruction versus traditional lectures, ibid., 56, (1992). (6) Kinade R. E., Mathews C. T., Draugalis J. R. and Erstad B. L., Evaluation of a computer stimulation in a therapeutics case discussion, ibid., 59, (1995). (7) Anderson-Harper H. M., Mason, H. L. and Popovich N. G., Attitudes and beliefs of pharmacy students about using computers for instruction, ibid., 54, (1990). (8) Chisholm, M. A., Dehoney J., Poirier S. Development and evaluation of a computer-assisted instructional program in an advanced pharmacotherapeutics course, ibid., 60, (1996). APPENDIX. DIRECTIONS: Using the following scale, please rate the quality of the software for each of the topics below. Explain any low ratings. P=Poor, F=Fair, A=Adequate, G=Good, E=Excellent P F (1) (2) A G E (3) (4) (5) 1. Percentages a. Objectives c. Examples d. Sample Questions 2. Solubility a. Objectives c. Examples d. Sample Questions 3. Conversions between concentrations a. Objectives American Journal of Pharmaceutical Education Vol. 62, Spring

5 c. Examples d. Sample Questions 4. Pharmacy Practice: Alligation a. Objectives c. Examples d. Sample Questions 5. Pharmacy Practice Reducing/Increasing Formula c. Examples a. Objectives d. Sample Questions 6. Pharmacy Practice: Number of Doses/percent of error a. Objectives c. Examples d. Sample Questions 7.Parts, Components, and Ratios c. Examples a. Objectives d. Sample Questions 8. Equivalence, Milliequivalence, Normality a. Objectives c. Examples d. Sample Questions 9. What do you like best about using this program? 10. What do you like least? 11. What materials/activities could be added to improve your learning? 12. How confident are you that you can pass an exam in this material? 13.When do you use this program? Please describe how often and how much time you spend. 66 American Journal of Pharmaceutical Education Vol. 62, Spring 1998