Web-Based Orthodontic Instruction and Assessment. Ruangrat Komolpis, D.D.S., M.S.; Richard A. Johnson, D.D.S., M.S.

Transcription

1 Educational Methodologies Web-Based Orthodontic Instruction and Assessment Ruangrat Komolpis, D.D.S., M.S.; Richard A. Johnson, D.D.S., M.S. Abstract: Orthodontic records study models, panoramic and cephalometric radiographs, and the patient s facial and intraoral photographs are used to collect data to establish a diagnosis and to develop problem-solution lists. These records, however, can be damaged or lost when dispensed to students and need to be stored and maintained every year. An orthodontic diagnosis web site, therefore, has been set up using digital records to provide students with an accessible source of complete, good-quality study materials. The web site is also used for clinical examination in orthodontic courses. The effectiveness of the web-based digital records in providing relevant information to students in comparison to the traditional records was evaluated by a randomized controlled trial involving ninety-nine second-year dental students. One group (fifty students) studied two cases from the web site; the other group (forty-nine students) studied the same two cases from traditional orthodontic records. Effectiveness was assessed by comparing test scores and the time spent on the tests by t-statistics. There were no significant differences between means for the two study groups in test performance or time. Attitudes of students toward the web site, assessed from post-test questionnaires, were positive. It was concluded that web-based digital orthodontic records were as effective in teaching clinical orthodontic diagnosis as were conventional records. Dr. Komolpis is Lecturer and Dr. Johnson is Clinical Associate Professor, Department of Orthodontics and Pediatric Dentistry, School of Dentistry, University of Michigan. Direct correspondence and requests for reprints to Dr. Richard A. Johnson, Department of Orthodontics and Pediatric Dentistry, School of Dentistry, University of Michigan, 1011 North University, Ann Arbor, MI ; phone; fax; cannes@umich.edu. Key words: computer-assisted instruction, web-based instruction, dental informatics, orthodontics, computer-based patient records Submitted for publication 9/17/01; accepted 4/8/02 Dental educators have sought for the last decade to integrate computers into the dental curriculum. The level of integration has varied, depending on the institution, the educational philosophies of individual instructors, and the subject material being taught. With the advent of faster and more powerful computers, as well as access to largescale networking such as via the Internet, the concept of information dissemination has taken on a whole new meaning. Consequently, the use of computers in the dental curriculum must be explored and approached with the understanding that, in the near future, computers will be not just a valuable asset, but a necessity. Although computer-assisted instruction (CAI) is not new, it has only recently begun to emerge as a mature educational tool in dental education. Past studies of the potential benefits of dental-related CAI have focused largely on computer programs that require a specific computer platform. In addition, many of these programs failed to incorporate networking features. In recent years, the availability of broadbandwidth networking and computer code that is not platform-specific has allowed the construction of programs that can be modified easily and will possess the flexibility of network interactivity. The most common example is in the HTML programming language, which has evolved into a universally accepted means of content creation. Internet users, including most dental educators, encounter HTML-encoded content by means of their interaction with materials available on the World Wide Web. This evolution of hardware and software has opened a vast resource to educators interested in implementing CAI into their curriculum. The Internet has enhanced the role of CAI in today s educational armamentarium as well as the potential integration of academic resources, patient care, and administrative tools in dental education. 1 Computer-assisted instruction in orthodontics has varied from text-based programs, with or without graphics, concentrating on cephalometric fundamentals 2-4 and current clinical procedures 5 to casebased expert systems that match a given case to a previously treated case for treatment planning. 6 Most 650 Journal of Dental Education Volume 66, No. 5

2 expert systems are commercial programs focusing on case assessment and treatment planning for general practitioners 7 or orthodontists in continuing dental education courses, rather than on teaching in dental school settings. Orthodontic education at the predoctoral level is largely designed to provide experience in identifying and evaluating orthodontic problems in the primary, mixed, and permanent dentitions. 8 Using case study material, students can practice developing problem lists and making diagnoses of a wide variety of occlusal abnormalities, many of which may not be seen during limited rotation time in the orthodontic clinic. Orthodontic records study models, panoramic and cephalometric radiographs, facial and intraoral photographs, health questionnaires, and clinical examination details have been used to present data needed to make diagnosis and develop problem-solution lists. These records, however, can be damaged or lost when dispensed to students and need to be stored and maintained every year. Storing records in digital form is an option to overcome problems in handling hard-copy records. Computer-based patient records are not new; however, they have evolved significantly in the past ten years because of the emergence of the Internet and related technologies. 9,10 The project of transforming orthodontic records into electronic form on the Web, therefore, was undertaken to provide students with an accessible source of complete, high-quality study materials. Digital orthodontic records for use in case diagnosis and treatment planning were used to set up an orthodontic diagnostic web site that can be accessed through the University of Michigan School of Dentistry Intranet (Internal Organizational Network a private version of the World Wide Web that is available only to members of a particular organization). The purpose of this paper is to describe the design and set-up of the web site and define how the site might be used as a resource for small-group learning in a preclinical orthodontic course for secondyear predoctoral students. In addition, the effectiveness of the web site in presenting diagnostic data when compared to hard diagnostic records will be tested by a randomized, controlled trial involving ninety-nine second-year dental students. Finally, the tangible costs of producing the web site in faculty time and programming expense will be discussed. Design of the Orthodontic Web Site The web site was developed in two phases. It was initially set up in 1998 with pre-treatment records of patients with various malocclusions for students to make orthodontic diagnoses in a problem-based learning format. In the second phase, post-treatment records, as well as progress records, where available, of the existing patients on the web site to teach orthodontic treatment outcome and appliance usage were added. Currently, the web site contains records of six adult and twelve child patients. The home page of the web site contains links to user instruction, to all patients pre-treatment records, and to an archive of images (or where appropriate, a video clip) of orthodontic appliances and clinical features related to orthodontic clinical examination. (See Figure 1.) Navigating tools in the left frame of the page allow for searching the entire web site for cases with specific orthodontic problems when matching one or more items provided in categories and pull-down menus. Pre-treatment orthodontic records were used to set up the web site. The records consisted of: 1) medical and dental histories, 2) facial photographs in relaxed frontal view, smiling frontal, and profile view, 3) intra-oral photographs in five views frontal, left, right, upper occlusal, and lower occlusal, 4) panoramic and lateral cephalometric radiographs, and 5) orthodontic study models in centric occlusion. To prepare the digital records, facial and intra-oral radiographs and photographs were digitized, and study models in centric occlusion were photographed digitally in five views frontal, left, right, upper occlusal, and lower occlusal. The upper occlusal image was taken with a millimetric ruler on the palate between the first molars to permit arch-width estimation in two dimensions. These digital images were placed in an orthodontic diagnostic web site to serve as web-based records. Each case has a pre-treatment index page containing links to patient medical and dental history page, clinical examination form, treatment plan form, extra-oral images, intra-oral images, study model images, and radiograph images (Figure 2). Each of the images can be enlarged for better viewing (Figures 3-5). A navigating bar on the bottom of each page contains links to group of images within the case, thereby precluding the need to go back to the case index page. May 2002 Journal of Dental Education 651

3 The clinical examination form on the web was modified from the form routinely used in orthodontic clinics at the School of Dentistry. All questions regarding patient function were removed, leaving only items that can be answered from the information provided by the pre-treatment records. This truncated examination form consists of extra-oral (assessment of skeletal relationships) and intra-oral sections (tooth and malocclusion identification). The treatment plan form contains blank lists of problems and possible solutions, open-ended questions regarding the case, and a blank space to fill out the treatment plan. The treatment plan form, when open, is in another window for ease of use and for possibility of reviewing the case while answering the questions on the form. Students can complete this form electronically and choose a printer-friendly version before printing it out for a course. When available, post-treatment records have been added to the case data. Post-treatment records are set up in another window to permit side-by-side viewing and comparison of the pre- and post-treatment records. Aside from post-treatment facial, intraoral, model, lateral cephalogram, and panoramic x-ray in the same fashion as pre-treatment records, post-treatment records also contain lateral cephalometric tracings, overall and regional superimpositions of the pre- and post-treatment tracings, measurements of the two cephalograms, and a treatment summary. Tracings of the lateral cephalograms were performed manually and then digitized using Dentofacial Planner 6.0 (Dentofacial Software, Toronto, Canada). The treatment summary includes a complete list of problems and treatment options, treatment plans, all treatment phases the patient received, and the analysis of treatment effects. Links to images of specific appliance(s) used for the case are also available on the treatment summary page. The Use of the Web Site The web site has been used for the past three years in the clinical orthodontics foundation level courses for first- and second-year dental students both in full-class, case-based discussion format and in smallgroup seminars in the school s computing laboratory. Figure 1. The web site home page contains links to user instruction, case diagnosis, and image archive of clinical features and appliances. Left navigating bar allows the search for case(s) with specific orthodontic problem(s). 652 Journal of Dental Education Volume 66, No. 5

4 Figure 2. Case index page contains links to the patient s health questionnaire, clinical examination form, treatment plan form, and facial, oral, model, and radiograph images. All thumbnail images are links to enlarged images. At the bottom of the page a link is available to show post-treatment records in another window. Figure 3. Enlarged intra-oral frontal image with links to other intra-oral views. May 2002 Journal of Dental Education 653

5 In the full classroom sessions, the instructor can access patient information from the web site to introduce the areas being studied that day, to provide material for the in-class exercise, or later in the session to clarify issues uncovered during the smallgroup discussions. Patient information is projected for all in the room to see, but student groups are free to move through the web site themselves at any point in their discussion, if they feel that doing so will aid them in reaching a consensus or lead to better understanding in their group. In the small-group seminars held in the school s computing laboratory, students usually work in pairs on one web-based case selected by the instructor, based on the orthodontic problem the case presents for teaching. Students view the patient records and fill out the clinical examination form. Presently, this online form is printed out and filled in manually by the students. The results of a pilot study showed that students preferred to use a paper form so they can make short notes for themselves next to the entries and do not have to switch between windows while reviewing the records and completing the form. Images of clinical features or orthodontic terms not familiar to the students can be viewed from the image archive on the web site. After finishing the clinical examination, the treatment plan form is completed electronically and printed out for student record in the seminar. The instructor and students then access the post-treatment pages together and discuss the orthodontic problems and treatment options for the case and view both the actual treatment the patient received and treatment results. Links to images of unusual orthodontic appliances used in the case are also available. At the graduate level, both orthodontic and pediatric dental residents have used the web site for orthodontic diagnostic exercises, as well as in an examination. For orthodontic treatment planning, a take-home examination can be considered comparable to the way a clinician actually treatment-plans a case. However, prior to web-based records, a takehome examination was not convenient because the hard-copy records had to be duplicated and kept in a place where all students had access to them during the exercise or examination. Web-based records, therefore, permitted the institution of a take-home examination. For example, an examination for a course in management of children with craniofacial anomalies was conducted using web-based pre-treatment records of three patients to evaluate the students ability to diagnose and treatment-plan. Eighteen students were given passwords to the web site and the passwords were disabled a week later when the examination was due. Most students accessed the web site from computers at the school or at home in the area; however, some completed it while out of town, and one student worked from home in Canada. Only one student experienced an access problem and that problem was easily resolved. In a survey done at the end of the course, acceptance of the web-based examination was positive. On a scale of 1 (low) to 5 (high), web-based case access was given a score of 4.4, and the image quality was rated 3.8. Evaluation of Web-Based Records To date, the computer-assisted teaching tool described here has been used successfully in several courses in orthodontics. Web-based records eliminate the problems of maintaining and updating hardcopy records for teaching; however, before transforming conventional records to a web-based digital form, it is necessary to examine the effectiveness of digital orthodontic records in providing clinical information needed for diagnosis. Methods The effectiveness of digital records was compared with that of conventional records by using records of two mixed dentition patients with Class II malocclusion. The study was conducted as part of small-group orthodontic seminars in a clinical foundation course. The 103 members of the second-year dental class were randomly assigned into two study groups, conventional and web-based. The students had taken classes on growth and development, dental radiology, etiology and classification of malocclusion, occlusal development, and orthodontic diagnosis, but had not attended orthodontic clinic. We evaluated the effectiveness of the records in providing clinical information by using an orthodontic examination form. For each patient, the students reviewed the records and completed the orthodontic examination forms. Students in the conventional group used hard-copy records while those in the web-based group used digital records on 654 Journal of Dental Education Volume 66, No. 5

6 PC-compatible computers. At the beginning of the study, students in the web-based group filled out a questionnaire to assess their computer experience to evaluate the computer literacy of students in this group. They were then given access to the web-based records of the same two patients, as were examiners in the conventional group. After completing the study, the students were asked to give their opinions about the web site in a survey. Students in both groups were told that their performance on the study would affect only their daily grade (2 percent of the total points for the course), and there was no time limit for completing the test. Each student recorded the time he or she started and finished the exercise to allow the total time spent to be calculated. Answer sheets from both groups were mixed and scored by an independent examiner who was calibrated by one of the authors. Before all the answer sheets were given to the examiner to score, ten answer sheets were randomly picked, copied, and replaced to test for intra-examiner reliability. The examiner was blinded by using student lab bench numbers instead of names or groups. Inferential and descriptive statistics were calculated with the aid of a commercial software program (SPSS 9.0 for Windows; Statistical Package for the Social Science, Chicago, IL, 1999). The differences between group means were compared statistically by way of t-tests. To test for intra-examiner reliability, error standard deviation (SD E ) for the double determinations was calculated with the aid of Dahlberg s formula,where d is the difference between two corresponding measures and N is the number of double determinations. 11 This formula is a method of estimating random errors in replicated cephalometric or dental model measurements. Descriptive and inferential statistics for the exercise performance of both groups are shown in Table 2. There were no statistically significant differences between mean test scores or mean test times. Responses from the survey are summarized in Table 3. On a scale of 1 (low) and 5 (high), all ratings were above 3 (neutral), with the lowest score of 3.2 given to quality of the radiographs. The overall rating of the web site was 4.3. The error standard deviation (SD E ) from double determination test for intra-examiner reliability using Dahlberg s Formula was 0.22 out of a total exercise score of 50 points, implying a high degree of reliability. 11 Discussion Computer-based patient records, especially in a web-based format, are thought to be an important step in improving health care delivery systems, as well as increasing the effectiveness of medical education and administration. The use of web-based patient records in orthodontic education was the main focus of this study. Although the study should be considered limited since the two student groups were assumed comparable based only on their random selection, the findings suggest that the diagnostic quality of the digital orthodontic records on the web does not differ significantly from that of conventional records. Storing records in the digital form, therefore, would seem to be an efficient, effective way of providing students with an accessible source of complete, good-quality study materials. In digital form, teaching material can be updated and edited easily. If presented in an interactive format, the web can be used as an exam-question or O.S.C.E. practice bank. Results Four students were absent on the study days and were excluded from the study, leaving forty-nine students in the conventional group and fifty students in the web-based group. The computer experience of forty-nine students in the web-based group was assessed using the questions shown in Table 1. In general, the web-based group reported that they used computers regularly and had basic knowledge of the Internet; therefore, the technical difficulty of using the web site was not a factor. Table 1. Profile of web-based study group Computer Experience Responses (n=49) Computer preference PC-compatible 37 Mac 10 Both 1 No response 1 Frequency of computer use Daily 30 Several times weekly 11 Once weekly 8 Years of computer experience Six years + 31 Five years 7 Four years 7 Three years 3 Two years 1 General knowledge/ Mean: 3.6 on 1 (low) to comfort usingthe Internet 5 (high) scale May 2002 Journal of Dental Education 655

7 Figure 4. Lateral cephalogram is placed side by side with the profile image for students to practice evaluating the antero-posterior relationship of jaws and teeth when patients are seen clinically. Figure 5. Tracing and measurements of the case lateral cephalometric radiograph to aid in diagnosis and treatment plan. 656 Journal of Dental Education Volume 66, No. 5

8 Table 2. Descriptive and inferential statistics Conventional Web-Based Variable Mean SD Mean SD Difference t-test Test scores Time spent (min) P>.05 Table 3. Students opinions of the web site Mean score (scale of 1 to 5) Ease of use of the web site 4.6 Quality of facial, oral, and model images 3.7 Quality of radiographic images 3.2 Appropriateness of digital records 3.9 in teaching orthodontics Overall quality of the web site 4.3 Our students reported that the least favorable aspect of the digital records was the quality of the radiographs. Even though the radiographs cannot be enhanced or spot magnified on the web, their brightness can be adjusted using the monitor s brightness control. Although the extraoral radiographs used in orthodontic diagnosis panoramic and lateral cephalograms are not intended to provide diagnostic dental details, the students tended to focus on finding small lesions (i.e., caries), as they were doing in another web-based, small-group exercise during that fall semester. The low score for radiograph quality therefore could be due, at least in part, to the students inexperience with panoramic x-rays. The size of the models in the present web site can be estimated from the millimetric ruler placed on the palate of the upper cast. The image quality is appropriate for clinical diagnosis, but is not intended for research. If necessary, model images can be improved by the use of new 3D technology. Hologram and three-dimensional imaging systems have been used in place of orthodontic casts to solve the problem of storing these bulky, fragile dental records. 12,13 The recent development of a 3D computer-aided design system for dental models permits a quantitative evaluation of casts, as well as a movement simulation for orthognathic surgery treatment planning. 14 Services are also now available to convert alginate impressions to 3D digital models that can be downloaded conveniently via the Internet. 15 Despite the moderate score on radiograph quality shown on the survey and the two-dimensional model images, the students were satisfied with the overall quality of the web site. The students had a positive attitude toward using web-based study cases, perhaps because they became familiar with computers prior to reaching dental school and also because they had not handled conventional orthodontic records in dental school. The site s point and click feature is user-friendly for students at any level of computer literacy. Although perhaps not as concrete an experience as searching through a patient folder or holding conventional study models, the flexibility of window manipulation on the computer allows viewing different pages of patient records at the same time. The convenient access to the study material on the computer, especially in web sites that are available anytime from anywhere, facilitates students independent study and accommodates their different learning paces. Past studies of the use of computer-assisted instruction (CAI) have shown its advantage as an adjunctive means of improving the knowledge base of both students and practicing general dentists, with a preference for CAI over traditional educational tools such as textbooks and audiotapes. 3,7,16,17 Less quantifiable benefits of CAI involve positive changes in the motivation level of students who have been exposed to CAI. Previous studies have reported an increased interest in the subject matter and fewer of the logistical obstacles that commonly impede access to adjunctive instructional aids. 18,19 Another long-term benefit of CAI could be to reduce faculty time needed to prepare lectures or problem-solving exercises. CAI in pathology has been shown to reduce faculty lecture hours by 30 percent. 20 This advantage is especially important given the growing shortage of full-time faculty in orthodontics and the other specialties in dental schools throughout the United States. If distance learning is to be a part of future dental education, CAI, especially on the Internet, could play a major role in student learning. 21 Although the design and set-up costs for this web site were relatively low ($1,850 for photography, copying, and programming) and funded mainly by a School of Dentistry instructional computing May 2002 Journal of Dental Education 657

9 grant, the cost in faculty time was high. Dr. Komolpis was given 40 percent release time from teaching and faculty practice for one academic year for the project. That effort included finding the case material and reviewing it with Dr. Johnson, rewriting the patient information, photographing the models, and scanning all of the photographs and x-rays. To reduce the cost of production, university students were hired as web technicians. Their dental knowledge was low, however, so that pages and links needed to be outlined for them and then each page checked and tested for accuracy. Although communication with the technicians was largely by , Dr. Komolpis estimates that two to three hours per week during the eight months of active programming were spent answering urgent questions. For faculty at schools with an active web production facility and staff, production time should be less than ours, although perhaps more costly. In addition, delegating much of the case selection and preparation to an orthodontic resident or knowledgeable upper-class dental student should reduce faculty time. If this type of instructional development is valued by the dental school and the university (as it is at Michigan), its documentation should also provide a significant addition to a junior faculty member s teaching portfolio for the reappointment or promotion process. In conclusion, for orthodontic diagnosis, digital orthodontic records appear to be comparable to conventional records. Storing records in a web-based form, therefore, is an efficient way to generate, maintain, and update orthodontic records for education. Acknowledgments The authors would like to thank Drs. Nicole Wright and Joseph Kim for participating in the research and Dr. Lysle E. Johnston, Jr. for his support of the project and for reviewing this manuscript. The web site set-up was supported by a University of Michigan School of Dentistry Instructional Computing Grant. A generous grant from the estate of Edward A. Cheney, former Director of the Predoctoral Orthodontic Program at the School of Dentistry, is funding additions and maintenance of the site. REFERENCES 1. Johnson LA. The Internet approaching a ubiquitous tool for dental education. J Am Coll Dent 1999;66: Clark RD, Weerakone S, Rock WP. A Hypertext tutorial for teaching cephalometrics. Br J Orthod 1997;24: Turner PJ, Weerakone S. An evaluation of a hypertext system for computer-assisted learning in orthodontics. Br J Orthod 1993;20: Turner PJ, Weerakone S. Computer-based learning in orthodontics a hypertext system. Br J Orthod 1992;173: Marsh CM, Hannum WH, Trotman CA, Proffit WR. Design and effectiveness of a computer-based continuing education program for orthodontists. Angle Orthod 2001;71: Hammond RM, Freer TJ. Application of a case-based expert system to orthodontic diagnosis and treatment planning. Aust Orthod J 1997;14: Benson PE, Stephens CD, O Brien KD. Current software for teaching orthodontics. Br J Orthod 1996;23: AADS Section on Orthodontics and the Council on Orthodontic Education of the American Association of Orthodontists. Curriculum guidelines for orthodontics. J Dent Educ 1993;57: Dick RS, Steen EB, Detmer DE. The computer-based patient record: an essential technology for health care. Washington, DC: National Academy Press, Schleyer TK, Dasari VR. Computer-based oral health records on the World Wide Web. Quintessence Int 1999;30: Houston WJB. The analysis of errors in orthodontic measurements. Am J Orthod 1983;83: Romeo A, Canal F, Roma M, de la Higuera B, Ustrell JM, von Arx JD. Holograms in orthodontics: a universal system for the production, development, and illumination of holograms for the storage and analysis of dental casts. Am J Orthod Dentofacial Orthop 1995;108: Ayoub AF, Wray D, Moos KF, et al. A three-dimensional imaging system for archiving dental study casts: a preliminary report. Int J Adult Orthodon Orthognath Surg 1997;12: Motohashi N, Kuroda T. A 3D computer-aided design system applied to diagnosis and treatment planning in orthodontics and orthognathic surgery. Eur J Orthod 1999;21: Marcel TJ. Three-dimensional on-screen virtual models. Am J Orthod Dentofacial Orthop 2001;119: Pollard DJ, Davenport JC. An evaluation of training general dental practitioners in partial denture design using a computer-assisted learning program. B Dent J 1994;177: Stephens C, Grigg P. A computer-based orthodontic learning package: report of a trial. Dent Update 1994;21: Mulligan R, Wood GJ. A controlled evaluation of computer assisted training simulations in geriatric dentistry. J Dent Educ 1993;57: Lechner SK, Lechner KM, Thomas GA. Evaluation of a computer-aided learning program in removable partial denture framework designing. J Prosthodont 1999;8: Klatt EC. Web-based teaching in pathology. JAMA 1997;278: Haden NK, Beemsterboer PL, Weaver RG, Valachovic RW. Dental school faculty shortages increase: an update on future dental school faculty. J Dent Educ 2000;64: Journal of Dental Education Volume 66, No. 5