Evaluation of a Radiology Picture Archiving and Communication System Laboratory Exercise

Current Developments in Technology-Assisted Education (2006) 1253 Evaluation of a Radiology Picture Archiving and Communication System Laboratory Exercise A. Tzavaras 1, E.Ventouras 1,* 1 Department of Medical Instrumentation Technology, Technological Educational Institution of Athens, Agiou Spiridonos Str., Egaleo, 12210, Athens, Greece This paper presents the pilot evaluation of a laboratory classroom simulation of radiology networking technology. The simulation focused on educating undergraduate students in applying radiology Picture Archiving and Communication Systems (PACS) in a networked environment, based on the Digital Imaging and Communications in Medicine (DICOM) standard. The laboratory exercise structure enables students to simulate the operation of radiology departments in terms of DICOM communication with imaging devices, storing and retrieving images, as well as viewing and processing them. The evaluation was applied to a group of students implementing an exercise and results, as provided by filling a questionnaire indicate the overall positive attitude of students in using such tools in the educational process, especially concerning the motivation of students for active participation in the exercises and incitation for further reading of theoretical material related to the course. Keywords Digital Imaging and Communications in Medicine (DICOM) ; Picture Archiving and Communication System (PACS); postgraduate biomedical engineering education; evaluation, questionnaires. 1. Introduction Picture Archiving and Communication Systems (PACS) have become necessary for storing, printing, retrieving and reviewing in an efficient manner the high volumes of data extracted from imaging devices, i.e. modalities, such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Digital Radiography (DR) etc. [1, 2]. The Digital Imaging and Communications in Medicine (DICOM) standard has been universally accepted by the medical industry for exchanging imaging data between modalities [3]. Although DICOM-PACS technology is well established in many radiology departments, it is difficult for Universities to access it without securing cooperation with hospitals that implement such technology. Even when such cooperation is established, finding a time slot in the radiology department s operation and educating in-depth students on systems that are in operation is difficult and raises ethical questions since real patient data are stored and processed. The DICOM-PACS laboratory implementation of the networked operation of radiology departments eliminates the need to purchase expensive DICOM medical devices. Furthermore it provides detailed education to undergraduate students based on the learner centred education model [4]. The present work concerns the pilot evaluation of a laboratory classroom exercise simulating a radiology department s networking technology. The implementation of the DICOM-PACS laboratory is described in detail by A. Tzavaras et al [5, 6]. The evaluation is based on a questionnaire filled by students implementing one of the laboratory course s exercises. The evaluation results, as provided by processing the answers of the participants, indicate the overall positive attitude of students in using information technology tools in the educational process, concerning the ease of use of the tools and the satisfaction of the real-life simulating capabilities of the exercise. The influence of the exercise on clarifying the relevant theoretical course material was also appreciated, as was the motivation of students for active participation in the exercises and incitation for further reading of the topics related to the course. * Corresponding author: e-mail: ericvent@teiath.gr, Phone: +302105385387

1254 Current Developments in Technology-Assisted Education (2006) 2. DICOM PACS overview DICOM has emerged in the 1980 s by a joint effort of the American College of Radiologists (ACR) and the National Electrical Manufacturers Association (NEMA) [3]. It is becoming trivial for Medical Diagnostic Imaging devices such as computed tomography (CT), magnetic resonance (MR) imaging, Ultrasonography (US), nuclear medicine, fluroscopy and digital radiography to retrieve and process medical images in digital format. ACR-NEMA standard enables equipment of different manufacturers and modalities (CT, MR, etc) to communicate medical images and relevant information. DICOM standard represents the format and protocol in which digital radiology devices and scanners must use to facilitate interoperability. DICOM-enabled devices require special software and viewers in order to receive and send DICOM information. DICOM works in a networked environment using industry-standard TCP/IP and enables digital medical image transfer containing relevant equipment, patient and medical data. Medical devices that conform to the latest DICOM protocol are capable of exchanging information over the network. Current DICOM standard is DICOM 3 [2]. PACS utilize DICOM to exchange information with imaging modalities. PACS systems act both as medical image diagnostic viewer systems, enabling commenting, zooming, trimming and other image processing functions, and as storing and archiving systems [1]. PACS have built-in databases that support the storage/archiving functions and are usually equipped with both short and long-term storage devices such as Hard Disks and optical compact disks juke boxes. Since printing and digitizing films is important for radiology departments, PACS can communicate with DICOM-compliant printers and scanners. Furthermore, in an advanced radiology department PACS provide connectivity with health information systems, such as Radiology Information Systems (RIS) and more generally with Hospital Information Systems (HIS). 3. PACS education and training Training of PACS was initially offered in the annual meetings of the Radiology Society of North America (RSNA). Manufacturers organized special workshops by application specialists, with emphasis on viewing workstations components. The target audience was the radiologists and physicians. As PACS became the golden standard in the Radiology Information Technology (IT), vendors offered advanced seminars on various PACS components [7]. The transition from film to digital technology is a challenging task that requires the knowledge of IT engineers and the experience of radiology department personnel to ensure successful implementation. To ensure a thorough education of radiology personnel it is important to provide training not only on basic PACS utilities such as viewing and manipulating medical images, but also on hidden PACS components such as DICOM gateway and network workflow. The application of an extensive and advanced training requires the introduction of troubleshooting techniques. Thus advanced training requires an installation site of a PACS system, which is usually available to clinical site and vendors installations. A variety of factors are prohibiting the training conducted in the clinical site. Since modalities are designed to run 24/7, it is very difficult to gain availability for training purposes. Issues such as system s stability are crucial in the operation of installed PACS systems. Finally the utilization of real patient data for training purposes posse medical ethical dilemmas [5-7]. New trends in PACS education suggest in-depth training in all its components, in PACS simulation labs. Simulation labs utilize modality simulators to generate medical images, viewing stations to process medical images and DICOM gateways for image storage and archive process. The training in a laboratory environment exhibits several advantages over the clinical site training. It is available on a daily basis, the installation is a replica of the clinical one, stability of the system could be sacrificed for familiarizing students with troubleshooting techniques, and medical images are not related to real patients, overcoming medical ethical dilemmas. Such installations are established in several Universities such as the Department of Medical Instrumentation Technology of the Technological Educational Institution of Athens [6], the University of

Current Developments in Technology-Assisted Education (2006) 1255 South Carolina School of Engineering [8] and the Department of Optometry and Radiography of the Hong Kong Polytechnic University [9]. 4. DICOM PACS laboratory structure The laboratory is structured on networked PCs equipped with TCP/IP, running Microsoft Windows 2000 TM operating system; each of them hosting a DICOM- compatible software tool. The structure is such as to mimic a radiology department. Two computers on the network are equipped with DICOM imaging and gateway software and represent the modalities. The PACS PC, is assigned with server (workstation) capabilities and has installed minipacs software. The workstation is equipped with dual flat screens in order to act also as a medical diagnostic viewing station, and CD/DVD storage devices for storing and archiving. The software installed on the workstation, is a DICOM server that accepts and transfers data to and from modalities. The software has built-in database capabilities for short and longterm storage and archiving. The PACS simulation lab is described in detail by A.Tzavaras et al [6]. The laboratory aims to provide in-depth training of undergraduate students on installing, setting up and utilizing DICOM PACS software. The theoretical background on DICOM PACS is provided by the relative theoretical course. The software used is fully functional and allows students to explore medical image transfer, processing and storage in a laboratory setting. Implementing a DICOM-PACS system in a laboratory site provides students with training on DICOM networks using a holistic experience, from installing to utilizing and troubleshooting DICOM-PACS networks. 5. Evaluation Methodology The present pilot study evaluated three components of the undergraduate students training in PACS technology. The first component was the evaluation of the training material, i.e. notes, in terms of comprehensiveness and relevance with the training software. The second component was the evaluation of the medical software chosen for the purpose of training, in terms of its ability to simulate the radiology environment and the friendliness to the user. The third component was the evaluation of the training process with the following criteria: students satisfaction; training methodology; students participation in the training process; and the appropriateness of media used in the training. For the purpose of the evaluation we have chosen a laboratory exercise named Medical Image Archiving and Processing. The choice of the exercise was made based on the expected knowledge gains of the participating students. Students were expected to familiarize with basic concepts of medical networks, principles of remote viewing and archiving of medical images, and basic image processing techniques. A questionnaire was developed and circulated to students that participated in the laboratory exercise. The questionnaire was designed with closed questions. Answers were scaled on Likert scale [10], except from the final question were students were urged to provide additional comments on the training process. The responders were presented with the questionnaire at the end of the laboratory training. The students sample was formed by students who volunteered for the evaluation phase (convenience sample). The number of students participating as well as their synthesis (sex, age), was representative of a typical laboratory course participants. The structure of the questionnaire was designed to provide us with data for statistical analysis for the following key points: Level of familiarity of participants with Information Technology. Degree of satisfaction with the training materials available. Students motivation in the learning process. Software s appropriateness for the purpose of the simulated PACS lab. Degree of realistic simulation of a radiology department.

1256 Current Developments in Technology-Assisted Education (2006) 6. Evaluation Question fields, statistical analysis and coding are presented in table 1. Figure 1, presents the median and mean scores of questionnaire statistics for fields 11 to 25. Interpretation of scoring values is accomplished with the help of the coding interpretation of table 1. Answers in fields 5 to 7 suggest that students familiarity with information technologies is high. Responders agree that the use of PC in laboratory teaching makes learning process pleasant and efficient (field 8). Questions 11 to 18 extract information relevant to the laboratory teaching materials. Answers to fields 11 to 14, support the good quality of the available laboratory notes. However in field 13, the score of a median value of 3, interpreted as no opinion, could be attributed to lack of experience in laboratory notes of similar context. Table 1 Questionnaire fields, Statistics and coding. Field Field Description Coding median mean SD 1 Semester 0=No answer 7 2 Sex 1=Male, 2=Female 1 3 High school type 1=General,2=Technological,3=Night, 5=Other 1 4 High school graduate degree Scale 0-20 16,05 15,84 1,30 5 PC at home 1=Yes, 2=No 1 1,00 0,00 6 Frequency of PC use 1=Daily, 2=Often, 3=Rare, 4=Never 1 1,36 0,63 7Α Use of PC for Study 1 0,93 0,27 7Β Internet 1 0,79 0,43 7C Entertainment 1 0,71 0,47 7D Communicate 1 0,57 0,51 7Ε Professional 0 0,07 0,27 8Α The use of PC could make lecturing: Pleasant 1=YES, 0=NO 1 0,57 0,51 8Β Efficient 1 1,00 0,00 8C Faster 0 0,36 0,50 9 Number of times participating in the laboratory 1,2,3,4=More than 3 1 1,86 1,41 10 Successfully pass the theory lectures 1=YES, 2=NO, 0=No answer 2 1,71 0,61 11 The laboratory covers the theoretical background 1=Strongly disagree 2= Disagree, 3=No opinion, 4=Agree, 5=Totally agree, 0=No answer 4 4,00 0,55 12 Laboratory theory is comprehensive 4 3,50 1,02 13 Manuals are well written 3 3,29 1,27 14 Laboratory experiments are suitable for the purpose of the lab 4 3,86 0,77 15 Presentation quality and motivation for students 4 3,79 1,25 16 Exercise steps are comprehensive 3,5 3,29 0,99 17 Motivation for future engagement 3,5 3,50 0,76 18 Gradual degree of lectures difficulty 4 3,79 0,97 19 The laboratory simulates well radiology departments 4 3,79 0,80 20 Software is easy to learn 4 4,21 0,58 21 Software motivates student 4 3,71 0,99 22 Software is appropriate for the purpose of the lectures 4 4,00 1,11 23 Software engages students into active participation 4 3,50 0,76 24 Laboratory structure helps students learn from their mistakes 4 4,00 1,18 25 Laboratory lectures enrich the theoretical knowledge 1=Not at all, 2=a little, 3=sufficiently, 4=a lot 3 3,07 0,62 26 Overall ranking of the laboratory scale 0-10 8 7,64 0,63 27 Comments 0=Very negative, 1=negative, 2=slightly positive, 3=very positive, 4=no comment 4 3,21 1,31 Questions 15 to 17, examine whether students are motivated by the laboratory process for future engagement with the lectured materials. Answers are showing that students appreciate the laboratory learning environment as challenging and motivating. Responders confirm (fields 16 & 18), that the exercises are comprehensively written and the degree of difficulty is gradually increasing. 5 Median Mean 4 3 2 1 0 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 1 12 13 14 15 16 17 18 19 Fig. 1 Questionnaire answers statistics for fields 11-25. Blue lines: median value. Purple: mean value. 20 21 2 23 24 25

Current Developments in Technology-Assisted Education (2006) 1257 Software evaluation is performed in fields 19 to 23. Although the software used is commercially available for medical DICOM-PACS applications and it is not designed for teaching purposes, results suggest that students find it easy, stimulating and suitable for the purpose of the laboratory. Furthermore responders believe that the simulated laboratory mimics efficiently the radiology setting. Questions 21 to 24, evaluate the appropriateness of the laboratory tools in a student-centered learning environment. Student centered learning enables students to learn in their own pace and from their mistakes [11]. Median and mean values of answers in fields 21 to 24, suggest that a student-centered environment was successfully implemented. Negative students comments were concentrated on the number of available computers for the purpose of the laboratory. They suggested a one to one analogy, so each student could experiment and complete exercises in his/her own pace. Overall the laboratory structure was highly ranked (field 26). 7. Conclusions The pilot evaluation was carried out targeting to improve the educational process of the undergraduate students in medical DICOM-PACS networks. The evaluation was primary concerned with the quality of the training materials, the suitability of the chosen software for the training purposes and the overall satisfaction of students from the laboratory application. Questionnaire statistical analysis results suggest that a student centered environment was successfully implemented. Students were satisfied by the quality of the hard copy materials as well by the software application tools chosen for the simulation of a DICOM-PACS laboratory. Student comments suggests the increase of the number of PCs and software licenses available in the laboratory, so each student carries the exercise individually. Since the student sample was relatively small and choice of participants was voluntary, further evaluation is planned, with an increased student sample which will include all laboratory participants. Acknowledgements Financial support for this work and its dissemination efforts was provided by the project Upgrading of Undergraduate Curricula of Technological Educational Institution of Athens, (APPS program - Τ.Ε.Ι. of Athens), financed by the Greek Ministry of Education and the European Union (Greek Operational Programme for Education and Initial Vocational Training - O.P. Education - action: 2.2.2. Reformation of Undergraduate Studies Programs ). The authors would like also to thank Mr. Nikolaos Kontodimopoulos for the help he provided in planning the evaluation process. References [1] Y.Kim and S.C.Horii (Eds.), Display and PACS (Handbook of Medical Imaging, vol.3), SPIE Press, [2000]. [2] K.J.Dreyer, A.Metha, J.H.Thrall, PACS: A Guide to the Digital Revolution, Springer-Verlag N.York, Inc, ISBN 0-387- 95291-8, Chapt. 5, pp 73-119, [2002]. [3] American College of Radiology National Electrical Manufacturers Association, Digital Imaging & Communications in Medicine (DICOM), Part 1: Introduction & Overview, PS 3.1, National Electrical Manufacturers Association, [2003]. [4] R.B.Gunderman, K.B.Williamson, M. Frank, D.E. Heitkamp, H.D. Kipfer, Learner centered Education, Radiology, Vol.227, pp. 15-17 [2003]. [5] A. Tzavaras, N. Kontodimopoulos, E. Monoyiou, I. Kalatzis, N. Piliouras, I. Trapezanidis, D. Cavouras, E. Ventouras, Upgrading Undergraduate Biomedical Engineering Laboratory Training, proc. IEEE-EMBS 05, Shanghai, China [2005]. [6] A.Tzavaras, M.Athanasiou, A.Hatjiiokakimidis, P.Diamandi, E.Ventouras, Implementation of a simulated Radiology DICOM PACS network, proc. WSEAS conference on Engineering Education,Venice, Italy [2004]. [7] M.Y.Y.Law, Z.Zhou, New direction in PACS education and training, Comput. Med. Imaging and Graphics, vol 27, pp 147-156, [2004]. [8] University of south Carolina, http://www.ipilab.org/clinical%20serives/pacssim.htm [9] Hong Kong Polythechnic University, http://www.polyu.edu.hk [10] R.Likert, A technique for the measurement of Attitudes, N.York: Archives of Physiology, [1932]. [11] J. Cook and L. Cook, How technology enhances the quality of student-centered learning, Qual. Prog., vol. 31, pp. 59-63, July [1998].