Comprehensive Evaluation of Radiation Oncology Information Systems (ROIS)

Transcription

1 Comprehensive Evaluation of Radiation Oncology Information Systems (ROIS) Luis Fong, Ph.D. Michael Herman, Ph.D.

2 Several Layers of Complexity Consultation Patient Information Prescription Main Hospital Treatment R & V Simulation Treatment QA Treatment Plan Other Sites Different types of cancer Different treatment techniques Several technologies Multi- vs. single-vendor environments Different users: Physicians Physicists Therapists Dosimetrists IS Staff Administrative Staff Technological Innovations: EPID OBI CBCT Other IGRT Research Clinical activities Analysis: On-line Off-line Paper vs. Paperless Lots, Lots of Information

3 Several Layers of Complexity Consultation Patient Information Prescription Main Hospital Treatment R & V Treatment Different types of cancer Different treatment techniques Several technologies Multi- vs. single-vendor environments Guarantee High Standards of Patient Care QA Different users: Physicians Physicists Therapists Dosimetrists IS Staff Administrative Staff Simulation Treatment Plan Functionality, Integration, Connectivity, Robustness, High Performance and Stability Technological Innovations: EPID OBI CBCT Other IGRT Research Clinical activities Other Sites Paper vs. Paperless Analysis: On-line Off-line Lots, Lots of Information

4 Some History

5 Paper Chart Clinac 4 control console X-Otron PMH/OCI kv x-ray tube housed in the treatment Head H.E. Johns et al.(1959)

6 TomoTherapy Hi-Art ROIS Elekta Synergy Varian OBI MRI-CT PET-CT

7 High Quality Level of Patient Care Patient Treatment ROIS Technological Innovations Information Management The successful implementation of the ROIS depends greatly on a clear understanding of the site-specific processes and IS/IT infrastructure as well as the capacity of the ROIS to match the constraints and clinical practice of a particular institution 1,2. 1 Ammenwerth E, Graber S, Herrmann G, et al. Evaluation of health information systems-problems and challenges. International Journal of Medical Informatics 2003;71: Lorenzi N.M, and Riley R. T Organizational Aspects of Health Informatics: Managing Technological Change, New York, Spring- Verlag, 1995, pp

8 Goal To develop a comprehensive methodology to systematically assess the processes and information flow as well as the performance of radiation oncology information systems (ROIS). ROIS Clinic Specific Road Map Clinic Specific Practice and Infrastructure

9 Radiation Oncology Classification Model Radiation Oncology Practice Peopleware Clinical Process Team of Experts CP Software - Hardware ROIS IM TII Team of Experts representing the different areas of the radiation oncology: Physicians Physicists Therapists Users Dosimetrists Administrative Staff IS team Evaluation Categories: Clinical Processes (CP), Information Management (IM) Technological Innovations Integration (TII)

10 Analytical Methods

11 Analytical Methods Analytical Method I IS/IT Infrastructure Maps. The purpose of these maps is to provide a clear understanding of all IS/IT processes and connectivity between the different components and internal systems in the department. Analytical Method II Process Flow Maps. These maps include all the patient care tasks and activities as well as the management of the patient information. These maps will be the basis for the development of a survey.

12 Analytical Method I - IS/IT Infrastructure Map Template Equipment or System A Connectivity Processes Equipment or System B Evaluate and identify potential interoperability problems and lack of functionality and connectivity

13 Analytical Method II - Process Flow Map Template INPUT System A Process X User(s) OUTPUT System B Survey Generic: Tasks Activities Sub-Processes

14 Analytical Method II Survey Process X Survey Act 1 Act 2 Act N ROIS X Tool 1 Tool 2 Tool N Three Major Areas Clinical Processes (CP), Information Management (IM) Technological Innovations Integration (TII) I. Quantitative Measurement of System Performance* II.Qualitative Measurement by Recording users Experiential Feedback *where Performance measures the level of integration between the ROIS and the site-specific clinical practice, needs and infrastructure

15 Analytical Method I IS/IT Infrastructure Maps Equipment or System A Connectivity Processes Equipment or System B

16 IS/IT infrastructures Two Types of Infrastructures: Clinical Multi-vendor: EXCI-IMPAC 4DITC-IMPAC Test Single-vendor 4DITC-ARIA

17 EXCI-IMPAC CT-SIM SIM WS GE Advantage TM Locally Mayo PACS system MIDIA QREADS Multi-ACCESS /ViewStation Brain Lab TPS Eclipse Process Locally Eclipse/VARIS Vision VARIS db (Sybase) :\\varsrv04 VARiS Images \\mfad.mfroot.org\ rchdept\filedata RTP to IMPAC RT Plan (*.rtp format) :\\rdosrv01\accessrt\apps\impac\rtp\ IMPAC db (Pervasive) :\\rdosrv01\accessrt\impac\db IMPAC DCM module IMPAC WS (Sequencer) NPV6 NAMER RT Image :\\rdosrv01\accessrt\apps\impac\stagedcm\ Record & Verification RT Image (per machine) :\\rdosrv01\accessrt\apps\impac\pv* LINAC Console DataTab MLC WS PortalVision WS IMPAC image :\\mfad.mfroot.org\rchdept\impac LINAC EPID MLC Controller IAS

18 4DITC-IMPAC CT-SIM SIM WS GE Advantage TM Locally Mayo PACS system MIDIA QREADS Multi-ACCESS /ViewStation Brain Lab TPS Eclipse Process Locally Eclipse/VARIS Vision VARIS db (Sybase) :\\varsrv04 VARiS Images \\mfad.mfroot.org\ rchdept\filedata RTP to IMPAC NAMER RT Plan (*.rtp format) :\\rdosrv01\accessrt\apps\impac\rtp\ IMPAC db (Pervasive) :\\rdosrv01\accessrt\impac\db RT Image :\\rdosrv01\accessrt\apps\impac\stagedcm\ MLC Controller IMPAC DCM module LVI Record IMPAC WS (Sequencer) VARIAN network switch LINAC Console DICOM Supervisor Verification 4DITC WS IAS CBCT WS OBI WS IAS IMPAC images :\\mfad.mfroot.org\rchdept\impac LINAC EPID OBI

19 4DITC-ARIA CT-SIM SIM WS GE Advantage TM Locally Mayo PACS system ARIA Brain Lab TPS MIDIA QREADS Locally ARIA Images ARIA db (Sybase) :\\varsrv05 Record Daemons VARIAN network switch Verification MLC Controller LVI LINAC Console Supervisor CBCT WS OBI WS IAS 4DITC WS IAS LINAC EPID OBI

20 IS/IT Infrastructure Maps EXCI-IMPAC CT SIM SIM WS GE Advantage Local Symbols Description: Mayo PACS system BrainSCAN Eclipse/VARIS Vision Server (Database) Local Storage Information System Workstation (WS) Connectivity Service RTP to IMPAC NAMER Multi-ACCESS RT Plan-Temporary from TPSs IMPAC db (Pervasive) RT Images-Temporary from TPSs Eclipse Process IMPAC DCM module Local VARIAN db (Sybase) Record & Verification IMPAC WS (Sequencer) NPV6 VARIAN Images (DFS) DFS (Distributed File System) Imager RT Images-Temporary per machine IMPAC images (DFS) LINAC Console DataTab LINAC MV MLC WS MLC Controller PortalVision WS IAS 4DITC-ARIA 4DITC-IMPAC CT SIM SIM WS GE Advantage Local CT SIM SIM WS GE Advantage Local Mayo PACS system Mayo PACS system VARIAN Images (DFS) MLC Controller LVI ARIA VARIAN db (Sybase) DICOM Daemons VARIAN network switch LINAC Console Supervisor LINAC MV kv CBCT WS BrainSCAN OBI WS IAS Local Record & Verification 4DITC WS IAS RTP to IMPAC NAMER Multi-ACCESS RT Plan-Temporary from TPSs IMPAC db (Pervasive) RT Images-Temporary from TPSs IMPAC images (DFS) MLC Controller BrainSCAN Eclipse Process IMPAC DCM module LVI Local Record IMPAC WS (Sequencer) VARIAN network switch LINAC Console Eclipse/VARIS Vision VARIAN db (Sybase) DICOM Supervisor LINAC MV kv Verification 4DITC WS IAS CBCT WS VARIAN Images (DFS) OBI WS IAS

21 Analytical Method II Process Flow Maps INPUT System A Process X User(s) OUTPUT System B

22 Clinical Processes (CP) Information Management (IM) Technological Innovations Integration (TII)

23 Site-Specific Clinical Processes Physician Consult Desk Notification Treatment Scheduling Simulation Treatment Planning Reports Mayo EMR Mayo GPAS Plan Second Check QA (IMRT) Rad Onc ROIS Treatment Delivery IMRT-S Review of Images Weekly Programming Completion of Treatment Weekly Management Visit Weekly Chart Check

24 Physician Consult Mayo EMR Physician Desk Notification Front Desk Mayo GPAS Treatment Scheduling Tx Scheduler ROIS Simulation Physician Therapist ROIS

25 Treatment Planning Physician Dosimetrist ROIS IMRT-S Report Report Plan Second Check Physicist ROIS IMRT-S IMRT QA Physicist ROIS IMRT Report

26 Daily IMRT Report Report Treatment Delivery Review Images Therapist ROIS Physician ROIS Weekly Programming Therapist ROIS Weekly Weekly Chart Check Physicist ROIS Mayo GPAS Weekly Management Visit Physician ROIS

27 Completion of Treatment IMRT Report Report Storage IMRT Report Report Therapist Front Desk Tx Scheduler ROIS Dose Summary Mayo EMR

28 Clinical Process Flow Map Patient information and process management systems: (A) Mayo EMR, (B) Physician Internal Electronic Ordering form, (C) Mayo Schedule Manager (D) IMRT QA Manager (ROIS)

29 Example of Process Flow Map and corresponding survey Simulation Physician Therapist ROIS Information System Evaluation Survey System: Process: ROIS X Simulation Please evaluate the performance of the system for: Adding patient ID photo Adding field specific set-up photos Adding patient set-up information Creating Reference Fields Sending reference images from GE Workstation into the system Sending images/field information from Acuity into the system Viewing/Accessing information from previous treatment courses Viewing/Accessing prescription and related notes Adding specific task lists to patient information Billing appropriate items performed Score

30 Clinical Processes (CP) Information Management (IM) Technological Innovations Integration (TII)

31 Information Management: I. Database management II. Long-term archiving solution III. Application availability IV. Customer support V. System integration VI. Data Conversion Information Management Survey

32 Clinical Processes (CP) Information Management (IM) Technological Innovations Integration (TII)

33 Technological Innovations Integration I. New imaging modalities: II. I. Cone Beam CT (3D matching), II. EPID-MV (2D matching), III. OBI-kV, kv MV (2D matching) Portal Dosimetry. Implementation Connectivity with ROIS Operability with ROIS Ability to keep updating with ROIS Ability to perform off-line review for research purposes Technological Innovations Integration Survey

34 Physicians Physicists Therapists Users Dosimetrists Administrative Staff IS team Clinical Process Survey Information Management Survey Technological Innovations Integration Survey ROIS X

35 Performance-Importance of Patient Care Space Importance for Patient Care Weighting Factor High Medium Low Poor Adequate Excellent High. The performance of this activity is essential to guarantee an accurate and safe patient treatment. It has a high effect on dose delivery as well as in the information and process flow. Medium. The performance of this activity helps in the process and information flow, and has medium dosimetric effect on the patient treatment. Low. The performance of this activity has very small effect on process and information flow and does not have any dosimetric effect at all on the patient treatment Excellent Performance Scoring Scale: Above Adequate Adequate Below Adequate Poor if Not Available Excellent. The system performs the activity and has the capability of improving the efficiency and efficacy of our department processes flow. Above Adequate. The system performs the activity with some extra features that make the activity flow smoothly. Adequate. The system performs the activity adequately Below Adequate. The system performs the activity with some minor difficulties that could be overcome. Poor. The system performs the activity in a way that could not be implemented in our department. Not Available. The system is unable to perform this activity. Importance for Patient Care Rejection Region (I) Rejection Region (II) Performance Magnitude Performance Acceptance Region (I) Acceptance Region (II) High Medium Low Patient Care Factor Direction

36 Performance Sorted by Patient Care Priority Region Evaluate the performance of the system for: PI IPC RR-I Review detailed treatment record RR-I Reviewing changes in plan per field AR-I Reviewing dose site summary AR-I Reviewing overrides AR-I Checking the chart since the last time checked RR-I If changes were made, tracking the history of the change RR-I If overrides were made, tracking the history of the override RR-I Reviewing action points RR-I Selecting BID patients AR-I Selecting patients per machine per site AR-I Signing off electronically that chart check was done Poor Below Adequate Adequate Above Adequate Excellent Low Medium High Performance Importance for Patient Care

37 Clinic Specific Performance vs. Patient Care Poor Adequate Excellent High PATIENT CARE Medium Low PERFORMANCE

38 Clinical Process Survey Information Management Survey Patient Care Site-Specific Master Priority List in terms of Importance for Patient Care Technological Innovations Integration Survey Focus on High Importance for Patient Care and Low Performance

39 Clinical Process Survey Information Management Survey Patient Care Lens 2 Lens 3 Alternative Priority Coefficient Performace Priority n Site-Specific Master Priority List in terms of n-priorities Technological Innovations Integration Survey Lens n

40 Quantitative Results ROIS A ROIS B Poor Adequate Excellent Poor Adequate Excellent RR-I AR-I High RR-I AR-I High Importance of Patient Care Medium Importance of Patient Care Medium RR-II AR-II Low RR-II AR-II Low Performance Performance In our practice, both evaluated systems showed, in average, above adequate performance

41 Qualitative Results Recommendation Clear definition of roles and responsibilities of vendor and user related to the management, service and support of ROIS need to be developed prior its deployment. Successful, effective and safe approach to ROIS issues shall be conducted by a team formed by: physics, IS staff and vendor service support. Effective channels of communication among users and between user and the vendor need to be developed in relation with ROIS issues, configuration and upgrades. Appropriate amount of resources need to be allocated toward the development of a strong IS team. The main role of the IS team will be to provide support for the everyday maintenance and basic trouble shooting of the non-clinical relevant hardware and software (e.g. virus configuration, operating system patches, workstations, etc.) Proficient knowledge and understanding of database management, DICOM RT and the components, connectivity and infrastructure of any clinically relevant information systems linked to the ROIS is needed by physicist and IS team Development of efficient and effective customer services program as well as training resources It is highly desirable to have the vendors provide a virtual environment (beyond a canned demo) ready for the users which will allow them to test the tools provided by the ROIS, simulating their current practice. Responsible Clinical Practice & Vendor Clinical Practice & Vendor Clinical Practice & Vendor Clinical Practice Clinical Practice Vendor Vendor

42 Who could benefit from this Evaluation Method? ROIS Vendors Developers Administrators Communication Channels & Taxonomy Specific Clinical Practice Physicians Physicist Administrators All Users Prioritize Develop better Solutions and Tools Ultimately Improve Patient Care Prioritize Develop better Processes

43 Conclusions This work provides for the first time an objective tool to understand and analyze clinical information flow and its compatibility with a given ROIS in a clinic specific patient care setting. Mapping provides insight into process communication, interoperability and efficiency. The hierarchy and importance of various characteristics are customizable to a given clinical practice and thus allow the tool s broad applicability. Proper assessment of information flow and matching to an ROIS will provide a more efficient and more effective care delivery setting

44 Acknowledgements Lori Buchholtz James Sorenson Janelle Miller Alan Kraling Jill Rendler Tim Peters Jessica Schlichting Michael Herman, Ph.D. Jon Kruse, Ph.D. Terry Harms Bob Dahl Janelle Molloy, Ph.D. Jack T. Cusma, Ph.D. Chris Hanna, Ph.D. Rob Mitchell Matt Adler Maureen Thompson Jeremy Cohen Richard Nusspickel David Ackerson Scott Johnson Sean Lapinski Chuck Pearce Kenneth Dick Cathy Tenda Diana Jones Project supported in part by Varian Medical Systems

45 Thanks! Ready for some questions?