GE Healthcare Intelligent Tools For A Productive Radiologist Workflow: How Machine Learning Enriches Hanging Protocols Authors: Tianyi Wang Information Scientist Machine Learning Lab Software Science & Analytics GE Global Research wangt@ge.com Alexandre Iankoulski Lead Software Engineer Business Integration Technology Lab Software Science & Analytics GE Global Research iankouls@ge.com For further information contact: Barbara Mullarky Sr. Marketing Manager Global Radiology GE Healthcare Barbara.Mullarky@ge.com imagination at work
INTRODUCTION Smart Reading Protocols (SRP) is a built-in feature of GE Healthcare s web-based radiology product Centricity TM PACS with Universal Viewer 1. SRP employs machine learning techniques to provide an innovative and simplified process for creating and applying hanging protocols in radiology workflow. SRP is designed to help to improve image setup efficiency compared to the traditional hanging protocol function in PACS software. This document provides an introduction to the SRP feature in Universal Viewer, explains briefly the SRP engine mechanism, and summarizes the key advantages of SRP over traditional hanging protocols. BACKGROUND Healthcare information systems, such as Hospital Information Systems (HIS), Electronic Medical Record (EMR) systems, Radiology Information Systems (RIS), Cardio-Vascular Information Systems (CVIS), and Picture Archiving and Communication Systems (PACS) are the infrastructure used by today s healthcare organizations to help provide efficient patient care. A PACS system may store images from different modalities, such as Computed Tomography (CT), Magnetic Resonance (MR) and others, while a RIS, EMR, CVIS, or HIS may contain non-image information, such as physician reports, disease history, and other patient associated data. Radiologists use the PACS system to review images and provide a report with a diagnostic conclusion to the patient s clinician. Clinicians also use the PACS to review images. In order to make a conclusion on the reviewed case, the radiologist/clinician organizes relevant data on a set of monitors, in a certain order. This order depends on many parameters, such as the imaging modality of the exam under review, the requested procedure, the body part scanned, the existence of prior studies and previous reports, and others. Traditional Radiology Workflow Figure 1 shows a typical radiology workflow. The radiologist logs into the system, reviews his/her work list and selects a study to review. The radiologist reviews other information such as patient history, prescriptions, clinical notes, etc., that are related to the case. Once the images are presented, the radiologist, in most cases, adjusts the Hanging Protocols to enable him/her to perform the value added steps of reading and positioning the results. Figure 1: Image setup is the typical bottleneck in radiology workflow. 1 Universal Viewer is a component of Centricity PACS
Hanging Protocol (HP) is a historic name, referencing the time when radiologists had a systematic method of hanging imaging film TM on a light box. Most PACS systems provide the functionality of both predefined and user configurable HPs. In PACS systems, the goal of the HP feature is to open the imaging data in an initial setup that is optimal for the radiologists/ clinicians, depending on the type of case under review. However, due to the large number of potential parameters and the variability of these parameters in the input data, in some cases the pre-configured HP does not display the images as expected. Furthermore, the tools for HP configuration can be very complex. Typically, the actual configuration is done by the vendor s product specialists, support engineers or IT administrators based on guidance from the primary users the radiologists. The complexity of the HP configuration tools and the dependence on expert knowledge to render them operational makes it difficult for users to modify or improve HPs to their own preference. In a lean six sigma study performed by GE Healthcare in 2010 with users of Centricity PACS 2, it was observed that review tasks with an average task time of 11.4 minutes, 9% of the time was spent on modifying the image layout. The number one issue for radiologists was the setup of hanging protocols, creating the highest non-value added component of the workflow show in Figure 1. In this study, radiologists were asked which feature/tasks result in the most productivity loss during their workflow. They named the HP feature as the number one problem in their PACS system. Figure 2 shows the complete results of the radiologists responses. In the past, different methods were proposed to address the need of automating HP creation. In one study [1] a method related to capturing one or more high level characteristics for the image study, based on relationship of the images in the study, is proposed. The method also includes image classifying based on low level characteristics of the images. Combining low and high level characteristics, the machine learning engine in [1] classifies the study and determines an appropriate hanging protocol, based on this classification. In a second study [2] it is proposed to monitor and calculate the productivity factor of each HP based on efficiency of the user during a reading of the study. The system may then advise the user to switch to another hanging protocol, defined by another user, if its efficiency factor is larger. Both of these prior articles do not refer to the users preferences while suggesting a new hanging protocol. Another system [3] allows the user to edit the default hanging protocol, create and apply additional display rules, and tracks the number of times the user selects different display rules for different image modalities. Then, based on thresholds, it automatically decides whether the default hanging protocol should be modified after the user confirms these changes. In this system although the user is involved in the process of creating new display rules, he is not involved in the process of gathering the statistics and can t provide the system information on how the HP of his choice meets his preferences. These methods have not provided a viable HP solution commercially. Figure 2: Results of C-PACS survey: radiologists perception on productivity impact of most common tasks performed as part of their workflow. 2 DOC1309398 CPACS Radiology Productivity Results Summary FINAL GE Healthcare Intelligent Tools For A Productive Radiologist Workflow 3
INTRODUCTION TO SMART READING PROTOCOLS The Smart Reading Protocols (SRP) technology represents a breakthrough in the way radiologists work with HPs. SRP provides a new, simplified process for creating and applying HPs in the radiology workflow, which is designed to improve image setup efficiency compared to the traditional HP function in today s current PACS software. Instead of manual pre-configuration of layouts with a large number of parameters, the new approach to HP is to create a system that learns the users preferences through an explicit teach action as they work, such that when they open a new study/exam, the system will setup the images in a way that is preferred by the user: viewport layout, image series placement, automatic post-processing, etc. The whole process of creating and applying HP with SRP is as simple as an occasional single mouse click added to the users normal review process, as illustrated by Figure 3. Upon the initial installation of the system, SRP is not yet taught by any users. When the first user opens an exam for the first time, the system will apply a traditional HP by default. The default HP may have come bundled with the product, or may have been pre-defined by a PACS administrator or GE HCIT Application Specialist. In many cases the default setup is not satisfactory so the user has to set up the images manually on the monitors in use. With SRP, once the exam has been set up to his/her preference and just before the user proceeds with the review process, he/she can click the learn button to teach SRP that this is a desirable HP for the current scenario. Then the user can continue to finish the rest of the review process for this exam. Later on, when the user opens another exam of a similar type, SRP will try to make HP suggestions based on all previously learned HPs created by all users of the system. If at least one suggestion is made, the first suggested HP will be applied automatically to set up images for this exam; otherwise, a default traditional HP will be applied again as it was for the very first exam however falling back to default will become less and less likely with the continuous learning of SRP from multiple users. If the suggested HP is in accordance with the user s preference, he/she can proceed with the normal review of this exam; otherwise, he/she may choose to make corrections/adjustments to the suggested HP currently applied, quickly change to other SRPs suggested (if any), or choose to use one of the traditional HPs. After the setup has been corrected to the user s preference, he/she can teach SRP this preference, expecting SRP will be correct next time. The process of creating and applying HP with SRP enabled is illustrated in Figure 4. Figure 3: Radiology workflow with SRP enabled. 4 GE Healthcare Intelligent Tools For A Productive Radiologist Workflow
Figure 4: Process of creating and applying HP with SRP enabled NO SRP SUGGESTIONS a) Open an exam for review from the work list. b) A traditional HP is applied before SRP is taught. CLICK THE LEARN BUTTON c) Setup images to the user s preference manually. d) Teach SRP the current HP settings, and proceed with the rest of the review process. ALL AVAILABLE HPS e) Open another exam (of similar type) for review; a desirable HP is applied automatically by SRP. f) If necessary, the user may manually choose secondary SRP suggestions, if any, or a traditional HP. GE Healthcare Intelligent Tools For A Productive Radiologist Workflow 5
Behind the SRP workflow is a set of machine learning algorithms that track the way a user or a group of users set up images for reading in particular workflows, collect relevant context information, and reproduce the layouts for new studies of this type. The goal is that, even if the algorithm does not produce image setups exactly as the user expects, it accepts corrections from the user, adapts and converges to the preferred image setups after a few teaching sessions. Given the typically great variety in exam types, data, and HP preferences, the learning engine may still need to be taught when new information is available. However, just like voice recognition systems, the success rate of SRP suggestions can be expected to reach a reasonably high level when the number of learns becomes sufficient. Figure 5 illustrates the potential SRP success rate. Success Rate Number of Learns Figure 5: SRP success rate is expected to improve with the increase of number of learns. HOW DOES IT WORK Considering the overall workflow, SRP is analogous to many learningpowered systems in other domains, such as movie, news, online shopping recommendation systems, etc. In such systems, the user preferences will be collected either explicitly by means of a like it action, or implicitly by tracking the buying/reviewing history in the background. By learning from past data, the learning system will be able to make inferences or recommendations to the user in a future time. The working mechanism of SRP is similar, although the input/output data structures are quite different due to application domain differences. Like any learning system, as illustrated in Figure 6, the SRP engine consists of two processes (operation modes): learning and inference (or prediction). The learning process is triggered explicitly by the user by clicking the learn button, at which time the system creates a snapshot of the current HP setup, such as viewport layout, series placement, and viewer parameters (e.g. window level), together with various context information, including but not limited to user name, global viewer settings, monitor configurations, order information, modality, body part, procedure name, study description, series descriptions, various DICOM tags, prior/comparison studies, etc. This snapshot becomes one example of preferred HP, called a Learn Case. Such learn cases are supplied to the SRP engine to update its internal machine learning model. The inference process is triggered whenever the user opens a new exam, at which time the SRP engine will make HP suggestions based on context information collected from the new exam (and the corresponding prior studies) to be opened. This context information is called a probe, and is similar to what is collected for the learn cases, but is used to infer hanging protocols. The core SRP engine is a customized machine learning algorithm that incorporates multiple techniques such as case based reasoning, text mining, information fusion, etc. The SRP engine characterizes each Learn Case and probe by features deemed relevant to the HPs, which are extracted from the associated context info as part of the learning and inference processes. The features can be computed from numerical variables (such as number of monitors), categorical variables (such as body part) and/or free-form text (such as series description). The extracted features are treated as inputs by the SRP engine while HPs are treated as outputs. The SRP engine derives machine learning models from input-output pairs and then is able to make predictions/inferences to the expected HPs given the input (represented by the probe). The SRP engine makes multiple HP suggestions at a time, which are sorted by a relative confidence level produced by the model, so that the user can try other HPs if the first one is not completely satisfactory. Figure 6: Learning and inference process of the SRP engine. 6 GE Healthcare Intelligent Tools For A Productive Radiologist Workflow
KEY ADVANTAGES OF SRP OVER TRADITIONAL HP SRP is expected to help improve radiologist productivity compared to traditional HP and help reduce the administrative effort to maintain a functioning HP system from the following perspectives. Ease of use SRP helps to improve the traditional HP workflow by adding a single click to learn. It unifies the two procedures of HP defining and HP applying of traditional methods into one, so that the user/radiologist can create an HP to their preference without the need to use an additional HP wizard or the involvement of site administrators. Robustness and Accuracy SRP has advantages over the traditional HP in terms of accuracy, or success rate, under a clinical environment with large data variations, i.e. in large enterprise hospitals. Data variation (missing, erroneous, and inconsistent records) and variation between modality vendors is a primary cause of failures of the traditional HP function. This is frequently seen in DICOM headers attributed to free-form user inputs, changes made by different equipment vendors, etc. SRP is able to handle data variations in a more robust way due to the utilization of both advanced text mining techniques and information fusion methods in the decision engine. For example, SRP automatically overcomes different labeling of the same study type (produced by images from modalities from different vendors, data entry by different technicians), which is achieved by taking into account certain combination of other parameters. SRP automatically overcomes the problem of series sequence order variations for the same type of exam. In certain solutions today, the system simply hangs image series based on their order, based on the assumption that the technician operating the modality creates the series in a predictable and deterministic order. However, this logic breaks down as soon as modalities from different vendors are used or technicians change. Based on observations from evaluations conducted at customers sites between October 2012 and January 2013, in most instances SRP was able to learn the user s hanging preferences in 1-4 teachings. After the teachings, more than 70% of the cases are considered as Success, which is claimed when the first SRP suggestion completely meets the user s expectation (no further hanging adjustments were necessary), while all the other cases are claimed as Failure, including those cases when the first suggestion is partially satisfactory or the second suggestion was completely satisfactory. Figure 7 gives an example of success/failure sequence in one evaluation session and the smoothed success rate, or learning curve. After a few learns, SRP started to make Success, with only occasional failures. When the user continued to teach SRP whenever it made mistakes, SRP s success rate continued to improve. 1 Ease of maintenance Because SRP combines the two operations of defining and applying traditional HPs into one, there is no need for a site administrator to maintain a predefined HP library. The library is maintained by a computer algorithm automatically for each individual user or user group through simple settings. Human involvement is decreased as the need to manually define rules decreases. DEVELOPMENT AND CLEARANCE SRP was developed in collaboration between GE Healthcare IT and GE Global Research. Currently, SRP is offered commercially as a standard feature of GE Healthcare s Centricity PACS with Universal Viewer, which was commercially released at RSNA 2012. Figure 7: Based on testing performed by GE s product development teams using a methodology whereby radiologists evaluate SRP on a Universal Viewer test system in their hospital setting using their production data and workflow, SRP was observed to be able to learn the user s hanging preferences in 1-4 teachings. After that, it was observed that more than 70% of the cases were considered a Success, which is claimed when the first SRP suggestion completely meets the user s expectation and no further adjustments were necessary. 1 1. Centricity Universal Viewer Smart Reading Protocol Evaluation Findings (DOC1414524)
About GE Healthcare GE Healthcare provides transformational medical technologies and services to meet the demand for increased access, enhanced quality and more affordable healthcare around the world. GE (NYSE: GE) works on things that matter great people and technologies taking on tough challenges. From medical imaging, software & IT, patient monitoring and diagnostics to drug discovery, biopharmaceutical manufacturing technologies and performance improvement solutions, GE Healthcare helps medical professionals deliver great healthcare to their patients. GE Healthcare Level 6, 1 Sentral, Jalan Travers, Kuala Lumpur Sentral 50470 Kuala Lumpur Tel: +603 2273 9788 Fax: +603 2273 6503 www.gehealthcare.com REFERENCE Systems and Methods for Machine Learning Based Hanging Protocols. US disclosure publication 20100080427. (12_240_733) Method for Providing Adaptive Hanging Protocols for Image Reading. US disclosure publication 20080166070. (11_619_915) Content Based Hanging Protocols Facilitated by Rules Based System. US patent 7525554. Shai Dekel, Alexander Sherman, Sohan Rashmi Ranjan, Viswanath Avasarala, Xiaofeng Liu, Alexandre Nikolov Iankoulski, Tianyi Wang, Smart pacs workflow systems and methods driven by explicit learning from users, US patent publication. Publication number US20130129165 A1. Publication date May 23, 2013. imagination at work 2013 General Electric Company All rights reserved. General Electric Company reserves the right to make changes in specifications and features shown herein, or discontinue the product described at any time without notice or obligation. This does not constitute a representation or warranty or documentation regarding the product or service featured. Timing and availability remain at GE s discretion and are subject to change and applicable regulatory approvals. Contact your GE representative for the most current information. GE, the GE Monogram, Centricity and imagination at work are trademarks of General Electric Company. All other products names and logos are trademarks or registered trademarks of their respective companies. GE Healthcare, a division of General Electric Company. DOC1388817