Clinical Content Interoperability White Paper The Evolution of PACS Data Migration A Discussion of Current Migration Strategies Shannon Werb Chief Strategy Officer Chief Operating Officer
Contents Introduction... 3 Historical Perspectives Early PACS Design Limitations... 3 Description of Migration Strategies and Usages... 5 Standard DICOM Query Retrieve Migration... 6 Media Migration... 7 Current Study Migration Combined With Historical Study Migration... 8 Priority Migration... 9 Data Cleansing... 9 Summary and Conclusions... 11 Acuo Technologies 2 Evolution of PACS Data Migration
Introduction Starting around 2000, for various technological and feature/functionality reasons, users of PACS systems began replacing their original PACS. The replacement decision included the need to move study data from the old system to the new system. The term migration became the popular term to describe this task. Over time, methods and approaches have been developed to address the various challenges that can be encountered during a PACS data migration. This paper will describe these strategies and discuss their applicability in migration projects. Learning objectives include: Historical perspectives early PACS design limitations Description of migration strategies and their use Historical Perspectives Early PACS Design Limitations When PACS solutions were first designed, system architects either completely overlooked or marginalized the requirements for migrating studies from their systems. Initially, there was really no way for an outside source to effectively request studies from a PACS. Query/retrieve (Q/R) services arrived with the adoption of the DICOM standard but few systems completely implemented it. Consequently, getting data out of the PACS was challenging to say the least. Even if DICOM Q/R was implemented, the systems could not efficiently respond to queries in high volumes over short periods of time. PACS study export and study import use the same DICOM interface. Computer processing resources can be added to balance the import and export resource requirements but the ability for the PACS to simultaneously respond to high volume export requests while managing current study imports is limited. Many PACS have limits set on the number of C-Find/C-Move transactions that the system will handle at any given time. Once that threshold has been met, additional requests beyond the limit will enter a queue. Once the queue is full no other requests will be accepted. Many times the requesting source is unaware that the request cannot be retrieved. There are constraints for getting data out of the old PACS which limits how fast and how much data can be exported. It is not a very effective way to move hundreds of thousands of studies in a short period of time. Along with these architectural challenges, early PACS were often delivered as a turnkey solution, including software, hardware and services. The turnkey solution was typically sized for a normalized ingest volume with periodic, if not rare, export requirements. Some PACS are entirely designed for import without a provision for export as previously mentioned. Since the system was sized for a specific ingest capacity and the ability to balance access to resource across ingest and export did not exist or was not mature, the need to rapidly export data using Q/R services simply was not reliable. In short, vendors and customers alike did not size systems to export data at rates of 5, 10 or 50 times faster than the expected normalized ingest volume. Acuo Technologies 3 Evolution of PACS Data Migration
To this day, PACS systems are limited in the ability to import studies. It appears that PACS architects focused on the number of studies to be received per hour from the modalities as the design requirement. A busy radiology department might send up to 100 studies per hour to the PACS. In order to migrate hundreds of thousands of studies in a reasonable period of time, several hundred studies per hour need to be imported into the new system. Some PACS simply limit the number of DICOM associations that can be managed at one time after which no others will be accepted. The import bottleneck is related to the gateway functions that every PACS performs as new studies are received from the modalities. There are study verification steps, database entry steps and DICOM transactions among others that the gateway applications address. This simply takes time. Once again computer processing power can be assigned that can increase the number associations but it usually ends of being a migration limitation that needs to be addressed. Early efforts to use DICOM Q/R services to migrate studies quickly revealed these limitations and their impact on migration project duration. The effort involved to manually manage the Q/R requests and verify successful receipt of all of the study images was greater than many institutions had resources to apply. Even scripted batch Q/R requests only marginally improved migration rates. Early PACS architects simply depended on the modality technologist to manually enter the required patient demographic, accession numbers, medical record numbers and other study information at the modality console. HIS/RIS interface support was not an early requirement of PACS systems and was not implemented until many years after the installation of many PACS. The resulting data integrity of study information was suspect. Transposition errors alone could account for 80% or more of study information errors. Missing information and duplication of information was common place. And study data hygiene efforts to correct these problems were not always undertaken largely due to the lack of availability of easy to use system administration tools (another design oversight). The implication of this so called dirty data became known as the new PACS systems were unable to verify the study data being imported from the old PACS. Unverifiable studies are then placed in an exception queue or folder where they will stay until the invalid or missing data elements are corrected (so called data cleansing ). This is a time consuming step that requires a substantial institutional effort to undertake. Most PACS have limits to the number of studies that can occupy the exception folder, either intentional limits or unintentional oversights. Once that limit is reached the ability to receive new studies into the PACS can be limited or becomes unruly for the end user to manage. The implication on migration is that the study transfer process slows down substantially or completely when the dirty historical study data causes exception folders to fill up. Over time, migration methods and approaches have evolved to address some of the idiosyncrasies of the way PACS deal with importing and exporting study data, the quality of the study data, the order in which studies can be transferred and the final verification and reconciliation of the migrated data. This white paper explores that evolution. Acuo Technologies 4 Evolution of PACS Data Migration
Description of Migration Strategies and Usages The graphic below depicts four approaches to be considered for performing PACS data migrations: These strategies can be used separately or in combination, depending on migration project requirements. They have been developed and refined by companies like Acuo Technologies that specialize in PACS data migration services. These approaches are intended to achieve fast, accurate and cost-effective migration project completion. An individual description of each approach is provided in the sections that follow. Acuo Technologies 5 Evolution of PACS Data Migration
Standard DICOM Query Retrieve Migration: This is one of the more common approaches currently in use for performing PACS data migrations. The requirement is that the source PACS is a DICOM query retrieve service class provider. With this approach, a dedicated migration server receives the studies via DICOM C-Move from the source PACS and then transmits them across the network to the target or new PACS. Transfer rates and project duration are dependent upon many variables, including the number of studies to be migrated, the extent of removable media storage, the capacity of the source PACS to respond to queries, the institutional constraints on migration activities, target PACS import resources and the quality of source PACS data. HIGH-LEVEL PROCESS DESCRIPTION Migration of studies from source archive to new archive using DICOM interface Establish VPN connection Install migration server(s) Create study inventory = interrogate source system: DICOM study root, study level query by study date tag, or database Start DICOM study-level C-MOVE automated migration Send studies to migration server then target PACS Monitor progress; intervene as necessary Verify source vs. new: use study level image counts or a database extract to verify image count success Create final verification report Acuo Technologies 6 Evolution of PACS Data Migration
Media Migration: This is another PACS data migration approach that bypasses the DICOM interface and reads the study files directly from removable media or spinning discs. This approach does not necessarily require that the source PACS studies are DICOM Part 10 files but rather that the data migration vendor has or can provide an adapter to read any proprietary files from the source PACS. This also requires access to the source PACS database (and possibly the RIS database) in order to ascertain the latest version of the patient/study meta-data which will not be present in the files but needs to be applied to the images prior to receipt by the destination PACS. The data migration vendor may be required to perform some custom development if an adapter does not already exist. The data migration vendor should be able to perform standard DICOM Q/R migration while any development is occurring to assure the most rapid extraction possible. Once the adapter is available the migration should be fully transitioned to a media migration. Directly reading the media significantly increases that rate at which studies can be migrated off of the source PACS. HIGH-LEVEL PROCESS DESCRIPTION Directly read removable media or disc storage systems, send to migration server for cleansing, send to new PACS Requires source database access Reverse engineering implications May require custom engineering Bypassing DICOM interface accelerates migration rate Target system must be configured to accept the high input rates Acuo Technologies 7 Evolution of PACS Data Migration
Current Study Migration Combined With Historical Study Migration: One of the problems encountered with many migrations is that current studies are often being added to the source PACS while historical studies are being migrated off of the source PACS on a last in, first out basis starting at a defined point in time. This creates a situation commonly referred to as gaps. The gap studies are those that have been received by the source PACS since the migration of the historical studies started. Periodically, the gap studies need to be migrated to the target PACS which requires a temporary suspension of the historical study migration so that the gap studies can be moved to the target PACS. This is referred to as gap-filling. Gap-filling slows down the historical migration. In order to eliminate gap-filling, an intelligent router is installed and all modality study traffic is redirected to the router. The router in turn sends the new studies to the source PACS and the target PACS, applying any necessary data cleansing policies to the newly acquired data as if it would be passed through the normal migration process. HIGH-LEVEL PROCESS DESCRIPTION Combine DICOM migration with current study routing Install router and route current studies to source and target PACS install router Eliminates the effort for periodic gap fills Reduces total migration time Gets modality redirect out of the way early in the implementation process Potential to accelerate time to first clinical use Dedicated router more efficient than through PACS or modality dual send Acuo Technologies 8 Evolution of PACS Data Migration
Priority Migration: Historical study migration is basically the movement of studies off of the source PACS in reverse chronological order. The last studies in are the first studies to be migrated out (LIFO), starting at a defined point in time and working backward until the source PACS is empty. While this is a practical approach, it does not reflect the practice of radiology where patients do not show up in reverse chronological order. They arrive in random order. The migration implication being that the most relevant study may not yet be available on the target PACS, requiring some sort of intervention. The priority migration approach has been developed to mitigate this situation. The approach involves pre-fetching studies based on schedule messages for scheduled studies (HL7) and DICOM modality work list (DMWL) messages for emergent studies, in many cases priority migration can be enhanced through custom HIS/RIS interfaces based on customer requirements. The priority migration application processes the schedule or DMWL requests and prioritizes the study migration. The entire study folder is migrated (or the folder for a specific date range), not just the relevant prior study, thus completing that study migration. Priority migration can easily be combined with standard migration. Priority migration transactions take priority over historical migration tasks. Radiologist confidence improves when there is reliable access to relevant prior studies. HIGH-LEVEL PROCESS DESCRIPTION Receive HL7 schedule messages and/or DMWL feeds to prioritize the retrieval of scheduled and/or emerging studies from source PACS Prioritize the pre-fetching of scheduled and/or worklist studies Potential to move either the entire patient folder or the relevant prior study Throttle up LIFO migration tasks after pre-fetch migration tasks are complete Most relevant prior studies on the target archive Builds radiologist confidence; Can decrease total number of studies that need to be migrated prior to go-live Potential to accelerate time to first clinical use May require additional interface costs and effort; May require high level of clinical staff involvement Acuo Technologies 9 Evolution of PACS Data Migration
Data Cleansing: As previously mentioned, the integrity of the study information in the source PACS can be suspect, particularly for the years of service that the source PACS operated without the benefit of admission, discharge and transfer (ADT) interfaces. Data integrity problems can be addressed with automated data cleansing tools that can correct or repair identification problems. Missing and duplicate medical record numbers, patient name misspellings and other common data integrity problems can be repaired by comparing the gold standard or system of truth data source to the source PACS data and then creating rules to define the conditions that need to be met that would in turn trigger the automatic correction of the suspect data. Rules can also be created to prepend or append medical record numbers with identifiers intended to differentiate the study data. Missing and duplicate accession numbers and duplicate or missing study instance unique identifier (SUID) can be corrected using a combination of study and patient-level matching conditions. Additional tags can be added to drive PACS workflow based on customer requirements. Automated data cleansing must be carefully undertaken and the changes must be closely scrutinized and approved by the institution prior to permanent commitment to the target PACS. It must be noted that automated data cleansing is never 100% successful. Certain exceptions will still exist after cleansing efforts requiring manual intervention. The effort to address these exceptions will be far less than that required to address all of them. HIGH-LEVEL PROCESS DESCRIPTION Interrogate legacy PACS system: DICOM study root, study-level query by study date tag Create study inventory Obtain RIS database extract; compare RIS database extract to PACS inventory Analyze results and establish automated cleansing rules; Cleanse action examples: o Clear contents of tag values; Delete tags from data stream; Copy/move/replace contents of tag values o Create new tags using existing tag or imported data set; Append/prepend data to tags o Normalize non-dicom conforming data Reduces exception handling; Improves migration efficiency Automated rules matching reduces manual correction effort prior to sending to target PACS Acuo Technologies 10 Evolution of PACS Data Migration
Summary and Conclusions PACS data migration projects present significant challenges. This white paper has attempted to describe the various clinical data migration approaches created to address these challenges. These approaches have been developed over time largely through the efforts of a handful of PACS data migration service providers, including Acuo Technologies. These approaches are frequently used in combination to affect the orderly and timely migration of study data from the source PACS to the target PACS. When planning a PACS data migration project, careful consideration should be given to each of these approaches in order to design a plan that incorporates the methods and approaches that best meet the technical, clinical and business requirements of the institution. Acuo Technologies 11 Evolution of PACS Data Migration
About Acuo Acuo Technologies, now part of Lexmark s Perceptive Software (NYSE: LXK), was founded in 2000 with the objective of developing the first enterprise-wide collaborative Universal Clinical Platform (UCP) solution for medical image content, both DICOM and Non-DICOM. Today, Acuo supports implementations of UCP around the world, including locations in Africa, Australia, Europe, North America and South America. For more information on providing superior clinical content management and data migration while simplifying operations and reducing costs, visit www.acuotech.com. ACUO TECHNOLOGIES, POWERED BY ACUO, ACUO, and Universal Clinical Platform are trademarks of Acuo Technologies, LLC. Acuo Technologies, LLC 2012. All rights reserved. DICOM is the registered trademark of the National Electrical Manufacturers Association for its standards publications relating to digital communications of medical information. Acuo Technologies 12 Evolution of PACS Data Migration