pophealth - Open source tool for ecqm calculation and dashboarding OSHERA Meeting September 4, 2014
Presenters Eric Whitley: Data Warehouse Manager, Northwestern University John Rancourt: Product Manager for pophealth, ONC s Office of Care Transformation
Background Prior to 2011, two CMS programs required the submission of CQMs PQRS and IQR. CQMs were calculated using claims data and manual chart reviews. Overall it was a very manual process. Meaningful Use Stage 1 required hospitals and providers to generate CQMs from their certified EHR technology (CEHRT), and then manually submit the information during the attestation process. The process was more automated, but less accurate, and still fairly manual.
Background In 2011, ONC sponsored the development of pophealth, an Open Source Reference Implementation for Meaningful Use CQMs, which included a web-based user interface. ONC contracted with the MITRE Corporation to perform the development and management of pophealth.
Background In 2014, CMS introduced electronic CQMs (ecqms), meant to make the CQM reporting process automated end-to-end. Providers and hospitals may use ecqms for PQRS and IQR reporting and are required to use them for Meaningful Use reporting. For Meaningful Use reporting, providers and hospitals must use CEHRT to capture, calculate, and export CQMs.
Certification Overview
pophealth Components Data importer GUI Fraction Calculator (Quality Measure Engine) Data exporter 9/8/2014 Office of the National Coordinator for Health Information Technology 7
pophealth Architecture 9/8/2014 Office of the National Coordinator for Health Information Technology 8
pophealth Screenshots 9/8/2014 Office of the National Coordinator for Health Information Technology 9
pophealth Current Activities ONC is transitioning the pophealth tool to the open source community. ONC has been working with the community to develop draft governance plans that would support the community as it moves forward. 9/8/2014 Office of the National Coordinator for Health Information Technology 10
pophealth Use Cases and Community Members Community Members Northwestern University MITRE/Veteran s Health Administration ehealth Connecticut Illinois Department of Health Wyoming Medicaid Agency Primary Use Case Meaningful Use CQMs Meaningful Use CQMs FQHC, PCMH, and PQRS quality measure reporting Quality measures for CDC healthy hearts grant and for FQHCs Accept CQMs from ambulatory providers to support Medicaid Meaningful Use program 9/8/2014 Office of the National Coordinator for Health Information Technology 11
ONC and CMS Future State - Takoma
pophealth Future The governance and development of pophealth will be transitioned from being a government-supported piece of software to one that is supported by the open source community. 9/8/2014 Office of the National Coordinator for Health Information Technology 13
ONC s Future Role in pophealth ONC supports and funds Cypress development and will work with the pophealth community to ensure continued alignment of the tools. ONC will promote and market pophealth as an option for ecqms for Meaningful Use and other CMS programs. ONC will engage with any potential governance entity that supports pophealth. ONC will continue to work to align CQMs across CMS programs, and support the alignment of the pophealth tool with future ecqms. 9/8/2014 Office of the National Coordinator for Health Information Technology 14
pophealth Transition For more information on the pophealth transition to the open source community, please visit http://projectpophealth.org/resources.html. For information on appropriately using the pophealth mark, see http://projectpophealth.org/documents/pop Health%20Transition%20memo%207-31- 14%20(FINAL).pdf. 9/8/2014 Office of the National Coordinator for Health Information Technology 15
Request pophealth will flourish as an open source piece of software. The speed of the tool s growth will increase as more entities join the community. Please consider using pophealth and please join our open source community of users. 9/8/2014 Office of the National Coordinator for Health Information Technology 16
Contact John Rancourt: john.rancourt@hhs.gov Kevin Larsen: kevin.larsen@hhs.gov 9/8/2014 Office of the National Coordinator for Health Information Technology 17
Certifying Clinical Quality Measures (CQM) 2014 Edition How to Succeed With Certification While Really Trying Eric Whitley: Data Warehouse Manager, Northwestern University
Setting Expectations This is a 45 minute presentation that is more appropriate for a week-long course We re going to fly through topics It s taken us two years of constant work to obtain a basic working understanding of the mechanics Don t be overwhelmed Please feel free to contact the presenters with follow-up questions
Certification is the Easy Part First, really think this through do you want to certify? This is a serious commitment It takes a lot of time and work It can also be expensive Certification doesn t deliver the results ~10% of the work - Certification & Tooling What we re talking about today ~90% of the work Process & Data
How do CQMs Fit Into MU? MU is about change. It s about using the EMR meaningfully. For the program to be successful, you must embrace change In process In data management Reframing the picture - put the patient at the core If you remember only one word for MU Stage 2, it should be interoperability. This is the linchpin for the entire program CQMs are the last mile that are only truly possible if you embrace a mature model for managing your clinical information assets and meet the other MU requirements All other MU Requirements CQMs
Why Do We Need This? I m doing just fine with abstraction. Really Define day to me. Take two clinicians and ask them the same CQM question tell me about your results In Stage 1, no two vendors could likely take the same data and give you the same result ecqms move us forward we need to measure the same thing the same way The same thing: A common information model & value sets The same way: Codified logic removes debate This is hard. But it s not hard because of emeasures it s hard because we refuse to change behaviors with EMRs This is version 1.0. Is it perfect? No. But it s far better than where we were.
A Quick Terminology Review ATL Accredited Testing Laboratory CEHRT - Certified EHR Technology QDM Quality Data Model emeasure Electronic measure HQMF Health Quality Measure Format QRDA Quality Reporting Document Architecture QRDA I: patient-level data QRDA III: aggregate data (for a measure) CMS Centers for Medicare & Medicaid Services CMS sets the requirements around customer possession and use of a CEHRT ONC The Office of the National Coordinator for Health IT ONC sets the policies and practices for vendor certification of a CEHRT
Differences Between 2011 and 2014 2011 2014 Prescribed Information model (QDM) Codified measure logic Formalized logical elements (EX: temporal relationships) Downloadable, strict value sets Mandated use of normalized vocabularies Downloadable test tool (Cypress) Reference patients Expected reference output Randomized test data per vendor, product, and version Online discussion community for test tool Transparent issue tracking (JIRA) Data formats for patient-level data Data formats for aggregate level data
How Do I Start? First, choose an ATL http://www.healthit.gov/policy-researchersimplementers/certification-bodies-testing-laboratories An ATL is authorized to administer the certification process They may (or may not) have resources that can help you (FAQs, testing tools, etc.) They must adhere to the ONC-mandated certification processes, so if you have questions, go to them but keep in mind they may not have the ability to be flexible Establish a relationship with them. They are your sounding board on what is / is not allowed if you need guidance An ATL isn t going to train you on the requirements, tools, or process that s your job
Model + Value Set + Logic = Measure A Measure is a set of logic + discrete values that answer a question It utilizes a shared information model (QDM) to represent key elements Patients with diagnosis of X who received drug Y within 20 minutes of end of procedure Z? Had DX? Had PX? DX List PX List Drug start PX End <= 20 (min)? Drug List Population
Common Information Model The NQF created the Quality Data Model (QDM) to represent widely understood (but not commonly defined) elements such as a diagnosis, drug application, drug order, etc. as well as common attributes (start time, end time, etc.) The QDM is an information model. It is descriptive, not prescriptive. In addition to the base QDM, additional clarification of concepts such as relative time (during, before, etc.) and other elements were created to help support common questions http://www.healthit.gov/quality-data-model
Coded Measure Logic Logic now coded in HQMF an XML format No more forced interpretation of logic from humanreadable text (HITSP specs) Can be transformed into a variety of tools (using XSLT or another mechanism) No more problems with mismatched open/close parentheses from Stage 1 (HITSP pseudo-code) No more debating AND, OR, <, before, during, day, etc. You can recode the measures they just have to come out with the same results
Value Sets To ask a question across systems requires you be able to refer to the same data the same way They are lists of drugs, procedures, etc. that act as groupers for measures They use standardized vocabularies (LOINC, RxNorm, SNOMED-CT, etc.) to ensure consistency across tools
What is QRDA? It s a child of CCDA Consolidated Continuity of Care Document It s an HL7 standard QRDA I patient-level Like a CCD, but far more granular and detailed Adheres to measure-specific data requirements QRDA III aggregate (replacement for PQRI XML) Can contain measure-specific counts and aggregates for IPP, DENOM, etc. Also contains performance results Each stratified measure must be represented NOTE: CMS has taken the core QRDA standard and extended it slightly for CEHRT
What is QRDA? We mentioned QRDA is a CCDA document It s very verbose and very complex One patient will usually (easily) have thousands if not tens of thousands of lines of XML Anticipate spending a lot of time on QRDA I The format itself the variations of templates, etc. QRDA is non-trivial. The very, very specific atomic data required It relies heavily on HL7 identifiers
Different Kinds of Measures Patient vs. Episode Patient-Based Is the patient in a population? Most EP measures are patientbased (56 of 64) Episode-of-Care Often centered around encounters or visits, but not limited to just that concept Most EH measures are episode-of-care 8 of 64 measures for EPs are episode-of-care Proportion vs. Continuous Variable Proportion % if patients or events in population Continuous Variable Measures a specific artifact (EX: median time for X) Can be based on patient or episode-of-care
Several Kinds of Populations We re trying to identify populations and possibly metrics around events http://blog.agilehealthservices.com/2013/05/an-explanation-of-meaningful-use-stage.html
Populations can be Stratified CMS172v3 - Prophylactic Antibiotic Selection for Surgical Patients CABG procedures Other cardiac surgery Hip arthroplasty Knee arthroplasty Colon surgery Abdominal hysterectomy Vaginal hysterectomy Vascular surgery CMS137v2 - Initiation and Engagement of Alcohol and Other Drug Dependence Treatment Treatment Treatment w/ AOD Treatment, RS1: 13-17 Treatment, RS2: >=18 Treatment w/ AOD, RS1: 13-17 Treatment w AOD, RS2: >=18 Each of these is effectively treated as a distinct measure within a measure - distinct logic and value sets
Each Measure Needs 3 Certifications 170.314.c.1 Capture patient data - represent it in the QDM Export in QRDA I (patient-level) This is more of an EMR function 170.314.c.2 Import patient data into calculation tool Calculate (correctly) 170.314.c.3 Export in QRDA III (aggregate)
Measures Have Versions A single measure can have multiple versions This includes Logic Value sets You certify to the specific version of a given measure and value set Measure tend to be versioned annually June 2013 CMS104v2 - Discharged on Antithrombotic Therapy June 2014 CMS104v3 - Discharged on Antithrombotic Therapy
Each Domain & Measure Needs to be Certified Domain: Eligible Hospitals (aka inpatient ) Product 1 Our Fantastic EH Tool v1.0 CMS104v2 - Discharged on Antithrombotic Therapy CMS107v2 - Stroke Education CMS91v3 - Thrombolytic Therapy CMS102v2 - Assessed for Rehabilitation CMS113v2- Elective Delivery Domain: Eligible Providers (aka outpatient ) Product 2 Our Amazing EP Tool v1.0 CMS148v2- Hemoglobin A1c Test for Pediatric Patients CMS130v2- Colorectal Cancer Screening CMS131v2 - Diabetes: Eye Exam CMS125v2 - Breast Cancer Screening CMS134v2 - Diabetes: Urine Protein Screening
Certification Pricing Consult your ATL on pricing they should be following a standard pricing guide From our experience Each product has a base cost, regardless of the measures ($5,000) This fee covers one hour for the certification If you exceed the hour, there s a $1,000 / hour charge to continue If you need to add or adjust measures, you start again from scratch it s a new product Original Certification (10 CQMs) EP v1.0 $5,000 (10 minutes over) $1,000 EP v1.0 Total: $6,000 We want another measure EP v1.1 $5,000 EP v1.0 Total: $5,000
You Can Mix and Match You don t have to certify for all three requirements, but a customer must possess a CEHRT for all three if they wish to attest This does get expensive if you split products Measure CMS104v3 - Discharged on Antithrombotic Therapy Product My Fantastic EH EMR Version: 1.0 Domain: EH My Awesome EH CQM Analytics Tool Version: 1.0 Domain: EH Requirement 170.314.c.1 170.314.c.2 170.314.c.3 Base Cost $5,000 $5,000
The Certification Process The ONC has supplied a tool called Cypress Cypress is the open source reference testing platform that allows anyone to Generate test patient data specific to a measure Download that data in QRDA I & HTML Test structure, conformance, and results of QRDA I format & patient data QRDA III format & measure calculation results Accessing Cypress Your ATL may have a local installation of Cypress You can also download Cypress and install it yourself http://projectcypress.org
The Certification Process Get Started 170.314.c.1 1. Register as a vendor on Cypress 2. Create your product 1. Choose domain (EH or EP) 2. Set product version 3. Choose measures you wish to attempt to certify 1. Download the QRDA I / HTML patient data from Cypress 2. Enter / capture data in EMR 3. Export your generated QRDA I files 4. Load QRDA I files into Cypress and verify results / format 170.314.c.2 1. Import patient data into EMR (if not done in 170.314.c.1) 2. Calculate results 170.314.c.3 1. Export your generated QRDA III files 2. Upload QRDA III into Cypress and verify results / format
The Certification Process Test, test, test, and test You really do get one shot at this Remember for $5,000 you get one hour It s in your best interest to rehearse and be prepared Your ATL should supply you with test data and anticipated results in advance (up to 30 days prior to your exam) Make it easy for your ATL to certify you If you petition for any exemptions, provide clear, concise rationale / impacts and links to known CQM JIRA tickets (if appropriate). Let them know about these in advance. These must be petitioned. Provide them with a simple PowerPoint overview of your technology and solution Follow the ONC/ATL test guide to the letter (and then some review those JIRA items!)
What Do I Do About Issues? For certification issues, consult your ATL The ONC has fully embraced transparency All issue tracking is public If you questions or suggestions with measure logic, value sets, or aspects of the QDM, post to the JIRA http://jira.oncprojectracking.org/browse/cqm Find a Cypress issue? There s a tracker for that http://jira.oncprojectracking.org/browse/cypress All source code for Cypress is open https://github.com/projectcypress/projectcypress.github.com Find a bug? Patch it and issue a pull request on github Northwestern has posted 25 patches to Cypress components to date
What did Northwestern Do? Early on we recognized certification and tooling isn t the challenge the absence of reliable data is We recognized our technology (SQL) wasn t going to be appropriate for much of this Good for staging information Not good for complex derivations & logic (not set based) We came up with a two-pronged plan
What did Northwestern Do? 1. Focus our time on data and process Built a new EDW (data marts) from the ground up, focused on better data capture and normalized vocabularies Worked hard to build relationships with EMR teams across campus educate them on the data problem and work to fix processes and tools. Again change is hard and will be met with resistance. Don t give up. 2. Find any alternative to writing a solution on our own The problem is too big and the maintenance costs too high There s already a significant national investment we weren t going to benefit from reinventing the wheel Waste of resources when we could embrace standards and just use them Focus less on a measure and more on solving the measurement problem We needed to focus on a future with national interoperability at the core
What did Northwestern Do? Thankfully, we found something wonderful We found an entire team of amazing people from the ONC and Mitre Corporation had been working on a CQM tool called pophealth for several years and had already started adapting it to 2014 CEHRT requirements It was 99% done, so we helped nudge it over the finish line and certify it so that we and everyone in the country could benefit We chose to stand on the shoulders of giants THANK YOU, ONC and Mitre
Where Do I Go for More Information? ecqm Library, QRDA Documentation, and more http://www.cms.gov/regulations-and- Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html CEHRT 2014 Edition Test Method http://www.healthit.gov/policy-researchers-implementers/2014-edition-final-testmethod United States Health Information Knowledgebase http://ushik.ahrq.gov Value Set Authority Center https://vsac.nlm.nih.gov/ Example QRDA Category 1 XML for Meaningful Use Stage 2 Clinical Quality Measure Reporting http://blog.agilehealthservices.com/2013/04/example-qrda-category-1-xml-for.html Example QRDA Category 3 XML for Meaningful Use Stage 2 Clinical Quality Measure Reporting http://blog.agilehealthservices.com/2013/11/example-qrda-category-3-xml-for.html