How To Improve Health Information Technology

Transcription

1 ASSESSING THE IMPLEMENTATION OF HEALTH INFORMATION TECHNOLOGY IN FLORIDA HOSPITALS: PROCESS AND IMPACT By LORI A. BILELLO A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA

2 2012 Lori A. Bilello 2

3 To my family for all of their support and encouragement 3

4 ACKNOWLEDGMENTS I would like to express my gratitude to my Dissertation Committee Chair, Dr. R. Paul Duncan, who encouraged me to pursue my doctorate degree and his confidence and support all throughout this process. My sincere appreciation to my Cochair, Christopher Harle, for sharing his knowledge of Health Information Technology and providing invaluable advice on my topic, as well as guidance and patience through the dissertation process. I want to thank Jeffery Harman, a committee member, whose was instrumental in helping me with the study design, analysis and interpretation of the results. Thirdly, I want to thank Robert Cook, a committee member, who provided a unique prospective of the topic as a practicing physician. I would also like to thank the WellFlorida Council and Christopher Sullivan, formerly from the Agency for Health Care Administration, for providing me the opportunity to be a part of the Florida HIT Environmental Scan project which resulted in the development of an unique database which was used for my study for this dissertation - the Florida Hospital Health Information Technology Survey. Lastly, I would like to thank my family and friends for their belief in me and their support in my decision to return to graduate school. 4

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS... 4 LIST OF FIGURES... 9 LIST OF ABBREVIATIONS ABSTRACT CHAPTER 1 INTRODUCTION Role of Health Information Technology in Healthcare Quality Rationale Policy Environment and Emphasis Study Objectives Significance of Study BACKGROUND Government Policies, Legislation and Programs Meaningful Use (MU) Objectives Electronic Health Records (EHRs) Definition LITERATURE REVIEW HIT Adoption Levels Predictors of Hospital HIT Adoption HIT and Quality CPOE Studies CDSS Studies EHR Studies Limitations of Prior Studies CONCEPTUAL FRAMEWORK Donabedian s Quality Framework Delone and McLean Information System Success Model Conceptual Model Development Hypotheses DATA AND METHODS Data Sources

6 Florida Health Information Technology Environmental Scan CMS Hospital Compare AHCA Hospital Beds and Services List Florida Hospital Uniform Reporting System American Hospital Association Annual Survey Study Objective Study Objective Dependent Variable Independent Variables Study Objective Dependent Variables Independent variable Control Variables Statistical Analysis HIT Adoption Model Quality Model RESULTS Descriptive Statistics HIT Adoption Analysis Bivariate Statistics Multivariate Analysis Sensitivity Analysis Quality Measures Analysis Bivariate Statistics Multivariate Analysis Sensitivity Analysis Specified Models DISCUSSION AND CONCLUSIONS Summary and Interpretation of Results MU Objectives Met HIT Adoption Impact on Quality Limitations Policy Implications APPENDIX A FLORIDA HOSPITAL INFORMATION TECHNOLOGY SURVEY B SUPPLEMENTAL STATISTICAL DATA LIST OF REFERENCES BIOGRAPHICAL SKETCH

7 LIST OF TABLES Table page 2-1 CMS Stage 1 meaningful use objectives and measures for eligible hospitals HIMSS EMR Adoption Model CMS Stage 1 Meaningful Use Measures matched to Florida Hospital Survey Measures Description and categorization of variables Survey respondents and population characteristics Bivariate statistics: hospital characteristics by MUSum Poisson regression estimates for Total MU objectives met (MUSum) Poisson regression estimates for core MU objectives met (MUCore) Logistic regression estimates for binary MUSum Dependent variable characteristics Bivariate statistics: Hospital Compare measures and MUSum Summary of GLM regression estimates for Hospital Compare measures GLM regression estimates for Pneumonia quality measures GLM regression estimates for heart failure quality measures Summary of GLM regression estimates for Hospital Compare measures (expanded model) GLM regression estimates for Pneumonia quality measures (expanded model) GLM regression estimates for heart failure quality measures (expanded model) Logistic regression estimates for Binary CMS hospital quality measures CPOE and CDSS analysis with CMS hospital quality measures Specified model for PNIAS Specified model for HFLVSF

8 7-1 Florida hospitals meeting meaningful use Florida hospitals receiving CMS EHR incentive payments

9 LIST OF FIGURES Figure page 4-1 DeLone and McLean IS Success Model, EHR Use/Quality Model Total number of MU objectives met by Florida hospitals Percent of each MU objectives met by Florida hospitals Residual versus fitted plot after Poisson regression Percent of each MU Objective met by low and high HIT Adopters A-1 Q-Q plot residual-fitted GLM regression for PNIAT A-2 Residual versus fitted plot after GLM regression for PNIAT A-3 Q-Q plot residual-fitted GLM regression for PNIAS A-4 Residual versus fitted plot after GLM regression for PNIAS A-5 Q-Q plot residual-fitted GLM regression for PNPVS A-6 Residual versus fitted plot after GLM regression for PNPVS A-7 Q-Q plot residual-fitted GLM regression for PNIVS A-8 Residual versus fitted plot after GLM regression for PNIVS A-9 Q-Q plot residual-fitted GLM regression for PNBC A-10 Residual versus fitted plot after GLM regression for PNBC A-11 Q-Q plot residual-fitted GLM regression for PNSC A-12 Residual versus fitted plot after GLM regression for PNSC A-13 Q-Q plot residual-fitted GLM regression for HFLVSF A-14 Residual versus fitted plot after GLM regression for HFLVSF A-15 Q-Q plot residual-fitted GLM regression for HFAIARB A-16 Residual versus fitted plot after GLM regression for HFAIARB A-17 Q-Q plot residual-fitted GLM regression for HFDI

10 A-18 Residual versus fitted plot after GLM regression for HFDI A-19 Q-Q plot residual-fitted GLM regression for HFSC A-20 Residual versus fitted plot after GLM regression for HFSC

11 LIST OF ABBREVIATIONS AHA AHCA American Hospital Association Agency for Health Care Administration ARRA American Recovery and Reinvestment Act of 2009 CMS CDSS CPOE EHR HHS HIMSS HIT HITECH IOM IT MU ONC Centers for Medicare & Medicaid Services Computerized Physician/Provider Decision Support System Computerized Physician Order Entry Electronic Health Record Department of Health and Human Services Healthcare Information and Management Systems Society Health Information Technology Health Information Technology for Economic and Clinical Health Act Institute of Medicine Information Technology Meaningful Use Office of the National Coordinator for Health Information Technology 11

12 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy ASSESSING THE IMPLEMENTATION OF HEALTH INFORMATION TECHNOLOGY IN FLORIDA HOSPITALS: PROCESS AND IMPACT Chair: R. Paul Duncan Cochair: Christopher A. Harle Major: Health Services Research By Lori A. Bilello August 2012 The Health Information Technology for Economic and Clinical Health (HITECH) Act, as part of the American Recovery and Reinvestment Act, allocated over $19 billion for the U.S. health care system to adopt and meaningfully use health information technology (HIT). This legislation provided specific Meaningful Use objectives that need to be met in order to receive these funds. The primary objectives of this study are to assess acute care hospitals current use of electronic health records and whether meeting the federally mandated objectives is associated with better quality of care. Specifically, this study ascertains the level of attainment of Florida hospitals in meeting the CMS Stage I Meaningful Use objectives, identifies hospital organizational characteristics which are associated with higher levels of meaningful use among Florida hospitals (HIT Adoption Model), and assesses the association between achievement of the Meaningful Use objectives and improved patient process measures (Quality Model). This study was conducted using a retrospective, cross-sectional design linking the 2010 Florida Hospital HIT Survey 12

13 database with the CMS Hospital Compare quality measures for pneumonia and heart failure. The results of the HIT Adoption Model indicate several significant positive relationships between key hospital characteristics and meeting the CMS Meaningful Use objectives including urban location of hospitals, affiliation with a hospital system, and the presence of a Chief Medical Information Officer on staff. The Quality Model analysis provided mixed results with only three of the six pneumonia quality measures and one of four heart failure measures showing a significant positive, although weak association with meeting the CMS Meaningful Use objectives. This study addresses an important gap in the literature, especially regarding the CMS Meaningful Use objectives and their implementation as a path to standardize and measure the impact on patient care and adds to the evidence regarding the effect of using health information technology on health outcomes. 13

14 CHAPTER 1 INTRODUCTION Role of Health Information Technology in Healthcare Quality There is a common belief that widespread adoption of health information technology (HIT) has the potential to improve health care quality, reduce costs and increase the efficiency of the health care delivery system (IOM, 2001; ONCHIT, 2004; Hillestad et al, 2005; Blumenthal, 2010). The basis for this belief derives from logic and several narrowly defined studies or anecdotal evidence. While the potential benefits of implementing health information technologies such as electronic health records and computerized provider order entry systems are clear in theory, identifying and measuring their impact on health care has been challenging. In this study, the process of HIT adoption and the relationship between the level of HIT adoption and the quality of care delivered in Florida hospitals will be systematically and rigorously examined. Rationale The implementation of health information technology has become a major priority in the health care industry due to: (1) rising health care costs; (2) escalating concerns for patient safety and reducing medical errors; (3) focus on improving the provision of evidence-based care; and (4) the increasing number of regulatory requirements placed on health care providers (Doebbeling, Chow & Tierney, 2006). The National Coordinator of Health Information Technology, appointed by the Secretary of Health and Human Services, envisioned that widespread use of health information technology will result in fewer medical errors, fewer unnecessary treatments or wasteful care, fewer variations in care, and will ultimately improve care (ONCHIT, 2004). It is believed the use of HIT will result in the prevention of many medical errors 14

15 because all of the providers involved in a patient s care could receive timely clinical data, accessible and readable by all, and that automated order entry systems and decision support systems would be able to check for errors and provide evidence-based clinical guidelines to aid health care providers in decision making at the point of care (IOM, 2001). There is evidence that having electronic health records that are readily accessible can reduce treatment errors that result from gaps in knowledge regarding past medical history, allergies, or medications, especially when patients are being treated by multiple providers. Additionally, there is evidence that decision support tools can integrate electronic patient information directly into the provision of care and can reduce errors of omission that result from gaps in provider knowledge or the failure to synthesize and apply that knowledge in clinical practice (Shekelle & Goldzweig, 2009). Furthermore, a nationwide electronic health information infrastructure will allow providers real time access to health records across health care settings, reduce duplication of services and help coordinate care during transitions of care. Through the standardization of information and processes in the delivery of care, there will be less opportunity for errors and omissions that may lead to poor outcomes. Although the preceding assumptions regarding the successes to be found when HIT is readily adopted are commonly held, during the last decade there has been an increased focus on studying the actual impact of HIT on the delivery of health care. The studies that have been conducted have resulted in mixed findings of the benefits of HIT adoption on improving care and/or reducing costs. One of the challenges in conducting HIT research and comparing results with other studies is the wide variety of HIT 15

16 systems and definitions. Most of these studies have been performed at a single facility testing a particular function of their health information system. The specific nature of these studies, especially the type of health information systems deployed in these facilities and the specific functions tested, can reduce the generalizability of their findings. The potential benefits of HIT are likely to be realized not just by investing in specific systems but by effectively managing and integrating those systems into patient care delivery processes across the patient care continuum. Therefore, to study the impact of HIT on health care quality, it is important to identify not only the components and functionality of health information systems but also how they are used. To provide strong, general evidence of HIT value in terms of quality, studies that span multiple hospitals and carefully measure HIT functionality and usage at a detailed level are needed. To date, there have been very few studies conducted that link actual usage of HIT to hospital performance or quality. Policy Environment and Emphasis Despite the lack of consistent and convincing evidence of the benefits of HIT, current national health policy reflects the belief in its potential and has committed a sizable investment in the implementation of HIT. The American Recovery and Reinvestment Act of Health Information Technology for Economic and Clinical Health Act (HITECH Act) appropriated $19.2 billion to improve health care delivery and patient care through investment in health information technology over the next five years. These funds are targeted to increase the use of Electronic Health Records (EHRs) by physicians and hospitals across the country by reimbursing providers for a large portion of the initial costs of purchasing or upgrading their health information systems. By providing this sizable appropriation, government officials support the 16

17 expected benefits of widely using electronic health records and made the promotion of a nationwide, interoperable health information system a national priority. The HITECH Act attempts to standardize the measurement of HIT system use through the development of meaningful use objectives, which providers will be required to perform in order to participate in the EHR incentive program (ARRA, 2009). These objectives were selected as measures that demonstrate the provision of care meeting nationally recognized standards of care from the National Quality Forum, the Agency for Healthcare Research and Quality and other organizations. The legislation provides an incremental approach in implementing these standards of care in three stages to be released in the first 3 years of the EHR incentive program with Stage 1 having been released by the Centers for Medicare & Medicaid Services (CMS) in July This study uses survey data gathered from Florida hospitals that collected information on the type of health information systems hospitals have in place and how these systems are used relative to the CMS Stage 1 Meaningful Use (MU) objectives and in doing so, examines the relationship between HIT usage and health care quality in these hospitals. Study Objectives The primary objective of this study is to assess acute care hospitals current EHR use in terms of the CMS Stage 1 MU objectives and whether meeting these objectives is associated with better quality of care. The specific objectives are: 1. To ascertain the level of attainment of CMS Stage I MU objectives among Florida hospitals. 2. To identify hospital organizational characteristics associated with higher levels of meaningful use. 17

18 3. To assess the relationship between hospitals achieving the MU objectives and quality of care. Significance of Study Even though the potential of HIT for improving the health care delivering system is very compelling and has now been adopted as a national strategy to improve health care quality and increase the efficiency of health care delivery system, there are many challenges in trying to assess the impact of HIT on the quality of care. In particular, the many disparate systems currently available to providers and the variation of system functionalities make it difficult to compare or measure the impact of these systems. To address this issue, the focus in this study is on the achievement of standardized MU objectives and not on which system is being used. This focus allows a more rigorous assessment of HIT impact on the delivery of care. This study will be an important contribution to the literature since the MU objectives are a new federal requirement to participate in the CMS EHR incentive program and little information is available on hospitals current ability to meet these objectives and the impact these objectives will have on improving the quality of care. Evidence of whether the Stage 1 MU objectives are associated with the quality of hospital care will be important as policy makers continue to add or modify these objectives and the corresponding reporting requirements for Medicare and Medicaid providers. Furthermore, identifying the characteristics of hospitals that are able to meet the MU objectives in the early stages of the EHR incentive program will provide valuable information to support efforts in targeting those hospitals that need the most assistance in implementing EHRs. 18

19 CHAPTER 2 BACKGROUND Throughout the last decade, health care researchers and policy makers have promoted the use of health information technology, especially electronic health records, as a way to transform the delivery of health care (Chaudhry et al., 2006). This discussion came to the forefront as a way to improve health care after the release of the Institute of Medicine s (IOM) landmark report, To Err Is Human: Building a Safer Health System which estimated that at least 44,000 deaths in the United States are caused by clinical errors each year (IOM, 2000). Many of these deaths are the result of process errors, medication errors, or failure to provide the standard of care for a given medical condition (IOM, 2000). The Institute of Medicine released a follow-up report, Crossing the Quality Chasm: A New Health System for the 21st Century, which outlined several initiatives to prevent medical errors and improve the quality of health care in the United States. Key recommendations included the widespread use of health information technology and providing medical care based on the best available evidence (IOM, 2001). The IOM advocated for a nationwide commitment of all stakeholders to building an information infrastructure to support health care delivery, consumer health, quality measurement and improvement, public accountability, clinical and health services research, and clinical education (IOM, 2001). Since the release of this report, many of the recommendations have been endorsed by industry leaders, health care associations, health care policy makers, and some have been incorporated into legislation. 19

20 Government Policies, Legislation and Programs Partly in response to the 2001 IOM report, several government policies and programs have been developed during the last decade to promote the use of information technology in the health care industry. President George W. Bush issued an Executive Order in April, 2004 which directed the implementation of a nationwide interoperable health information technology infrastructure and widespread adoption of EHRs within ten (10) years and established the National Coordinator for Health Information Technology (Executive Order No , 2004). The Executive Order directed the National Coordinator to produce a report on the development and implementation of a strategic plan to guide the nationwide implementation of interoperable HIT in both the public and private sectors. This strategic plan, called The Decade of Health Information Technology: Delivering Consumer-centric and Information-rich Health Care was released in July 2004 and outlined four major goals, corresponding strategies and action steps in realizing the vision for improved health care through the widespread use of health information technology. The goals outlined in the plan included informing clinical practice through EHR adoption, electronically connecting clinicians to other clinicians, using information tools to personalize care delivery, and advancing surveillance and reporting for population health improvement (ONCHIT, 2004). The benefits of a consumer-centric and information-rich health care system include the reduction of medical errors, decrease in the variation in the quality of care, and increase in consumer access and knowledge of their medical information (ONCHIT, 2004). This plan was updated in 2008 and was the framework for much of the HIT related legislation that followed. 20

21 In 2009, the Health Information Technology for Economic and Clinical Health (HITECH) Act, as part of the American Recovery and Reinvestment Act, allocated over $19 billion for the U.S. health care system to adopt and meaningfully use HIT. This legislation firmly established the national agenda and timelines for the widespread adoption of HIT. The HITECH Act gives Department of Health and Human Services (HHS) the authority to establish programs to improve health care quality, safety, and efficiency through the use of electronic health records and health information exchange. This legislation provided funding for EHR incentive programs that pay eligible health care professionals and hospitals to purchase or upgrade certified EHRs (ARRA, 2009). Eligible professionals (physicians, dentists, certified nurse-midwives and nurse practitioners) who meet CMS-established EHR meaningful use criteria and patient volume thresholds may receive as much as $44,000 under the Medicare EHR incentive program or $63,750 under the Medicaid EHR incentive program. Eligible hospitals may access payments through both the Medicare and Medicaid EHR incentive programs. Through these programs, hospitals can receive millions of dollars for the meaningful use of federally-certified EHRs (CMS, 2010). Meaningful Use Objectives As provisioned by the HITECH Act, the HHS was directed to develop specific MU objectives that providers must report on in order to participate in the EHR incentive program (ARRA, 2009). The goal of the HITECH Act is not only adoption of EHR technology but also using this technology to achieve significant improvements in health care processes and outcomes (Blumenthal, 2010). HHS announced in July 2010 the first of three stages of objectives that hospitals must meet in order to qualify for 21

22 Medicare and Medicaid incentive payments for becoming meaningful users of EHRs (CMS, 2010). Stage 1 MU objectives include 14 core or required objectives and 10 menu objectives which hospitals must choose 5 to report on. These objectives are grouped by major policy priorities: 1) improve quality, safety, efficiency, and reducing health disparities; 2) engage patients and family in health care; 3) improve care coordination; 4) ensure adequate privacy and security protections for personal health information; 5) improve population and public health (CMS, 2010). Specific measures are outlined in Table 2-1 and address activities such as recording patient information, ordering and receiving test results, computerized order entry for medications and medication checks, decision support systems, and exchanging information with patients, other providers and health agencies. Most of these MU measures were derived from the National Quality Forum s (NQF) previously endorsed standards. These standards developed through a consensus process involving medical associations, purchasers, health care professionals, and others, have been identified and accepted by healthcare professionals as useful, achievable and relevant to improving health care quality and performance (NQF, 2011). Many of these measures are already in use by physicians and hospitals through Medicare s physician quality reporting initiative (PQRI) and hospital value based purchasing (HVBP) programs. Electronic Health Records (EHRs) Definition EHRs are complex health information systems that have evolved over time. Because of this evolutionary process, there are many ways to define what comprises an electronic health record, and these definitions oftentimes depend on who is using it and how it is used. It is important to note, EHRs are also often referred to as Electronic 22

23 Medical Records or EMRs. For the purpose of this study, any reference to EMRs in the literature will be reported as EMRs/EHRs to reduce confusion and maintain consistency in the discussion of these systems. The most commonly cited definition of an EHR, especially in the hospital field, is the Health Information Management Systems Society s (HIMSS) definition of an EHR as provided below: The Electronic Health Record (EHR) is a longitudinal electronic record of patient health information produced by encounters in one or more care settings. Included in this information are patient demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports. The EHR system automates and streamlines the clinician s workflow. The EHR has the ability to generate a complete record of a clinical patient encounter - as well as supporting other care-related activities directly or indirectly via interface - including evidence-based decision support, quality management, and outcomes reporting. (EHR consensus definition at HIMSS web page The adoption of information technology in the health care field has been a slow and costly process as compared to other industries. Hospital information systems were first initiated in the 1960s and became prevalent in U.S. hospitals in 1970s but were mostly confined to certain departments or functions in the hospital such as registration and billing, scheduling systems and laboratory information systems (Collen, 1995). Over the years, silos of health information in hospitals have evolved. Most hospital EHRs combine data from these disparate systems - administrative services such as registration and billing with large ancillary services, such as pharmacy, laboratory, and radiology, and other various clinical care components such as nursing plans, medication administration records, and physician orders (NIH, 2006). The number of integrated components and features of hospital EHRs is dependent on how information systems evolved in a given facility, their financial ability to purchase or custom build these systems and the vendors they have selected for the various components. 23

24 To meet the CMS MU objectives, hospitals must have an integrated EHR system that has the ability to: collect and store a wide range of clinical and demographic information; provide computerized physician order entry (CPOE) to electronically order laboratory, pharmacy, and radiology services; include an integrated decision support system that can incorporate clinical practice guidelines; and have the ability to exchange information with others inside and outside of their network. Only a small percentage of hospitals in the U.S. currently meet these criteria due to the substantial investment in human and financial resources that is required to achieve this level of EHR adoption (HIMSS, 2010). 24

25 Table 2-1. CMS Stage 1 meaningful use objectives and measures for eligible hospitals Policy Priority Core Objectives Measures Improving quality, safety, efficiency, and reducing health disparities Use CPOE for medication orders directly entered by any licensed healthcare professional who can enter orders into the medical record per state, local and professional guidelines More than 30% of unique patients with at least 1 medication in their medication list admitted to the hospital s inpatient or emergency department have at least one medication order entered using CPOE Implement drug drug and drug allergy interaction checks Record patient demographics (sex, race, ethnicity, date of birth, preferred language, and date and preliminary cause in the event of death) Maintain up-to-date problem list of current and active diagnoses Maintain active medication list Maintain active medication allergy list Record and chart changes in vital signs (height, weight, blood pressure, body-mass index, growth charts for children) Record smoking status for patients 13 years of age or older Implement one clinical decision support rule and ability to track compliance with the rule The hospital has enabled this functionality for the entire EHR reporting period More than 50% of all unique patients admitted to the hospital or emergency department have demographics recorded as structured data More than 80% of all unique patients admitted to the hospital or emergency department have at least one entry or an indication that no problems are known for the patient recorded as structured data More than 80% of all unique patients admitted to the hospital or emergency department have at least one entry (or an indication that the patient is not currently prescribed any medication) recorded as structured data More than 80% of all unique patients admitted to the hospital or emergency department have at least one entry (or an indication that the patient no known medication allergy) recorded as structured data For more than 50% of all unique patients age 2 and over admitted to the hospital or emergency department, height weight and blood pressure are recorded as structured data More than 50% of all unique patients 13 years old or older admitted to the hospital or emergency department have smoking status recorded as structured data One clinical decision support rule implemented 25

26 Table 2-1. Continued Policy Priority Core Objectives Measures Report hospital clinical quality measures to CMS or States For 2011, provide aggregate numerator and denominator and exclusions through attestation; for 2012, electronically submit measures Engage patients and families in their health care Improve care coordination Ensure adequate privacy and security protections for personal health information On request, provide patients with an electronic copy of their health information (including diagnostic-test results, problem list, medication lists, medication allergies, and for hospitals, discharge summary and procedures) Provide patients with an electronic copy of their discharge instructions at time of discharge, upon request Capability to exchange key clinical information (for example, problem list, medication list, medication allergies, diagnostic test results) among providers of care and patient authorized entities electronically Protect electronic health information created or maintained by the certifies EHR technology through the implementation of appropriate technical capabilities Policy Priority Menu Objectives Measures More than 50% of requesting patients receive an electronic copy of their health information within 3 business days More than 50% of all patients who are discharged from the inpatient or emergency department of a hospital or who request an electronic copy of their discharge instructions are provided with it Perform at least one test of certified EHR technology s capacity to electronically exchange key clinical information Conduct or review a security risk analysis, implement security updates as necessary, and correct identified security deficiencies as part of the risk management process Improving quality, safety, efficiency, and reducing health disparities Implement drug formulary checks Record advance directives for patients 65 years of age or older Incorporate clinical laboratory test results into certified EHR technology as structured data The hospital has enabled this functionality and has access to at least one internal or external drug formulary for the entire EHR reporting period More than 50% of all unique patients 65 years of age or older admitted to a hospital have an indication of an advance-directive status recorded More than 40% of clinical laboratory test results whose results are in positive/negative or numerical format are incorporated into EHRs as structured data 26

27 Table 2-1. Continued Policy Priority Menu Objectives Measures Improve care coordination Improve population and public health Generate lists of patients by specific conditions to use for quality improvement, reduction of disparities, research, or outreach Use certified EHR technology to identify patientspecific education resources and provide those to the patient as appropriate The hospital who receives a patient from another setting of care or provider of care or believes an encounter is relevant should perform medication reconciliation The hospital who transitions their patient to another setting of care or provider of care or refers their patient to another provider of care should provide summary of care record for each transition of care or referral Capability to submit electronic data to immunization registries or immunization information systems and actual submission in accordance with applicable law and practice Capability to submit electronic data on reportable (as required by state or local law) lab results to public health agencies and actual submission in accordance with applicable law and practice Generate at least one report listing patients with a specific condition More than 10% of all unique patients seen by the EP or hospital are provided patient-specific education resources The hospital performs medication reconciliation for more than 50% of transitions of care The hospital who transitions or refers their patient to another setting of care or provider of care provides a summary of care record for more than 50% of transitions of care or referrals Perform at least one test of certified EHR technology s capacity to submit electronic data to immunization registries and follow-up submission if the test is successful (where registries can accept electronic submissions) Perform at least one test of certified EHR technology s capacity to provide electronic submission of reportable lab results to public health agencies and follow-up submission if the test is successful (where public health agencies can accept electronic submissions) Capability to submit electronic syndromic surveillance data to public health agencies and actual submission in accordance with applicable law and practice Perform at least one test of certified EHR technology s capacity to provide electronic syndromic surveillance data to public health agencies and follow-up submission if the test is successful (where public health agencies can accept electronic submissions) 27

28 CHAPTER 3 LITERATURE REVIEW The scientific literature on health information technology has grown considerably in the last twenty years with most of the studies focused on two main themes: 1) adoption of HIT; and 2) its impact on the quality, safety and efficiency of heath care provision. It is important to examine both of these themes since the rate and extent of HIT adoption has an impact on how well hospitals and providers might use HIT in the delivery of patient care, independent of any influence it may have on the quality, safety and efficiency of the care provided. HIT Adoption Levels Over the past decade, there has been significant pressure from the federal government and other payers on hospitals and physicians to adopt HIT, especially electronic health records. However, the adoption rate remains fairly low, especially in comparison to hospitals in Europe (Anderson et al., 2006). Several national surveys are available that examine the rate and level of HIT adoption in hospitals. Most notable are the annual surveys performed by Health Information Management Systems Society (HIMSS) and the American Hospital Association (AHA). These studies indicate there are two major levels at which technology is adopted: 1) the organization level, at which the IT system is purchased and installed; and 2) the user level, where the intended users of the information system decide whether or not to incorporate that technology in their daily practice (Fonkych & Taylor, 2005). Generally, these national studies tend to capture types of systems available within health care organizations but do not capture the extent to which a system is used in patient care delivery. The national HIMSS Analytics survey showed that the vast majority of hospitals are still only in the low or 28

29 medium stage of EHR adoption process. They categorize adoption levels into seven stages with hospitals at Stage 1 only having major ancillary clinical systems installed (i.e., pharmacy, laboratory, radiology) and Stage 7 where hospitals no longer uses paper charts to deliver and manage patient care. The HIMSS EMR/EHR Adoption Model and specific characteristics of each stage are illustrated in Table 3-1. The 2010 HIMSS survey showed that 90% of hospitals had clinical information systems for ancillary services but only 1.0% of hospitals in 2010 had reached Stage 7. The majority of the hospitals in the middle stages (HIMSS, 2010). Jha and his colleagues at the Harvard School of Public Health reported similar results with 2.7% of the hospitals they surveyed in 2009 having a comprehensive electronic health record, according to their definition where twenty-four specified clinical functions are fully functional across all departments within the hospital (Jha et al., 2010). However, national surveys by the American Hospital Association and RAND have reported significantly higher EHR adoption rates with the AHA survey citing 11% of hospitals being fully functional while RAND reported between 20-30% (AHA 2007, Fonkych & Taylor, 2005). The estimates vary depending on how EHRs are defined, how well the questions can distinguish between EHR adoption and use, and the proper characterization of the multitude of features available in hospital information systems. Predictors of Hospital HIT Adoption Various studies have examined the characteristics of hospitals that have adopted HIT using data from national studies such as the HIMSS or AHA surveys as well as from state or specific surveys of hospitals types (academic institutions, children s hospitals). 29

30 Understanding the factors or characteristics that may influence hospital adoption of EHRs can guide policymakers in the distribution of funding to incentivize providers to adopt and meaningfully use EHRs. Many of the hospital characteristics examined include size (number of beds), system affiliation, ownership status, geographic location (rural or urban) and financial status. Using data from the 2004 HIMSS Analytics survey, Kazley and Ozcan (2007) performed a national study of over 4,000 hospitals which examined how the organizational and environmental factors correlate with a hospital's EMR/EHR adoption. Their findings showed EMR/EHR adoption is significantly associated with hospital system affiliation, size (larger) and urban location. Other characteristics such as market competition, non-profit or public ownership, teaching status, public payer mix, and operating margin were not statistically significant. A RAND study, also using 2004 HIMSS Analytics data, found that HIT adoption and use is more common for academic and pediatric hospitals and that system affiliation is the strongest predictor of hospital HIT use. They also found that EHRs are less likely in small hospitals, rural hospitals and for-profit hospitals (Fonkych & Taylor, 2005). However, Li and colleagues (2008), who examined multi-hospital system affiliation on the level of EMR/EHR adoption in greater detail, found that small hospitals owned by multihospital systems had a significantly higher EMR/EHR level compared with independent hospitals. They speculate that smaller hospitals in multi-hospital systems have an advantage over small independent hospitals in HIT capacity because of the greater availability of capital, access to shared HIT capacity, and technical expertise. 30

31 Jha et al. (2009) collaborated with the American Hospital Association (AHA) to survey all acute care member hospitals in 2008 for the presence of specific electronic health record functionalities. They examined the relationship of adoption of EHRs to hospital characteristics such as size, geography, ownership, teaching status, and presence of markers of a high technology institution (dedicated coronary care unit, burn unit, or PET scanner). Larger hospitals, those located in urban areas, and teaching hospitals were reported to have a significant higher number of electronic health record functionalities. Wang et al. (2002) studied the factors influencing hospital HIT adoption including several financial factors, using a sample of 1,441 U.S. hospitals in Their results show that hospitals affiliated with a multi-hospital system and those that are for-profit are more likely than others to have IT applications as well as hospitals with higher cash flows, and operating margins. A Florida study resulted in similar findings as the national studies. Hikmet et al. (2008) collected HIT data from a survey of 98 Florida hospitals in Their study examined whether specific organizational characteristics, such as hospital size, geographic location, system affiliation, and tax status influence adoption of health care information technologies. They found that non-profit hospitals, larger hospitals (regardless of system affiliation), as well as small hospitals affiliated with multi-hospital systems, had higher levels of HIT adoption. Geographic location of hospitals was not a factor in HIT adoption. The authors attributed this result to the high number of small, rural hospitals affiliated with multi-hospital systems in Florida. 31

32 Major barriers to EHR adoption has been identified in several surveys. The American Hospital Association s 2006 survey of its hospital members found major barriers to adoption include capital costs of implementing an EHR (54%), ongoing costs of maintaining a system (32%), interoperability with current systems (27%), acceptance by the clinical staff (23%), and the availability of well-trained IT staff (16%) (AHA, 2007). Similar barriers have been identified by Jha and his colleagues based on a 2008 national survey of hospitals. Their findings showed that among hospitals without EHRs, the most commonly cited barriers were inadequate capital (74%), maintenance costs (44%), physician resistance to EHRs (36%), unclear return on investment (32%), and lack of availability of HIT staff (30%) (Jha et al, 2009). The Lewin Group report from the Health Information Technology Panel findings commissioned by the National Coordinator for Health Information Technology in 2005 also found that ongoing operating costs, including specialized staff for the ongoing operation and maintenance of systems, as a major factor in affecting HIT adoption (Lewin Group, 2005). Hersh and Wright (2008), using the HIMSS Adoption Model, found that the amount of IT staff varies by level of EHR adoption, with FTE per bed at the lowest level of the HIMSS Adoption Model (Stage 0) and increasing to FTE bed at higher levels (Stage 4). In summary, most of these studies found that larger hospitals, non-profit hospitals, hospitals in urban settings and hospitals affiliated with multi-hospital systems had greater levels of HIT adoption. Issues regarding having adequate capital, physician support and adequate HIT staff are also known to be factors in HIT adoption. 32

33 HIT and Quality Much of the HIT literature begins with a discussion regarding the consensus among policy makers, health care researchers and quality experts that widespread adoption of HIT will lead to increased efficiency and improved patient care (IOM, 2001; ONCHIT, 2004; Blumenthal, 2010). The enthusiasm regarding the potential benefits of HIT on improving the delivery of health care has led to a national policy that providers should adopt HIT, although the evidence in support of these benefits varies greatly by type of application. There has been some evidence that HIT has a positive impact on quality by eliminating poorly written and poorly organized paper medical records, standardizing physician orders, and incorporating clinical pathways and other tools to help providers better manage patients care. In an extensive global review of the literature for the National Health Service in the United Kingdom, Car and colleagues (2008) found evidence of primary studies showing positive benefits from EHRs in particular institutions, but the nature and magnitude of benefits were not consistent across studies, nor was there clear findings on how benefits might be applied to other institutions and settings. This is especially true with studies with favorable results that were based on home grown health information systems such as the one developed by the Veteran Health Administration. Two systematic reviews of the literature, one funded by AHRQ in 2006 and one follow-up study published in 2009 found some evidence of cost and quality benefits of comprehensive HIT systems at a few organizations (Chaudhry et al., 2006; Goldzweig et al., 2009; Shekelle & Goldzweig, 2009). In particular, these studies found that HIT, especially more advanced EHR systems with computerized physician order entry (CPOE) and clinical decision support systems (CDSS) have been shown to decrease 33

34 medication errors and improve quality by providing decision support tools and increasing adherence to clinical practice guidelines. Much of the relevant research on health information technology s impact on health services delivery focus on evaluating CPOE and CDSS, either alone or as integrated systems. Evidence of CPOE and CDSS s impact on quality of care and improved patient outcomes are presented in detail in the following sections. CPOE Studies Computerized physician order entry (CPOE) is an important component of a comprehensive HIT system and has been defined as an electronic application used by physicians to order drugs, laboratory tests, radiology and other medical procedures as well as requests for consultations (Poon et al., 2004). It is important to note that CPOE systems offer a variety of capabilities depending on the vendor. The successful use of these systems also depends on how well the hospitals have integrated CPOE throughout their facility and patient care processes. Several systematic reviews of the literature show that there is strong and consistent evidence that CPOE is an important intervention in the reduction of medication errors and adverse drug events. Medication errors are one of the most common errors that occur in hospitals due to the complexity of the medication process with ordering, dispensing, and delivery/administration of medications to patients requiring the participation of numerous health care providers (IOM, 2000). Ammenswerth et al. (2008) performed a systematic and quantitative review of the literature to determine the effect of e-prescribing/cpoe on the risk of medication errors and adverse drug events (ADE). They identified 27 studies that met their inclusion criteria and the majority of these studies were conducted in hospital inpatient units. Two 34

35 of the studies were randomized trials and the rest were before/after implementation studies. Twenty-three of the 25 studies that evaluated the effects on the medication error rate showed a significant decrease in medication errors with the use of e- prescribing/cpoe with relative risk ratios between 0.01 and They also saw a reduction in actual and potential adverse drug events. A systematic review of studies that examined the impact of CPOE on prescribing errors for pediatric and adult inpatients was performed by Reckmann et al. (2009). They identified 4 pediatric studies and 9 adult studies that met their inclusion criteria. The pediatric studies were performed in either the pediatric or neonatal intensive care units and all the studies showed a decrease in medication errors and an increase in the proper administration of IV drug therapies. All but one of the adult studies demonstrated lower medication errors with the use of CPOE. The use of CPOE in the management of hospital patient orders is not confined to only medications but can include many hospital departments. Besides reducing medication errors or adverse drug events, the use of CPOE in hospitals can also reduce turnaround time for procedures, overuse of diagnostic tests, and reduce length of stay. Kuperman and Gibson (2003) examined the literature on CPOE by major outcome or effect. They identified several randomized control trials on the use CPOE for laboratory test ordering which resulted in a 5% to 18% reduction in laboratory orders due to the identification of duplicate orders or other unnecessary orders. They found mixed evidence with regards to CPOE in reducing the use of radiology procedures where one study had positive results and another study showed minimal impact on reducing radiology orders even though there was an improvement with procedure turnaround 35

36 time. Kuperman and Gibson (2003) identified two studies that demonstrated a reduction in inpatient length of stay, a randomized control trial by Tierney et al. (1993) and a time series study by Mekhjian et al. (2002). The study by Tierney and colleagues showed that the use of CPOE with decision support capabilities in certain medical units resulted in a significant reduction of length of stay by 0.89 days while the Mekhjian study only had a decrease of 0.20 days in one of the two hospitals that participated in the study. A recent study was done to try to link the CMS MU objectives to quality. Jones and colleagues (2011) used a national survey to estimate data for one particular MU objective - electronic medication order entry - which served as the primary variable of interest for their study and examined its impact on mortality rates. Their results suggest that the initial meaningful-use threshold for hospitals using electronic orders for at least 30% of eligible patients did not have a significant impact on deaths from heart failure and heart attack. However, the proposed threshold for the stage 2 of Meaningful Use using the orders for at least 60% of patients, was associated with lower mortality. CDSS Studies Computerized decision support systems (CDSS) can vary greatly in their form and functionality as well as the extent of their integration in the care process and the hospitals clinical information system. Basic decision support applications can provide tools for checking the completeness and accuracy of patient orders while more sophisticated systems integrate patient data with evidence based practice guidelines for the comprehensive management of patient care. These applications can produce patient specific output in the form of care recommendations, assessments, alerts and reminders to actively support clinical decision making (Car et al., 2008). Many researchers view embedding CDSS into well-developed, comprehensive EHRs as a 36

37 way to truly harness the full potential of HIT to provide timely, relevant information, guide clinical decisions and improve patient safety and outcomes (Staggers, Weir, & Phansalkar, 2008). Garg et al. (2005) performed a comprehensive systematic review of the literature and identified 100 studies that included randomized and nonrandomized controlled trials from 1973 to 2004 that evaluated the effect of a CDSS compared with care provided without a CDSS on practitioner performance or patient outcomes. These studies included inpatient and outpatient settings and were grouped by major activity preventive care, diagnosis, disease management and organizational efficiency. There were 10 trials evaluating diagnostic capabilities and 4 of these studies found CDSS to improve practitioner performance. Two of the 4 successful CDSS studies examined diagnostic capability for cardiac ischemia in the emergency department and found that it significantly decreased the rate of unnecessary hospital admissions by 15%. Of the 97 studies they identified as assessing practitioner performance, the majority (64%) improved diagnosis, preventive care, disease management, or drug prescribing. Of the 52 studies they identified as assessing patient outcomes, the researchers only found 7 studies that improved patient outcomes with CDSS; however, they noted many of these studies did not have adequate statistical power to detect differences. Damiani et al. (2010) performed a systematic review of available literature and focused their research on a key component of CDSS, computerized clinical guidelines, and its impact on the process of care compared with the use of non-computerized clinical guidelines such as paper guidelines, peer-to-peer consultation or previous experience. Of the 45 articles they selected for the study, 64% showed a positive effect 37

38 with the use of computerized clinical guidelines. Specifically, they found a significant impact on the process of care with the automatic provision of recommendations provided by the computerized clinical guidelines as part of clinician workflow as compared to the use of non-computerized clinical guidelines. Kaushal, Shojania and Bates (2003) also examined studies using stand alone CDSS (without CPOE) and their impact on medication safety. They identified 7 studies in their literature review, 6 were randomized control trials and one was a prospective, before-after analysis. These results showed that 6 of the 7 studies demonstrated improvements in medication safety including lower numbers of toxic levels of medications, and improved antibiotic drug selection (greater pathogen susceptibility). A meta-analysis of research involving 311 unique studies using CDSS and Knowledge Management Systems (KMS) from 1976 to 2010 was conducted by the Duke Evidence-based Practice Center under contract to the Agency for Healthcare Research and Quality (AHRQ, 2012). This meta-analysis examined three main constructs of these systems: their impact on clinical effectiveness, their impact on outcomes and to identify features that impact their success. The researchers found 22 studies that assessed the impact on morbidity and found moderate success in reducing patient morbidity with a combined relative risk of 0.88 (95% CI 0.80 to 0.96). The researchers found 6 studies that assessed the impact on mortality and found limited evidence that CDSSs were effective in reducing patient mortality. The meta-analysis did find that CDSS and KMS can improve the process of delivering care, especially in the areas of performing preventive services (n = 25; OR 1.42), ordering clinical studies (n = 20; OR 1.72), and prescribing therapies (n = 46; OR 1.57). 38

39 However, some studies have illustrated that CPOE and CDSS can have unintended consequences or introduce other types of errors. Kaushal and colleagues (2003) found that early in the implementation phase of these systems, incorrect default dosing or medication recommendations may create potentially erroneous orders. Some CDSSs may be based on flawed or outdated clinical practice guidelines, have faulty algorithms or were not thoroughly tested (Car et al., 2008). User error is always an issue with computers such as inputting incorrect patient data which will result in erroneous CDSS recommendations, clicking on the wrong patient or item selection (often select the item above or below the intended choice), or ignoring alerts generating by the system (Ash et al., 2009). Improper use may occur when the technology is a poor fit with the current workflow processes. Careful testing, documentation and ongoing scrutiny of these systems by clinicians, managers and software designers is necessary to identify emerging issues and remedy them before there is serious patient consequences (Harrison, Koppel & Bar-Lev, 2007). EHR Studies As noted earlier, there have been many studies on certain components of HIT such as CPOE and CDSS but there are only a limited number of studies that have focused on hospital EHRs and their impact on health care quality and outcomes. Recent studies have shown some degree of success but also some mixed results. Early success with electronic medical records can be found with the Veterans Health Administration (VHA) and their implementation of a national, interoperable electronic medical record system in the early 1990s. Asch et al. (2004) compared the quality of care received by patients at 12 VHA facilities with a sample of almost a 1,000 patients from community hospitals. They collected data on 348 indicators targeting 26 39

40 conditions and found that the VHA facilities were associated with higher levels of overall patient care, chronic disease management and preventative care but not in acute care. However, this study was also during the same period when the VHA was implementing their quality performance measurement system which may have influenced these results. Kazley and Ozcan (2008) performed a nationwide cross-sectional study using the HIMSS Analytics hospital HIT adoption database and Hospital Quality Alliance (HQA) quality measures for 2004 to investigate the impact of EMRs/EHRs and hospital quality performance. Four of the ten HQA process measures selected showed a statistically significant positive relationship for hospitals with EMRs/EHRs as compared to hospitals without EMRs/EHRs and one measure had a negative relationship. The positive measures included the use of beta blocker at arrival to the hospital, use of aspirin at discharge from hospital, and use of beta blocker at discharge from hospitals for acute myocardial infarction patients, and the assessment of left ventricular function for congestive heart failure patients. The negative result was a measure for pneumonia that examined if patients were given antibiotic within 4 hours of admission. DesRoches and colleagues (2010) performed a study of U.S. hospitals using EHR adoption information from a national survey to investigate the relationship between the adoption of EHRs and select measures of health care quality and efficiency from HQA and Medicare using 2008 data. In particular, their investigation looked at whether EHR adoption was associated with better performance on standard process-of-care measures, lower mortality and readmission rates, shorter lengths-of-stay, and lower inpatient costs. Their results showed no significant relationship between EHRs and 40

41 mortality or readmission rates, as well as most of their process of care measures except for the prevention of surgical complications which had a slight, but statistically significant, improvement. The authors noted that further study is needed to look beyond adoption of EHRs and more on how EHRs are being used in hospitals. Parente and McCullough (2009) performed a longitudinal study using HIMSS Analytics database and MEDPAR data for the period from Specifically, they studied whether EHRs, electronic nurse charting, and picture archiving and communications systems (PACS) had an impact on 3 outcome measures: infection due to medical care, postoperative hemorrhage, and postoperative pulmonary embolism or deep vein thrombosis. Their results showed that EHRs was the only HIT application to have a statistically significant positive effect on one measure - infection due to medical care. McCullough et al. (2010) built upon their earlier study by examining HIT adoption decisions over a 3 year time period ( ) using the HIMSS Analytics database and Medicare s Hospital Compare database for process/quality measures. Their findings showed that for most of their selected measures, quality was higher for hospitals with EHRs but only two of these measures were statistically significant higher rates of pneumococcal vaccination and appropriate antibiotic selection for pneumonia. Similar to the McCullough study, Jones et al. (2010) also performed a longitudinal study using HIMSS hospital HIT adoption data from 2003 and 2006 and Hospital Compare data from 2004 and 2007 to examine a difference in quality over time. They used a difference-in-difference analytic approach to estimate the relationship between 41

42 hospital EHR adoption levels and improvement in quality measures during the two time periods. Over the specified time period, the researchers saw an increase in the adoption of EHRs and a movement towards more advanced EHRs. They found that hospitals that maintained a basic EHR during this time period had statistically significant improvements in their quality scores for heart failure, AMI and pneumonia as compared to hospitals with no EHRs. However, hospitals that adopted EHRs during this time period did not have improved quality scores as compared to hospitals with no EHRs, indicating that it takes time for the benefits of EHRs to take effect. Himmelstein et al. (2010) examined both the cost and quality impacts of HIT in hospitals. Using HIMSS Analytics data, they created composite scores for each hospital based on the number of computer applications implemented in their facility including EHRs. They used Medicare cost reports and the 2008 Dartmouth Atlas data on costs and quality of care as their outcome measures. The Dartmouth Atlas data included 4 quality scores for pneumonia, congestive heart failure, and acute myocardial infarction and a quality composite score. The researchers found that hospitals with higher overall computer application scores had slightly better composite quality scores, especially those hospitals with EHRs and CPOE. For specific quality measures, more computerized hospitals scored higher on process measures of care for acute myocardial infarction, but not for pneumonia or heart failure. Menachemi, Chukmaitov, Saunders, & Brooks (2008), performed a similar study among Florida hospitals and found that hospitals that adopted a greater number of IT applications were significantly more likely to have better quality outcomes on certain 42

43 inpatient quality indicator measures, including risk-adjusted mortality from coronary angioplasty, gastrointestinal hemorrhage, and acute myocardial infarction. Limitations of Prior Studies While the potential benefits of health information technology are clear in theory, identifying its impact on health care quality has proven difficult and rates of use within the hospital industry have been limited. Most of the studies discussed in the systematic reviews have been performed at single institutions, oftentimes with hospitals that have developed their own HIT applications or are considered HIT leaders. Many of the HIT studies on quality in the literature focused primarily on CPOE and CDSS (Chaudhry et al., 2006; Goldzweig et al., 2009). The specific nature of these studies on HIT adoption, especially the type of health information systems deployed in these facilities and the specific functions tested, can reduce the generalizability of their findings. Moreover, the potential benefits of HIT are likely to be realized not just by investing in specific systems but by effectively managing and integrating those systems into patient care delivery processes. Therefore, to study the impact of HIT on health care quality, it is important to identify not only the components and functionality of health information systems but also how they are used. Only recently national studies have been performed on the impact of hospital EHRs and quality. Many of these studies show mixed results. To provide strong, general evidence of HIT value on health care quality, especially with EHRs, studies that span multiple hospitals and carefully measure HIT functionality and usage at a detailed level are needed. There have been very few studies to date that link actual usage instead of adoption of HIT to hospital performance or quality and to my knowledge, none that have specifically tested the CMS MU objectives. 43

44 Table 3-1. HIMSS EMR Adoption Model Stage Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Description The organization has not installed all of the key ancillary department systems (e.g. laboratory, pharmacy, radiology). Major ancillary clinical systems are installed (i.e., pharmacy, laboratory, radiology). Major ancillary clinical systems feed data to a clinical data repository (CDR) that provides physician access for retrieving and reviewing results. The CDR contains a controlled medical vocabulary, and the clinical decision support/ rules engine (CDS) for rudimentary conflict checking. Information from document imaging systems may be linked to the CDR at this stage. The hospital is health information exchange (HIE) capable at this stage and can share whatever information it has in the CDR with other patient care stakeholders. Nursing/clinical documentation (e.g. vital signs, flow sheets) is required; nursing notes, care plan charting, and/or the electronic medication administration record (emar) system are scored with extra points, and are implemented and integrated with the CDR for at least one service in the hospital. The first level of clinical decision support is implemented to conduct error checking with order entry (i.e., drug/drug, drug/food, drug/lab conflict checking normally found in the pharmacy). Some level of medical image access from picture archive and communication systems (PACS) is available for access by physicians outside the Radiology department via the organization s intranet. Computerized Practitioner Order Entry (CPOE) for use by any clinician is added to the nursing and CDR environment along with the second level of clinical decision support capabilities related to evidence based medicine protocols. If one patient service area has implemented CPOE with physicians entering orders and completed the previous stages, then this stage has been achieved. The closed loop medication administration environment is fully implemented. The emar and bar coding or other auto identification technology, such as radio frequency identification (RFID), are implemented and integrated with CPOE and pharmacy to maximize point of care patient safety processes for medication administration. Stage 6 Stage 7 Full physician documentation/charting (structured templates) is implemented for at least one patient care service area. Level three of clinical decision support provides guidance for all clinician activities related to protocols and outcomes in the form of variance and compliance alerts. A full complement of PACS systems provides medical images to physicians via an intranet and displaces all film-based images. The hospital no longer uses paper charts to deliver and manage patient care and has a mixture of discrete data, document images, and medical images within its EMR environment. Clinical data warehouses are being used to analyze patterns of clinical data to improve quality of care and patient safety. Clinical information can be readily shared via standardized electronic transactions (i.e. CCD) with all entities who are authorized to treat the patient, or a health information exchange (i.e., other non-associated hospitals, ambulatory clinics, subacute environments, employers, payers and patients in a data sharing environment). The hospital demonstrates summary data continuity for all hospital services (e.g. inpatient, outpatient, ED, and with any owned or managed ambulatory clinics). Source: 44

45 CHAPTER 4 CONCEPTUAL FRAMEWORK As demonstrated in the review of the literature, a few national studies been performed on the impact of hospital EHRs and quality and only a limited number of these studies have examined the link between actual usage of HIT, instead of simple adoption of HIT, and hospital performance or quality. The purpose of this dissertation is to study the possible linkages between the actual usage of EHRs in hospitals and the quality of care at these organizations. A conceptual model derived from two overreaching frameworks of investigation, one focused on quality (Donabedian) and the other on information systems (Delone and McLean), will be employed which will allow this study to both build upon and contribute to the literature. The following provides an overview of these frameworks and the coalescence of these into this study's conceptual model. Donabedian s Quality Framework Although Florence Nightingale and Dr. Ernest Codman are consider early pioneers for evaluating and monitoring health care quality and patient outcomes, Avedis Donabedian is considered the father of the quality movement for the health care industry (Iezzoni, 2003; Car et al., 2008). Donabedian's seminal paper introduced the concepts of structure, process, and outcome for the evaluation of the quality of health care in 1966 (Car et al., 2008). Donabedian s quality framework asserts that organizations implement structures, which influence processes and outcomes (Donabedian, 2005). Donabedian defined structure as the qualifications of the providers of care, the tools and resources they 45

46 have available to care for patients and the physical and organizational environment in which they work (Donabedian, 1980). He defined processes as the set of activities that take place between providers and patients and can include protocols and procedures in the diagnosis, treatment and management of patient conditions (Donabedian, 1980, p. 82). And, outcomes refer to the impact of the processes working within the structure. To view HIT usage and quality care in Donabedian's framework, hospitals that invest in the resources to adopt EHRs (structure), provide the incentives and/or requirements to staff and physicians to use EHRs effectively (process), would see changes in the quality of care (outcome). In this model, the use of EHRs influence the process of care with the use of CPOE and CDSS as well as standardizing the information collected from patients such as demographic data, problem lists, medication lists etc. Improved and automated processes are expected to reduce medical errors, and support and inform clinical decisions, thus improving health care quality outcomes (Chaudhry et al., 2006; Goldzweig et al., 2009; Shekelle & Goldzweig, 2009; Kazley & Ozcan, 2008). Other structural elements in hospitals can also influence the quality of care including organizational and financial structures and will need to be considered in developing a model to assess the impact of EHRs on hospital quality. Hospital size (patient volume), ownership, and especially nursing staffing ratios have been found to impact the quality of care. Stanton and Rutherford (2004) performed a review of studies funded by the Agency for Healthcare Research and Quality (AHRQ) as well as other research on the relationship of nurse staffing levels to adverse patient outcomes. They identified a broad array of research on this topic and have found several studies that 46

47 have shown an association between lower nurse staffing levels and higher rates of poor patient outcomes such as pneumonia, shock, cardiac arrest, and urinary tract infections. Delone and McLean Information System Success Model Donabedian s quality framework provides the unifying concept on how hospital structure, including EHRs, can impact the process of care delivery and quality of care. To provide an information systems perspective, the Delone and McLean Information System Success Model (D & M IS Success Model) provides a framework to examine the use of information technology (Delone & McLean, 1992). Their model is based on the pioneering research on communications by Shannon and Weaver (1949) and as well as the more current work on information influence theory of Mason (1978). Shannon and Weaver (1949) addressed three levels of communication: the technical, level, the semantic level and the effectiveness level. The technical level of communications is the accuracy and efficiency of the communication system, the semantic level is the success of the information in conveying the intended meaning, and the effectiveness level is the effect of the information on the receiver. The original D & M IS Success Model (shown in Figure 4-1) posits that information system quality and information (data) quality impact both the use of the system and the user s satisfaction of the information system which impacts the user s performance and ultimately, the organization s performance (Delone & McLean, 1992). The model uses system quality as a construct for technical success, information quality as a construct for semantic success, and the constructs of use, user satisfaction, individual impacts, and organizational impacts to measure effectiveness success. Delone and McLean define system quality as the usability, reliability, functionality, availability and adaptability of the information system to the organization 47

48 and define information quality as the accuracy, completeness, relevance, and timeliness of the information to the user (Delone & McLean, 2003). Their model is grouped into three dimensions: the IS system (system quality and information quality), the actual use of the system, and the consequences of system use (individual and organizational impact). They argue that their model is a causal model and each of these steps is a necessary condition for the resultant outcomes. These dimensions align with Donabedian s quality framework where the IS system dimension is analogous with structure, the use dimension is analogous to process and the consequences dimension is comparable to outcomes. For example, HIT usage and quality care in the D & M IS Success Model may be represented as hospitals that value, invest resources, and communicate with users regarding the adoption of EHRs (system quality and information quality), provide the training, feedback, incentives and/or requirements to use components of EHRs (use of the system), would see changes in the way the practitioners use the technologies (individual impact) that then affect the quality of care (organizational outcome). Conceptual Model Development The proposed conceptual model for this study builds upon Donabedian s quality framework and incorporates output elements from the D & M IS Success Model to show the relationship between structure and the use of EHRs and how the use of EHRs impact the process of care which then impacts the quality of care. Figure 4-2 illustrates this study's conceptual model and explains how the variables included in the adoption of EHR system which includes hospital characteristics and EHR system functions (structure) influence the actual usage of HIT in the delivery of health care (process) resulting in the number of MU measures met by each hospital 48

49 (Objectives 1 and 2). By meeting the MU objectives, clinicians are adhering to clinical practice guidelines and have access to structured patient data (individual impact) to improve informed decision making and therefore increase the quality of care delivered in the hospitals (organizational impact). It is posited that this model will show that EHR enhanced health care processes will demonstrate better quality outcomes than those who do not have EHRs or use them ineffectively (Objective 3). This model also shows the relationship between hospital characteristics, HIT adoption and HIT use. Hypotheses Hypotheses have been articulated to evaluate the association of financial and organizational characteristics that are linked to higher levels of meaningful use among hospitals. Several studies described in the literature review section (Chapter 3) have shown that key hospital organizational characteristics such as non-profit status, system affiliation, large bed size and positive operating margins are positively associated with higher levels of EHR adoption; however, some of these findings are inconsistent, especially regarding profit status of hospitals (Fonkych & Taylor, 2005; Li et al., 2008; Wang et al., 2002). For this study, it is expected that for-profit hospitals, especially those hospitals owned by large national chains, are expected to have a higher level of adoption than non-profit hospitals due to their market power in negotiating competitive prices for EHR systems and a pool of expertise in implementing a national strategy for EHR adoption among their hospitals. Furthermore, with the availability of the CMS incentives, forprofit hospital chains will likely take advantage of these resources and have a systemwide approach in achieving the CMS MU objectives. Therefore, it is postulated as follows: 49

50 Hypothesis 1: For-profit hospitals will be associated with a greater number of the CMS MU objectives than non-profit hospitals. Hospitals that are affiliated with hospital systems are expected to have a higher level of adoption due to economies of scale and shared technical expertise, as well as more financial resources and access to capital (Kazley & Ozcan, 2007; Li et al., 2008). Furthermore, group purchasing of common systems and interoperability are more likely to occur in hospital systems. As discussed for the for-profit chains, hospitals affiliated with a health system will likely have a system-wide approach in achieving the CMS MU objectives Therefore it is postulated as follows: Hypothesis 2: System affiliated hospitals will be associated with a greater number of the CMS MU objectives met than independent hospitals. Hospital size can impact a hospital s ability to adopt new technology. Larger organizations tend to have more financial resources and personnel dedicated to health information technology (Fonkych & Taylor, 2005; Jha et al., 2009; Kazley & Ozcan, 2007). Larger hospitals also tend have the administrative structure in place to effectively monitor and manage HIT systems, including the provision of ongoing training. Therefore, it is postulated as follows: Hypothesis 3: Large hospitals will be associated with a greater number of the CMS MU objectives met than small or medium size hospitals. Urban hospitals are generally larger than rural hospitals and have more access to specialized staff, capital, and equipment in their community (Fonkych & Taylor, 2005; Jha et al., 2009; Kazley & Ozcan, 2007). HIT vendor support and other specialized technical support will also be more available in the urban areas. Furthermore, local 50

51 competition drives hospitals to be more technologically advanced than hospitals that have no competition. Therefore, it is postulated as follows: Hypothesis 4: Urban hospitals will be associated with a greater number of the CMS MU objectives met than rural hospitals. Hospitals with positive operating margins have the financial resources and cash reserves for capital investments such as health information technology (Wang et al., 2002). They also have the funds to maintain and upgrade health information systems. Therefore, it is postulated as follows: Hypothesis 5: Hospitals with higher positive operating margins will be associated with a greater number of the CMS MU objectives. Hospitals with in-house information technology staff should have the capability to implement and manage electronic medical records and will have greater ability to integrate systems within the hospital. Based on the Hersh and Wright (2008) hospital IT staff analysis, higher ratios of IT FTE per bed were found in hospitals who had achieved higher levels of HIT adoption (0.210 FTE bed at HIMSS Stage 4 Adoption Level). Therefore, it is postulated as follows: Hypothesis 6: Hospitals with higher ratios of IT staff (FTE) per bed will be associated with a greater number of the CMS MU objectives. Medical staff leadership has been found to be an important catalyst in EHR implementation (Boonstra & Broekhuis, 2010; Goroll et al, 2009; Saathoff, 2005). A Chief Medical Information Officer (CMIO) on staff can provide insights on how systems can improve physician workflow and provide the leadership to the medical staff to move adoption more quickly. Therefore, it is postulated as follows: 51

52 Hypothesis 7: Hospitals with Chief Medical Information Officers will be associated with a greater number of the CMS MU objectives met than hospitals without CMIOs. The conceptual model discussed above leads to the following hypothesis reflecting the main objective of this study: is there a relationship between hospitals achieving the MU objectives and better quality of care? If the MU objectives are good indicators of relevant use of health information technology, it is expected that meeting these objectives will have a positive impact on the quality of care measures. EHR s potential impact on quality is believed to lie with clinicians having immediate access to standardized patient information and increased adherence to clinical practice guidelines which will improve the process of care and therefore improve the quality of care. Hypothesis 8: Hospitals with higher numbers of MU objectives met will be associated with higher scores on Hospital Compare quality measures. Based on past evidence, hospitals that are effectively using CPOE and CDSS are expected to have higher scores on quality measures and will also be tested individually. 52

53 Figure 4-1. DeLone and McLean IS Success Model, 1992 Figure 4-2. EHR Use/Quality Model 53

54 CHAPTER 5 DATA AND METHODS The primary objective of this study is to assess Florida acute care hospitals current EHR use in terms of the CMS Stage 1 MU objectives and whether meeting these objectives is associated with better quality of care. Specifically, this study ascertains the level of attainment of Florida hospitals in meeting the CMS Stage I MU objectives, identifies hospital organizational characteristics which are associated with higher levels of meaningful use among Florida hospitals, and assesses the association between achievement of the MU objectives and improved patient process measures. This chapter describes the data sources and variables used for this study and the method for analysis to address the study objectives. Data Sources Due to the timeliness of the subject matter with the release of the Stage 1 MU objectives in July of 2010, there has not been any national publicly available data regarding hospitals ability to meet the Stage 1 MU objectives. However, for a state to implement the Medicaid EHR Incentive Program, the Centers for Medicare and Medicaid Services (CMS) required states to submit a Medicaid Health Information Technology Plan (SMHP). The SMHP included an environmental scan to identify the health information technology baseline of Medicaid providers in the state. This environmental scan included questions regarding the hospitals current ability to meet the Stage 1 MU objectives. Other data sources used in this study include the CMS Hospital Compare database for the dependent variables and the Agency for Health Care Administration (AHCA) Hospital Beds and Services List, the Florida Hospital Uniform Reporting 54

55 System, and the American Hospital Association Annual Survey as sources for the independent variables. Florida Health Information Technology Environmental Scan As part of the state s SMHP, a comprehensive assessment was performed that surveyed acute care hospitals, federally qualified health centers (FQHCs), rural health clinics (RHCs), Regional Health Information Organizations (RHIOs)/Health Information Exchanges (HIEs), and select healthcare professionals participating in Medicaid and who may be eligible for the Medicaid EHR incentives (physicians, dentists, certified nurse-midwives, and nurse practitioners). This study utilizes the survey data from the Florida Hospital Health Information Technology survey conducted as part of a statewide HIT Environmental Scan in August This survey provides detailed information on acute care hospital EHR adoption levels in Florida including information on the type of health information systems hospitals have in place and how these systems are used. The survey also addresses the hospital s ability to exchange information with other providers and the functionality of their system, especially the ability to meet the CMS Stage 1 MU guidelines for Medicare HIT incentive payments. I served as the project manager for the Environmental Scan and was responsible for the overall development, implementation and analysis of this research. The survey tool was developed by researchers at the University of Florida and University of Alabama-Birmingham and implemented by the WellFlorida Council. A draft final survey was pilot tested in five Florida hospitals, including urban and rural hospitals. The pre-testing was performed by administrators and Chief Information Officers who provided written or verbal feedback that was used to improve the survey s 55

56 interpretability. A copy of the 2010 Florida Hospital Health Information Technology Survey instrument is included in the Appendix. The survey population included all 211 acute care hospitals in Florida. Specialty hospitals, long-term care hospitals and federal hospitals (VA, military) were excluded because only acute care hospitals are eligible for the CMS EHR incentive program. The survey was ed directly to hospital CEOs and was fielded for 64 days (August 9 October 13, 2010). Hospital CEOs were asked to direct the survey to the most knowledgeable person in their institution on information technology for completion. Extensive follow-up procedures, including direct s and phone calls to hospital CEOs, led to a 76% response rate (161 hospitals). This survey was administered using an on-line survey tool and the data were converted into SAS and STATA for analysis. The survey instrument includes several sections that are relevant to this study. The first section asked for identifying information about the hospital (hospital name, address, AHCA hospital identification number and type of hospital). The third section asked about the hospital s organizational structure for the management of health information systems and the financial and human resources devoted to the management of these systems. The fourth section asks for detailed information about the current clinical information systems at their facility including EHRs, laboratory, pharmacy and radiology systems, the functionality of these systems and the extent to which they may meet the federal guidelines for meaningful use. The fifth section asks detailed questions on the medication management system and the type of patient alerts built into the system for the prevention of medication errors. 56

57 The 2010 Florida Hospital HIT survey question response categories use ranges (0, 1-25%, 26-50%, 51-75%, 76-99%, 100%) which resulted in six of the survey questions responses not perfectly aligning with the MU objective threshold. For instance, the CMS measure may specify that 80% of patients have at least one entry recorded as structured data while the Florida Hospital HIT Survey had quartile ranges for responses. In this case, hospitals that reported 76-99% for the measure is considered to meet the CMS standard of 80%. See Table 5-1 for a crosswalk between the CMS Stage 1 MU objectives and the hospital survey questions. The hospital survey data is used to determine how many of the Stage 1 MU measures each participating hospital has met. CMS Hospital Compare The Hospital Compare website was created through the efforts of CMS and the Hospital Quality Alliance (HQA) and includes consensus-derived set of hospital quality measures appropriate for public reporting established by the National Quality Forum. Hospital Compare displays rates for Process of Care measures for patients being treated for a heart attack, heart failure, pneumonia, asthma (children only) or patients having surgery. Hospitals voluntarily submit data on a quarterly basis about the treatments their patients receive for these conditions as part of the CMS Hospital Inpatient Quality Reporting Program. The measures selected for this study include all of the process measures for heart failure and pneumonia and are indicative of processes that can be enhanced by having an effective health information system in use. The heart failure measures are endorsed by the American College of Cardiology/American Heart Association and incorporated in their Heart Failure Practice Guidelines (Hunt et al, 2009). The pneumonia measures are part of the Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of 57

58 community-acquired pneumonia in adults (Mandell et al, 2007). The data includes all patients treated with these conditions, not just Medicare beneficiaries. The dataset used in the analysis were released on August 5, 2011 and include the reporting period of October 2009 through September AHCA Hospital Beds and Services List The AHCA Hospital Beds and Services List is produced quarterly by AHCA as part of their Certificate of Need hospital bed inventory process. The July 2010 AHCA Hospital Beds and Services List document was used to identify the hospitals for participation in the Florida Environmental Scan survey (AHCA, 2010). In order to accurately match the hospital characteristics information to the time of the survey, this database was used to determine hospital bed size, ownership status and geographic designation (urban or rural). Florida Hospital Uniform Reporting System Florida requires all hospitals in the state to report detailed information about its patient discharges and financial information. Florida Hospital Uniform Reporting System (FHURS) collects the financial data on an annual basis based on the hospital s fiscal year. This data includes a basic hospital profile as well extensive financial data of hospitals revenues and expenses including their income statement and balance sheet. The 2010 database was used to obtain hospital operating margin data and system affiliation data and was provided by Arlene Schwahn, Office of Data Dissemination, Florida Agency for Health Care Administration. American Hospital Association Annual Survey The annual survey performed by the American Hospital Association contains over 800 hospital-specific data items on hospitals across the country and collects data 58

59 covering organizational structure, personnel, hospital services, and financial performance. Hospitals are identified through state heath care agencies, Medicare and Medicaid participation, and state and local associations. The AHA Annual Survey is completed online by most hospitals in the U.S. with over 5,700 hospitals responding to the survey (85% response rate). The 2009 database was used to calculate the nurse staffing to bed ratio (AHA, 2011). There were 142 Florida hospitals reporting nurse staffing information from the 161 hospitals who participated in the Florida Hospital HIT survey. Study Objective 1 The first objective was to ascertain the level of attainment of Florida hospitals in meeting the CMS Stage I MU objectives according to the guidelines provided by CMS. The Stage 1 MU objectives include 14 core or required objectives and 10 menu objectives. The AHCA hospital survey had questions relating to 13 of the 14 core objectives and all 10 of the menu objectives (the MU measure that required hospitals to report to CMS was excluded from the survey since it was not in effect yet). The estimated total number of MU objectives met per hospital was used to formulate the dependent variable for Study Objective 2 and used as the main predictor or explanatory variable of interest for Study Objective 3. Study Objective 2 The second objective is to identify hospital organizational characteristics that are associated with higher levels of meaningful use among Florida hospitals and is addressed as the HIT Adoption Model. The following variables included in the HIT Adoption Model are described below and variable specifications are provided in Table

60 Dependent Variable The dependent variable is the total number of the MU objectives (MUSum) met by each hospital as reported in the 2010 Florida Hospital HIT Survey. Independent Variables Variables that have been previously documented as influencing HIT adoption and hospital quality (as discussed in Chapter 4) are included in the analysis. These include hospital ownership status (for-profit versus non-profit), system affiliation, geographical location (rural versus urban), hospital size (small, medium, and large), nurse staffing to bed ratio, and operating margin. Hospital operating margin is defined as the difference between the net operating revenue and net operating expense. Additional variables obtained from the Florida Hospital HIT survey was tested including whether having a Chief Medical Information Officer (CMIO) or a high number of hospital IT staff influence HIT adoption. In particular, a HIT staff per bed ratio was used to provide an adequate measure across all hospitals (Hersh & Wright, 2008). Questions regarding these variables were included in the Florida Hospital HIT survey to determine if organizational structure and specialized resources influence hospitals ability to implement HIT. Study Objective 3 The third and main objective of this study is to assess the relationship between hospitals achieving the MU objectives and quality of care and is addressed as the Quality Model. The following variables in Quality Model included in the analysis are described below and variable specifications are provided in Table

61 Dependent Variables The dependent variables or outcomes of interest include 10 hospital process measures that cover 2 clinical conditions pneumonia and heart failure. Pneumonia and circulatory conditions including heart attacks and heart failure are in the top 5 most common causes for hospital admissions patients in the U.S. for those who are 65 and older (Wier et.al, 2010). Using evidence based medicine to address these issues can improve patient outcomes. The following process of care measures represent a subset of best practices for the treatment of these conditions and were selected from the database as having the most qualified responses from hospitals surveyed in Florida. A qualified response indicates that all of the required data elements for that patient s admission were submitted to CMS. These measures are calculated in percentage terms with the numerator as the sum of all eligible cases during the reporting period where the recommended care was provided and the denominator is the sum of all eligible cases (QualityNet, 2010). The following descriptions of measures are abbreviated descriptions from the Hospital Compare Technical Appendix (CMS, 2011). Pneumonia Initial Antibiotic Timing (PNIAT): Pneumonia in-patients receive antibiotics within 6 hours after arrival at the hospital. Appropriate Initial Antibiotic Selection (PNIAS): Immuno-competent patients with pneumonia receive an initial antibiotic regimen that is consistent with current guidelines. Pneumococcal Vaccination Status (PNPVS): Pneumonia inpatients age 65 and older who were screened for pneumococcal vaccine status and were administered the vaccine prior to discharge, if indicated. 61

62 Influenza Vaccination Status (PNIVS): Pneumonia patients age 50 years and older, hospitalized during October through February who were screened for influenza vaccine status and were vaccinated prior to discharge, if indicated. Blood Cultures Performed in the Emergency Department (PNBC): Pneumonia patients whose initial emergency room blood culture specimen was collected prior to first hospital dose of antibiotics. Smoking cessation advice/counseling (PNSC): Pneumonia patients with a history of smoking cigarettes were given smoking cessation advice or counseling during a hospital stay. Heart Failure Evaluation of left ventricular systolic function (HFLVSF): Heart failure patients with documentation that an evaluation of the left ventricular systolic function was performed before arrival, during hospitalization, or is planned for after discharge. ACE inhibitor or ARB for left ventricular systolic dysfunction (HFAIARB): Heart failure patients with left ventricular systolic dysfunction and without contraindications to these medications are prescribed an ACE inhibitor or an ARB at hospital discharge. Discharge instructions (HFDI): Heart failure patients discharged home with written instructions or educational material given to patient or care giver at discharge or during the hospital stay. Smoking cessation advice/counseling (HFSC): Patients with a history of smoking cigarettes are given smoking cessation advice or counseling during a hospital stay. Based on CMS recommendations for the use of these data, hospitals that treated less than 25 qualified patients in a particular measure were excluded from that measure s analysis. Independent variable The MU objectives were designed to demonstrate that proper use of electronic health records would result in better patient care and include many basic functions of EHRs as well some more sophisticated uses. Many of the MU objectives are interrelated such as CPOE for medications and checks for drug-drug interactions or 62

63 drug-allergy interactions and hospitals use these functions together for maximum benefit. The total number of the MU objectives met by each hospital will be used as the independent variable (MUSum). Control Variables Covariates that have been previously documented as influencing HIT adoption and hospital quality (as discussed in Chapter 4) are included as control variables. These include hospital ownership status, system affiliation, geographical location, hospital size, nurse staffing to bed ratio, and operating margin. Statistical Analysis This study was conducted using a retrospective, cross-sectional design linking primary and secondary data sources with the hospital as the unit of analysis. Descriptive and bivariate analyses were performed on key hospital characteristics of hospitals participating in the 2010 Florida Hospital HIT Survey database and include the following variables: MUSum, hospital ownership status, system affiliation, geographical location, hospital size, operating margin, IT staff ratio per bed, RN staff ratio per bed and the existence of a CMIO. To explore the potential for non-response bias, several of these variables (urban/rural, bed size, ownership status, hospital affiliation) were examined to determine whether respondents differed from non-respondents. HIT Adoption Model An analysis was performed to examine the correlations of independent variables and descriptive statistics were calculated using frequencies, means, percentages, and standard deviations for all variables. All 161 hospitals participating in the 2010 Florida Hospital HIT Survey were included in the model. Poisson regression or negative binomial regression techniques are considered the methods of choice for this model 63

64 since the dependent variable MU objectives met is count data. Both Poisson regression and negative binomial regression were performed to test model fit since the distribution is slightly over-dispersed where the variance of the dependent variable exceeds the mean (mean = 14.7 and variance of =18.6). Based on similar HIT adoption models (Kazley & Ozcan, 2007; Hikmet et al., 2008; Jha et al., 2009) the specification for the HIT Adoption Model is provided below: Log e (MUSum) = β 0 +β 1(OWNERSHIP) + β 2(SYSTEM) + β 3(RURAL) + β 4(SIZE) + β 5(NURSESTAFF) + β 6(OPMARGIN) + β 7(CMIO) + β 8(ITSTAFF) + ε Tests for linearity and model fit were performed. An analysis using the sum of only the core MU objectives was also performed since the core MU objectives must be met by all hospitals in order to be eligible for the incentives. This was followed by a sensitivity analysis that converted the outcome measure, MUSum into a binary variable using the mean as the cutoff point which provides a broader view of hospitals who are above average in achieving the MU objectives. Quality Model The Quality Model tests each CMS Hospital Compare measure individually and results in 10 separate analyses. The distributions of the dependent variables are predominantly non-normal and skewed to the left. Generalized Linear Model (GLM) regression was chosen which allows analysis of non-normally distributed dependent variables where models are fitted via Maximum Likelihood estimation. There are 3 components of GLM: 1. Random Component identifies the dependent variable (Y) and specify/assume a probability distribution for it. 2. Systematic Component specifies the explanatory or independent variables. 64

65 3. Link specifies the relationship between the mean or expected value of the random component and the systematic component (Dobson & Barnett, 2008). The random component is the Hospital Compare measures which are proportional data based on a dichotomous response (yes/no) which are best represented as a binomial distribution. The systematic component is the independent variables already described in a previous section and specified below. A logit link was specified due to the proportional nature of the outcome measures and will result in odds ratios being reported for each independent variable. The general specification for the Quality Model that is the basis for each Hospital Compare Measure of interest uses the following basic model to test the hypothesis that hospitals that have achieved a greater number of the CMS MU objectives will perform better on quality of care measures. Logit (Hospital Compare Measure) = β 0 +β 1(MUSum) +β 2(OWNERSHIP) + β 3(SYSTEM) + β 4(RURAL) + β 5(SIZE) + β 6(NURSESTAFF) + β 7(OPMARGIN) + ε Depending on which Hospital Compare measure is tested, the number of hospitals in the dataset varies due to lack of reporting or not meeting the 25 patient per hospital threshold. The primary objective of this study is to assess acute care hospitals current EHR use in terms of the CMS Stage 1 MU objectives and whether meeting these objectives as a whole, is associated with better quality of care. It is believed that having a fully functional, integrated HIT system in use in hospitals will improve care; therefore, this study focused on how many MU objectives hospitals are able to achieve. As noted in Chapter 3, two major components of a hospital EHR system have been found to have a positive effect on quality: CPOE and CDSS. A targeted analysis was performed to 65

66 specifically test the MU objectives for CPOE and CDSS to determine if there is a relationship with the CMS Hospital Compare quality measures. Additionally, for each CMS Hospital Compare quality measure, there may be specific MU objectives that are more likely to influence the provision of care. A more specific analysis for two of the CMS Hospital Compare measures was performed that examined specific MU objectives that are thought to be influential in the process of delivering care. The pneumonia measure Initial Antibiotic Selection (PNIAS) and the heat failure measure evaluation of the left ventricular systolic function (HFLVSF) were selected for this analysis and the MU objectives selected were based on the standard of care for these procedures. Based on the Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults, the following MU objectives were selected for the analysis: demographics, vital signs, problem list, medication list, allergy list, CPOE for medication orders, drug-drug interaction checks, CDSS, drug formulary check, and electronic lab results (Mandell et al., 2007). Rationale for the selection of these objectives include the need to know what the patient s demographics (especially weight and age), vital signs, other health problems, allergies, current medications and blood culture results are in order to select the proper antibiotic and dosage amount and to avoid any allergic reactions or drug interactions. The CPOE, CDSS, and drug formulary objectives are also included since they can provide guidance in the proper ordering and administration of medications. Similarly, for the heat failure measure (HFLVSF), selection of MU objectives was based on the American College of Cardiology s guidelines and includes: demographics, vital 66

67 signs, problem list, CDSS, and electronic lab results. Initial work-up in the emergency department for heart failure patients requires an initial exam (vitals, history, and problem list) and lab tests such as a complete blood count, urinalysis, serum electrolytes, blood urea nitrogen, serum creatinine, fasting blood glucose, lipid profile, liver function tests, and thyroid-stimulating hormone(hunt et al., 2009). Initial work-up also includes an electrocardiogram and two-dimensional echocardiography with Doppler during initial evaluation of patients which should be identified as a recommended procedure through the CDSS as well as the other tests noted above. Sensitivity analysis was performed on the quality models by converting the quality measures into binary variables and testing their association with MUSum and covariates. Lastly, tests for linearity and model fit were performed including testing for selection bias. Selection bias is a possible concern with this model since hospitals that are already high quality providers may be more likely to adopt EHRs than those hospitals that have lower quality. The potential endogeneity problem between MUSum and the quality measures was tested using the Hausman procedure by regressing MUSum on all exogenous variables, then adding the residual as a new variable into the initial structural equation of the quality models (Hausman, 1978).The coefficients for the residuals (MUSum_resid) in each of the quality measure models were not significant, indicating that there is no evidence of endogeneity. 67

68 Table 5-1. CMS Stage 1 Meaningful Use Measures matched to Florida Hospital Survey Measures Core CMS MU Objectives Measure Hospital Survey Measure 1. Record patient demographics Over 50% of patients demographic Please indicate the approximate (sex, race, ethnicity, date of data recorded as structured data percentage who have the following birth, preferred language, date clinical information documented in an and preliminary cause of death) electronic record 2. Record vital signs and chart changes (height, weight, blood pressure, body-mass index, growth charts for children) Over 50% of patients 2 years of age or older have height, weight, and blood pressure recorded as structured data Please indicate the approximate percentage who have vital signs recorded electronically and plotted over time. Survey Response Categories 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 3. Maintain up-to-date problem list of current and active diagnoses Over 80% of patients have at least one entry recorded as structured data 4. Maintain active medication list Over 80% of patients have at least one entry recorded as structured data 5. Maintain active medication allergy list 6. Record smoking status for patients 13 years of age or older 7. Computer provider order entry (CPOE) for medication orders Over 80% of patients have at least one entry recorded as structured data Over 50% of patients 13 years of age or older have smoking status recorded as structured data Over 30% of patients with at least one medication in their medication list have at least one medication ordered through CPOE Please indicate the approximate percentage who have the following clinical information documented in an electronic record. Please indicate the approximate percentage who have the following clinical information documented in an electronic record. Please indicate the approximate percentage who have the following clinical information documented in an electronic record. Out of all of your hospital s patients who have an electronic health record & are 13 years old or older, please indicate the approximate percentage who have their smoking status recorded. Please indicate the approximate percentage who have at least one medication ordered through CPOE. 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 8. Implement drug drug and drug allergy interaction checks Functionality is enabled for these checks for the entire reporting period Do your hospital s current computer systems have drug-allergy interaction checks enabled? Do your hospital s current computer systems have drugdrug interaction checks enabled? Yes No Unsure 68

69 Table 5-1. Continued Core CMS MU Objectives Measure Hospital Survey Measure 9. Implement one clinical decision One clinical decision support rule support rule and ability to track implemented compliance with the rule 10. Report clinical quality measures to CMS or states 11. Provide an electronic copy of hospital discharge instructions on request 12. On request, provide patients with an electronic copy of their health information (including diagnostic-test results, problem list, medication lists, med. allergies, and for hospitals, discharge summary and procedures) 13. Implement capability to electronically exchange key clinical information among providers and patientauthorized entities 14. Implement systems to protect privacy and security of patient data in the EHR For 2011, provide aggregate numerator and denominator through attestation; for 2012, electronically submit measures Over 50% of all patients who are discharged from the inpatient or emergency department of an eligible hospital or critical access hospital and who request an electronic copy of their discharge instructions are provided with it Over 50% of requesting patients receive electronic copy within 3 business days Perform at least one test of EHR s capacity to electronically exchange information Conduct or review a security risk analysis, implement security updates as necessary, and correct identified security deficiencies Do your hospital s computer systems have at least one clinical decision support rule (such as an alert, reminder, diagnostic support or clinical guideline) related to a high priority hospital condition? Ability to provide patients with an electronic copy of discharge instructions at time of discharge, upon request. Ability to provide patients with an electronic copy of their health information, upon request. Do your hospital s current computer systems exchange (send or receive) clinical data or other information (e.g. clinical messages) with any of the following? Does your hospital conduct or review a security risk analysis and implement security updates as necessary in order to protect electronic health information? Survey Response Categories Yes No Unsure 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% List of entities Yes No Unsure 69

70 Table 5-1. Continued Menu CMS MU Objectives Measure Hospital Survey Measure 1. Implement drug formulary checks 2. Incorporate clinical laboratory test results into EHRs as structured data 3. Generate lists of patients by specific conditions to use for quality improvement, reduction of disparities, research, or outreach 4. Record advance directives for patients 65 years of age or older 5. Use EHR technology to identify patient-specific education resources and provide those to the patient as appropriate 6. Perform medication reconciliation between care settings Drug formulary check system is implemented and has access to at least one internal or external drug formulary for the entire reporting period Over 40% of clinical laboratory test results whose results are in positive/negative or numerical format are incorporated into EHRs as structured data Generate at least one listing of patients with a specific condition Over 50% of patients 65 years of age or older have an indication of an advance-directive status recorded Over 10% of patients are provided patient-specific education resources Medication reconciliation is performed for over 50% of transitions of care Do your hospital s current computer systems have drug-formulary checks enabled? Out of all clinical lab test results ordered by an authorized provider for patients (inpatient or emergency) in your hospital, please indicate the approximate percentage for which the results are recorded electronically. Do your hospital s computer systems have the capability of generating lists of patients with a given condition (e.g., admitted patients with a diagnosis of pneumonia)? Please indicate the approximate percentage who have an indication of their advanced directive status documented electronically. Please indicate what approximate percentage of your total patients received patient-specific education resources through an EHR or other computerized systems. Do your hospital s current computer systems support medication reconciliation at transitions of care (such as inpatient admission, emergency admission, or discharge to another provider s care)? Survey Response Categories Yes No Unsure 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% Yes No Unsure 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% 0, 1-25%, 26-50%, 51-75%, 76-99%, 100% Yes No Unsure 70

71 Table 5-1. Continued Menu CMS MU Objectives Measure Hospital Survey Measure Summary of care record is provided for over 50% of patient transitions or referrals 7. Provide summary of care record for patients referred or transitioned to another provider or setting Do your hospital s current computer systems allow summary of care records to be shared (in either electronic or paper form) when patients are transitioned to other providers or care settings? Survey Response Categories Yes No Unsure 8. Submit electronic immunization data to immunization registries or immunization information systems 9. Submit electronic syndromic surveillance data to public health agencies Perform at least one test of data submission and follow-up submission (where registries can accept electronic submissions) Perform at least one test of data submission and follow-up submission (where public health agencies can accept electronic data) Do your hospital s current computer systems exchange (send or receive) clinical data or other information (e.g. clinical messages) with any of the following? (Please check all that apply) Do your hospital s current computer systems exchange (send or receive) clinical data or other information (e.g. clinical messages) with any of the following? (Please check all that apply) List of entities List of entities 10. Submit electronic data on reportable laboratory results to public health agencies Perform at least one test of data submission and follow-up submission (where public health agencies can accept electronic data) Do your hospital s current computer systems exchange (send or receive) clinical data or other information (e.g. clinical messages) with any of the following? (Please check all that apply) List of entities 71

72 Table 5-2. Description and categorization of variables Dependent Description Type Identification Variables MUSum Sum of MU objectives met Count Total number of MU objectives met by each hospital MUCore Sum of core MU objectives met Count Total number of core MU objectives met by each hospital PNIAT PNIAS PNPVS PNIVS PNBC PNSC HFLVSF HFAIARB HFDI HFSC Pneumonia - Initial Antibiotic Timing Pneumonia - Appropriate Initial Antibiotic Selection Pneumonia - Pneumococcal Vaccination Status Pneumonia - Influenza Vaccination Status Pneumonia - Blood Cultures Performed in the Emergency Department Pneumonia - Smoking cessation advice/counseling Heart Failure - Evaluation of left ventricular systolic function Heart Failure - ACE inhibitor or ARB for left ventricular systolic dysfunction Heart Failure - Discharge instructions Heart Failure - Smoking cessation advice/counseling Proportional (%) Proportional (%) Proportional (%) Proportional (%) Proportional (%) Proportional (%) Proportional (%) Proportional (%) Proportional (%) Proportional (%) Pneumonia cases who received antibiotics within 6 hrs./all eligible pneumonia cases in each hospital Pneumonia cases who received appropriate antibiotics./all eligible pneumonia cases in each hospital Pneumonia cases who has been vaccinated./all eligible pneumonia cases in each hospital Pneumonia cases who has been vaccinated/all eligible pneumonia cases in each hospital Pneumonia cases who had blood cultures done in the ED./all eligible pneumonia cases in each hospital Pneumonia cases who received smoking cessation advice/all eligible pneumonia cases in each hospital HF cases with eval. of LVSF/all eligible HF cases in each hospital HF cases received ACEI or ARB at discharge/all eligible HF cases in each hospital HF cases with discharge instructions/all eligible HF cases in each hospital HF cases who received smoking cessation advice/all eligible HF cases in each hospital 72

73 Table 5-2. Continued Independent Variables Description Type Identification MUSum Sum of MU objectives met Count Total number of MU objectives met by each hospital OWNERSHIP Ownership - for-profit or nonprofit Dummy For-profit=1 Nonprofit =0 SYSTEM System affiliation Dummy Affiliated = 1 Not affiliated = 0 RURAL Geographical location Dummy Urban = 1 Rural = 0 SIZE Hospital bed size Categorical Small =150 beds or less Medium = 151 to 350 beds Large = >351 beds NURSESTAFF Nurse staffing to bed ratio Continuous Total nurse FTEs/beds OPMARGIN Operating Margin Continuous Net operating revenue - net operating expense. CMIO ITSTAFF Chief Medical Information Officer on staff Number of IT staff employed per hospital bed Dummy Yes = 1 No = 0 Continuous Ratio of IT staff per bed 73

74 CHAPTER 6 RESULTS The results of the study are presented in three sections. The first section provides the hospital characteristics of Florida hospitals who responded to the HIT survey. The second section provides the results of the HIT Adoption analysis which examines key hospital organizational characteristics that may be positively associated with higher levels of EHR adoption. The third section assesses the relationship between hospitals achieving the MU objectives as a whole and quality of care using the CMS Hospital Compare quality measures. Specific MU objectives are also tested to see if there is a relationship to the quality measures. Descriptive Statistics The survey population included all 211 acute care hospitals in Florida and excluded specialty hospitals, long-term care hospitals and federal hospitals (VA, military) of which 161 hospitals responded (76% response rate). Table 6-1 provides the hospital characteristics of those hospitals who responded to the survey compared to the population of acute care hospitals in Florida. The survey respondents characteristics were similar as the population as a whole which indicated no need for weighting the survey results. Of those hospitals that responded to the survey, 32.9% (53) were small hospitals with 150 beds or less, 37.9% (61) were medium size hospitals with beds between 151 to 350, and 29.2%(47) were large hospitals with beds greater than 350. The majority of hospitals were located in urban areas (87.0%), not-for-profit (63.4%) and affiliated with a hospital system (76.4%). Non-respondents tended to be smaller, more rural hospitals and/or for-profit hospitals. 74

75 Other sample characteristics included in the study include hospitals operating margin (mean 3%, std. deviation 12%), Information Technology (IT) staff per bed ratio (mean 0.12 staff, std. deviation 0.11) and nurse staff per bed ratio (1.2 staff, 0.64 std. deviation). There were 101 hospitals (62.7%) who had responded that they had a Chief Medical Information Officer (CMIO) at their facility. To ascertain the level of attainment of CMS Stage I MU objectives among Florida hospitals (Objective 1), a histogram of the number of MU objectives met is presented in Figure 6-1. Only two hospitals reported that they are able to meet all the core MU objectives and at least 5 of the menu objectives. The mean number of MU objectives met is 14.7 and the median is 17 with a minimum of 0 and a maximum of 22. An examination of individual MU objectives depicted in Figure 6-2 show that most hospitals were able to achieve two core objectives, demographic information (94% of all hospitals surveyed), and security controls (94%), and 3 menu objectives: electronic lab results (96%), generate patient lists with specific conditions (94%), and generate patient specific education (93%). Hospitals were least likely to achieve the following MU objectives: electronic discharge instructions (20%), electronic copy of records to patients (30%), CPOE (30%), and submit electronic immunization data to registries (25%). HIT Adoption Analysis The HIT Adoption analysis examined key hospital organizational characteristics that may be positively associated with higher levels of EHR adoption. A bivariate analysis was performed on the individual hospital characteristics as well as a multivariate analysis which included subcategories of some of these hospital characteristics. Furthermore, an analysis of only the core or required MU objectives 75

76 was performed to examine characteristics of hospitals who are achieving high levels of these objectives. Bivariate Statistics Bivariate analysis were performed using simple Poisson regression to test the association between the total number of MU objectives met (MUSum) and each hospital characteristic. As reported in Table 6-2, these results indicate that there is a statistically significant positive relationship between MUSum and hospitals affiliated with a hospital system (p< 0.01), hospitals located in an urban area (p< 0.01), and hospitals with a CMIO, (p< 0.01). However, there were no statistically significant relationships between MUSum and hospital profit status or hospital size except with medium size hospitals (p< 0.01). Additionally, there was a positive statistical relationship with MUSum and hospital operating margin (p = 0.01) and a marginally positive relationship with nurse staffing per bed ratio (p = 0.06) but not with the IT staff per bed ratio. Multivariate Analysis Since the dependent or outcome variable, MUSum is a count variable whose variance was slightly greater than the mean, both negative binomial regression and Poisson regression were performed to determine model fit. The results of the two regressions were nearly identical with the likelihood ratio test of the over dispersion parameter alpha for the negative binomial regression equal to zero indicating that it is equivalent to the Poisson distribution. The LR Chi-square statistic and the Pseudo R 2 were larger for the Poisson model than the negative binomial model which led to the selection of the Poisson regression for the remainder of the analysis. The final Poisson model had a LR Chi-square(10) of and a Pseudo R 2 of

77 Diagnostics were performed to check the fit between the data and the assumptions of Poisson regression. The Hosmer-Lemeshow test provided evidence of linearity (p=0.72) while visual inspection of the residuals versus predicted values showed signs of heteroskedasticity at the lower values (Figure 6-3). However, the Poisson regression incorporates observed heterogeneity into the Poisson distribution function, Var(y/x) = E(y/x) = mμ =exp(xβ), where as the mean increases, the variance increases (unlike OLS which assumes constant variance). The dispersion of data increases as μ increases, thus the errors in a Poisson regression are inherently heteroskedastic (Maddala, 1983). Table 6-3 summarizes the results of the regression analysis for the HIT Adoption model with the incidence rate ratios (IRR) reported for easier interpretation of results. The results indicate several significant positive relationships including urban location (IRR= 1.32, p<0.01), system affiliation (IRR = 1.22, p<0.01), and the presence of a CMIO on staff (IRR= 1.16, p<0.01). These results show that urban hospitals have about 32% or 4.7 more MU objectives met than rural hospitals, holding covariates constant. Hospitals affiliated with a hospital/healthcare system have 22% or 3.2 more MU objectives met than non-affiliated hospitals. System affiliation was further categorized to examine the differences between local/regional systems and national systems. Results indicate that hospitals in a regional system have 20% or 2.9 more MU objectives (IRR = 1.20, p<0.01) than non-affiliated hospitals while hospitals in a national system have 30% or 4.4 more MU objectives (IRR = 1.30, p<0.01) than non-affiliated hospitals. Finally, hospitals with CMIOs on staff have 16% or 2.4 more MU objectives (IRR = 1.16, p<0.01) than hospitals without a CMIO. Large hospital size and for-profit 77

78 ownership had negative relationships with the number of MU objectives met, although the observed coefficients are not statistically significant. Stage 1 MU objectives are categorized as Core or required objectives and Menu objectives. For the latter, hospitals have a choice of selecting and reporting on 5 of the 10 possible measures. Further analysis of the sum of the Core MU measures as the dependent variable (MUCore) are depicted in Table 6-4 and show only two significant positive relationships: geographic location (IRR= 1.38, p=0.01), regional system affiliation (IRR = 1.18, p= 0.05). Unlike the original analysis, the CMIO relationship proved not to be significant for the number of Core MU objectives met. Sensitivity Analysis The number of MU objectives hospitals achieved ranged from zero to 22 with a mean of 14.7 and a median of 17. Based on the mean, a cutoff point of 15 was selected to convert MUSum into a binary variable. In order to analyze more closely what the relationships are between the high HIT adopters (those who met 15 or more MU objectives) and hospital characteristics, both a probit and logit regression were performed after converting MUSum into a binary variable. The results between the two analyses were very similar and the results of the logistic regression are summarized in Table 6.5. The results indicate several significant positive relationships including hospital size-medium hospital (OR = 3.55, p=0.05), national system affiliation (OR = 6.00, p=0.04), and the presence of a CMIO on staff (OR = 3.20, p=.02). This analysis differed from the original analysis with the addition of medium hospital size as a significant variable and the loss of regional system affiliation and urban location as a significant variable. Upon examination of the odds ratios, medium size hospitals have 3.6 times the odds of being high HIT adopters (those who met 15 or more MU 78

79 objectives) than small hospitals and hospitals in a national system have 6.0 times the odds of being high HIT adopters than non-system affiliated hospitals. Hospitals with CMIOs on staff are 3.2 times the odds of being high HIT adopters than hospitals without a CMIO. In comparing the MU objectives met by low and high HIT adopters (Figure 6-4), hospitals that are considered low adopters were least likely to achieve the following MU objectives: core objective 7 electronic discharge instructions (10%), core objective 8 electronic copy of records to patients (10%), core objective 9 CPOE (6%), and menu objective 7 submit electronic immunization data to registries (5%). In summary, the results of the HIT Adoption analysis indicate several significant relationships that may influence Florida hospitals level of HIT adoption, including urban location, system affiliation, and the presence of a CMIO on staff. The sensitivity analysis confirmed two of these relationships, system affiliation and the presence of a CMIO on staff but not the urban versus rural relationship. Additionally, the sensitivity analysis indicated that medium size hospitals (151 to 350 beds) are associated with high adoption of HIT. Differences in the two analyses are based on the statistical method where Poisson regression is examining the differences at each count level while the logistic regression examined the differences of hospitals who achieved 15 or more MU objectives versus hospitals that achieved less than 15 MU objectives. Quality Measures Analysis The third and main objective of this study is to assess the relationship between hospitals achieving the MU objectives and quality of care using the CMS Hospital Compare quality measures. The Quality Model tests each CMS Hospital Compare measure individually with HIT adoption (MUSum) and results in 10 separate analyses. 79

80 Bivariate Statistics The CMS Hospital Compare quality measures indicate the percentage that each hospital is able to achieve for all qualified patients with pneumonia or heart failure. They are calculated in percentage terms with the numerator as the sum of all eligible cases during the reporting period where the recommended care was provided and the denominator is the sum of all eligible cases (QualityNet, 2010). Table 6-6 provides a description of the distribution of each of the Hospital Compare measures which shows low variability with many of the measures having small ranges and mean percentages in the high 90s (left skewed distributions). Bivariate analysis were performed to test the association between each Hospital Compare measure and the total number of MU objectives met (MUSum). As reported in Table 6-7, these results indicate that there is a statistically positive significant relationship between MUSum and four pneumonia measures: PNIAT- Initial Antibiotic Timing (p = 0.02), PNIAS - Appropriate Initial Antibiotic Selection (p = 0.01), PNPVS - Pneumococcal Vaccination Status (p = 0.02), and PNSC - Smoking Cessation advice/counseling (p = 0.02). A statistically significant relationship was also found between MUSum and one heart failure measure - evaluation of left ventricular systolic function (HFLVSF) with a p value of < Multivariate Analysis A multivariate analysis was performed on each CMS Hospital Compare measure. The control variables in these analyses are similar to the hospital characteristics variables in the HIT Adoption model. Control variables include hospital ownership status, system affiliation, geographical location, hospital size (beds), nurse staffing to bed ratio, and operating margin. The IT staff ratio and presence of CMIO were not 80

81 included in the analysis since theoretically, IT staff and having a Chief Medical Information Officer should not have a direct relationship with hospitals providing the appropriate standard of care for pneumonia and heart failure patients. Furthermore, bivariate analysis with these two variables and the quality measures showed no significant correlation. The multivariate models employed GLM with a binomial family and a logit link due to the proportional nature of the dependent variables. The dependent variables, the CMS Hospital Compare measures are calculated in percentage terms with the numerator as the sum of all eligible cases during the reporting period where the recommended care was provided and the denominator is the sum of all eligible cases. Therefore, for each case, it is a binary variable where the patient received the recommended care or they did not. Tests for linearity and model fit showed heteroskedacity for all of the quality measures and some non- normality of residuals for a few of the models especially at the tail ends of the distributions (see Figures A-1 through A-20 in Appendix). Therefore, robust standard errors were used because the models do not meet standard assumptions. The Hosmer-Lemeshow goodness of fit test provided evidence of linearity for all of the quality measures analyses (all Chi- square p values were not significant). A summary of the results of all ten multivariate analyses of MUSum with each CMS Hospital Compare measure are presented in Table 6-8 and show a marginally significant positive association between MUSum and three of the pneumonia quality measures: PNIAT- Initial Antibiotic Timing (OR = 1.03, p = 0.08), PNPVS - 81

82 Pneumococcal Vaccination Status (OR = 1.05, p = 0.05), and PNSC - Smoking Cessation advice/counseling (OR = 1.09, p = 0.06). One heart failure measure, HFLVSF - evaluation of left ventricular systolic function (OR = 1.08, p=0.01), shows a significant positive association. The odds ratio in this analysis is interpreted as one additional MU objective met by a hospital is associated with an X change in the odds of a given patient receiving the care specified by the quality measure. For example, using the HFLVSF results with an odds ratio of 1.08, a hospital having one additional MU objective met results in an 8% increase in the odds that a given heart failure patient in that hospital will receive the prescribed standard of care for the evaluation of the left ventricular systolic function. The probability of receiving the recommended care for HFLVSF is 98%, which is the odds of.98/(1-.98) or 49 to 1. Therefore, if we increase the odds by 8% we get 52.9 resulting in the odds of getting the recommended care for HFSLVSF is now 52.9 to 1 for a given heart failure patient. The complete multivariate regression results for each measure is provided in Tables 6-9 and 6-10 and show covariates that have a significant positive impact on a majority of the quality measures: urban location for six of the measures (PNIAS, PNPVS, PNBC, PNSC, HFAIARB, HFSC), for-profit ownership for eight of the measures (PNIAT, PNIAS, PNPVS, PNIVS, PNSC, HFAIARB, HFDI, HFSC), affiliation with a hospital system for all pneumonia measures and two heart failure measures HFLVF, HFDI), and operating margin for six of the measures (PNIAT, PNIAS, PNPVS, PNIVS, PNBC, HFLVSF). The nurse staffing ratio has a negative significant association with all but two measures (PNPVS, HFLVSF) which implies that as the nurse staffing increases, the ability to meet these measures decreases. 82

83 The model was revised to include the expanded versions of the control variables bed size (small, medium and large hospitals) and system affiliation (local and national) similar to those used in the HIT Adoption model, and a summary of these analyses is presented in Table It showed a marginally significant positive association between MUSum and three of the quality measures: PNPVS - Pneumococcal Vaccination Status (OR = 1.05, p = 0.10), and PNSC - Smoking Cessation advice/counseling (OR = 1.09, p = 0.08), and HFLVSF - evaluation of left ventricular systolic function (OR = 1.08, p = 0.01). The addition of four more variables in the models may have resulted in a decrease in the power to detect significant differences. The results of the complete regression analyses are presented in Tables 6-12 and 6-13 and show covariates that have a significant positive impact on a majority of the quality measures include: seven measures for affiliation with a regional system (PNIAT, PNIAS, PNPVS, PNIVS, PNSC, HFLVSF, HFDI), and affiliation with a national system for all but one measure (HFAIARB). The addition of the expanded hospital size and affiliation variables reduced some of the associations with the quality measures but system affiliation (whether regional or national) remained strongly predictive of quality. Sensitivity Analysis Sensitivity analysis was performed on the quality models by converting the quality measures into binary variables and testing their association with MUSum and covariates. Two separate analyses were performed using different cutoff points. One analysis examined only those who reported 100% on the quality measure and the other analysis used those who reported 99% or above on the measure. The 100% cutoff point analysis led to four of the ten quality measures having less than 20 cases positive for this measure and three of the logistic regression models did not achieve model 83

84 significance (Prob> Chi 2 of 0.05) due to separation. Separation occurs when one or more predictors perfectly predict the outcome. Therefore, the cutoff point of 99% or above was used. The results of the analysis are provided in Table The results showed a significant positive association between MUSum and three of the quality measures: PNIAT (OR = 1.19, p = 0.07), HFLVSF (OR = 1.12, p = 0.06), and HFAIARB (OR = 1.18, p = 0.02) and a negative association with HFDI (OR = 0.89, p = 0.06). The regression model for HFSC did not achieve model significance due to a separation issue with two variables. A targeted analysis was performed to specifically test the MU objectives related to CPOE and CDSS to determine if there is a relationship with the Hospital Compare quality measures. Table 6-15 shows that CPOE has a significant positive association with two quality measures, PNIAT (OR = 1.69, p< 0.01) and PNIAS (OR = 1.52, p= 0.01) while CDSS has a negative association with PNIVS (OR = 0.58, p= 0.09) and HFDI (OR = 0.40, p= 0.01). Specified Models Two quality measures were selected for a more detailed analysis: initial antibiotic selection for Pneumonia (PNIAS) and evaluation of the left ventricular systolic function (HFLVSF). Particular MU objectives were identified for each quality measure and summed into one variable to maintain power in the regression analysis. Tables 6-16 and 6-17 provide the results for these analyses. For the PNIAS model, the specified MU objectives show a significant positive association with the PNIAS quality measure (OR = 1.07, p value = 0.03). However, the HLVSF specified model did not show a significant association with the quality measure even though the original analysis with all of the MU objectives showed positive results. 84

85 Table 6-1. Survey respondents and population characteristics Hospital Characteristic All Florida acute care hospitals (N=211) Survey respondents (N=161) Survey nonrespondents (N=50) Location Urban 85.3% 87.0% 80.0% Rural 14.7% 13.0% 20.0% System Affiliation Affiliated 75.8% 76.4% 68.2% Not Affiliated 24.2% 23.6% 31.8% Size Small (150 beds or less) 35.2% 32.9% 40.1% Medium (151 to 350 beds) 40.0% 37.9% 43.6% Large (>350 beds) 24.8% 29.2% 16.3% Ownership For-profit 42.9% 36.6% 38.1% Non-profit 57.1% 63.4% 51.9% Chief Medical Information Officer Yes Not available 62.7% Not available No Not available 37.3% Not available Other Hospital Characteristics (mean) Operation Margin 0.03^ IT staff per bed ratio Not available 0.12 Not available Nurse staff per bed ratio 1.28^ ^Sample sizes vary due to missing data (N= 187 for operation margin, N= 178 for nurse staff per bed ratio) 85

86 Table 6-2. Bivariate statistics: hospital characteristics by MUSum Hospital Characteristic MuSum (mean) P Value Location Urban <0.01 Rural System Affiliation Affiliated <0.01 Not Affiliated Size Small (150 beds or less) Medium (151 to 350 beds) <0.01 Large (>350 beds) Ownership For-profit Non-profit Chief Medical Information Officer Yes 15.8 <0.01 No 12.9 Hospital Characteristic Correlation (r) P value Operation Margin IT staff per bed ratio Nurse staff per bed ratio Note: bivariate analysis were done with Poisson regression except for operating margin, IT staff ratio and Nurse staff ratio (Pearson correlation) 86

87 Table 6-3. Poisson regression estimates for Total MU objectives met (MUSum) Variable IRR Std. Error P Value Size Small (150 beds or less) Reference Medium (151 to 350 beds) Large (>350 beds) Ownership Non-profit Reference For-profit System Affiliation Not Affiliated Reference Affiliated <0.01 Regional <0.01 National <0.01 Location Rural Reference Urban <0.01 Chief Medical Information Officer No Reference Yes <0.01 Operation Margin IT staff per bed ratio Nurse staffing to bed ratio Two models were performed one with system affiliation (yes/no) and one with system affiliation in 3 categories (none, regional, national). 87

88 Table 6-4. Poisson regression estimates for core MU objectives met (MUCore) Variable IRR Std. Error P Value Size Small (150 beds or less) Reference Medium (151 to 350 beds) Large (>350 beds) Ownership Non-profit Reference For-profit System Affiliation Not Affiliated Reference Affiliated Regional National Location Rural Reference Urban Chief Medical Information Officer No Reference Yes Operation Margin IT staff per bed ratio Nurse staffing to bed ratio Two models were performed one with system affiliation (yes/no) and one with system affiliation in 3 categories (none, regional, national). 88

89 Table 6-5. Logistic regression estimates for binary MUSum Variable OR Std. Error P Value Size Small (150 beds or less) Reference Medium (151 to 350 beds) Large (>350 beds) Ownership Non-profit Reference For-profit System Affiliation Not Affiliated Reference Affiliated Regional National Location Rural Reference Urban Chief Medical Information Officer No Reference Yes Operation Margin IT staff per bed ratio Nurse staffing to bed ratio Two models were performed one with system affiliation (yes/no) and one with system affiliation in 3 categories (none, regional, national). 89

90 Table 6-6. Dependent variable characteristics Variable N Mean Minimum Maximum Std. Dev. Total MU objectives met (MUSum) Total Core MU objectives met (MUCore) Pneumonia - Initial Antibiotic Timing (PNIAT) Pneumonia - Appropriate Initial Antibiotic Selection (PNAIS) Pneumonia - Pneumococcal Vaccination Status (PNPVS) Pneumonia - Influenza Vaccination Status (PNIVS) Pneumonia - Blood Cultures Performed in the Emergency Department (PNBC) Pneumonia - Smoking cessation advice/counseling (PNSC) Heart Failure - Evaluation of left ventricular systolic function (HFLVSF) Heart Failure - ACE inhibitor or ARB for left ventricular systolic dysfunction (HFAIARB) Heart Failure - Discharge instructions (HFDI) Heart Failure - Smoking cessation advice/counseling (HFSC)

91 Table 6-7. Bivariate statistics: Hospital Compare measures and MUSum Robust Dependent Variable Coeff. Std. Error P Value PNIAT PNAIS PNPVS PNIVS PNBC PNSC HFLVSF <0.01 HFAIARB HFDI HFSC Table 6-8. Summary of GLM regression estimates for Hospital Compare measures Robust Dependent Variable OR Std. Error P Value PNIAT PNAIS PNPVS PNIVS PNBC PNSC HFLVSF HFAIARB HFDI HFSC Note: These results represent 10 separate statistical models one for each quality measure. All GLM regressions used a binomial distribution with logit link. 91

92 Table 6-9. GLM regression estimates for Pneumonia quality measures Variable PNIAT PNIAS PNPVS PNIVS PNBC PNSC MUSum 1.03 (0.02)* 1.02 (0.02) 1.05 (0.03)** 1.02 (0.03) 1.01 (0.02) 1.09 (0.05)* Urban Location 1.27 (0.39) 1.98 (0.42)*** 2.47 (1.11)** 1.91 (0.76) 2.02 (0.45)*** 7.90 (4.65)*** Size (Beds) 1.00 (0.00)** 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00)** 1.00 (0.00)** For-profit Ownership 1.96 (0.37)*** 1.54 (0.26)** 2.02 (0.75)* 2.06 (0.76)* 1.42 (0.41) 5.45 (3.47)** Affiliation with Hospital System 1.66 (3.97)** 2.01 (0.33)*** 2.46 (0.75)*** 2.44 (0.71)*** 1.55 (0.40)* 4.45 (2.09)*** Operation Margin 6.84 (1.97)*** 2.93 (1.81)* (15.9)*** 7.13 (5.47)** 3.71 (2.51)** 0.51 (1.09) Nurse staffing to bed ratio 0.83 (0.07)** 0.81 (0.08)** 0.83 (0.12) 0.77 (0.07)** 0.79 (0.11)* 0.67 (0.15)* Results reported as Odds Ratio (robust std error) Significance levels: * p < 0.10, ** p < 0.05, *** p < 0.01 Table GLM regression estimates for heart failure quality measures Variable HFLVSF HFAIARB HFDI HFSC MUSum 1.08 (0.03)** 1.03 (0.03) 1.00 (0.02) 1.01 (0.06) Urban Location 2.10 (1.14) 2.12 (0.68)** 1.02 (0.55) 4.13 (2.76)*** Size (Beds) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) For-profit Ownership 1.18 (0.55) 2.64 (0.81)*** 2.17 (0.63)** 9.6x10 5 (5.0x10 6 )*** Affiliation with Hospital System 2.74 (1.13)** 1.41 (0.36) 2.22 (0.54)*** 1.78 (1.14) Operation Margin (0.01)** 3.94 (4.70) 1.74 (1.72) 0.82 (1.46) Nurse staffing to bed ratio 0.72 (0.16) 0.81 (0.09)* 0.70 (0.08)*** 0.48 (0.06)*** Results reported as Odds Ratio (robust std error) Significance levels: * p < 0.10, ** p < 0.05, *** p <

93 Table Summary of GLM regression estimates for Hospital Compare measures (expanded model) Robust Dependent Variable OR Std. Error P Value PNIAT PNAIS PNPVS PNIVS PNBC PNSC HFLVSF HFAIARB HFDI HFSC Note: These results represent 10 separate statistical models one for each quality measure. 93

94 Table GLM regression estimates for Pneumonia quality measures (expanded model) Variable PNIAT PNIAS PNPVS PNIVS PNBC PNSC MUSum 1.02 (0.02) 1.01 (0.02) 1.05 (0.03)* 1.01 (0.03) 1.01 (0.02) 1.09 (0.05)* Location Rural Reference Reference Reference Reference Reference Reference Urban 1.16 (0.35) 1.82 (0.46)** 1.88 (1.02) 2.04 (1.14) 1.34 (0.41) 4.43 (3.05)** Size Small (150 beds or less) Reference Reference Reference Reference Reference Reference Medium (151 to 350 beds) 0.87 (0.18) 1.04 (0.22) 1.11 (0.44) 0.77 (0.34) 1.01 (0.30) 1.16 (1.49) Large (>350 beds) 0.69 (0.13)* 0.87 (0.17) 0.92 (0.37) 0.66 (0.27) 1.05 (0.33) 1.14 (0.70) Ownership Non-profit Reference Reference Reference Reference Reference Reference For-profit 1.26 (0.33) 1.14 (0.22) 1.47 (0.92) 1.38 (0.92) 0.94 (0.43) 3.71 (2.42)** System Affiliation Not Affiliated Reference Reference Reference Reference Reference Reference Affiliated Regional 1.45 (0.31)* 1.82 (0.33)*** 2.17 (0.74)** 2.16 (0.61)* 1.30 (0.30) 4.25 (2.02)*** National 3.07 (0.83)*** 2.99 (0.79)*** 4.43 (2.81)** 4.20 (3.01)** 3.05 (1.53)** (21.73)*** Operation Margin 7.61 (5.12)*** 2.84 (1.84) (17.42)** 7.62 (6.91)** 3.67 (3.64) 0.45 (0.92) Nurse staffing to bed ratio 0.85 (0.08)* 0.84 (0.08)* 0.85 (0.12) 0.79 (0.07)** 0.78 (0.12) 0.71 (0.16) Results reported as Odds Ratio(robust std error) Significance levels: * p < 0.10, ** p < 0.05, *** p <

95 Table GLM regression estimates for heart failure quality measures (expanded model) Variable HFLVSF HFAIARB HFDI HFSC MUSum 1.08 (0.03)*** 1.03 (0.03) 1.00 (0.02) 1.00 (0.06) Location Rural Reference Reference Reference Reference Urban 1.26 (0.74) 2.16 (1.00)* 1.27 (0.71) 5.36 (5.33)*** Size Small (150 beds or less) Reference Reference Reference Reference Medium (151 to 350 beds) 2.72 (1.12)** 1.00 (0.41) 0.83 (0.27) 1.00 (1.13) Large (>350 beds) 2.65 (1.13)** 1.03 (0.38) 0.74 (0.22) 0.78 (0.85) Ownership Non-profit Reference Reference Reference Reference For-profit 1.04 (0.65) 2.57 (1.38)* 1.53 (0.61) 2.3x10 5 (1.4x10 5 ) *** System Affiliation Not Affiliated Reference Reference Reference Reference Affiliated Regional 2.36 (0.96)** 1.39 (0.40) 2.05 (0.54)* 1.67 (1.07) National 4.06 (2.82)** 1.43 (0.81) 3.11 (1.25)** 4.2 x10 5 (2.6x10 5 )*** Operation Margin (29.91)*** 3.86 (4.43) 1.84 (1.79) 0.87 (1.66) Nurse staffing to bed ratio 0.91 (0.21) 0.81 (0.09)* 0.72 (0.08)* 0.50 (0.07)*** Results reported as Odds Ratio(robust std error) Significance levels: * p < 0.10, ** p < 0.05, *** p <

96 Table Logistic regression estimates for Binary CMS hospital quality measures 99% and above Variable OR Std. Error P Value PNIAT PNAIS PNPVS PNIVS PNBC PNSC HFLVSF HFAIARB HFDI HFSC Regression model not significant Table CPOE and CDSS analysis with CMS hospital quality measures CPOE CDSS Variable OR Std. Error P Value OR Std. Error P Value PNIAT < PNAIS PNPVS PNIVS PNBC PNSC HFLVSF HFAIARB HFDI HFSC Note: GLM regressions using a binomial distribution with logit link. 96

97 Table Specified model for PNIAS Variable OR Std. Error P Value PNIAS Specific MU objectives Urban Location <0.01 Size (Beds) For Profit Ownership Affiliation with Hospital System <0.01 Operation Margin Nurse staffing to bed ratio Note: GLM regressions using a binomial distribution with logit link. Table Specified model for HFLVSF Variable OR Std. Error P Value HFLVSF Specific MU objectives Urban Location Size (Beds) For-profit Ownership Affiliation with Hospital System Operation Margin Nurse staffing to bed ratio Note: GLM regressions using a binomial distribution with logit link. 97

98 elabs demog security pt lists pt ed CDSS allergies med list smoke adv direct xchng data vitals med recon care summ reportable labs syndromsurv drugchecks prob list drug formchk Pt copy CPOE immun reg discharge inst % Hospitals Met Measure 0 5 Frequency MUSum Figure 6-1. Total number of MU objectives met by Florida hospitals 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Figure 6-2. Percent of each MU objectives met by Florida hospitals 98

99 -10-5 residual predicted mean total_score Figure 6-3. Residual versus fitted plot after Poisson regression Figure 6-4. Percent of each MU Objective met by low and high HIT Adopters 99