The Impact of Electronic Health Record Adoption and Integration on Physician Productivity and Health Outcomes

Transcription

1 The Impact of Electronic Health Record Adoption and Integration on Physician Productivity and Health Outcomes By Chad D. Meyerhoefer 1,2, Mary E. Deily 1, Susan A. Sherer 3, Shin-Yi Chou 1,2, Lizhong Peng 1, Michael Sheinberg 4, and Donald Levick 4 November Department of Economics, Lehigh University, Bethlehem, PA; 2 National Bureau of Economic Research; 3 Department of Management, Lehigh University, Bethlehem, PA; 4 Lehigh Valley Health Network, Allentown, PA. Acknowledgements: We are grateful to Tianyan Hu and Jeffrey McFetridge for excellent research assistance, and to the physicians and staff members of the Lehigh Valley Health Network who took time to share their thoughts about and experience with the new information systems. We note in particular the important contribution of Marion Nihen, RN, who both collected key data for the study and served as the principal source of information on the progress of the installation. Finally, we acknowledge the financial support we received from the Agency for Health Research and Quality under Grant PARA

2 Abstract We investigate the impact of the installation of an outpatient electronic health record (EHR) and its subsequent integration with the hospital s medical record on physician productivity and health outcomes. In contrast to previous studies, we use a mixed methods approach and analyze a direct measure of physician productivity: relative value units. Specifically, we combine qualitative data from multiple rounds of interviews with physicians and staff with estimates from fixed effects regression models to identify the impact of the new record system as well as the underlying changes in work processes leading to these effects. Our estimates indicate that the EHR installation led to a four-year reduction in productivity, but also reduced the severity of adverse birth outcomes and influenced clinical practice norms. Qualitative data from physician interviews suggest that much of the productivity reduction resulted from additional documentation requirements associated with the new system, as well as from information overload resulting from the integration of information into a single record. 2

3 I. Introduction Health information technology has the potential to reduce waste and improve the quality of care in the U.S. health care system, but evidence of its performance benefits is equivocal (Agarwal et al., 2010). Electronic health records (EHRs), and in particular integrated EHRs that transmit information across care settings, could facilitate speedy transmission of accurate and timely clinical data (Eden 2008; Campbell et al., 2008; McCartney, 2006; Cherouny et al., 2005). However, the capital investment to purchase and install new systems, particularly EHRs, is significant (DesRoches et al., 2008). And, while it is expected that productivity will fall in the short term as physicians learn the new systems (Poon, et al., 2006; Motashari, et al., 2009), EHRs may continue to affect physician productivity as they adapt their work processes and learn to make the most of the system s capabilities. Studies of information technology (IT) investments in business suggest that IT improves productivity, but that those improvements take time to fully develop, requiring intensive process reengineering to adapt work practices to the new technology (Brynjolfsson & Hitt, 2003; Devaraj & Kohli, 2003). In healthcare, the majority of research on the impact of IT has examined instead its effect on the quality of patient care, generally finding that the benefits created by health information technology (HIT) are relatively modest 1 and may take several years to be realized (Parente & McCullough, 2009; Deily et al., 2013). While there is little study of productivity effects directly, there is some evidence that HIT can reduce costs in healthcare settings, although again it may take several years, suggesting that the necessary adjustments and learning take time (Borzekowski, 2009; Devaraj & Kohli, 2000; Garrido, et al., 2005; Javitt, et al., 2008; Frisse et al. 2012). In this paper we study the impact of the implementation of an integrated commercial EHR on the productivity of OB/GYN physicians. New policy initiatives have created incentives for major investments in HIT, with the expectation that such investments will result in better care at lower cost. However, there has been little or no work examining the impact of EHRs on physician productivity jointly with the quality of medical services. In particular, it is unclear 1 Most of these studies examine whether HIT improves adherence to guideline-based care or monitoring, or contributes to fewer medication errors; some studies examine the impact of HIT through direct examination of patient outcomes. See Chaudhry, et al., (2006), Garg, et al., (2005), Gluck (2009), and Buntin, et al., (2011) for references to this literature. 3

4 how long the learning period might last, and whether physician productivity will recover or exceed levels achieved before adoption of the EHR. For example, if using such systems actually takes more of a physician s time, then ultimately the physician will be able to see fewer patients, in essence reducing the effective supply of physician services. On the other hand, even if a physician sees fewer patients, his services may be more effective when combined with better information, and the quality of health outcomes for patients may improve. We investigate these questions by studying the implementation of an integrated commercial EHR adopted by a large health network for its outpatient OB/GYN practices and hospital-based OB/GYN physicians between July 2007 and December The EHR, and its subsequent integration with the associated hospital s IT system, was implemented in several stages. We examine the associated changes in productivity and health outcomes at the individual physician level before and after each stage of implementation. Unlike previous studies, we use a direct measure of physician productivity, relative value units (RVUs), and track productivity over five and half years to observe learning effects and changes in work processes over time. We also examine whether these new systems were associated with changes in practice patterns by examining the rate of induced births and of C-sections over time, again at the individual physician level. Finally, we use qualitative data from multiple rounds of physician and staff interviews in addition to estimating fixed effect regression models so that we can both measure the impact of the new EHR system on productivity and health outcomes and identify the underlying mechanisms generating that impact. II. EHRs and Physician Productivity The prior literature suggests that EHRs may reduce physician productivity in the short term. Visits may take longer as providers learn to use a new system (Mostashari et al., 2009), but ideally physicians learn how to use the system and quickly return to their previous levels of productivity. They may now be able to provide higher quality care because easy access to more standardized and current data improves clinical decision making and prevents delays in care delivery, leading to better prevention of problems and more efficacious treatment. In addition, more accurate and timely patient data may allow physicians to reallocate their effort across patients, giving more time and attention (of more highly skilled labor) to patients in greater need, 4

5 and less, for example, to those whose pregnancy is progressing normally. It is even possible that the availability of better data may cause physicians to spend less time acquiring missing information about a patient and fewer resources duplicating tests, resulting in an increase in both the quantity (on a per patient basis) and the quality of services. There is evidence that EHRs may improve the quality of care by improving adherence to guidelines, coordination of care in the practice, and transfer of information over time (Adams et al., 2003; O Malley et al., 2009; Shachak et al., 2009), and in some cases by improving the quality of medical record documentation (Soto et al., 2002). Researchers have also noted the potential benefits of integrating outpatient EHRs with hospital systems. Physicians in primary care offices frequently lack discharge summaries that they feel would improve care (Kripalani et al., 2007; Moore et al., 2003; O Malley et al., 2009; Motamedi et al., 2009); and Miller, et al. (2003) report that prenatal records are either pending or never available 57% of the time at birth, and that it generally takes 1.4 hours to retrieve missing data. Bernstein et al. (2005) also note that integrated EHRs may significantly improve communication among providers at different sites. In addition to saving time and facilitating information flow, there is some evidence that EHRs may reduce costs (Garrido et al., 2005; Adler-Milstein, et al., 2013). However, use of EHRs may also reduce physician productivity as well as the quality of care. Office visits may take more time (Adams et al., 2003) even when physicians know how to use new documentation systems, partly because EHRs, and integrated EHRs in particular, may create information overload (Javitt, et al, 2008; O Malley et al., 2009). EHRs may also hinder communication between doctor and patient by inhibiting eye contact and serving as a distraction during physician-patient interactions (Adams et al., 2003; Shackak, et al., 2009), and introduce errors if incorrect information is entered into the record (Shackak et al., 2009). Realizing the benefits of the EHR may depend on physicians satisfaction with the system (Menachemi et al., 2010; DiMatteo et al, 1993; Grol et al., 1985); in ambulatory settings, evidence suggests that physician satisfaction and improvements in the quality of care are affected by the design and capabilities of the EHR, and on the characteristics of the practice and of the physicians (Menachemi et al., 2010; Fleurant et al., 2012; DeRoches et al., 2008; Laxmisan et al., 2012). Finally, if productivity does not quickly recover to its original levels, there is little evidence on how long this will take, but research showing that physicians with two or more years of 5

6 experience with their EHR are most likely to be satisfied (Menachemi et al., 2010) suggests that adjustments may be lengthy. To the best of our knowledge, there has been no published study of the direct effect of installing an outpatient EHR, 2 or the impact of integrating that EHR with a hospital s system, on physician productivity over an extended period. In addition, previous studies typically consider costs rather than direct measures of productivity, and do not also measure the quality of the health outcomes produced by physicians during the same time period. III. EHR Implementation at the Lehigh Valley Health Network We study the implementation of an EHR at the Lehigh Valley Health Network (LVHN), a large health system located in Eastern Pennsylvania. LVHN operates four OB/GYN practices, where all clinicians are employees of the network, and who collectively deliver about 3,900 babies each year. Table 1 provides an overview of these practices. Practices A and B are conventional OB/GYN outpatient practices. The Outpatient Clinic serves a low income, primarily Hispanic population, who traditionally had poor access to care, and visited the clinic during pregnancy, but not regularly for gynecological care. 3 The fourth practice, High Risk, is based at the hospital and provides specialized care for difficult pregnancies (e.g., multiple births, ectopic pregnancy, or eclampsia). Prior to the EHR implementation, patient record systems differed across practices. Perinatal care in particular may be improved by information technology because this care is provided by different physicians and at different sites. At LVHN, a woman normally visits her primary care office times during her pregnancy, and, while she may see the same support staff, the physician she sees may not be the same at each visit. Further, when expectant mothers think they may be in labor, they go to a subunit of the hospital s Labor & Delivery Unit called Triage, where their condition is evaluated. The physician on duty either formally admits the patient to Labor & Delivery or discharges her back home; again, the physician may not be one 2 Garrido et al., (2005) report that internal analyses by Kaiser Permanente and the Geisinger Health System suggest that EHRs may have a neutral or positive effect on physician productivity, but these studies are not publicly available. 3 Maintaining accurate and current health information for these patients can be more challenging because of language differences and less stable insurance arrangements. 6

7 she has seen before. Patients who are not admitted after a visit to Triage continue office appointments until a subsequent visit to Triage results in their admission to the hospital for delivery, or the outpatient physician directly admits them. Ideally, information would flow quickly from an OB/GYN practice site to Triage and from Triage back to the practice site every time a patient is seen but is not admitted, and again from the practice site, through Triage, to Labor & Delivery once a patient is formally admitted. 4 But prior to the EHR implementation, data transmittal was costly and slow. Records were sent from the ambulatory OB/GYN office via courier or fax to Triage at specific pregnancy milestones, or when the patient arrived, and physicians and medical staff spent time acquiring the records and synthesizing information recorded in different formats. Since records were transmitted at certain points in a pregnancy, they could be out of date, for example, if a patient whose records were transmitted at 36 weeks showed up in Triage at 41 weeks. Furthermore, the maintenance of a "library" of paper records at Triage from the physician practices sometimes led to information loss, and reduced the efficacy and efficiency of care provision. Finally, the lack of a universally available record of a patient's medical history and past treatment often led to the unnecessary duplication of diagnostic tests. In turn, information from Triage did not flow back to the primary care offices. After a Triage visit, the physician seeing the patient back at a primary care office would have no information in the patient record about the patient s Triage complaint or condition, Triage tests or results, or even that the Triage visit had occurred, unless told by the patient. 5 In 2006 LVHN decided to replace all existing record systems at their primary care OB/GYN offices with a single commercial EHR that could be integrated with a companion system at the network s main hospital. The commercial system selected was GE s Centricity Physician Office (CPO) EHR. 6 At the start of this study in July 2007, CPO had just been 4 Approximately 10% of pregnant patients visit Triage well before they are due, during weeks of their pregnancy, and about 50% have a visit during weeks for a labor check ; of these, some return home and some are admitted. 5 LVHN is not unique. As noted above, previous work has documented that timely information is frequently lacking during an episode of care (Kripalani et al., 2007; Miller et al., 2003; Moore et al., 2003; Motamedi et al., 2009; O Malley et al., 2009). 6 This technology enables ambulatory care physicians and clinical staff to document patient encounters, streamline clinical workflow, and securely exchange clinical data with other providers, patients, and 7

8 installed at the Outpatient Clinic sites, while the other three practices moved to the new EHR later (see Table 1). Training was provided at each installation, but LVHN was prepared for reduced productivity as providers learned to use CPO and permitted reductions in scheduled appointments at the primary care offices of the two conventional practices, A and B. However, they anticipated this shock would be absorbed within a few weeks. 7 Installation of CPO in the primary care offices gave providers of obstetric services at the hospital two new sources of information. 8 First, an upgraded version of the system used in the L&D unit, a GE product called the Centricity Perinatal Network (CPN), 9 became operational in May The new CPO records were partially integrated with the hospital system so that data from them became available to the Triage physician accessing CPN. 10 In addition, outpatient CPO terminals were installed in Triage in October and November 2007 so that providers at the hospital could directly access a patient s record at the primary care office. 11 information systems. The application provides a broad range of clinical templates and allows the flexibility to design and modify forms used for documentation, health maintenance, and decision support. The system can be engineered to provide nationally accepted evidence-based guidelines for use at the point of care, and clinical decision support tools that automatically remind providers about required/recommended tests and procedures. 7 For Practice A, which migrated to CPO from an existing legacy system, schedules were reduced to 50% of capacity for the first week and about 25% of capacity the 2nd week. For Practice B, which migrated from paper, schedules were reduced by 50% for the first 2 weeks and 25% for the following 2 weeks. 8 While in theory it was possible to access the legacy EHR records for patients from Practice A and the High Risk Practice from the L&D Triage unit prior to the installation of CPO, doing so was cumbersome, and our discussions with Triage staff suggest few people actually attempted to do this. Again, physicians relied on paper copies messengered or faxed from the primary care office. 9 The CPN system is site tailorable, which allowed the staff to develop custom templates, forms, and reports. The system also has capability for protocol-based alerts, evidence-based decision support, and other clinical decision support tools; global alerts can notify users of abnormal fetal monitoring strips through visual and/or audible alerts. 10 Data moved automatically once the CPO record of a visit was signed, in that once a patient was assigned a medical record number in a primary care office a CPN record was automatically created and data was copied. These data included such items as blood pressure, cervical exam information, non-stress test results, and group B strep test results; the list expanded over time. 11 As CPO was installed in the Outpatient Clinic sites in May and June of 2007, there was a lag of a few months before it became accessible from the hospital. 8

9 Discussions with providers revealed that most used the second source of information to access a patient s prenatal record. Physicians became comfortable with the CPO user interface once they began using it in their offices, and they had greater trust in the data (this point being of particular importance). The unit clerk printed out data from CPO and many physicians used this printout to review the prenatal information. Eventually, as physicians became familiar with CPO, they began accessing it directly through the CPO terminals, although this was not commonplace until The integrated EHR system was designed to allow physicians or other clinical staff to obtain information entered into CPO at outpatient practices through CPN on Triage, but our interviews revealed that physicians were much less likely to know how to access information from CPO that had migrated to CPN. In addition, data integration problems prevented a significant amount of the data in CPO records from being transmitted to CPN, as intended. Over time data flow improved, but this took several years, and our interviews suggest that providers had already established a protocol for obtaining patient records directly from CPO by the time CPO-to-CPN data connectivity was close to design specifications. As a result, the practice of using records sent by courier or fax was gradually replaced principally by users accessing CPO from Triage (and other units of L&D), rather than searching within the CPN interface for data transmitted automatically from CPO. The second phase of the integration involved sending data from Triage visits documented in CPN back to the CPO system in each office, so that physicians in the primary care offices were automatically informed about any visit to Triage that had occurred since their last primary care appointment. Two types of data moved from CPN to CPO. One was a discharge summary of the visit to Triage in the form of a text document containing a written note and a list of clinical data measures (e.g. blood pressure). The summary included information on new treatments and diagnoses generated in the hospital Triage Unit and recorded in CPN. This capability was initially enabled for practices A, B, and the Outpatient Clinic during the summer of 2011, but was suspended in January 2012, before finally becoming fully operational in March The primary reason for the suspension was the discovery of an error in the document interface that allowed data from previous Triage visits to populate the document if these data fields were not updated again during the latest visit. At the same time, the original document format was improved to make it easier to read. The second type of information, a set of discrete clinical data 9

10 items transmitted directly from CPN to specified locations in the CPO interface, 12 also encountered difficulties during implementation, being briefly turned on and off twice before becoming fully operational by April Therefore, as of April 1, 2012, both types of information were moving from the CPN system in Triage to the CPO in all primary care offices. In our empirical analysis, we separately model each of the following three stages of CPO implementation: 1) Initial installation (for all practices except the Outpatient Clinic our data sample begins before the initial installation of CPO at the practice sites); 2) Transmission of Triage summary document from CPN to CPO; and 3) Transmission of discrete clinical data elements from CPN to CPO. While these stages were sequential, re-transmission of a corrected Triage discharge summary occurred just one month before the transmission of the discrete clinical data. As a result, what our empirical models measure in stage two is largely the impact of an initially poorly-formatted Triage discharge summary with inaccurate data for some patients on physician productivity and health outcomes. Because we believe there is also an overall learning effect that begins when the new system is installed and persists through all stages of integration, we also include separate measures of learning throughout the entire sample period. IV. Measuring Physician Productivity Examining the effect of IT on productivity in the economics literature is generally approached through estimation of a production function, frequently a Cobb-Douglas specification, which links inputs such as labor and capital (both measured in dollars) to some measure of output, commonly value-added (operating revenues minus intermediate inputs) for a panel of firms. In this framework, the impact of IT may be examined by separating out investment in IT systems from the rest of a firm s capital, and examining the effect of this type of capital on changes in firms value added. (See Brynjolfsson & Hitt, 2003, for a description of the basic approach, and Lee et al., 2012, for a recent application to hospital IT.) This 12 These items included: urine protein, glucose, nitrates, and leukocytes, headache, edema, vaginal bleeding, ROM-Triage, fundal height, FHR baseline fetus A, fetal presentation admit, dilation, effacement, station, gestation estimate, and triage final assessment. 13 The primary reason for the suspension of the discrete data was that multiple visits to triage on the same day were not apparent on the CPO flow sheet, which was based upon daily reporting. A program fix alerted physicians to change their view from days to hours to see all relevant information if a patient had multiple Triage visits on the same day. 10

11 methodology is particularly valuable for investigating the contribution of IT to a firm s overall productivity growth, but is less useful for gauging the effect of a particular IT system on the quantity and quality of services provided by a single input, in our case physicians. We take a different approach. We assume that the LVHN physicians work in an environment in which the supply of patients is infinitely elastic in the patient, i.e., that they can treat as many patients as they have time to treat. Therefore, the quantity of healthcare services supplied by physicians is determined by their capacity to see patients. On the inpatient Labor & Delivery Unit and the Mother-Baby Unit (where women go after delivery) the supply of patients is limited by the number of beds, which are typically near full capacity. Our principal measure of the amount of health services produced by a physician in a month is the amount of work relative value units (wrvus) credited to a physician that month. wrvus are a standard measure of physician effort that account for the time, technical skill, physical and mental effort, and stress required by a particular type of health service (National Health Policy Forum, 2009). 14 While wrvus were developed to facilitate reimbursement, we use them here as a measure of the amount of health care produced by a physician. For example, suppose a physician is able complete eight prenatal appointments in four hours, each generating three wrvus for a total of 24 wrvus. If use of an EHR slows the physician, so that he can now only complete seven such appointments in the same four hours, his wrvus will fall to 21. In this sense, the wrvus measure the amount of medical services delivered. Our measure of total wrvus includes all wrvus generated by the physician, whether they were recorded for gynecological or obstetric services. All the physicians we study provided both types of care, spending roughly 50% of their time on each, and any slowdowns created by the transition to CPO would affect provision of both types of care. Further, physician wrvus might be affected by changes in the mix of service type provided by a physician in a month. Therefore, we examine total wrvus so that we can have a full measure of the total healthcare services provided by the physician The Center for Medicare and Medicaid Services (CMS) also uses RVUs to measure practice expenses, such as capital costs, and liability insurance expenses. Our RVU measure excludes these other types of RVUs, and includes only physician work effort, which is why they are denoted wrvu. 15 While the goal of the IT investment by LVHN was to improve data flow across the perinatal continuum, and we consequently focus on obstetric outcomes to assess quality effects, the new system may also have 11

12 Physicians may respond to reductions in the wrvus they are generating by moving some of the tasks that they previously carried out to other medical staff, such as residents, physician assistants, midwives, or nurses so that they can increase the number of patients they see or procedures they provide. To account for this, we include wrvus generated by the medical staff supervised by the physician in the physician s total wrvu amounts. In other words, we assign all wrvus for which a physician was the attending physician to that patient. By measuring wrvus in this manner we obscure what might be interesting patterns of substitution among different types of medical staff, but we avoid undercounting the total amount of labor-related medical services provided in patient care. We categorize wrvus along several different dimensions. First, our data allow us to differentiate between wrvus earned for patient visits as opposed to procedures. Visits result from an encounter between a patient and the physician, or a member of the physician s medical team, whereas procedures are generated through some type of medical intervention or the use of a medical technology. Examples of the former include a regular prenatal appointment at an outpatient practice site or a trip to the inpatient Triage Unit, while examples of the latter include an ultrasound or non-stress test. Sometimes procedures are performed during the visit; other times, patients have a test performed at a practice site, but they do not see the physician. Second, we differentiate visits and procedures occurring in the outpatient setting from those that occur in the hospital. Physicians spend about 70% of their time seeing patients in the primary care offices and about 30% at the hospital or on the phone to the hospital providing principally obstetric care. At LVHN, all deliveries occur on the inpatient Labor & Delivery Unit. However, visits to the Triage Unit at the hospital are coded in our data in the same manner as visits to the outpatient practices, because the tests and procedures performed on the Triage Unit, and the type of physician-patient interaction there, are very similar to the services provided at outpatient practices. As a result, we cannot differentiate outpatient and Triage visits and procedures, and so both are represented in our outpatient wruv category. Although total wrvus and visits measure the total amount of health care services provided by a physician each month, they do not account for variation in work schedules due, for improved the quality of gynecological services, given its greater capabilities. However, we have no clear way to measure quality in the provision of gynecological services to test for such an effect. 12

13 example, to vacation, conference travel, or continuing medical education, or for budgeted changes in patient loads. Therefore, we use wrvus/visit as our main measure of productivity: this normalization accounts for these factors and allows us to measure the intensity of services provided for each visit. We examine this measure for both outpatient and inpatient services. However, when we calculate wrvus/visit separately for outpatient care, we divide by the number of visits to outpatient facilities, but when we calculate wrvus/visit for inpatient care we divide by total visits. We do this because very few non-triage visits occur at the hospital, and wrvus earned at the hospital are ultimately the result of visits made to the outpatient practices or to Triage. V. Methods Quantitative Approach, Specification, and Variables We examine the impact of the new EHR by estimating the following fixed effects specification: Y kit = β 0 + β 1 *EHR it + β 2 *LEARN kit + β 3 *P kit + CC it + ST it + STS it + ρ k + π i, + η t + ε ikt, where k is the individual physician, i is the primary care practice site, t is month, Y is a measure of either the production of health care services by physician k of practice site i in month t, or the health outcomes of that physician s patients in that month. EHR is a vector of three dummy variables indicating the current level of installation and integration achieved at practice site i at time t, LEARN is a vector of variables included to measure learning by physician k in practice site i, P is the average health status of the physician s patients at a site in a month, CC t is a dummy included to control for an accounting change that occurred at Practices A and B during our study, ST it is a site-specific linear trend and STS it is the site-specific trend squared, and ρ k, π i, and η t are physician, practice-site, and quarter fixed effects, respectively. Note that while there are four OB/GYN practices, there are multiple sites within each practice, and that not all sites within a practice necessarily had CPO installed on the same date. 16 Dependent Variables 16 Although CPO installation was staggered across practices and sites, there is not enough difference in install dates to estimate a difference-in-differences specification. 13

14 The first dependent variable, wrvu/visit kit, is our measure of the production of health services and is based on the total wrvus earned by a physician (and their treatment team) at a particular site each month. To derive this variable we multiplied the wrvu weight associated with each specific procedure or type of visit by the number of those activities performed, and summed the result for each physician in each month during the July, 2007 December, 2012 study period, generating a total of 4,599 physician-month observations. We then divided the total wrvus by the total number of visits to a physician in the given month to find the average wrvus earned per visit for each physician-month observation. We calculated this average for all wrvus, and separately for those wrvus associated with inpatient and outpatient care. Our panel of physician-month observations is unbalanced because not all physicians are represented for the full five and a half year study period, and because some physicians did not earn any wrvus in certain months. In addition, when physicians work at multiple sites they enter the dataset separately for each site. 17 Our second dependent variable is a measure of the quality of obstetric outcomes produced by the physicians each month. Using chart review, LVHN has tracked 10 preidentified adverse events for all deliveries since July, Each adverse event is weighted to reflect its severity and these numbers are then summed across mother and baby(ies) to calculate a weighted adverse outcomes score (WAOS) for each delivery (Mann et al. 2006; Mohr et al. 2003). 18 The dependent variable is the average WAOS across all the physician s deliveries in each month, and so is also measured at the physician-month level. Our adverse outcomes data only run through June, 2012 and only apply to physicians with deliveries in the given month, resulting in 3,798 physician-month observations. Because the distribution of WAOS is quite skewed, we analyze it using two separate models. First we model the proportion of physicianmonth observations with a positive WAOS, and then estimate a model on the ln(waos) using those observations where WAOS > We account for this when computing the standard errors of our estimates by cluster-correcting at the physician level. 18 The adverse outcomes used to construct the WAOS include maternal death (weight 750); intrapartum and neonatal death > 2500 g (weight 400); uterine rupture (weight 100); material admission to ICU (weight 65); birth trauma (Erb s palsy; vacuum of forces injury) (weight 60); return to OR/L&D (weight 40); admission to NICU > 2500g & for > 24hours (weight 35); Apgar < 7 at 5 minutes (weight 25), blood transfusion (weight 20), and 3 or 4 perineal tear (weight 5). 14

15 Finally, we also examined the impact of the new EHR system on two measures of treatment intensity and clinical decision making:, the proportion of a physician s patients where labor was medically induced in a month, and the proportion of a physician s patients delivered using Cesarean section (C-sections). 19 Since wrvus and insurance payments are higher for inductions and C-sections than vaginal births, it is possible that physicians could compensate for a productivity decrease caused by CPO by increasing the intensity of treatment through these procedures. However, greater intensity of care may be warranted when risks are detected. CPO and its integration with CPN might increase inductions and C-sections if, for example, earlier information about birth risks contraindicated vaginal delivery. In this case, we would also expect to observe an improvement in the quality of care. IT and Learning variables EHR is a set of three dummy variables. The first dummy capturing CPO installation equals 1 for a practice site if the CPO is live at the site, but integration with the hospital system has not yet occurred. The second dummy, Triage summary, equals one if, in addition to CPO being installed, a summary of any visit by a patient to Triage is automatically sent to CPO from CPN, but discrete data elements are not yet being transmitted. The third dummy variable, Discrete elements, is one if the discrete clinical data elements are being sent from the hospital to CPO after a patient has visited Triage. During this time period the corrected Triage discharge summary is also being transmitted from CPN to CPO. Therefore, each dummy reflects the cumulative effect of the EHR capabilities installed at that time. We calculate marginal effects corresponding to each of the three stages of the EHR installation and integration by differencing the relevant dummy variable coefficients. For example, the impact of the Triage Summary alone is calculated as the difference between the coefficients of the Triage Summary dummy and the CPO installation dummy. We model learning by the physician as a quadratic function, and using the number of months after CPO installation at his or her practice site, and the number of months squared. We interpret the effects of learning in the broadest sense. Not only do the variables capture the 19 Because the inpatient units of LVHN have used electronic record systems since 1993, we had access to a long and ongoing time series of detailed information on the characteristics of deliveries and birth outcomes. In particular, these patient-level data include the number of inductions and C-sections. 15

16 changing ability of physicians to use the EHR system, but they may also capture changes in clinical practice patterns that result from greater knowledge of CPO and its capabilities. Control Variables There are several factors other than the EHR that could affect both wrvus and health outcomes. In particular, patients with pre-existing conditions that complicate pregnancy, such as diabetes or high blood pressure, require more physician management during pregnancy and are more likely to have adverse birth outcomes. To control for pre-existing conditions and overall illness severity among a physician s patients we use the Diagnosis Cost Groups/Hierarchical Condition Categories (DCG/HCC) method to generate risk scores for each patient. DCG/HCC risk scores are derived from data on patient age, sex, and physician-reported diagnosis codes (ICD-9-CM). They have been validated as a proper measure of risk adjustment in the inpatient and outpatient settings (Ash et al., 2003; Petersen et al., 2005; Chukmaitov et al., 2009) and are also used by CMS to risk adjust Medicare payments to private insurers under Part C (Pope et al., 2000; Pope et al., 2004). So that the risk scores capture baseline illness severity and not adverse events occurring during the current pregnancy, we computed the risk scores using diagnosis codes documented up through the year prior to the delivery. The variable P kit in the specification is the average of the individual risk scores for all women treated by a physician in a given month. We also include a dummy variable, CC it, that equals one for those physicians associated with Practices A and B in the months after July This variable is included to control for a change in the way in which LVHN allocated wrvus for obstetric care at these practices. After a birth, insurance companies typically pay LVHN a global fee that is meant to cover all the care delivered during the preceding nine months of the pregnancy. Because this care will have been provided by a variety of different physicians and medical staff, possibly from different practices, the global fee must be allocated amongst these different providers in accordance with earned wrvus. In July 2007 when our study began, pregnancy wrvus were split, with half going to the provider who delivered the baby and half going to the providers of the prenatal care. 20 This, 20 Non-bill generating pseudo-codes were created to capture the work effort provided during prenatal care. These pseudo-codes allow the work provided to be "credited" to the appropriate provider, although no fee is collected until after the delivery, when the pregnancy bill is created. An additional complicating factor relates to the allocation of the actual payment for pregnancy services. The reimbursement was posted to the originating practice location (cost-center) that the patient first presented to for the newob exam. 16

17 however, made it difficult to generate budgets for the practices since the physicians on the inpatient unit could potentially deliver babies whose mother received prenatal care at any practice, not just their own. As of July 1, 2011, all the wrvus tied to the delivery and standard package of prenatal care were allocated to the physician who delivered the baby, while payments were allocated to the practice where the women was registered for prenatal care. While we expect that, on average, the allocation of wrvus should be similar between the new and old accounting systems, we include the control variable CC it to account for any differences in coding that might threaten comparability between the two time periods. Previous research has demonstrated that successful implementation and use of an EHR may depend on characteristics of both the physicians and the practices to which they belong (Soto et al., 2002; Menachemi et al., 2010; Fleurant, et al., 2012; Laxmisan et al., 2012). In addition, it is possible that the timing of the CPO installation across practice sites was not random, but correlated with certain practice characteristics. Furthermore, practice norms may change over long periods of time and result in deterministic productivity and health outcome trends. In order to control for the unobservable attributes of physicians and practice sites that are potentially correlated with our productivity and health outcome measures as well as the EHR implementation and integration variables, we include fixed effects for individual physicians and for practice sites, while linear and quadratic time trends for each practice site capture deterministic trends. Finally, we also include fixed effects for quarters to control for common shocks, such as severe weather or LVHN-wide policies, which might affect physician productivity or the quality of their services across all practices. Qualitative Analysis Simultaneous to our quantitative data collection, we qualitatively investigated the process changes that impacted productivity during this time period. We reviewed archival data such as hospital staff meetings, and conducted approximately 75 interviews of personnel both in the offices and the hospital, at three points in time, 2010 Q2, 2011 Q3, and 2013 Q2. Interviews were transcribed, coded, and analyzed to interpret key concepts related to technology acceptance, process change, changes to roles and relationships, complementary investments, and impact on outcomes such as productivity 17

18 VI. Results Descriptive statistics for our analysis variables, reported in Table 2, indicate that physicians earn most of their wrvus (80%) providing outpatient care. Outpatient wrvus are earned both on the hospital s Triage Unit, where patient volumes are relatively high, and at the outpatient practices. The proportion of visits that are for outpatient care (96%) is even more skewed, because nearly all OB/GYN events classified as visits that occur in hospitals are visits to Triage. On average, an attending physician and the residents under his supervision deliver roughly nine babies per month, and an adverse outcome occurs in approximately nine percent of all deliveries. In Figure 1 we plot time series of total, outpatient, and inpatient wrvus/visit separately for each of the four practices as well as pooled over all practices. In all of the figures there is considerable variability from month-to-month in outpatient wrvus whereas inpatient wrvus are more stable. Nonetheless, the plot of total wrvus/visit across all practices suggests that productivity declines at the beginning of the sample period; that this decline slows in the middle of the sample period; and then increases at the very end of the sample, a pattern that is also present in both outpatient and inpatient wrvus/visit. Examination of the separate plots for each practice reveals that the decline at the beginning of the sample period is driven mostly by a drop in productivity at the Outpatient Clinic. At both Practices A and B productivity is relativity flat until the end of the sample period where it starts to increase steadily at Practice A while dropping sharply at Practice B, when the variance also shrinks noticeably. These changes in the trend of wrvus/visit at the end of the sample period at Practices A and B is coincident with the introduction of new cost centers for these practices. Finally, there was an increase in outpatient productivity among the High Risk practice physicians because of a shift in responsibility whereby staffing of the Labor & Delivery Unit was increased with physicians from this practice. Table 3 contains coefficient estimates and marginal effects from our fixed effect productivity regressions. We estimated three sets of specifications. The first set includes only the dummy variables indicating CPO installation and integration with the hospital system, CPN, while the second set contains only the learning variables. In the third set we include all of the EHR variables. Within each set we estimate two regressions, with the first containing practice site-specific linear time trends, and the second containing site-specific linear time trends and 18

19 their square. All of the specifications additionally contain physician, site, and quarter fixed effects, and the standard errors of all estimates are cluster-corrected at the physician level. Finally, panel A of table 2 is based on total wrvu/visit earned at all locations, while panels B and C present results for outpatient (including Triage) and inpatient wrvus/visit, respectively. Because the coefficient estimates are similar irrespective of whether we include squared time trends in the models or include both the CPO indicators and learning in the same specification, we focus on column (6), which contains our most comprehensive and flexible model. The coefficient estimates in panel A are generally precisely estimated and suggest there are productivity decreases associated with each stage of CPO implementation. The coefficients on the learning variables measuring month and months squared since installation indicate that learning is initially associated with productivity reductions, but then with productivity gains after the turning point in the convex learning function. The coefficient estimates themselves, however, are difficult to interpret, so we focus in these tables on the marginal effects of each stage of CPO implementation and integration, and subsequently present simulations of the full EHR effect on productivity that take all five variables into account. Only the marginal effect for the initial CPO installation is precisely estimated in panel A, and indicates that CPO was associated with a 49 percent reduction in total wrvus/visit. The analogous marginal effect for outpatient services (Panel B) is similar in magnitude, indicating that CPO led to a 45 percent reduction in outpatient wrvus/visit. Finally, as with the total and outpatient productivity measures, neither the marginal effect of the initial Triage discharge summary or the effect of transmission of discrete data elements from CPN to CPO is precisely estimated for inpatient services (Panel C), but the impact of the initial CPO installation has a much larger effect on inpatient wrvus/visit, decreasing them by 84 percent. In order to gain additional insight into these effects we estimated the same models on wrvus (the numerator) and on visits (the denominator) separately and report these in the appendix. These results show that the installation of CPO reduced both wrvus and visits, but that there was no statistically significant impact of the initial Triage summary document on either wrvus or visits. Transmission of the discrete elements did increase wrvus, but visits also increased during this time period due to both the direct effect and learning, resulting in a the lack of a significant effect once wruvs are normalized by visits. 19

20 Simulations of the full effect of all of the EHR variables on total RVUs/visit are shown in Figure 2. The simulations reveal that physician productivity decreased immediately after the installation of CPO at a decreasing rate. The initial reduction resulted from the increase of CPO at the Outpatient Clinic, but a clear increase in the magnitude of the reduction can be seen when CPO was installed at Practices A and B in the middle of This reduction continued until the end of 2010 when productivity began to climb at an increasing rate. The transmission of the initial Triage discharge summary resulted in a productivity increase, which brought productivity back to its pre-installation level, and although productivity dropped again after transmission of the discrete clinical data elements began, it recovered quickly and started to reach levels above the pre-installation level at the very end of the sample period. Ultimately, the productivity drop lasted approximately four years, and although the end-of-sample trajectory is upward, the end-ofsample productivity increase is not statistically different from zero. The simulations for outpatient and inpatient RVUs/visit mirror that for total productivity. The main difference is that inpatient productivity drops to a lower level, but then rebounds to a higher level than outpatient productivity by the end of the sample period. 21 An important question is whether these observed changes in productivity occurred contemporaneously with changes in health outcomes. To answer this question we report the estimates from fixed effect regressions of our measures of adverse delivery outcomes on the CPO implementation variables in Table 4. While none of the EHR variables have a statistically significant impact on the proportion of deliveries with a positive WAOS (Panel A), the transmission of discrete data from CPN to CPO had a large and precisely estimated negative effect on the size of the WAOS (Panel B), decreasing it by 122 percent in our preferred specification. Simulations of the effect of the EHR variables on the WAOS (Figure 3) confirm that WAOS dropped significantly at the end of the sample period as a result of the discrete data transmission from the Triage unit to the practice sites. Finally, we investigate whether the initial drop in productivity or availability of new information through the EHR system caused providers to alter their practice norms by analyzing 21 Note that all three graphs have a localized peak at the beginning of This is the result of the suspension and then resumption in transmission of the Triage discharge summary. Although the second version of the summary document contained changes, it was only in place one month before transmission of the discrete data elements began, so the Triage summary effect is dominated by the impact of the first summary document. This jump occurs in the graphs of all outcome variables. 20