Kidney International, Vol. 57, Suppl. 74 (2000), pp. S-7 S-13 Collecting and using patient and treatment center data to improve care: Adequacy of hemodialysis and end-stage renal disease surveillance 1 WILLIAM M. MCCLELLAN and JENNA OSBORNE KRISHER Georgia Medical Care Foundation and Rollins School of Public Health, Emory University, Atlanta, Georgia, and Southeastern Kidney Council, Raleigh, North Carolina, USA Collecting and using patient and treatment center data to im- the control of infectious diseases, and, during the last two prove care: Adequacy of hemodialysis and end-stage renal decades, has been increasingly used in chronic diseases disease surveillance. The capacity to use patient and treatment control programs [4, 5]. ESRD is unusual among the center information about the patterns and outcomes of care of end-stage renal disease (ESRD) to improve the care and chronic diseases in the United States in that a compre- prevention of kidney failure has been greatly enhanced in the hensive surveillance system has been established for this United States by the national ESRD surveillance system. This categorical disease [6]. The U.S. ESRD surveillance syssystem consists of 18 regional Networks, Health Care Financing tem consists of two parts, the ESRD Networks and the Administration (HCFA), United Network for Organ Sharing (UNOS), and the United States Data System (USRDS). This United States Renal Data System (USRDS). The Netreport describes how ESRD Network 6 used different data works originated in the 1972 legislation establishing the sources to implement and evaluate a program to improve the Medicare ESRD program, and there are currently 18 of adequacy of hemodialysis prescription in a three state region. these regional systems (Fig. 1) [7]. Each Network collects regional data and produces surveillance information about the occurence and outcomes of treatment of End-stage renal disease (ESRD) surveillance systems ESRD within the population in its defined geographic provide information that promotes action to improve the area. care of patients with kidney failure. ESRD surveillance The USRDS was established in 1988 and is responsible systems have two essential characteristics. First is the for the collection and analysis of information about the systematic and ongoing collection, aggregation, analysis, incidence, prevalence, treatment and outcomes of ESRD and interpretation of data about the occurence and out- in the entire U.S. Networks collect and transmit data to comes of kidney failure in a defined population [1, 2]. HCFA where it is combined with other data files and Second, the resulting information is disseminated and forwarded to the USRDS. The USRDS compiles an anused to improve the treatment and control of the ESRD nual report on the epidemiology and outcomes of care [3]. Surveillance information can be used to define the of ESRD within the United States [8]. As described epidemiology of the disease, detect epidemic occurences, elsewhere in these proceedings, the USRDS also conidentify high risk populations, assess outcomes of treat- ducts studies of various topics and the data for these ment, and to plan, implement and evaluate disease con- studies are largely collected by the ESRD Networks [9]. trol activities. Surveillance systems can also facilitate A recent use of the U.S. ESRD surveillance system observational, clinical and laboratory research. has been to improve the care of hemodialysis patients Public health surveillance was originally developed for [10]. The purpose of this report is to give a brief account of how one ESRD Network, Network 6, developed and 1 used surveillance information to implement and evaluate The views expressed in this article are those of the authors and do not reflect official policy of the Health Care Financing Administration. an intervention to improve the care of hemodialysis pa- As an unpublished document the contents may not be cited, quoted, tients in the three state region of North Carolina, South or reproduced without the express permission of the authors. Carolina and Georgia. This report will focus on a series Key words: quality of care, mortality, informatics, ESRD Core Indicators of studies and interventions conducted by the Network Project, HCFA, ESRD Networks. to illustrate the sources and types of data used in this 2000 by the International Society of Nephrology surveillance system. S-7
S-8 McClellan and Krisher: Dialysis adequacy Fig. 1. Geographic location of the 18 ESRD Networks. The networks are regional ESRD surveillance systems under contract to the Health Care Financing Administration. They are responsible for quality assurance. Data collected by the networks are transmitted to the United States Renal Data System (USRDS) for compilation of the USRDS annual data report. DATA SOURCES AND ATTRIBUTES FOR ESRD SURVEILLANCE Administrative and registry data The administrative data used by the individual Net- works is collected using standard HCFA forms. ESRD Network 6, like other Networks, also uses individual patient tracking tools which were designed as part of the ESRD Networks use multiple data sources for surveillance activities. These include administrative data, rou- tinely collected registry data, recurring surveys and special data collection. Billing claims data are an additional source of information that is available but not easily accessed by the Networks. Administrative data are col- lected by the Networks routinely for all ESRD patients for enrollment into the Health Care Financing Adminis- tration (HCFA) Medicare ESRD program. Registry data are collected from all persons in the system for the pur- poses unique to the surveillance system. There are several important attributes of these data used for ESRD surveillance [11]. These data should orig- inate within a defined geographic area, allowing estima- tion of population-based rates. These data should be collected in a timely manner, insuring that reports are current and relevant to clinicians. These data should be valid and reliable and should allow standard casedefinitions to be assigned to all patients. Absence of standard case-definitions results in misestimation of rates for cause-specific, but not all-cause ESRD. Case-ascertainment (identification of new ESRD patients) should be complete to avoid biased rate estimation. These data should be readily available for use by the system. Finally, and most importantly, strict confidentiality of patient- and provider-specific surveillance data should be main- tained [11]. surveillance system. It pinpoints where each patient is treated and their current modality. The utility of these data is that they are routinely and uniformly collected for each eligible ESRD patient. In contrast, the shortfall of these data is that they are not primarily designed for use by the surveillance system. Nonetheless, surveil- lance information derived from administrative and claims data can provide important insight into the occurence and outcomes of care of ESRD within a population. National administrative data and mortality rates. An example of this usefulness was a symposium convened in Dallas, Texas in 1989 to examine international patterns of mortality, morbidity and prescription of dialysis [12]. At that meeting, several national ESRD registries reported information for the annual entry (incidence) rate of newly-treated patients, the prevalence of treated patients on a selected day of 1987, and transplantation and mortality rates for 1987. Substantial registry-to- registry differences in acceptance, prevalence, trans- plantation and mortality rates were reported by the national registries. Further, mortality trends for hemodialy- sis patients in the U.S., but not other countries, were increasing. Among the hypotheses advanced at the Dallas Symposium to explain these marked country-to-coun- try variations in risk of death, two caught our attention in ESRD Network 6. One, the case-mix hypothesis, was that the differences in risk of death in the different ESRD populations could be explained by registry-to-registry variations in patient age, severity of illness and cause of renal failure. The other, the adequacy of hemodialysis hypothesis, raised the possibility that systematic differences in the prescription and delivery of the per-treatment dose of hemodialysis were largely responsible for the differences in mortality that were noted among the different registries.
McClellan and Krisher: Dialysis adequacy S-9 Fig. 2. Facility-specific unadjusted ( ) and crude case-mix adjusted ( ) relative risk for death by dialysis facility. Unadjusted and adjusted estimates of the risk for death are relative to those of the largest facility. Estimates are ranked by adjusted mortality (bars indicate CI adjusted relative risk). The adjusted rates were controlled for age, race, diabetic renal failure, angina pectoris, congestive heart failure, nutritional status and functional status. Figure reprinted with permission from [13]. Regional registry data and case-mix adjusted mortality risk for mortality for individual treatment centers still rates. Following the Dallas meeting, ESRD Network 6 differed significantly from unity [13]. This suggests that conducted a series of studies of the role of case-mix other factors associated with treatment centers might factors in explaining variations in mortality among influence the risk of death within a facility. ESRD populations [13, 14]. The Network routinely collected comorbidities, functional status, nutrition, employment Survey data and socioeconomic status information from The administrative and registry data collected by Net- all new patients. These registry data were used by the work 6 didn t capture information about dialysis pre- Network to track rehabilitation of ESRD patients and scriptions for individual hemodialysis patients. Thus it their use as case-mix factors illustrates a common theme, was not possible following the Dallas symposium to use the novel application of data for purposes other than the existing data sources to determine if ESRD patients that for which they were collected. were under-dialyzed. It is not uncommon for surveillance Our first analysis examined facility-to-facility varia- systems to require additional data and a common means tions in risk of death among hemodialysis patients before of supplementing routinely collected administrative and and after controlling for treatment center differences in registry data is a survey. Surveys use probability sampling the distribution of case-mix factors [13]. We reasoned to obtain a representative subpopulation and collect the that, if variations in center-specific death rates did occur, needed data from these patients. Surveys can be conthen controlling for differences in case-mix would reduce ducted either at a single point in time or on an ongoing the variability in mortality rates. We found that unad- basis. A series of surveys conducted by the ESRD Netjusted mortality rates for the 31 treatment centers provid- work 6 and HCFA to study the adequacy of hemodialysis ing hemodialysis in one of the Network s member states treatment illustrates how surveys can be used to obtain during 1987 were quite variable. Adjusting for facility- surveillance data [15]. to-facility differences in case-mix did little to reduce the National survey data. The ESRD Core Indicators variability of rank order of mortality among the centers. Project was begun during 1993 to provide the ESRD Figure 2 illustrates this variability by comparing individ- Networks information about the care delivered in the ual treatment center mortality rates before (crude) and U.S [16, 17]. Information at the regional, but not treat- after (adjusted) accounting for case-mix differences [13]. ment center level, was obtained about a number of processes After adjusting for center-to-center case-mix differences of care, including the adequacy of hemodialysis. the rank order of relative risk persisted and the relative The baseline ESRD Core Indicators survey was con-
S-10 McClellan and Krisher: Dialysis adequacy Linking surveillance information to quality improvement. The information from the Core Indicator baseline survey was explicitly intended to initiate efforts by ESRD Networks and treatment centers to improve care [10]. ESRD Network 6 used the facility-specific URR results described above to identify the 10% of treatment 58% were less than 65%. The mean URR was 63% centers with the combination of the lowest average URR and URR was noted to be lower among males, African and highest proportion of patients with a URR less than Americans and younger individuals. There was also sub- 60%. These treatment centers would be found in the stantial Network-to-Network variation in the proportion left-hand portion of the distribution of preintervention of patients with URR less than 65% and ESRD Network mean URR shown in Fig. 3. Among the 23 centers identi- 6 ranked 15th of 16 Networks with 68% of its patients fied, the mean facility-specific URR was 58.9% and an having a URR less than 65% [18] These results were distributed to each of the 18 Networks and disseminated average of 26% of the patients in these centers had a by them by mail to each facility s head nurse, administrabegan by providing all centers comparisons of facility- URR of less than 60%. The Network 6 intervention tor and medical director in their respective region Regional survey data. The baseline ESRD Core Inditended one of four regional workshops designed to teach specific URR. The centers in the intervention group atcators survey provided region- but not facility-specific estimates of URR. Because of the previously recognized facility staff how to identify and correct reasons for their center-to-center variations in case-mix adjusted mortal- poor URR levels. The workshops were open to any other ity rates, our second analysis was a survey of all treatment centers that cared to attend and 40% (N 77) of the centers within its region to obtain center-specific esti- non-intervention centers also attended the workshops. mates of dialysis adequacy [19]. During October, 1994 Finally, Network staff and volunteer physicians and ala sample of 30 patients per treatment center, sufficient lied health professionals worked individually with each to estimate mean URR with a 2.5% 95% confidence of the 23 intervention centers until they achieved and interval was chosen and these data were gathered in the sustained a mean URR of 65% and had reduced by 50% same manner as in the baseline Core Indicators survey. the proportion of patients with a URR of less than 50%. Two hundred and fourteen treatment centers were sam- During the intervention period, the mean URR in pled with a mean of 27 patients per center. There was considerable heterogeneity among the treatment centers these 23 centers increased from 58.9% to 65.9% (P with respect to URR (Fig. 3). The facility-specific pro- 0.001); in the remaining centers from 65.7% to 67.1%. portion of patients with URR less than 50% varied from Multivariate analyses that controlled for other facility 0 to 14% and 34% of the facilities had a mean URR characteristics found a small, additional improvement less than 65%. These results were distributed through in URR had occured in the context of Network-wide both mailed reports and regional presentations to treatment centers within the Network. tion (r 2 0.03, P 0.006) feedback of center-specific adequacy of dialysis informa- [19]. ducted by each Network on a random sample of adult (aged 18 years and older) hemodialysis patients who were alive on December 31, 1993 [18]. The sample size of about 400 patients per Network was calculated to give an estimate of the proportion of patients with a urea reduction ratio of 65% or greater with a 95% confidence interval of 2.5% in each Network. A one-page questionnaire for each selected patient was sent to the individual s treatment center and was completed by a staff member. Two of the 18 Networks did not participate in the initial survey. Surveys were sent to 1728 treatment centers and included 6358 patients. The completion rate was 96.6%. Treatment center staff abstracted data from medical records for the last three months of 1993, including the first pre- and post-treatment blood urea nitrogen levels in each month. A three-month average URR was then calculated for each patient. The Networks independently requested and re-abstracted a 6% replicate sample of the same patient records [17]. There was a high inter- rater reliability on all data fields abstracted by treatment center staff and the concurrence between URR levels was 94%. During the last quarter of 1993, 35% of U.S. hemodialysis patients had an average URR less than 60% and Fig. 3. Distribution of preintervention mean URR among 213 treatment centers in Network 6 in 1994. Figure reprinted with permission from [19].
McClellan and Krisher: Dialysis adequacy S-11 about URR to target interventions had faster rates of increase in URR than did other Networks [21]. NEXT STEPS: IMPROVING THE DATA LINK BETWEEN FACILITY AND SURVEILLANCE SYSTEM A major consequence of the Core Indicators Project was the recognition that informatics technology might be used to improve data collection within the ESRD surveillance system [22]. The multiple data collection demands at all levels of the project (dialysis center, Network office and nationally) that the Core Indicators project entailed led us to develop informatics systems that Fig. 4. Kaplan-Meier plot of release from close monitoring of 22 treatment centers in Network 6 that were selected for targeted intervention aggregation, analysis, and interpretation of data [1, 2]. would facilitate the systematic and ongoing collection, in March 1995. Facilities were released from close monitoring when they Because it must interface initially with an existing sysreached target dialysis adequacy goals. Figure reprinted with permission from [19]. EVALUATION AND SURVEILLANCE INFORMATION The ongoing nature of data collection by a surveillance system allows evaluation of trends in occurence of disease and outcomes of care. This information can be used to evaluate efforts to improve care. Special studies and evaluation The Network 6 intervention included a special study where Network staff collected monthly URR information from each of the 23 intervention centers. These data were used to provide feedback to the individual treatment centers in the intervention and to identify facil- ities that were having difficulty improving care. For centers in this latter group, additional assistance was provided by the Network staff and volunteers [19]. The rate of attainment of the intervention goals, a facility mean URR of 65% and a 50% reduction of patients with a URR of less than 50% varied. The mean time to goal attainment was 273 days, and the extreme was 435 days for the last center (Fig. 4) [19]. Survey data and evaluation The ESRD Core Indicators surveys have been conducted annually since 1994. The distribution of the threemonth mean URR among ESRD patients in the U.S. has shifted substantially toward higher values (Fig. 5) and the national mean has increased from 62.7% in 1993 to 68.0% in 1997 [20]. ESRD Network 6 increased its URR rank from 15th of 16 Networks to 5th in 18 Networks over the same interval. Analysis of the individual Network intervention strategies using these data suggests that interventions that used facility-specific information tem of data collection, this development has been an involved process. Initially, Networks had unique data collection systems that were used to track patient treatment histories (treatment center and modality changes and mortality). The Networks used this information for health services planning and medical care evaluations. The early systems required considerable manual processing, but were quickly computerized. Subsequently, HCFA and the Networks developed regional systems with similar elements and functionality. However, the development of these systems was not centrally organized, and the national data collection system was characterized by multiple data entry platforms and little uniformity in data definitions. The first step toward standardizing Network data sys- tems came with the consolidation to 18 Networks in 1988. The Networks were required to have comparable information systems to enter and transmit forms to HCFA, continue patient tracking, and to support quality assurance and improvement activities like those de- scribed above. HCFA identified a core set of data ele- ments to be maintained. A data entry system called the EDEES (Electronic Data Entry and Editing System) was provided by HCFA to standardize data entry of the three HCFA forms. The shortcoming of the EDEES system was that it only provided for forms entry and did not interface well with the other components of the Networks information systems. ESRD Network 6, with guidance by the Forum of ESRD Networks and under contract with HCFA, began the development of a data entry and transmission system called the Standard Information Management System (SIMS). This system will standardize both the elements being collected and the definitions of those elements. Networks will use SIMS to enter and track HCFA forms, patient events, clinical indicators and facility information; produce core reports needed for business processes; and provide e-mail, Internet and bulletin
S-12 McClellan and Krisher: Dialysis adequacy Fig. 5. Annual distribution of urea reduction ratios for nationally representative samples of adult in-center hemodialysis patients alive on December 31 of each year and who had been treated for at least three months. Reprinted with permission from [20]. board capability. The system will standardize data ele- CONCLUSION ments and definitions, provide standard analysis tools Surveillance systems can play a vital role in efforts to and establish a national, standardized repository of Net- reduce the burden of ESRD by improving the care and work data at the patient level. This will allow Networks reducing the occurence of renal failure within a populafor the first time to electronically track patients across tion. We have described the way one surveillance system Network boundaries. SIMS is now in beta testing and is employed different data sources to develop and use inscheduled to be delivered to the Networks in December formation for a program to improve the quality of hemo- 1999. dialysis treatments. This limited example is intended to Plans are already in progress to expand SIMS with suggest the rich possibilities inherent in the current syssoftware called the Vital Information System to Improve tem of national registries and ESRD surveillance. Outcomes in Nephrology (VISION). VISION will allow facilities to enter and transmit data like that collected ACKNOWLEDGMENTS by the serial Core Indicators surveys electronically to an The analyses upon which this publication is based were performed ESRD Network. The Networks, in turn, can include under Contract Number 500 97-E024, entitled End Stage Renal Disthese data in SIMS. VISION is intended to reduce paing Administration, Department of Health and Human Services. The ease Networks Organization #6, sponsored by the Health Care Financ- perwork burden on facilities by allowing single reporting/ content of this publication does not necessarily reflect the views or entry of data, provide a system for storing and using policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply data for quality improvement, and improve the quality endorsement by the U.S. Government. The authors assume full responof data through standardization, single entry, pretesting sibility for the accuracy and completeness of the ideas presented. This of data. article is a direct result of the Health Care Quality Improvement Program initiated by the Health Care Financing Administration, which VISION will provide the treatment center analysis has encouraged identification of quality improvement projects derived tools and reports to use in quality improvement activities. from analysis of patterns of care, and therefore required no special For example, national and regional comparative practice funding on the part of this contractor. Ideas and contributions to the authors concerning experience in engaging with issues presented are feedback can be transmitted to facilities from the Net- welcomed. works for comparison to local quality of care. VISION will create a virtual private Network using Internet lines Reprint requests to William M. McClellan, M.D., M.P.H., Georgia Medical Care Foundation, 57 Executive Park South NE, Atlanta, GA for transmission. HCFA and ESRD Networks 6 and 9 30329 2224, are working to develop a prototype by the first quarter E-mail: bmcclell@gmcf.org 2000. With the completion of this informatics system, the systematic and ongoing collection, aggregation, analysis, REFERENCES and interpretation of data about the occurence and out- 1. Thacker SB, Berkleman RL: Public health surveillance in the comes of kidney failure in a defined population will be United States. Epidemiol Rev 10:164 190, 1988 2. Thacker SB: Historical development. In: Principles and Practice available in a timely manner for quality improvement of Public Health Surveillance, edited by Teutsch SM, Churchill activities throughout the ESRD system. RE, New York, Oxford University Press, 1994
McClellan and Krisher: Dialysis adequacy S-13 3. Goodman RA, Remington PL, Howard RJ: Communicating infor- stage renal disease patients: Implications for quality improvement. mation for action. In: Principles and Practice of Public Health Am J Kidney Dis 24:280 286, 1994 Surveillance, edited by Teutsch SM, Churchill RE, New York, 15. McClellan WM, Frederick PR, Helgerson SD, Hayes RP, Bal- Oxford University Press, 1994 lard DJ, McMullan M: A data-driven approach to improving 4. Berkleman RL, Buehler JW: Public health surveillance of non- the care of in-center hemodialysis patients. Health Care Financ infectious diseases: The potential to detect rapid changes in disease Rev 16:129 140, 1995 burden. Intern J Epidemiol 19:628 635, 1990 16. Health Care Financing Administration: 1994 ESRD Core Indi- 5. Thacker SB, Stroup DF, Rothenberg RB: Public health surveilmore, MD, Department of Health and Human Services, Health cators Project Initial Results. Opportunities to Improve Care. Baltilance for chronic conditions: a scientific basis for decisions. Stat Med 14:629 641, 1995 Care Financing Administration, Health Standards and Quality Bu- 6. McClellan W: Hypertensive end-stage renal disease in blacks: reau, 1994 17. Health Care Financing Administration: 1994 ESRD Core Indi- The role of end-stage renal disease surveillance. Am J Kidney Dis cators Project Supplemental Report #1: Results of Validation Study: 21 (Suppl. 1):25 30, 1993 Comparison of Data Abstracted by ESRD Facility Staff and by 7. Rettig RA, Levinsky NG: Kidney Failure and the Federal Govern- ESRD Network Staff. Baltimore, MD, Department of Health and ment. Washington DC, National Academy Press, 1991 Human Services, Health Care Financing Administration, Health 8. U.S. Renal Data System: USRDS 1998 Annual Data Report. Standards and Quality Bureau, 1995 Bethesda, MD, National Institutes of Diabetes, Digestive, and 18. Helgerson SD, McClellan WM, Frederick PR, Beaver SK, Kidney Diseases, National Institutes of Health, 1998 Frankenfield DL, McMullan M: Improvement in adequacy of 9. Port FK: Role of observational studies versus clinical trials in delivered dialysis for adult in-center hemodialysis patients in the ESRD research. Kidney Int 57(Suppl 74):S-3 S-6, 2000 United States. 1993 95. Am J Kidney Dis 29:851 861, 1997 10. McClellan WM, Helgerson SD, Frederick PR, Wish JB, 19. McClellan WM, Soucie JM, Krisher J, Caruana R, Haley W, McMullan M: Implementing the Health Care Quality Improvesis: Farmer C: Improving the care of patients treated with hemodialyment Program in the Medicare ESRD Program: a new era of a report from the Health Care Financing Administration s quality improvement in ESRD. Adv Renal Replac Ther 2:89 94, ESRD Core Indicators Project. Am J Kidney Dis 31:584 592, 1998 1995 20. Health Care Financing Administration: 1998 ESRD Core Indi- 11. Stroup NE, Zack MM, Wharton M: Sources of routinely collected cators Project Initial Results. Opportunities to Improve Care. Baltidata for surveillance. In: Principles and Practice of Public Health more, MD, Department of Health and Human Services, Health Surveillance, edited by Teutsch SM, Churchill RE, New York, Care Financing Administration, Health Standards and Quality Bureau, 1998 Oxford University Press, 1994 21. McClellan WM, Frankenfield D, Frederick PR, Flanders 12. Hull AR, Parker TF III: Proceedings from the Morbidity, Mortal- WD, Alfaro-Correa A, Rocco M, Helgerson SD: Can dialysis ity and Prescription of Dialysis Symposium, Dallas, TX, September therapy be improved? A report from the ESRD Core Indicators 15 17. Am J Kidney Dis 15:375 383, 1989 Project. Am J Kidney Dis, in press 13. McClellan WM, Flanders WD, Gutman RA: Variable mortality 22. Mattern W, Krisher J, Garrett L, Hogan S, Gunn J, Twito C: rates among dialysis treatment center. Ann Intern Med 117:332 From the facility to the Network: DxBase moves on to SIMS and 336, 1992 VISION to help track patient outcomes. Nephrol News Issues 14. McClellan WM, Soucie JM: Facility mortality rates for new end- 13:24 26, 1999