The Value of Provider-to-Provider Telehealth

ORIGINAL RESEARCH The Value of Provider-to-Provider Telehealth Eric Pan, M.D., M.Sc., 1,2 Caitlin Cusack, M.D., M.P.H., 1,2 Julie Hook, M.A., M.P.H., 2 Adam Vincent, M.P.P., 2 David C. Kaelber, M.D., Ph.D., 1,2 David W. Bates, M.D., M.Sc., 1,2 and Blackford Middleton, M.D., M.P.H., M.Sc. 1 3 1 Division of General Medicine and Primary Care, Brigham and Women s Hospital, Harvard Medical School, Boston, Massachussetts. 2 Center for Information Technology Leadership, Partners Healthcare, Boston, Massachussetts. 3 Clinical Informatics Research and Development, Partners Information Systems, Partners HealthCare, Wellesley, Massachussetts. Abstract Telehealth has great potential to improve access to care, but its adoption in routine healthcare has been slow. The lack of clarity about the value of telehealth implementations has been one reason cited for this slow adoption. The Center for Information Technology Leadership has examined the value of telehealth encounters in which there is a provider both with the patient and at a distance from the patient. We considered three models of telehealth: storeand-forward, real-time video, and hybrid systems. Evidence from the literature was extrapolated using a computer simulation, which found that the hybrid model was the most cost effective. The simulation predicted savings of $4.3 billion per year if hybrid telehealth systems were implemented in emergency rooms, prisons, nursing home facilities, and physician offices across the United States. We also conducted a sensitivity analysis to determine which factors most influence costs and savings. Payers, providers, and policymakers should work together to remove the barriers to the adoption of telehealth so that this cost savings can be realized in the U.S. healthcare system. Key words: providers, telehealth, computer stimulation Introduction A mericans face an ongoing healthcare crisis. Mounting costs, 1 misaligned reimbursement policies, 2 diminishing access, 3,4 an impending primary care and specialty physician shortage, 5 an aging population, and concerns about quality of care 6 command the attention of both patients and policy-makers. Healthcare information technology (HIT) may play a key role in addressing these problems, but the relative value of many HIT systems is difficult to determine. Telehealth, the transmission of medical information and clinical expertise from one location to another, allowing patients to receive care when and where it is needed, may be a critical component of HIT solutions, but the body of evidence supporting the use of telehealth has been slow to evolve. In 2001, Hersh and colleagues completed an extensive, systematic review of the existing telehealth literature and updated that review in 2006. 7,8 The authors reported that while the number of telehealth programs is growing, evidence of their cost effectiveness was insufficient to make definitive statements supporting the use of telehealth. Studies with small sample sizes and poor research methodologies were cited as key deficiencies. Whitten et al. recently completed a comprehensive review on research methodology in telehealth studies and noted similar issues with the methodologies being used in telehealth research. 9 As a result, significant questions about the value, efficacy, and effectiveness of these technologies remain unanswered. This uncertainty has prompted providers, payers, and policy-makers to call for evidence to guide decision-making regarding the feasibility for and implementing of telehealth systems. To fill this important information gap on the value of telehealth, Center for Information Technology Leadership (CITL) undertook a comprehensive study to estimate the financial impact of telehealth 446 TELEMEDICINE and e-health JUNE 2008 DOI: 10.1089/tmj.2008.0017

PROVIDER-TO-PROVIDER TELEHEALTH technologies in which there is a provider both with the patient and remote to the patient. Store-and-forward, real-time, video, and hybrid technologies were considered in four environments: emergency departments, correctional facilities, nursing homes, and physician offices. A cost benefit model was constructed to estimate the impact of telehealth across the nation and to whom the costs and benefits would accrue. Materials and Methods CITL performed multiple steps to develop a value-based model for provider-to-provider telehealth technologies. 10 First, CITL conducted a literature review to elucidate where value had been reported in the literature. An advisory board was then formed to provide guidance and advice on the current evidence base, analytic framework, telehealth taxonomy, and the telehealth value model and estimates. Finally, a taxonomy was developed to form a framework for the model and necessary data were gathered to project the cost and benefits of telehealth technologies. CITL subsequently combined the cost and benefit projections to predict the national value of providerto-provider telehealth. Details on CITL s approach are outlined in the sections below. SCOPE OF ANALYSIS AND MAJOR ASSUMPTIONS This project looked at interactions in which a registered healthcare professional ( provider ) was on both ends of an outpatient encounter. As such, CITL did not model inpatient care, home monitoring, or medical education. In addition, only clinical encounters were modeled. Interpretive services such as telepathology and teleradiology were excluded. LITERATURE REVIEW CITL used systematic review methods and standard techniques to find, review, and analyze telehealth-related literature in MEDLINE, CINAHL, and EMBASE (K. Kuntz, personal communication, 2003 2004). 11,12 Additional literature sources were also suggested by CITL s telehealth advisory board. Detailed search strategies are beyond the scope of this paper, but can be found in CITL s full technical report. 13 ADVISORY BOARD CITL worked with an advisory board of nationally recognized telehealth experts throughout this project. The board provided review and advice on the analytic framework, taxonomy, model estimates, and preliminary findings. The members of the board are listed in the Acknowledgments. STATISTICAL METHODS CITL used the Delphi consensus technique developed by the RAND Corporation (Santa Monica, CA) to summarize expert opinions and estimates where published evidence on telehealth impact was not available. 14 23 The final telehealth value model was produced using Analytica 3.1 (2006) decision-analysis software (Lumina Decision Table 1: Telehealth Technology Taxonomy LEVEL STORE-AND- REAL-TIME TYPE OF DATA MINIMUM FORWARD TRANSMITTED BANDWIDTH Advanced telehealth (fast broadband) IV Convergence of traditional Convergence: images, High telehealth functionality high-resolution video, EMR (512 Kbps or greater) throughout medicine, including integration with interoperable- EMR systems Modern telehealth (broadband) III Hybrid with high-resolution Images, high-resolution video Medium video and image (364 Kbps) II a. High-resolution b. Low-resolution Images, low-resolution video Low still images video (128 Kbps) Pre-telehealth (narrow band) I E-mail of textual Faxing of textual Electronic transmission Analog modem information information of textual data (<50 Kbps) 0 Postal mail Verbal report Traditional, nonelectronic, Voice network via telephone methods of communication EMR, electronic medical record. MARY ANN LIEBERT, INC. VOL. 14 NO. 5 JUNE 2008 TELEMEDICINE and e-health 447

PAN ET AL. Systems, Los Gatos, CA), and the resultant graphics produced with Microsoft Excel (Microsoft Corp., Redmond, WA). Results TAXONOMY AND FRAMEWORK CITL developed a taxonomy of telehealth technologies based on the minimum bandwidth needed per connection between the two endpoints of the telehealth encounter. Bandwidth availability not only impacts the method with which patient information is transmitted, e.g., store-and-forward (asynchronously) versus real-time (synchronously), but also the amount and the speed at which the data may be transmitted. As such, bandwidth became the basis for CITL s telehealth taxonomy. Based on required bandwidth per connection, we divided the technologies into pre-telehealth, modern telehealth, and advanced telehealth. Pre-telehealth is based on narrow-band networks such as the common voice telephone network or analog modems. Modern telehealth typically use broadband networks such as Integrated Service Digital Networks (ISDN) and consumer-grade Digital Subscriber Line (DSL) networks. Advanced telehealth technologies uses fast broadband networks such as private enterpriselevel network, high-bandwidth business-grade DSL, or optical fiber networks. These telehealth levels are summarized in Table 1. TELEHEALTH BENEFITS For the four healthcare environments in this analysis emergency departments, correctional facilities (CF), nursing homes (NH), and physician offices CITL considered six possible encounter pairings that demonstrated potential values of telehealth technology: (1) correctional facilities to emergency departments (CF to ED); (2) nursing homes to emergency departments (NH to ED); (3) emergency departments to emergency departments (ED to ED); (4) correctional facilities to physician offices (CF to MD); (5) nursing homes to physician offices (NH to MD); and (6) physician offices to physician offices (MD to MD). For each of the encounter pairing types, CITL determined the annual number of encounters and then projected the number of encounters that could be avoided with each type of modern telehealth technologies store-and-forward, real-time, and hybrid based on existing evidence (Table 2). For types 1 and 2, transportation from CF and NH to ED for acute evaluation and treatments, the use of realtime and hybrid telehealth systems could avoid the transports of some subacute cases by managing the patients using local resources guided by remote ED physicians. For type 3, transportation between ED for subspecialty care, real-time and hybrid telehealth systems could be used to bring in subspecialty consultants virtually to advise the local ED personnel, such as in the case of tele-stroke care. For type 4 and 5, the transportation from CF and NH to MD, physicians could use telehealth technologies to conduct virtual visits with patients with the help of local physician extenders to avoid transporting the patients physically. Finally, type 6 is unique in that the pairing does not reflect actual transports from MD to MD; therefore type 6 is not included in Table 2. Instead, type 6 encounters represent the use of telemedicine to avoid the need for patients to physically travel to a consultant s office with the use of virtual visits or teleconsultations from a local healthcare facility. Table 2: Estimated Annual Transports Avoided with Telehealth by Technology TELEHEALTH ENCOUNTER TYPE ENCOUNTERS AT TRANSPORTS AVOIDED WITH TELEHEALTH BASELINE STORE-AND-FORWARD REAL-TIME VIDEO HYBRID CF to ED 94,180 a 0 34,900 39,900 NH to ED 2,699,000 b 0 337,000 387,000 ED to ED 2,204,320 b 0 646,000 850,000 CF to MD 691,000 c 411,000 452,000 543,000 NH to MD 10,100,000 d 4,090,000 5,420,000 6,870,000 CF, correctional facilities; ED, emergency departments; NH, nursing homes; MD, physicians offices. a Center for Information Technology Leadership (CITL) calculated the average inmate transports to emergency departments, (Ref. 30; AH Vo, personal communication, February 16, 2007; C.A. Martino, personal communication, June 13, 2007), 6.2%, and multiplied this by the U.S. correctional population, 1.5 million. 30 b Ref. 40. c CITL multiplied a baseline rate of correction facility to physician offices, 0.45 transports per inmate per year (D. Kendrick, personal communication, May 8, 2007), by the U.S. Federal and State prison population, 1.5 million inmates. 30 d CITL multiplied the average number of physician visits, 6.75, made by the U.S. population aged 65 or older 26 by the U.S. average nursing facility population, 1.5 million. 28 448 TELEMEDICINE and e-health JUNE 2008

PROVIDER-TO-PROVIDER TELEHEALTH Based on the encounters avoided with telehealth, we then estimated cost saving resulting from these avoided encounters (Table 3). These savings were based on the cost and reimbursement of a telehealth encounter being the same as the equivalent face-to-face encounter. In addition, these cost savings took into account that some encounters will not be able to completely replace a face-toface encounter, resulting in the patient needing to see the provider twice, once virtually, and another time in person, effectively doubling the time and professional charges for the care. For example, the consultant may decide that in-person physical examination or minor procedures such as biopsy are needed for the initial assessment, requiring the patient to travel to the consultant in spite of the virtual encounter. CITL derived the visit avoidance rates from literature experiences and expert interviews. On the other hand, seeing a specialist via telemedicine could lead to better communication between the referring physician and the consultant, and could result in more cost-effective care. CITL combined these factors in producing the estimated annual cost avoided with telehealth shown in Table 3. SYSTEM COSTS CITL represented each modern telehealth level with the cost of typical midrange systems. For sensitivity analysis, CITL considered both the least expensive telehealth setup (low-end) and the most sophisticated telehealth system (high-end) in use today. Hardware was assumed to be replaced every 5 years. CITL estimated the expenses involved in implementing and operating each of the telehealth technologies across store-and-forward, real-time video, and hybrid systems. The cost estimates did not include costs such as governance, sales/pre-sales activities (i.e., contracting), directories of providers, and program planning and development, but did include both acquisition (purchase) and annual system maintenance costs. For each NH, ED, MD, and CF in the United States, CITL projected costs over a 10-year period including a 5-year implementation schedule. All acquisition costs were assumed to occur during the first year of implementation and annual costs occurred for each of the following years. Only the store-and-forward, real-time video, and hybrid scenarios were modeled, with the assumption that telephone, fax, and e-mail are already in place and that more advanced telehealth systems do not widely exist today. For installation costs, CITL included costs associated with telehealth implementation. According to the experience of University of Texas Medical Branch (UTMB), the installation for a high-end system takes approximately 2 days for a technician and costs $2,000. 24 For low-end and midrange systems, CITL assumed that the installation would take a half day for the former and a full day for the latter, with an associated cost of $500 and $1,000, respectively. Annual costs represent recurring costs required to maintain these systems, which included warranty, service contracts, upgrades, troubleshooting, equipment depreciation and replacement, training, and support staff. Organizations typically budget for these expenditures as a fixed percentage of the cost of the capital equipment. CITL has estimated these costs as 20% of acquisition costs. Aggregation of costs. To extrapolate costs to the nation, CITL determined the telehealth equipment needed by the facility and the number of installations required at each facility type. The assumptions for costs across each type of care setting are as follows: Emergency Department: One telehealth equipment set for each of the 4,516 emergency departments in the United States. 25 With just over 2.2 million patients a year transported between U.S. Table 3. Estimated Annual Costs Avoided with Telehealth by Technology Level TELEHEALTH ENCOUNTER TYPE COST AT COST AVOIDED WITH TELEHEALTH BASELINE STORE-AND-FORWARD REAL-TIME VIDEO HYBRID CF to ED $158,000,000 0 $51,700,000 $60,300,000 NH to ED $3,620,000,000 0 $259,000,000 $327,000,000 ED to ED $1,390,000,000 0 $408,000,000 $537,000,000 CF to MD $302,000,000 $162,000,000 $171,000,000 $210,000,000 NH to MD $1,290,000,000 $261,000,000 $305,000,000 $479,000,000 MD to MD $28,700,000,000 $3,000,000,000 ($709,000,000) $3,610,000,000 CF, correctional facilities; ED, emergency departments; NH, nursing homes; MD, physicians offices. MARY ANN LIEBERT, INC. VOL. 14 NO. 5 JUNE 2008 TELEMEDICINE and e-health 449

PAN ET AL. emergency departments, 26 this equates to approximately 500 transfers per ED annually, or almost 1.4 per day, which a single set of telehealth equipment could support. Physician Offices: One telehealth equipment set for up to 10 providers to share per site would lead to 312,000 telehealth equipment sets to cover all practices and physicians in the United States. CITL used the American Medical Associations s physician data in order to estimate the number of provider offices and providers in the United States, as well as the average number of providers per office. 27 Nursing Homes: One telehealth equipment set for each of the estimated 16,000 NH in the United States with an estimated daily population of 1.5 million. 28 In a 40-hour work-week, Cumulative net value (billions of dollars) $25 $20 $15 $10 -$10 one set of telehealth equipment is capable of supporting up to 160 follow-up visits at the average visit time of 15 minutes for an established patient, 29 which is more than one visit per resident given the average NH size of 94 residents. Correctional Facilities: One telehealth equipment set for each of the 1,668 prisons in the United States, 30 given that a typical prison has only one healthcare facility. National cost projection. The national projected costs for the midrange telehealth installation are presented in Table 4. The totals were $5 $0 -$5 Store-and-Forward Real-Time Video Hybrid 1 2 3 4 5 6 7 8 9 10 Fig. 1. Cumulative national net benefit projection. Year derived by multiplying the number of installations by the capital, installation, and annual costs for each installation over the national projection timeframe (10 years). National net benefit projection. Figure 1 projects the cumulative net benefit for the nation, combining the benefits for each interaction type shown in Table 3 with the total costs of telehealth installations shown in Table 4. The cumulative net return during the initial 5-year implementation period, the 10-year rollout period, and the future steady-state annual net return is shown in Table 5. Table 4. Total Projected Costs of Telehealth Installation by Site Type, Mid-Range ACQUISITION COSTS (IN THOUSANDS) ANNUAL COSTS (IN THOUSANDS) NUMBER OF STORE-AND- REAL-TIME INSTALLATION STORE-AND- REAL-TIME INSTALLATIONS FORWARD VIDEO HYBRID COSTS FORWARD VIDEO HYBRID MD 312,400 $477,000 $4,180,000 $4,430,000 $312,000 $95,500 $835,000 $887,000 ED 4,516 N/A $60,400 $64,100 $4,520 N/A $12,100 $12,800 NH 16,100 $24,600 $214,000 $228,000 $16,100 $4,920 $42,900 $45,500 CF 1,668 $2,550 $22,200 $23,600 $1,670 $510 $4,440 $4,720 Total 334,684 $504,150 $4,476,600 $4,745,700 $334,290 $100,930 $894,440 $950,020 MD, physicians offices; ED, emergency departments; NH, nursing homes; CF, correctional facilities; N/A, not applicable. 450 TELEMEDICINE and e-health JUNE 2008

PROVIDER-TO-PROVIDER TELEHEALTH Limitations While CITL based our conclusions on systematic review of literature evidences, the results are the product of a computer model that extrapolated the existing evidence, not the actual experience of implementing telehealth systems nationwide. In addition, we only modeled provider-to-provider telehealth and focused on four care settings and six encounter types. There are many other telehealth encounters such as teleradiology, telepathology, and remote medical education that were not included in this model. Therefore, these results should not be interpreted as representing the total benefits of all telehealth activities. In addition, this analysis is focused on the costs and benefits of the technical system, and does not include potential cost and benefits from workflow re-engineering, credentialing and privileging, and cross-state licensure. Discussion Overall, CITL has found that the potential benefits of telehealth systems far outweigh the implementation costs. From our extensive literature review, this is the first study to examine the potential value of national adoption of telehealth technologies across a wide range of clinical settings. Whereas earlier studies and reviews were limited by small sample sizes and poor research methodologies, 7,8 this systematic review leveraged existing studies, domain expert inputs, and modeling methodologies to integrate and project potential value of telemedicine systems. Of the three scenarios, the hybrid scenario is projected to be the most cost-effective system. Equipping all U.S. emergency departments with hybrid telehealth technologies could potentially be covered by savings from a reduction in transfers between emergency departments. The $64 million acquisition cost for equipping all emergency departments with hybrid telehealth systems is 12% of the estimated annual savings of $537 million, and could potentially be recovered in less than 2 months of full system operation. For correctional facilities, the costs of hybrid telehealth equipment could be covered by savings from a reduction in transporting patients to EDs and to MDs, and by avoiding the costs of the ED visit. Assuming that EDs and MDs have the appropriate telehealth equipment, CFs only need to invest $23.6 million in acquisition costs to avoid nearly 600,000 transports with the projected potential annual savings of $270 million, over a 10-fold return. For NHs, the costs of implementing hybrid telehealth equipment could be covered by savings from a reduction in transferring residents to EDs and MDs, and by avoiding the costs of the ED visit. Hybrid systems could avoid over 7.25 million transports and save over $800 million. Since outfitting all NH with hybrid telehealth systems cost only $228 million, telehealth is clearly a smart investment for NH once the local MDs and EDs are equipped to support telehealth visits. For teleconsults between MDs, there is a loss to the system from provider-to-provider teleconsults when considering only professional fees; however, when patient care costs are considered, providerto-provider teleconsults potentially achieve savings far exceeding the loss in professional fees. More professional fees are incurred in telehealth systems because some teleconsultations will still require in-person assessment or procedures, which result in a higher number of visits. However, CITL also found that this loss could be far outweighed by involving specialists early in the care of a patient and reducing the number of redundant or unnecessary tests. In reducing face-to-face visits and redundant and unnecessary tests, hybrid technologies are projected to have a cost savings of $3.61 billion. From an overall cost benefit perspective, the potential benefit of implementing telehealth technologies in these areas far outweighs the costs. With a 5-year roll-out, the first-year national cost for hybrid technology is $254 million, with a mid-implementation, third-year peak of $2.78 billion, and a steady-state, ongoing annual cost of $950 million. Nationwide implementation of hybrid technologies reaches a break-even point in year 5, with a total annual net savings of $4.28 billion in the steady-state. Beyond what we demonstrated from this model, evidence in the literature suggests additional benefit from the use of telehealth tech- Table 5. National Net Benefits IMPLEMENTATION (5-YEAR) INITIAL 10-YEAR STEADY STATE CUMULATIVE NET CUMULATIVE NET ANNUAL NET Store-and-forward $4,940,000,000 $20,800,000,000 $3,330,000,000 Real-time video -$5,750,000,000 -$7,900,000,000 -$409,000,000 Hybrid $2,130,000,000 $22,300,000,000 $4,280,000,000 MARY ANN LIEBERT, INC. VOL. 14 NO. 5 JUNE 2008 TELEMEDICINE and e-health 451

PAN ET AL. nologies in provider-to-provider care settings. Telehealth technologies can lead to a reduction in hospital admissions from emergency departments, 31 as well as a reduction in the need for referrals from EDs to outside specialists. 32,33 The impact of increasing access to care through telehealth is potentially large, improving the quality of care provided, and likely improving clinical outcomes. Increasing the speed of a diagnosis in cases where rapid diagnosis is linked to improved outcomes also is impacted by telehealth technologies, such as is the case with management of acute strokes. 34,35 The use of telehealth technologies in ambulances can also help speed the time to diagnosis and the initiation of important, potentially lifesaving interventions. 36 CITL has found in previous studies that healthcare technologies frequently have costs borne by provider organizations (e.g., hospitals and physician offices), resulting in savings that accrue to payers. 37 39 This also holds true for provider-to-provider telehealth technologies. This finding has critical implications for the entire healthcare system and is a reflection of the traditional third-party payer system. However, in this study, CITL found one clear exception to this finding: As closed systems, correctional facilities bear both the costs and the benefits of telehealth technologies. The reduction in the need to transport prisoners outside of the facility adds directly to the facility s bottom line. This, however, is not the case for EDs, NHs, or MDs, where the costs of the telehealth system are borne by those facilities, while much of the benefit accrues to the payers. These findings should provide an impetus for payers to support the implementation of these systems and begin to reap the benefits that they provide. Despite these positive economic findings, CITL recognizes the existence of other barriers to the implementation and full adoption of telehealth technologies. Given the potential impact that telehealth can have on improving access to and quality of care, it is imperative that these other barriers to adoption be addressed head on and steps taken to remove them. Major barriers needing to be addressed include: (1) a reimbursement model that favors face-to-face visits; (2) concerns around medical liability; and (3) a lack of cross-state licensure. In the end, even considering all the limitations on the current evidence on telemedicine, it is clear that the broad integration of telehealth technologies into clinical practice could produce quantum leaps in the efficiency of the healthcare system. Healthcare stakeholders, providers, and payers alike should not be fearful of whether telehealth might lead to an increase in the number of visits or increase utilization from demands previously unmet. Any of those increases will be overshadowed by the dramatic reduction in costs associated with decreased unnecessary tests, improved disease prevention, and improved chronic disease management that will come from a broad telehealth deployment, where we can virtually bring the collective wisdom of the entire healthcare system to any patient, anywhere, any time. Acknowledgments CITL would like to acknowledge the contributions of our telehealth advisory board members: Karen E. Edison, M.D.; Joseph C. Kvedar, M.D.; Jonathan Linkous; Hon S. Pak, M.D.; Jay H. Sanders, M.D.; Joseph A. Tracy, M.S.; and Ronald S. Weinstein, M.D. REFERENCES 1. CMS (Centers for Medicare and Medicaid Services). Public payors share of national health spending 1980 2005. CMS 2005. 2. American College of Physicians. Reform of the dysfunctional healthcare payment and delivery system. Philadelphia, PA: American College of Physicians; 2006. 3. Lesser CS, Ginsburg PB, Felland LE. Initial findings from HSC s 2005 site visits: Stage set for growing health care cost and access problems. Washington, DC: Center for Studying Health System Change, 2005. Report no.: Issue brief no. 97. 4. Shortage designation: HPSA & MUA/MUP [homepage on the Internet]. [cited 7/11/2007]. Available from: http://bhpr.hrsa.gov/shortage (Last accessed February 5, 2008). 5. Council on Graduate Medical Education. Sixteenth report: Physician workforce policy guidelines for the United States, 2000 2020. Rockville, MD: U.S. Department of Health and Human Services, 2005. 6. Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press, 2001. Report no. 0-309-07280-8. 7. Hersh W, Patterson PK. Telemedicine for the medicare program. Rockville, MD: AHRQ Publication, 2001. Report no. Evidence Report/Technology Assessment no. 24. 8. Hersh W, Hickam DH, Severance SM, Dana TL, Krages K, Helfand M. Telemedicine for the medicare population: Update. Rockville, MD: Agency for Healthcare Research and Quality, 2006. Report no. 06-E007. 9. Whitten P, Johannessen LK, Soerensen T, Gammon D, Mackert M. A systematic review of research methodology in telemedicine studies. East Lansing, MI: Michigan State University, 2007, Tuesday, May 15. 10. Kendrick DC, Bu D, Pan E, Middleton B. Crossing the evidence chasm: Building evidence bridges from process changes to clinical outcomes. J Am Med Inform Assoc 2007;14:329 339. 11. Mulrow CD, Cook D. Systematic reviews: Synthesis of best evidence for health care decisions. London: Royal Society of Medicine Press, 1998. 12. Evidence-based practice center website [homepage on the Internet]. Available from: http://healthpolicy.stanford.edu/stanford-ucsf-epc (Last accessed February 5, 2008). 13. Cusack CM, Pan E, Hook JM, Vincent A, Kaelber DC, Bates DW, et al. The value of provider-to-provider telehealth technologies. Boston, MA: Center for Information Technology Leadership, 2007. 14. Linstone HA, Turoff M. The delphi method: Techniques and applications. Newark, NJ: New Jersey Institute of Technology, 2002. 452 TELEMEDICINE and e-health JUNE 2008

PROVIDER-TO-PROVIDER TELEHEALTH 15. Sackman H. Delphi assessment: Expert opinion, forecasting, and group process. Santa Monica, CA: Rand Corp., 1974. Report no. R-1283-PR. 16. Dalkey NC, Rourke DL. Experimental assessment of delphi procedures with group value judgments. Washington, DC: RAND Institute, 1971. Report no. R- 612-ARPA. 17. Brown B, Cochran SW, Dalkey NC. The delphi method, II: Structure of experiments. Santa Monica, CA: Rand Institute, 1969. Report no. RM-5957-PR. 18. Dalkey NC. The delphi method: An experimental study of group opinion. Santa Monica, CA: RAND Institute, 1969. Report no. RM-5888-PR. 19. Dalkey NC, Brown BB, Cochran SW. The delphi method, III: Use of self-ratings to improve group estimates. Santa Monica, CA: RAND Institute, 1969. Report no. RM-6115-PR. 20. Brown BB. Delphi process: A methodology used for the elicitation of opinions of experts. Santa Monica, CA: RAND Institute, 1968. Report no. P-3929. 21. The delphi method, II structure of experiments [homepage on the Internet]. Available from: http://rand.org/pubs/research_memoranda/rm5957 (Last accessed February 5, 2008). 22. Linstone HA, Turoff M. The delphi method: Techniques and applications. Newark, NJ: New Jersey Institute of Technology, 2002. 23. Delphi assessment expert opinion, forecasting, and group process [homepage on the Internet]. Available from: http://rand.org/pubs/reports/r1283 (Last accessed February 5, 2008). 24. Hammack G. Galveston, TX: University of Texas, medical branch (UTMB). 2007. 25. Burt CW, McCaig LF. Staffing, capacity, and ambulance diversion in emergency departments: United States, 2003 04. Adv Data 2006;376:1 23. 26. Hing E, Cherry DK, Woodwell DA. National ambulatory medical care survey: 2004 summary. Adv Data 2006;374:1 33. 27. American Medical Association. Physician characteristics and distribution in the US. 2005 ed. Chicago, IL: AMA Press, 2005. 28. National Center for Health Statistics. National nursing home survey (NNHS). Centers for Disease Control and Prevention. Atlanta, GA: Centers for Disease Control and Prevention, 2004. Report no. 2004 Facility Tables, Table 1. 29. Sophocles A. Time is of the essence: Coding on the basis of time for physician services. Fam Pract Manag 2003;10:27 31. 30. Bureau of Justice Statistics. Prisoners in 2005. Washington, DC: US Department of Justice; 2006. Report no. NCJ 215092. 31. Stamford P, Bickford T, Hsiao H, Mattern W. The significance of telemedicine in a rural emergency department. IEEE Eng Med Biol Mag 1999;18:45 52. 32. Darkins A, Dearden CH, Rocke LG, Martin JB, Sibson L, Wootton R. An evaluation of telemedical support for a minor treatment centre. J Telemed Telecare 1996;2:93 99. 33. Beach M, Goodall I, Miller P. Evaluating telemedicine for minor injuries units. J Telemed Telecare 2000;6(suppl 1):S90 S2. 34. Schwamm LH, Rosenthal ES, Hirshberg A, Schaefer PW, Little EA, Kvedar JC, et al. Virtual TeleStroke support for the emergency department evaluation of acute stroke. Acad Emerg Med 2004;11:1193 1197. 35. LaMonte MP, Bahouth MN, Hu P, Pathan MY, Yarbrough KL, Gunawardane R, et al. Telemedicine for acute stroke: Triumphs and pitfalls. Stroke 2003;34:725 728. 36. Haskins PA, Ellis DG, Mayrose J. Predicted utilization of emergency medical services telemedicine in decreasing ambulance transports. Prehosp Emerg Care 2002;6:445 448. 37. Johnston D, Pan E, Walker J, Bates DW, Middleton B. The value of computerized provider order entry in ambulatory settings. Chicago: Health Information Management and Systems Society, 2003. 38. Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The value of health care information exchange and interoperability. Health Aff (Millwood) 2005;(suppl Web exclusives):w5,10-w5-18. 39. Bu D, Pan E, Johnston D, Walker J, Adler-Milstein J, Kendrick D, et al. The value of information technology-enabled diabetes management. Chicago, IL: Healthcare Information and Management Systems Society, 2007. 40. McCaig LF, Nawar EW. National hospital ambulatory medical care survey: 2004 emergency department summary. Adv Data 2006;1 29. Address reprint requests to: Eric Pan, M.D., M.Sc. Center for Information Technology Leadership Partners Healthcare One Constitution Center Boston, MA 02192 E-mail: epan@partners.org Received: February 6, 2008 Accepted: April 18, 2008 MARY ANN LIEBERT, INC. VOL. 14 NO. 5 JUNE 2008 TELEMEDICINE and e-health 453