Informing the National Public Health Accreditation Movement

Informing the National Public Health Accreditation Movement Informing the National Public Health Accreditation Movement: Lessons From North Carolina s Accredited Local Health Departments Mary V. Davis, DrPH, MSPH, Margaret M. Cannon, MPH, David O. Stone, MS, Brittan W. Wood, MPH, Joy Reed, EdD, RN, and Edward L. Baker, MD, MPH, MSc To advance understanding of public health accreditation, we analyzed data on the North Carolina Local Health Department Accreditation program. We surveyed accredited health departments on barriers to and supports of accreditation preparation, performance on accreditation standards, and benefits and improvements after accreditation. All 48 accredited agencies responded to the survey. All agencies improved policies to prepare for accreditation and met most accreditation standards. Forty-six percent received local funds for accreditation preparation. The most common barrier to accreditation preparation was time and schedule limitations (79%). Fifty percent of agencies acted on suggestions for improvement, and 67% conducted quality improvement activities. Benefits of accreditation included improvements in local partnerships. Agencies of all sizes conducted accreditation activities, were successfully accredited, and experienced benefits resulting from accreditation. (AmJPublicHealth.2011;101: 1543 1548. doi:10.2105/ajph. 2011.300199) A KEY STRATEGY FOR IMproving the functioning of local health departments (LHDs) is to institute a public health accreditation system that requires LHD capacity and performance to be measured against benchmarks or standards. 1 A voluntary national accreditation program for local public health agencies, the Public Health Accreditation Board (PHAB), was recently established. 2 PHAB s mission is to facilitate improvement of state, local, and tribal public health departments while emphasizing that accreditation is not the ultimate goal but rather part of an overall strategy to improve the health of the nation. Although the PHAB standards have been developed, the PHAB process beta test is complete, and state accreditation programs are under way, several public health accreditation issues are still being debated. These include the level of performance at which standards are written (i.e., capacity vs outcomes), accreditation s costs relative to its benefits, barriers to preparing for accreditation, benefits of accreditation, and the ability of health departments of varying sizes and structures to meet accreditation standards. 3-7 Chief among the debated issues is how to design an accreditation program that supports and fosters a quality improvement mindset in public health, rather than encouraging practitioners to see accreditation as an end in itself. 2 Literature from the health care, education, social service, and public service fields suggests that public health accreditation can have positive effects on service quality, operations, and service-related outcomes. 8 However, the literature on public health accreditation s impact on service capacity, delivery, and quality is in its nascent phase. 9 The PHAB program is modeled on several state-based LHD accreditation systems, including the North Carolina Local Health Department Accreditation (NCLHDA) program. The NCLHDA history and program elements have been previously described. 6,10 Initiated in 2004, with final state rules in place in 2006, this legislatively mandated program s primary objectives are to increase the capacity, accountability, and consistency of the policies and practices of all North Carolina LHDs. Accreditation benchmarks are written primarily at the capacity-achievement level. 10 As of July 2009, 48 of the 85 North Carolina LHDs had achieved accredited status under final state rules. (Two additional agencies received accreditation under pilot standards but had not achieved accreditation under the final rules.) Ten LHDs per state fiscal year will go through initial accreditation review, and all LHDs are required to have been reviewed by 2014. Each LHD was allowed to choose the first fiscal year during which it would undergo initial accreditation. Accreditation status is valid for 4 years, after which LHDs must go through a reaccreditation process. Crosswalks between PHAB and North Carolina accreditation standards yield 95% overlap on the content of the standards. NCLHDA partners built evaluation into the program from its inception to ensure that the program performed as intended and September 2011, Vol 101, No. 9 American Journal of Public Health Davis et al. Peer Reviewed Government, Politics, and Law 1543

met the needs of stakeholders. 11 Program evaluation reports reveal that 86% of participants (health directors, LHD staff, site visitors, and North Carolina Division of Public Health staff) are very satisfied with the accreditation administrator s management of the program, and 92% of LHD directors report that they are satisfied with the accreditation program s outputs relative to the time they and their staff spent to prepare for accreditation. 12-14 The NCLHDA program s outputs are LHD achievement of accreditation standards and LHD activities after accreditation to continue infrastructure and service improvements. To date, no study has examined outputs of public health accreditation programs in light of LHD preparation activities, barriers to preparation, and benefits of participating in the program. To advance understanding of public health accreditation, we examined the extent to which the 48 accredited North Carolina LHDs (1) conducted accreditation preparation activities, experienced barriers to accreditation preparation, received support for accreditation preparation, and met the accreditation program standards; (2) conducted quality improvement activities after achievement of accredited status; and (3) experienced benefits as a result of being accredited. METHODS We reviewed data on LHD achievement of accreditation standards from program site-visit reports, and we collected data on accreditation preparation activities from evaluation reports. Both types of reports covered the years 2006 through 2009. We also conducted an online survey of the 48 LHDs that achieved accreditation through the final rules process. We developed survey items on the basis of questions in the literature, review of evaluation reports, and consultation with program stakeholders. Survey items included 2 accreditationpreparation questions: we asked whether LHDs had received local funds to prepare for accreditation, and we asked whether LHDs had faced barriers to preparing for accreditation. The remaining survey items assessed the extent to which LHDs had 1. updated policies and procedures after achieving accreditation; 2. acted upon site visitors suggestions for improvement; 3. conducted quality improvement activities after the site visit; 4. experienced benefits after being accredited; and 5. experienced barriers to conducting improvements after achieving accreditation. The survey included quantitative questions about specific activities and benefits, followed by open-ended questions that asked respondents to explain how activities were conducted or benefits were experienced. We reviewed the survey items with the quality improvement and accreditation subcommittee of the North Carolina Association of Local Health Directors, and we solicited their feedback on the items. We e-mailed the survey invitation and link to the 48 local health directors, along with instructions for survey completion. Multiple e-mail and face-to-face reminders encouraged LHDs to respond. Three health directors from accredited agencies also contacted their colleagues to encourage them to respond. Data analysis included summarizing program and evaluation data frequencies and percentages, compiling survey frequencies and percentages, and reviewing openended survey items to identify themes and explanations for closed-ended items. RESULTS The 48 accredited North Carolina LHDs served populations that ranged from 13851 to 894290. Most were single-county LHDs, but 2 were multicounty LHDs serving 3 and 7 counties each. Table 1 shows the population sizes served by accredited and nonaccredited North Carolina LHDs. More than two thirds of the accredited LHDs served medium-sized populations (50000---499999). The percentage of accredited LHDs that served medium-sized populations was higher than was the percentage of nonaccredited LHDs serving those populations; conversely, the percentage of accredited LHDs serving small populations (<50 000) was lower than was the percentage of nonaccredited LHDs serving those populations. These differences were not statistically significant. As a group, accredited LHDs met nearly all the accreditation standards: there were 60 total occurrences of unmet standards out of 7104 observations, for a rate of unmet standards of less than 1%. Fourteen agencies met all standards, 19 met all but 1 standard, 8 missed 2 standards, 5 missed 3, and 2 missed 4. Of the 148 standards in the self-assessment, 25 standards were not met by at least 1 LHD, and 3 standards were not met by 5 or more LHDs (Table 2). As part of the program evaluation, LHDs reported their activities to prepare for accreditation. All 48 LHDs updated or created new policies or TABLE 1 Local Health Departments, by State Accreditation Status and Size of Population Served: North Carolina, 2006 2009 Population Size Accredited LHDs (n= 48), No. (%) Not Accredited LHDs (n=37), No. (%) All LHDs (n=85), No. (%) <50 000 14 (29.2) 16 (43.2) 30 (35.3) 50000 499999 33 (68.7) 20 (54.1) 53 (62.3) 500 000 1 (2.1) 1 (2.7) 2 (2.4) Note. LHD= local health department. 1544 Government, Politics, and Law Peer Reviewed Davis et al. American Journal of Public Health September 2011, Vol 101, No. 9

TABLE 2 State Accreditation Standards Not Met by 5 or More Accredited Local Health Departments: North Carolina, 2006 2009 Accreditation Standard Language No. of Agencies Not Meeting Standard Activity 7.3: The local health department shall investigate and respond to environmental health complaints or referrals. 9 Activity 30.10: The local health department shall make efforts to prohibit the use of tobacco in all areas and grounds within fifty 5 (50) feet of the health department facility. Activity 31.4: The local health department shall have current written position descriptions and qualifications for each staff position. 6 procedures during the preparation process, and most LHDs reported updating or creating multiple policies or procedures to prepare for accreditation. Accreditation and Funding All 48 accredited LHDs completed the accreditation follow-up survey. In response to a survey item asking whether agencies had received funding to prepare for accreditation, 22 LHDs (46%) reported receiving local funds to prepare for accreditation. These funds were used to cover salary costs for accreditation preparation, facilities improvements (e.g., signage, medical records storage and security), and office supplies. One LHD received extensive funding ($1 million) from its county commissioners to cover the costs of filling new positions and renovating the health department building to comply with standards and improve workspace organization. One survey item asked whether LHDs had, as a result of being accredited, experienced increase or maintenance of funding levels from the county government or receipt of new funding. Agencies that said any of these things had happened were asked by an open-ended item to explain the funding benefit. Two agencies reported that funding from county commissioners had increased as a result of accreditation, 4 agencies had maintained county funding, and 2 agencies had received new grant funding as a result of being accredited. Increased funding went to support staff salaries and building improvements. The LHDs that had maintained funding used these resources to support administrative staffing needs and general services. The LHD that provided information on new grant funding indicated that it had received several small grants (sources not identified) to support a variety of activities, including online registration of applications for septic systems, foreignlanguage interpreters, and HIV services. Improvements and Benefits After Accreditation Table 3 provides data on LHDs improvement activities after accreditation and relationship benefits they experienced after achieving accreditation. Twenty-four LHDs (50%) reported acting on suggestions for improvement identified by site visitors, including updating position descriptions, organization charts, and strategic plans; improving client privacy in waiting and clinic areas; and increasing board of health diversity. Thirty-two agencies (67%) reported conducting quality improvement activities after achieving accreditation. The majority of these agencies (56%) conducted 1 or 2 quality improvement projects, 25% conducted 3 or more projects, and 19% conducted 5 or more projects. Processes used included the Institute for Healthcare Improvement s model for improvement, Lean, Six Sigma, and quality improvement tools such as Pareto charts. LHDs explained their projects as follows: Used team improvement program to conduct H1N1 response, and it was extremely successful and flexible enough to meet the changing circumstances of this major outbreak. Improved customer service by reducing wait time and total patient visit time by evaluating clinic patient flow and identifying areas for improvement. TABLE 3 Improvement Activities or Relationships That Benefited After State Accreditation for Local Health Departments: North Carolina, 2010 Improvement or Benefit, No. (%) No Improvement or Benefit, No. (%) Improvement activities Updated polices 45 (94) 3 (6) Acted on suggestions for QI 24 (50) 24 (50) Conducted a QI project 32 (67) 16 (33) Relationships that benefited County commissioners 11 (23) 37 (77) Community partners 15 (32) 33 (68) Local hospital 11 (23) 37 (77) Board of health 27 (56) 21 (44) Note. QI= quality improvement. September 2011, Vol 101, No. 9 American Journal of Public Health Davis et al. Peer Reviewed Government, Politics, and Law 1545

We created a short service clinic to streamline services such as TB skin testing.... This resulted in a significantly shortened waiting period and improved customer satisfaction. Accredited LHDs reported improved relationships with county commissioners (23%), community partners (32%), and local hospitals (23%) after achieving accreditation. LHDs provided the following accounts of these relationship improvements: Accreditation brought forth good press, thus the commissioners recognized the value of the health department. It has improved the general image of the department by substantiating the level of quality and professionalism of the organization and its people. The accreditation process and our achievement are viewed by our partners as being grounded in best practice principles and therefore validate the organization s commitment to its vision of being a model of best practice in public health. Twenty-seven (56%) agencies reported improved relationships with their local board of health, which included general improvements in relationships, increased attendance at board of health meetings, and improved board member understanding of their role as it relates to the LHD s function. On an open-ended item, 22 agencies identified additional benefits of accreditation, which we coded into 13 themes. The most commonly reported benefits were pride as a result of achieving accreditation (7 LHDs), improved policies and processes (5 LHDs), team building and teamwork (4 LHDs), and improved staff appreciation of public health services and functions (4 LHDs). One respondent commented that being accredited had helped agency staff feel that, when it came to LHD capacity, LHD size did not matter: In essence, we realized that whether large or small, things equate out when it comes to benchmarks, organization and structure, and staffing (even if some of our staff do several jobs). Barriers to Accreditation and Implementing Improvements The survey provided a list of barriers to accreditation and implementing improvements after accreditation, and LHDs were asked to check those that applied to them. The most common barriers to preparing for accreditation were time and schedule limitations (79%), resource limitations (50%), and lack of perceived value or benefit of accreditation (42%; Table 4). The most common barriers to implementing improvements were the same as preparing for accreditation, but with fewer agencies identifying each of these as barriers 52%, 46%, and 15%, respectively. These survey items specifically assessed whether lack of county support or board of health support were barriers to preparation or conducting improvements. Only 1 LHD reported that lack of support from the county was a barrier to accreditation preparation. DISCUSSION This review of NCLHDA program data and survey results provides new insights regarding what can be expected from public health accreditation programs. The agencies accredited through the NCLHDA program met nearly all accreditation standards, even though agencies only needed to meet 80% of standards to be accredited. This level of achievement, however, resulted from considerable accreditation preparation, a fact that may validate the original intent of the program: to improve the capacity, accountability, and consistency of North Carolina LHDs policies and practices by setting the accreditation bar at capacity level. As North Carolina LHDs go through reaccreditation, they will demonstrate quality improvement to meet accreditation standards, and LHD activities will be monitored accordingly. It is possible that the North Carolina LHDs that chose to undergo accreditation in the early years of the program were already high performers, a pattern that has been observed in the public service industry accreditation literature. 8 An examination of the first 37 North Carolina agencies accredited found that these agencies tended to serve larger populations, have higher public health expenditures, and have more full-time---equivalent employees. 15 These are factors identified in the public health systems and services research as being associated with higher performance. 16-18 Although the North Carolina---accredited agencies may be more likely to have factors present indicating the potential for high performance, all North Carolina-accredited agencies needed to conduct considerable work to prepare for accreditation. Thus, factors associated with high performance may not indicate readiness to apply for accreditation. A critical reported barrier to preparing for accreditation is the perceived cost of employee time and other resources needed to be successful, which is exacerbated by the concern that local funding will not be available to assist agencies with the process. 4 Nearly half of the North Carolina---accredited LHDs TABLE 4 Barriers to Preparing for and Implementing Improvements After State Accreditation: Local Health Departments, North Carolina, 2010 Barrier Experienced During Preparations, No. (%) Experienced While Implementing Improvements, No. (%) Resource limitations 24 (50) 22 (46) Time and schedule limitations 38 (79) 25 (52) Lack of county support 1 (2) 3 (6) Lack of staff support 6 (13) 3 (6) Lack of perceived value or benefit 20 (42) 7 (15) Not seen as a priority 13 (27) 4 (8) 1546 Government, Politics, and Law Peer Reviewed Davis et al. American Journal of Public Health September 2011, Vol 101, No. 9

reported receiving local funds to prepare for accreditation. Local governments may be more willing to support accreditation preparation activities than was previously thought, especially when an LHD needs funds to make improvements to meet standards, such as facilities improvements. Nevertheless, half of the accredited agencies reported that resource limitations were a barrier to preparing for accreditation, but this item did not specify whether these were financial limitations. Moreover, 79% of LHDs reported time and schedule limitations as a barrier. Additional research may be needed to further explore the relative importance of these barriers. Among North Carolina--- accredited LHDs, 50% reported acting on suggestions for quality improvement identified by site visitors, and two thirds reported conducting quality improvement projects. Previous surveys of North Carolina agencies and of health departments nationwide found that agencies were conducting quality assurance activities rather than quality improvement activities (North Carolina Institute of Public Health, unpublished data, 2010). 19 Quality improvement activities reported in the current study included the use of more true quality improvement techniques, including aim statements and use of methodologies specific to quality improvement. It is difficult to discern whether this shift to quality improvement work is attributable to participation in the accreditation program or to the plethora of public health efforts to advance quality improvement in LHDs. 20 Nevertheless, in NCLHDA program evaluations and research projects, LHD directors and staff indicated that accreditation served as a platform for performance improvement. Thus, there may be at least some relationship between participating in accreditation and conducting quality improvement activities. Previous reviews of NCLHDA evaluation data revealed that agencies experienced considerable benefits as a result of preparing for accreditation, such as increased teamwork and improvements in partnerships with hospitals and community groups. 4 Our results show improvements in relationships with these partners as well as with county commissioners and boards of health. These benefits, which are related to improved community perception of LHDs, may help balance the perceived cost/benefit ratio of public health accreditation. 4,8 In contrast to previously reported perceptions of the ability of small LHDs to meet accreditation standards, 3-5 North Carolina--- accredited agencies of all sizes, including 14 that served populations of less than 50000, were able to meet accreditation standards. According to 2008 profile data from the National Association of County and City Health Officials, among health departments that served populations of less than 50000 only 12% of respondents strongly agreed that their LHD would seek accreditation, and 35% agreed. 21 This is likely because of previous reports on the perceived costs and benefits of accreditation and the perceived ability of small agencies to meet accreditation benchmarks. 3-5 Results from the review of NCLHDA program data and this survey may dispel the perception that LHDs that serve small populations will experience undue difficulties in achieving accreditation standards. Because of the cross-sectional nature of this study, we cannot assert that participation in the accreditation program caused any of the specific outputs studied. Limited resources for this work and a small sample size precluded us from conducting more sophisticated analyses on these data. Future studies could examine differences in outcomes (particularly among LHDs that serve different sizes of populations), performance on accreditation activities, benefits experienced as a result of accreditation, and factors associated with performance in the public health systems and services literature. Although the NCLHDA program activities are written primarily at a capacity level, the North Carolina---accredited agencies profiled in this study conducted considerable improvement activities before and after accreditation. These agencies reported experiencing a variety of benefits from being accredited, including improvementsinpartnershipswiththeir county governments and boards of health. Our findings suggest that agencies of all sizes are able to meet accreditation standards and experience the benefits of accreditation. j About the Authors Mary V. Davis, David O. Stone, Brittan W. Wood, and Edward L. Baker are with the North Carolina Institute of Public Health, Gillings School of Global Public Health, University of North Carolina at Chapel Hill. At the time of the study, Margaret M. Cannon was with the North Carolina Institute of Public Health, Gillings School of Global Public Health, University of North Carolina at Chapel Hill. Joy Reed is with the Division of Public Health, North Carolina Department of Health and Human Services, Raleigh. Correspondence should be sent to Mary V. Davis, North Carolina Institute of Public Health, CB 8165, Chapel Hill, NC 27599 (e-mail: Mary_davis@unc.edu). Reprints can be ordered at http://www.ajph.org by clicking the Reprints/Eprints link. This article was accepted February 14, 2011. Contributors M. V. Davis, M. M. Cannon, D. O. Stone, B. W. Wood, and E. L. Baker conceptualized the study and created data-collection instruments and procedures. M. V. Davis and M. M. Cannon implemented the survey. M. V. Davis conducted data analyses and prepared article drafts. M. M. Cannon, D. O. Stone, B. W. Wood, J. Reed, and E. L. Baker reviewed versions of the article and provided substantive comments on them. Acknowledgments This work was supported by funds from the North Carolina General Assembly. The authors would like to thank Leah Devlin, DDS, for her contributions to the development of this work. Human Participant Protection The public health---nursing institutional review board of the University of North Carolina at Chapel Hill determined that this study protocol did not require institutional review board approval. References 1. Institute of Medicine. The Future of the Public s Health in the 21st Century. Washington, DC: National Academies Press; 2003. 2. Bender K, Halverson PK. Quality improvement and accreditation: what might it look like? J Public Health Manag Pract. 2010;16(1):79---82. 3. Bender K, Benjamin G, Carden J, et al. Final recommendations for a voluntary national accreditation program for state and local health departments: steering committee report. J Public Health Manag Pract. 2007;13(4):342---348. September 2011, Vol 101, No. 9 American Journal of Public Health Davis et al. Peer Reviewed Government, Politics, and Law 1547

4. Davis MV, Cannon MM, Corso L, Lenaway D, Baker EL. Incentives to encourage participation in the national public health accreditation model: a systematic investigation. Am J Public Health. 2009;99(9):1705---1711. 5. Meit M, Harris K, Bushar J, Piya B, Molfino M. Rural Public Health Agency Accreditation: Final Report. Bethesda, MD: Walsh Center for Rural Health Analysis, NORC at the University of Chicago; 2008. 6. Davis MV, Reed J, Devlin LM, Michalak CL, Stevens R, Baker E. The NC accreditation learning collaborative: partners enhancing local health department accreditation. J Public Health Manag Pract. 2007;13(4):422---426. 7. Nolan P, Bialek R, Kushion ML, Lenaway D, Hamm MS. Financing and creating incentives for a voluntary national accreditation system for public health. J Public Health Manag Pract.2007; 13(4):378---382. 8. Mays G. Can Accreditation Work in Public Health? Lessons Learned From Other Industries. Princeton, NJ: Robert Wood Johnson Foundation; 2004. 9. Joly BM, Polyak G, Davis MV, et al. Linking accreditation and public health outcomes: a logic model approach. J Public Health Manag Pract. 2007;13(4): 349---356. 10. Beitsch LM, Mays G, Corso L, Chang C, Brewer R. States gathering momentum: promising strategies for accreditation and assessment activities in multistate learning collaborative applicant states. J Public Health Manag Pract. 2007;13(4): 364---373. 11. Tremain B, Davis M, Joly B, Edgar M, Kusion ML, Schmidt R. Evaluation as a critical factor of success in local public health accreditation programs. J Public Health Manag Pract. 2007;13(4): 404---409. 12. North Carolina Local Health Department Accreditation Program. July 2006---June 2007 Stakeholder Evaluation Report. Chapel Hill, NC: North Carolina Institute for Public Health; 2007. 13. North Carolina Local Health Department Accreditation Program. July 2007---June 2008 Stakeholder Evaluation Report. Chapel Hill, NC: North Carolina Institute for Public Health; 2008. 14. North Carolina Local Health Department Accreditation Program. July 2008---June 2009 Stakeholder Evaluation Report. Chapel Hill, NC: North Carolina Institute for Public Health; 2009. 15. Cilenti D. North Carolina Public Health Agency Accreditation and Performance: The Climb From Good to Extraordinary [dissertation]. Chapel Hill: University of North Carolina; 2009. 16. Beitsch LM, Grigg M, Menachemi N, Brooks R. Roles of local public health agencies within the state public health system. J Public Health Manag Pract. 2006;12(3):232---241. 17. Erwin PC. The performance of local health departments: a review of the literature. J Public Health Manag Pract. 2008;14(2):E9---E18. 18. Mays GP, McHugh MC, Shim K, et al. Institutional and economic determinants of public health system performance. Am J Public Health. 2006;96(3):523---531. 19. National Association of County and City Health Officials. 2005 National Profile of Local Health Departments. Washington, DC: National Association of County and City Health Officials; 2006. 20. Davis MV. Opportunities to advance quality improvement in public health. J Public Health Manag Pract. 2010;16(1): 8---10. 21. National Association of County and City Health Officials. 2008 National Profile of Local Health Departments. Washington, DC: National Association of County and City Health Officials; 2009. 1548 Government, Politics, and Law Peer Reviewed Davis et al. American Journal of Public Health September 2011, Vol 101, No. 9