Assessing Practices against COA Standards I. Introduction A. About This Course B. Course Objectives II. Start Where You Are A. Evaluate and Begin III. The Assessment Process A. Three Stages in the Assessment Process B. Review IV. Track Your Progress A. A Systematic Approach B. Evidence of Implementation Worksheet B. Standards Assessment and Tracking Worksheet C. Use Available Tools V. Standards in the Accreditation Process A. Standards Are Goals B. You Don't Have to Be Perfect C. Standards Describe WHAT Not HOW D. The Stricter Standard Applies VI. How to Assess a Standard A. Before You Start B. The Method C. Gaining Efficiencies Along the Way VII. Conclusion A. Congratulations! B. Resources 1
Section 1: Introduction About This Course Welcome to this training course titled, Assessing Practices against COA Standards. This course is designed for your organization or program's primary contact, as well as other staff who will be reviewing standards during the accreditation process. After completing this training, you will have a more comprehensive understanding of how to assess your practices against COA s accreditation standards. Before taking this course, it is highly recommended that you take the two prerequisite courses of COA Accreditation: Getting Started and The Format and Structure of COA's Accreditation Standards. Let's begin! Course Objectives After completing this course, you should be able to: 1. Describe the three stages of the self-assessment process. 2. Adopt a method for tracking progress. 3. Summarize the method for how to assess a Standard. Please keep in mind that this is a general training and that there may be differences in terminology and approach that are specific to your organization or program that you can discuss with your COA Accreditation Coordinator. Evaluate and Begin Section Two: Start Where You Are Regardless of which Standards you are using, most of the work of becoming accredited involves an in-depth self-evaluation of your organization or program's practices against COA's standards. This self-assessment is at the heart of the accreditation process. The results of this assessment will form the core of your game plan for meeting and demonstrating implementation of the standards. Once they begin their assessment, many organizations and programs find that they already meet most of COA's Standards. Others find that while their day-to-day practices generally 2
conform to the Standards, they need to formalize much of what they do into written policies and procedures. Section Three: The Assessment Process Three Stages of the Assessment Process Since the self-assessment and its associated follow-up will take up the majority of the time you spend in the accreditation process, this course will give you the "big picture" before we get into the details. The self-assessment coincides with the Self-Study and Site Visit Preparation phases of the accreditation process and has three overlapping stages. During the first stage, staff will closely review the standards and carefully assess how well your current practices meet the standards. Once your staff start reviewing the standards you will begin to receive the results of their initial assessments. During the second stage, staff will be updating practices that do not meet the standards, formalizing informal practices, and putting new practices into place where the assessment revealed that no practice currently exists. The third stage ensures that you can demonstrate that each of the standards has been implemented. In other words, the task is to make sure that there is evidence that the standards have become part of the way that your organization or program does business on a day-to-day basis. Recap During the self-assessment almost every organization discovers that it needs to modify at least some of its practices or implement new ones. Depending on the area of practice or the standard, this can take time and may involve multiple staff at different levels of the organization. That s why it is important to stay on top of the work and not let it pile up. A Systematic Approach Section Four: Track Your Progress We recommend that you take a systematic approach to managing the various tasks associated 3
with the self-assessment, and to meet the milestones and deadlines you agreed to during Intake. A number of different staff or teams of staff might be in various stages of assessing practices, updating or creating new policies or procedures, initiating entirely new practices, undergoing training, etc. At any given moment you should know what has been finished, what is still being worked on, and what work still needs to be started. So we recommend that you adopt a method to keep track of work assignments, along with due dates, and any follow-up that may be needed. Some people create their own spreadsheets to track progress. Others prefer to keep track of assignments on hard copies of the standards. COA makes a couple of tools available that can be useful: Evidence of Implementation Worksheet The first is the Evidence of Implementation Worksheet, which is on your MyCOA Dashboard in your Portal. It provides a customized list of the standards that apply to your organization or program, the evidence associated with each of the standards, and space to track progress. Note: The Worksheet link in your Portal is blank until after your Intake call. Standards Assessment and Tracking Worksheet Another tool you can use is the Standards Assessment and Tracking Worksheet. It was developed at the request of organizations who wanted a more detailed tracking tool. Whatever method you decide to use, set up your tracking system before assigning sections of standards to staff for review. Use Available Tools There are many tools available to you that are designed to help you organize your work. A list of recommended tools that may be helpful to you during your self-assessment can found on the Resources page of this course. Also included are instructions on how to get the most out of the Tools Search function on COA's website. 4
Section Five: Standards and the Accreditation Process Standards Are Goals A standard is a goal. Within COA's accreditation system, a standard is a goal toward which an organization strives. It is a statement that represents current thinking as to what constitutes best practice in a given practice area. You Don't Have to Be Perfect The second thing to understand about the standards is that COA does not expect organizations or programs to be perfect. You will be rated on the extent to which you have made progress toward achieving the goal that is embodied in the standard. In the COA rating system there are two possible "passing" ratings. An organization does not have to receive the highest possible rating (i.e., a "1") in order to pass the standard. In fact, it is possible to become accredited without receiving a rating of "1" on any of the standards. Of course COA's Rating System is a bit more involved than that... but you get the idea. For more information about COA's rating System, see the "How the Reviewers Assign Ratings" section of the Accreditation Guidelines. A link to the Accreditation Guidelines can be found on the menu bar on the right hand side of every page of the Standards. Standards Describe WHAT Not HOW COA standards describe WHAT COA wants you to implement, and, for the most part, leaves the HOW it is implemented up to you. Take for example, ETH 101: "The public has access to clear, timely, accurate information, as appropriate to the type of organization, about the organization s programs, activities, service recipients, and finances." This standard applies, as written, to all types of organizations from the small, rural, counseling organizations with ten staff and a two hundred thousand dollar annual budget, to the one hundred million dollar organization with thousands of staff and programs in every part of two states. 5
The WHAT, as outlined in the language of the standard applies, as written, to both organizations. HOW the standard is implemented will be quite different for both of these organizations. This, by the way, is what is meant by "Contextual Accreditation." The standard is the standard. It is the evidence that can change in relation to context. The Stricter Standard Applies If there is a conflict between a COA standard and a regulatory requirement, the stricter standard applies. For example, if the state requires a staff to client ratio of 1:10, and COA standards require a 1:8 ratio, COA's 1:8 ratio will be the standard against which you will be rated. Section Six: How to Assess Standard Meet Rhonda Rhonda is the primary contact at one of our agencies undergoing accreditation. In this chapter, I will be providing guidance to Rhonda on how to work with her staff through the process of assessing their practices against the standards. Before You Start When instructing your staff on how to approach their assignments, you'll want to give them a method for approaching the task. But before we get to that, there a few things to keep in mind as you approach this work: 1. Get into the habit of looking at a standard and the evidence together. Yes, the standards represent best practices and you may appreciate the ideas and values they represent. But during the accreditation process you always should be thinking in practical terms about implementation. Remember, every core concept has a table which groups evidence into 3 categories. Self-Study evidence which is typically submitted 6-8 weeks before the Site Visit, On-Site Evidence which is reviewed onsite, and On-site Activities that involve the reviewers like interviews, tours, etc. Here is an example of the Evidence using ASE-ML 5: Safety and Security. 2. When assessing practices, look at all the documents listed in both the Self-Study Evidence and On-Site Evidence columns of the Tables of Evidence. You want to get as complete a picture as you can as to where you stand with regard to the standards and required evidence. 6
3. Don't forget the evidence listed in the On-Site Activities column. The Site Visit may be months away, but if the Reviewers are going to check for passenger restraints in any vehicles used to transport clients, better to send someone to take a look sooner rather than later. Now let s look at a method for approaching the task of assessing practices against the standards. The method involves a series of five questions: 1. What is the standard asking us to do? "Looks like MIL-AM 8.05 requires us to prohibit certain practices related to making referrals. OK, that's clear enough..." [Guidance: Instruct staff to summarize for themselves what they think the standard is asking. Now, let s see what evidence applies to the standard MIL-AM 8.05.] 2. What evidence is COA asking for? "Referral Policies and Procedures are listed in the Self-Study Evidence column that s relevant to MIL-AM 8.05. There's nothing that s relevant to MIL-AM 8.05 in the On-Site Evidence column.and I already know that implementation of MIL-AM 8.05 will be verified onsite by interviews with staff and other stakeholders." [Guidance: The staff members listed in the On-Site Activities column should be prepared to answer questions about your referral practices if asked about it during the Site Visit.] 3. Read any notes and interpretations. [Guidance: Stress to staff that they should always read any notes or interpretations associated with the standard.] "I know that none of our staff have private practices, but I need to check about family members then take a look at our referral list." For more information about notes and interpretation, see How to Read the Standards and Table of Evidence on the Recommended Tools and Resources document in the Resources tab of this course. 4. How well does our current practice meet the standard? "Next, I ll do a crosswalk between our documents and MIL-AM 8.05. It looks like our policy does address elements (a) and (c), but I found nothing that addresses element (b)." 7
5. If not, what is needed? "Since the standard has three elements and we only meet two of them I think we should get someone to update our referral policies and procedures. Even though my review so far shows that we're doing pretty well with the other standards in MIL-AM 8, this looks like an important standard and I wouldn't want to take any chances of not passing this section." [Guidance: For the purposes of assessing practices, it is better to err on the side of caution and be strict with your assessments.] Gaining Efficiencies Along the Way I contacted our program directors and asked if they had referral policies and procedures in effect. Turns out we have three different sets of policies and procedures that apply to different services. And they are all pretty similar with only minor differences. [Guidance: One of the benefits of getting accredited is, when appropriate, to develop common sets of policies and procedures that apply across all programs and services.] You get the idea. Now try your hand at reading a standard and see how you do! Congratulations Section Seven: Conclusion Congratulations! You have come to the end of this course. Now that you have completed this course, you should be able to: 1. Describe the three stages of the self-assessment process. 2. Adopt a method for tracking progress. 3. Summarize the method for how to read a Standard. Resources We hope you have found this training useful and informative. Click Here to access a list of helpful resources. We urge you to visit the Training Calendar and Tools Search pages on the COAnet website for more useful accreditation tools and resources. If at any time you have questions, please contact your coordinator or use the contact us link at www.coanet.org. 8
Post-Training Survey Thank you for participating in this training. This course was developed in collaboration with Relias Learning, online training partner to the Council on Accreditation. Please click on the survey link in the email confirmation you received for this course to share your feedback about this training. We greatly appreciate your input. 9