Contact Center QA Guide: Building a World-Class Quality Assurance Program. Sponsored by:

Transcription

1 : Building a World-Class Quality Assurance Program Sponsored by:

2 About DMG Consulting LLC DMG Consulting LLC is the leading analyst firm in the contact center and analytics markets. We are a strategic advisor to companies large and small. Our mission is to help companies build effective contact center and service environments that provide an outstanding customer experience. We achieve this goal by helping our clients leverage technology, process and people to optimize operational efficiency, sales and profits. Our actionable strategies and tactics effect change that enable companies to make strategic and tactical improvements with a rapid return on investment. Our customers include Global 2000 companies, government agencies, non profit organizations, and more than 150 contact center and analytics vendors. DMG Consulting LLC s business strategists have an average of 20 years experience in customer experience management (CEM), customer relationship management (CRM), contact centers, building customer focused businesses and financial analysis. We understand the power of customer data and the contributions that quality management, recording, speech analytics, performance management, surveying, elearning, coaching and workforce management systems make to the contact center and the entire enterprise. Our consulting experience with more than 2000 end user organizations and our handson operational, technology and financial expertise give us deep insight into what customers want and need from enterprises and vendors. This report is protected by United States copyright law and may not be copied, reproduced, displayed, published, transmitted or distributed in whole or in part in any form or medium without the express written permission of DMG Consulting LLC. You may not alter or remove any trademark, copyright or other notice from this report. Reproduction in whole or in part in any form or medium without express written permission of DMG Consulting LLC is prohibited. Substantial effort went into verifying and validating the accuracy of the information contained within this Report, however, DMG Consulting LLC makes no warranties as to the accuracy or completeness of this information. DMG Consulting LLC is not liable for any damages, consequential or otherwise, arising from use of this information. Copyright 2009 DMG Consulting LLC 2009 DMG Consulting LLC - i - April 2009

3 Table of Contents Executive Summary Introduction What is Contact Center Quality Assurance? Quality Assurance Program Benefits Components of a QA Program Getting Started Creating a Quality Assurance Leadership Team Staffing QA Positions Implementation Roadmap Developing a QA Evaluation Form Form Categories QA Evaluation Form Questions Assigning Weights to QA Evaluation Forms Validation Example QA Evaluation Forms Quality Assurance Monitoring Criteria and Guidelines Quality Monitoring Criteria and Guidelines Why are Quality Monitoring Criteria and Guidelines Necessary? Calibration What is Calibration? Benefits of Calibration The Calibration Process Quality Assurance Program Mechanics and Processes Transaction Selection Criteria Determining the Number and Frequency of Evaluations Coaching Evaluation Feedback and Escalation Process Ongoing Training Addressing Agent Performance Issues Rewards and Recognition Updating Procedures/Training DMG Consulting LLC - ii - April 2009

4 9.9 Monitoring Quality Assurance Reviewers Reporting QA Database Kicking Off the QA Program Agent Training QA Program Pilot Advanced Quality Assurance Initiatives Surveying Customer Experience Monitoring First Call Resolution (FCR) Quality Management/Liability Recording Suites...81 Appendix A: Procedure Format Sample...85 Appendix B: Policy Format Sample DMG Consulting LLC - iii - April 2009

5 Table of Figures Figure 1: Quality Assurance Process...5 Figure 2: Quality Assurance Program Development Roadmap...10 Figure 3: QA Implementation Roadmap...19 Figure 4: Common Quality Evaluation Form Sections/Categories...24 Figure 5: Financial Services Customer Service QA Evaluation Form...30 Figure 6: Healthcare Customer Service QA Evaluation Form...34 Figure 7: Technical Support QA Evaluation Form...38 Figure 8: Sample Customer Service Quality Monitoring Form...42 Figure 9: New Order Precision Quality Monitoring Form...45 Figure 10: The Calibration Process Figure 11: Coaching Methods...56 Figure 12: Average Contact Center QA Score: December Figure 13:Contact Center Average Scores by Evaluation Section: December Figure 14: Average Scores by Evaluation Question: December Figure 15: Average QA Scores by Agent: December Figure 16: QA Scores by Agent by Evaluation: December Figure 17: Average QA Scores by Agent by Category: December Figure 18: QA Agent Training Outline...76 Figure 19: Workforce Optimization Suites DMG Consulting LLC - iv - April 2009

6 Executive Summary Contact center quality assurance (QA) is an ongoing business activity that provides valuable insights into agent, customer and contact center service delivery performance and opportunities. The primary goal of any QA program is to verify and ensure that a company s contact center services are being performed in a manner that meets or exceeds internal requirements and customer expectations. At a fundamental level, QA programs measure how well agents comply with internal policies and procedures and interact with customers in phone, and chat sessions. The more advanced QA programs go well beyond these basics. They combine the results of customer satisfaction surveys with internal measurements to provide a 360 degree view of the customer experience. Well designed and effective QA programs demonstrate a company s commitment to its customers and agents, and are essential for building world class contact centers. This Guide is a cookbook for building a strong, effective and well received QA program. It is ideal for managers and supervisors in contact centers of any size who are either building their first quality assurance program or want to enhance an existing program DMG Consulting LLC April 2009

7 1. Introduction Quality assurance isn t an option for contact centers; it s essential for the success of the contact center, customer and agent satisfaction, improving agent and supervisor productivity and effectiveness, and keeping management in touch with their staff s performance. To achieve the best results and foster confidence in the program, managers must ensure that they are evaluating the right components of agent performance during customer interactions and using appropriate measurements and weights. Building an effective QA program is an iterative, multi step process that requires senior management support, planning and input, and buy in from all levels of contact center staff. Automation is helpful for formalizing, standardizing and institutionalizing the initiative, but QA a program will succeed only if the staff is on board and believes in its value DMG Consulting LLC April 2009

8 2. What is Contact Center Quality Assurance? Contact center quality assurance, also known as Quality Management (QM), is a process where managers, supervisors and QA specialists monitor and evaluate how well agents handle customer transactions. The monitoring process includes a simultaneous review of a call and the system screens used to handle the interaction. (If QA is being done on an or a chat session, it will review just the servicing screens and the content of the agent s response.) QA can be done in a real time mode where the supervisor or QA specialist live monitors calls. For live monitoring QA, reviewers can either access calls through their automatic call distributor (ACD) technology, or can sit next to agents and watch and evaluate them as they handle calls. QA is often performed on recorded transactions. There are pros and cons to both live monitoring and using recorded transactions. When live monitoring, the QA specialist/supervisor can provide immediate feedback to the agent. This can become an effective coaching opportunity, as the feedback is provided in real time when the call is still fresh in the agent s mind. On the other hand, doing QA from recorded transactions allows an organization to schedule calls, making the process more efficient for the reviewer. It allows QA specialists/supervisors to find interactions that require their attention, either because they are really good or really bad, instead of wasting time on satisfactory calls. It allows the reviewer to be able to carefully review the interaction/screens, including the ability to go back and re review a portion of the interaction. And, lastly, it is an anonymous process, so agents are not aware that they are being evaluated and will not necessarily be on their best behavior. When doing QA, the reviewer completes an evaluation form that measures how the agent handled each component of the call or online transaction. The form can be on paper or part of a QA system. The fundamental purpose of QA programs is to measure how well agents adhere to contact center departmental policies and procedures. Contact/call center managers have traditionally live monitored or listened to recorded interactions in order to gauge agent performance and internally evaluate the customer experience. To be effective, the program should be a formalized ongoing process designed to: 2009 DMG Consulting LLC April 2009

9 1. Measure agent adherence to internal policies and procedures 2. Improve consistency and quality of customer interactions across all channels (telephone, , chat/im, etc.) 3. Assess business execution detect and fix broken or inefficient policies, processes or operational issues throughout the company 4. Improve agent performance 5. Identify agent training needs 6. Identify policies or processes that frustrate and alienate customers 7. Maximize every customer interaction 8. Identify business trends 9. Improve the customer experience The underpinnings of a good QA program are consistency, accurate measurement, and a cycle of continuous feedback resulting in improvements. An effective QA program provides the contact center with a vehicle for measuring the quality and consistency of service delivery, capturing customer insights, and identifying trends, service issues and training/coaching opportunities to improve agent performance and productivity. Quality assurance is a dynamic and iterative process that must be adapted as a business changes. See Figure DMG Consulting LLC April 2009

10 Figure 1: Quality Assurance Process Contact C Center QA Guide Source: DMG Consulting LLC, April 2009 Action Item: Identify the company s reasons for performing contact center quality assurance. Build a program that delivers continuous feedback to the contact center and other operating departments DMG Consulting LLC April 2009

11 3. Quality Assurance Program Benefits When quality assurance programs are well designed and used consistently, they yield great benefits for customers, enterprises, contact centers and agents. The benefits include: Improved agent productivity, by reducing the average handle time of transactions Reduced operating expenses Better transaction quality Increased customer satisfaction Enhanced customer experience Identification of business opportunities, cross sell, up sell, new products and services Enhanced operating policies and procedures Reduced enterprise risk Improved agent satisfaction and reduced attrition Automated reporting for tracking and trending Action Item: Set up a process for capturing, quantifying and reporting the benefits from your QA program. Be sure to share successes with agents, supervisors, managers and senior executives on an ongoing basis, so that everyone appreciates the program s contributions DMG Consulting LLC April 2009

12 4. Components of a QA Program While every company should customize their QA program to reflect their corporate culture and values, the fundamental building blocks for contact center QA are standard across all companies. Effective QA programs should include the following components: Procedures and policies: Document all existing transaction procedures and policies so that the contact center staff knows the appropriate steps for handling all types of inquiries. QA specialists/supervisors can also use the procedures and policies as a standard to make sure that they are evaluating different transactions properly. It s important to set up a process that keeps the policies and procedures up to date. Quality monitoring criteria and guidelines: Specify the criteria to apply when evaluating transactions and performing evaluations. The criteria should define what QA specialists/supervisors are looking for in each type of contact center transaction. The easiest way to create criteria is to use the contact center s documented procedures and policies and note on each the most important aspects for each type of transaction. It s also a good idea to identify agent actions that would cause them to lose points in a quality evaluation. Program mechanics: Define the mechanics of the QA program, including who (manager, supervisor, team leader, QA specialist, trainer) is responsible for conducting evaluations, the number and frequency of evaluations, how many coaching sessions per agent per month, and how to select calls, s and chat sessions for evaluation. (If a QA application is being used, most of the mechanics will be automated, including the transaction selection process.) Training: Establish a closed loop training process that addresses new content, system issues, updates and agent performance issues. The trainers and QA staff must work closely together to ensure that the staff is fully trained. (In many small/mid sized contact centers, the same people do both QA and training.) Prior to kicking off a QA program, all contact center staff agents, supervisors, QA specialists, trainers, managers must be fully trained so that they know how to 2009 DMG Consulting LLC April 2009

13 handle all types of customer interactions. If new or enhanced procedures and policies are drafted to support the QA program, they should be reviewed with the staff before starting the QA program. In addition to agent procedural training, it s important to build a training program that introduces the new or enhanced QA program to the staff. The more informed the staff is about the program, the more effective the initiative. Coaching: Provide frequent feedback to agents about their performance. Feedback should address where agents are performing well and areas where they have opportunities to improve. Coaching is one of the critical success factors in QA programs and plays a very important role in agent satisfaction and retention. However, as it can be very challenging to provide negative feedback to agents, it s important to train the QA staff to produce effective coaching sessions and to make sure that management delivers coaching sessions consistently. Calibration: Build consistency into the QA program through calibration. Calibration is the process of teaching all people involved in performing QA evaluations how to score transactions on a consistent and equitable basis. To make a QA program fair for agents, it s essential for all QA reviewers to agree on the meaning and value of each question in a monitoring form. To achieve consistency, it s important to run calibration sessions where all reviewers listen to the same call, score it, identify variance in scoring approaches, reconcile their differences, and set a standard measurement that all will use going forward. The only way to reach consensus is to run calibration sessions. Calibration is an ongoing process and should be run on a monthly basis. It s also a good idea to involve agents in the calibration process so that they can appreciate the challenges associated with consistently evaluating transactions. Evaluation feedback: Provide a process to facilitate two way communication between agents and reviewers. Agents need to have a mechanism for responding to their quality evaluations so that they feel empowered and not put upon. They also need a formal process for filing complaints when they believe that a QA reviewer is not being fair or is not listening to their input. QA reviewers should welcome discourse, as it will ultimately yield a better and more effective program DMG Consulting LLC April 2009

14 Rewards and recognition: Recognizing and rewarding top performers is essential for the success of a QA program. While agents should deliver outstanding performance because it s their job, recognizing when they do encourages them to keep up the good work and motivates others to strive for recognition, as well. Rewards do not have to be large; they could include a plaque, a parking spot, lunch with the CEO, a gift card, movie tickets, etc. Action Item: Implement an Executive QA program, where senior managers from sales, marketing, operations and all other supporting areas sit with contact center agents as they handle calls. This has proven to create customer focused awareness and foster collaboration between departments. (Rapid process change is facilitated when senior executives hear firsthand the impact of their processes and programs on customers.) When senior managers take this program seriously, it has a very positive impact on agent morale and satisfaction because it underscores the importance of their job DMG Consulting LLC April 2009

15 5. Getting Started Figure 2 depicts a high level project plan for building a QA program. This figure provides the high level steps that each organization should customize to meet the needs of their company. The first step in the process is identifying and assigning resources to oversee and manage the program Contact C Center QA Guide on an ongoing basis. This step is critical because every company needs a champion to push the program through, or it is unlikely to happen. (The project champion often becomes the manager or supervisor responsible for the program.) It takes a significant amount of work to build a contact center QA program and there must be an individual responsible for making sure that everyone involved in the program is motivated and the project stays on track and on time. Figure 2: Quality Assurance Program Development Roadmap Identify QA manager and reviewer staff Pilot the program for 2 months and review results Train all staff members on the Quality Assurance program Build Quality Assurance training program Define the coaching and feedback process Develop quality monitoring criteria and guidelines 11 Implement the formal QA program Adjust program and repeat pilot as needed Test the form using real calls, conduct calibration and refine the form as needed Develop the evaluation forms; determine section and question weights Communicate to all staff that a formal Quality Assurance program is under development Source: DMG Consulting LLC, April DMG Consulting LLC April 2009

16 The 11 high level steps for building a QA program are explained below. It s important to note that departmental procedures for handling all types of customer transactions and inquiries should be drafted or enhanced prior to developing the QA process. Step 1: Identify resources from the contact center to serve as the QA manager and quality review staff. (The QA manager will also play a major role in building or enhancing the program.) Ideally the manager should have prior experience in setting up and running a QA program and conducting contact center QA evaluations. Minimally, the QA reviewer(s) must demonstrate complete and accurate knowledge of all contact center policies, procedures and systems, as well as having excellent interpersonal and coaching skills. The number of resources dedicated to the process varies based on the size of the contact center. Step 2: The institution of a formal QA program should be communicated to all staff members as early as possible. The success of the program depends upon agent cooperation, which will be enhanced if agents are invited to participate in program development and all changes are clearly communicated. Communication should include information about why the program is being developed, roles and responsibilities, impact on agents (their reviews, raises, and incentives), and program benefits. Step 3: Develop the quality evaluation forms. Typically, managers create one for each channel supported (phone, , chat), and/or one for different call classifications, i.e. service, sales, collections, fraud, etc. Once the number and types of forms are decided, determine the sections required in each form, i.e., opening, verification, problem resolution, communications, closing, and then write the questions for each section. Once the sections and questions are drafted, assign weights to each question and possibly to each section, reflecting their relative importance. It s easiest to use a 100 point scale but this is not a requirement. Step 4: Using real calls, test the evaluation forms to ensure all call components that need to be measured are captured and that the intended goals are achieved fairly and accurately. Conduct calibration sessions involving supervisors and QA specialists to ensure that everyone participating in the QA program is using the form correctly and with the same rigor, consistently 2009 DMG Consulting LLC April 2009

17 and fairly across all questions. It generally takes a few calibration sessions to finalize the questions and weights for each evaluation form. Step 5: Document the quality monitoring evaluation criteria and guidelines for how to apply them. Step 6: Define the agent coaching and feedback process. Coaching is considered the most effective method for training adults; it should take place in a one on one personalized session with an agent. During coaching sessions, agents should receive specific and targeted feedback to learn what they are doing well as well as the areas where they can improve. This is a critical part of successfully managing agents and motivating them to improve their performance. Step 7: Build the QA training program. Training should include an in depth review of the quality monitoring process and QA form, evaluation criteria and metrics/measurements, frequency and number of evaluations conducted, scoring methodology, how to access, review and respond to evaluations, and how to provide feedback to improve the program. Optionally, the QA training program may also include a review of documented policies, procedures and guidelines that govern agents behavior in processing transactions or advising customers. Step 8: Train staff on the QA program. Roll out the training program so all agents and other relevant contact center staff managers, supervisors, trainers, QA specialists, agents, and possibly participants from other departments become familiar and comfortable with the new program. If possible, conduct calibration sessions with agents during the training session to demonstrate application of the quality criteria and use of the evaluation form. (Note: Be sure to ask the agents whose calls will be used during calibration for their permission.) Agents who understand how their calls are being evaluated and know what is expected of them are more likely to perform well. Step 9: Pilot the QA program and monitor results. The pilot provides an opportunity to see what works and what needs to be enhanced. It also gives QA reviewers and agents an opportunity to get accustomed to the program DMG Consulting LLC April 2009

18 DMG recommends a 2 to 3 month pilot to work the kinks out of the process and give everyone time to get used to the new program before going live. Step 10: Adjust the program throughout the pilot and, if changes are significant enough, it may make sense to keep the pilot going longer. This step allows time to refine the program before it is formally launched. Based on participant feedback and pilot results, the QA manager should revise processes, evaluation criteria, evaluation forms, metrics, mechanics, etc. The pilot should be run for as long as necessary to test all components of the QA program, including conducting calibration exercises. Step 11: Implement the QA program. Launch the program and begin to execute the QA processes on a daily basis. QA reviewers should work closely with management, supervisors and trainers to review quality assurance results and identify areas for training and development. Periodic reviews to evaluate the QA process should be conducted to ensure compliance with ever evolving procedures, policies and protocols. It is also a good idea to establish a reward program or agent appreciation events, so that management can show special recognition for outstanding employee performance based on QA program reviews. After the QA program is implemented, contact centers should consider doing the following activities on a monthly basis. They are not a requirement but will help to enhance agent moral and improve the results of your QA program. Conduct monthly calibration sessions with all supervisors/qa reviewers to ensure program consistency. Conduct monthly agent team meetings to review QA results and do group training, as necessary. Have QA reviewers and trainers meet on a monthly basis to review training needs and other improvement opportunities uncovered in the QA process. (In many contact centers, the QA specialists and trainers speak daily.) 2009 DMG Consulting LLC April 2009

19 Publish a monthly QA newsletter (or other vehicle) with quality tips to help the department achieve quality goals. The newsletter should also recognize top performers. Action Item: Implement a process where top performing agents conduct side by side training sessions with agents who are not meeting their potential and are under performing. Agents learn best from their peers. Using the top performers for this activity will recognize their outstanding performance and help get them on board in support of management objectives DMG Consulting LLC April 2009

20 5.1 Creating a Quality Assurance Leadership Team Building an effective QA program is an iterative process that requires senior management support, in addition to thorough planning, input and buy in from all levels of contact center staff. It s a best practice to form a QA leadership team with representation from all contact center constituents including managers, supervisors, trainers, agents, and possibly other departments, such as marketing and sales. This team could be led by the head of QA or the contact center director. This is a good way to keep the head of the contact center engaged in the QA program on an ongoing basis. Action Item: Use the quality assurance monitoring process to identify areas for business process optimization. Establish a cross functional team to address contact center and enterprise business process opportunities identified during the quality monitoring process. This team should work together to change business processes that upset customers DMG Consulting LLC April 2009

21 5.2 Staffing QA Positions The success of any QA program is largely dependent upon the commitment and skills of the quality assurance specialist who administers the program and coaches agents. Quality assurance specialists must be highly skilled individuals who possess outstanding job knowledge and excellent communication, interpersonal and coaching skills. Because a vast amount of job knowledge is required for this role, most contact centers staff these positions by promoting agents who demonstrate exceptional job knowledge and call handling skills. (Other contact centers transfer a supervisor or manager into this position.) However, being an outstanding customer service or sales representative is not the same as knowing how to deliver effective one on one feedback to coach and motivate a contact center agent. The ability to coach is not innate, and generally has to be taught. QA specialists should have the following skills and knowledge: Job knowledge: To accurately, effectively, and objectively evaluate agents, quality assurance specialists must possess strong knowledge of all products, services, systems, processes and procedures. If quality assurance specialists are hired externally, they should be required to complete a comprehensive training course that includes in depth coverage of all products, services, policies procedures and guidelines that agents are required to adhere to when processing transactions or advising customers. System knowledge: Quality assurance specialists must be fully trained and knowledgeable about all systems that agents use to handle customer inquiries or process orders. Additionally, they must be fully trained to navigate the quality management application so that they can search, retrieve and play back calls, create evaluations, and respond to agents evaluation feedback. QA specialists should also be trained to access and use reports from the quality application as well as to create ad hoc reports. QA program mechanics: Quality assurance specialists must possess a strong understanding of the quality monitoring process, including how to select calls/ s/chat sessions, the number/frequency of evaluations that they are required to do, and how to coach agents DMG Consulting LLC April 2009

22 QA criteria/calibration: To maintain measurement integrity, it s essential for all QA specialists to uniformly apply quality monitoring criteria when evaluating calls. All QA reviewers should be thoroughly trained to complete evaluations in a consistent manner. The success and effectiveness of the program and its reception by agents depends upon the ability to consistently, objectively and fairly evaluate contact center interactions. To achieve these essential goals, it s also recommended that calibration sessions be conducted with all quality reviewers on a monthly basis. Coaching/motivation: QA specialists must demonstrate strong interpersonal and coaching skills so that they can work one on one with agents to recognize areas of strengths as well as provide directed feedback on areas that require improvement. All quality assurance specialists should be required to take courses in effective coaching methods and motivation techniques. Action Item: When staffing QA programs, use highly respected staff members with strong product, service, system knowledge and expertise in coaching when staffing QA programs. Agents are more welcoming of feedback from people they respect DMG Consulting LLC April 2009

23 5.3 Implementation Roadmap Once the leadership team is established and the quality manager and staff have been appointed, a detailed breakdown of all program deliverables should be developed and assigned to the responsible parties. Figure 3 provides a list of the required tasks, recommendations for ownership of each initiative, and estimated time frames for completion. The tasks reflected in Figure 3 have already been discussed in other sections of this Guide. They are repeated here in order to put them into a project plan and to show the responsible party and estimated time frame. DMG recommends using this list of initiatives to build a detailed project plan. Your project plan will likely include other steps and initiatives that are important for your organization. Once the project plan is drafted, review it and get the buy in for all involved parties, particularly those assigned tasks, to make sure that they are committed and able to dedicate time to the projects. It s also a good idea to have a weekly project review meeting to assess progress and to address items that are slipping. It is ideal, but not necessary for senior contact center management to participate in the weekly progress meeting. However, if a contact center manager is not able to participate in the weekly meetings, the project manager should make an effort to keep him/her updated on the team s progress DMG Consulting LLC April 2009

24 Figure 3: QA Implementation Roadmap Task/Initiative Responsibility Timeframe Status Development Phase Communicate to all staff that a quality assurance program is under development and will soon be implemented; communication should be frequent Promote or hire a QA specialist and quality reviewers Develop a call monitoring evaluation form (Note: In multi-channel contact centers, a QA form will need to be created and tested for each channel) Test the form using real calls to ensure that all call elements that need to be evaluated are captured Document quality monitoring evaluation criteria and guidelines for how to apply them Determine weights for each section of the monitoring form Test the form(s) using real calls to validate sections, questions, weights and ranges Hold calibration sessions with agents, supervisors and managers to make sure everyone uses the evaluation form the same way Enhance QA evaluation form based on input from calibration team Determine the volume of transactions (calls, s, chat sessions, other) to be monitored per agent/month Decide who will be conducting the evaluations (quality assurance specialists, supervisors, or both) Conduct calibration sessions to ensure rating reliability and consensus of all reviewers Define the agent coaching process Develop a process with an escalation/review procedure for agents to provide feedback or dispute evaluations Develop reports to support the QA process; if using a QA application, build the reports in the application Contact Center Director Contact Center Director Quality Assurance Leadership Team QA Manager and reviewers Quality Assurance Leadership Team Quality Assurance Leadership Team Quality Assurance Leadership Team Quality Assurance Leadership Team QA Manager and reviewers Quality Assurance Leadership Team Quality Assurance Leadership Team Quality Assurance Leadership Team QA Manager, contact center managers, supervisors and reviewers Quality Assurance Leadership Team Quality Assurance Leadership Team 2 days 3 weeks 2 weeks per form 2 days 2 weeks 1 week 1 week 2 weeks 3 days 1 day 1 day 2 weeks 1 day 1 day 2 weeks Test reports QA Manager 1 week 2009 DMG Consulting LLC April 2009

25 Figure 3: QA Implementation Roadmap Task/Initiative Responsibility Timeframe Status Development Phase Train all QA specialists to use the QA application and reports Develop QA training program Train all contact center staff on the quality assurance program Kick off the program Pilot program for 2 to 3 months and review results Revise departmental Agent Performance Evaluation form to include quality monitoring metrics Implement QA program Establish a process for identifying and recognizing agents who achieve quality monitoring excellence Perform monthly calibration sessions QA Manager QA Manager and trainers QA Manager and trainers Contact Center Director/QA Manager Quality Assurance Leadership Team Quality Assurance Leadership Team/HR Contact Center Leadership Team Contact Center Leadership Team QA Manager, all QA reviewers 2 days 3 weeks Variable depending on size of staff and program length 1 day 2 3 months 1 week On-going 2 weeks Monthly/ on-going Conduct monthly training sessions Trainer Monthly/ on-going Create and issue monthly QA newsletter QA Manager Monthly/ on-going Have QA staff meet with training to enhance training programs QA Manager and trainers Monthly/ on-going Action Item: Involve all levels of contact center staff in creating the program to avoid unnecessary skepticism and surprises. This will help agents appreciate the positive aspects of the QA program and speed up adoption DMG Consulting LLC April 2009

26 6. Developing a QA Evaluation Form When managers think about developing a QA program, one of the first things they must consider is creating a QA evaluation form. The evaluation form is the most visible component of the program and is necessary for all QA programs, whether an organization is doing QA manually or using an automated system. The QA evaluation form needs to capture all interaction components that a contact center wants to measure. Its questions and weights should reflect the culture of the company and what is most important to their service strategy. In general, QA evaluation forms contain the following components: 1. Call/evaluation details Examples: Name of agent, date of transaction, reviewer name, date of evaluation, call type, customer identifier (account number, social security number, etc.) 2. Sections (skill categories) Examples: Call opening/closing, verification, product/plan knowledge, procedure knowledge, system knowledge, hold/mute/transfer, communication skills, resolution skills, etc. 3. Questions (to objectively assess skill proficiencies) Examples: Did the representative identify the caller according to the verification policy prior to releasing information? Did the representative log a summary of the call according to the policy? 4. Scoring Point values for each question and section of the evaluation form; includes point values, points available, and points earned. Point values should be assigned based on the relative importance of each section and question to the business and the customer DMG Consulting LLC April 2009

27 5. Coaching comments Free form text box to allow reviewers to provide feedback on performance excellence or opportunities. 6. Recommendations Free form text box to allow reviewers to document an action or follow up items for agents to complete. Examples: taking a specific elearning course, reviewing specific procedures, recommendations, etc. 7. Acknowledgement (optional) Signature boxes so that the reviewer and agent can sign off and acknowledge that they had a discussion. After learning how to create a QA evaluation form and deciding what sections are required, it s relatively easy to modify an existing form or draft a new one. However, creating one for the first time can be daunting. Therefore, we suggest getting started with an existing QA form and then modifying it to meet the organization s specific needs. The easiest way to obtain sample quality evaluation forms is by asking a QA vendor, another contact center manager, or a consulting firm. (Most vendors have sample forms and are generally happy to share them, particularly if they think you are interested in purchasing their solution.) While it s relatively easy to obtain a sample QA evaluation form, the catch is that there is no "one size fits all" form. For example, a technical help desk needs a different QA evaluation form from a sales or customer service contact center. The QA evaluation form should be customized to meet the needs of each particular contact center DMG Consulting LLC April 2009

28 Here are a few best practices to employ when developing QA evaluation forms: Involve all levels of your staff agents, supervisors, trainers and managers. Contact C Center QA Guide Develop a separate QA form for each channel that you will be monitoring. Test your QA form(s) using real transactions and refine them before putting them into production. Assign weights that correspond to the importance of all categories on your QA form. Importance may be relative to the channel of communication, customer and/or the business. Include a comments box for each section of the evaluation so that coaching comments on performance strengths and opportunities and agent feedback can be captured. Action Item: To modify or create a QA evaluation form, start with an existing or sample form. Then listen to a selection of different call types. While scoring these calls, identify form components that need to be changed or added to fit your environment. (This process applies to s, chats or other types of transactions received by the contact center.) 2009 DMG Consulting LLC April 2009

29 6.1 Form Categories Figure 4 provides an example of sections or categories typically found in a QA form for calls. It also includes example criteria that can be used to evaluate if an agent properly performed a particular skill. Depending on how the form is structured, sections can be established based on skills, call flow, competencies, or call segments. In Figure 4 below, the sections are based on skills. Figure 4: Common Quality Evaluation Form Sections/Categories Quality Evaluation Form Call Opening/Closing Provided name/company name; thanked the customer for calling or purchasing [brand] products Verification Obtained verification prior to releasing information according to established data security procedures Demonstration of System Knowledge/Usage Effectively accessed information on system; utilized all appropriate systems to obtain information Demonstration of Product Knowledge and Information Demonstrated thorough knowledge of product/part/services/warranty, etc. Demonstration of Resolution Skills Accurately diagnosed problem/issue Hold/Transfer Procedure Use hold/transfer effectively and only as necessary Communication Skills Spoke clearly and confidently Agent First Contact Resolution (FCR) Did the agent do everything possible to ensure a callback was not necessary? Business Process FCR Is there a policy/procedure/service issue that prevented First Contact Resolution? Source: DMG Consulting LLC, April 2009 Action Item: Be sure to include a section in the evaluation form to reflect how well the agent performed in regard to compliance or disclosure regulations (if applicable) DMG Consulting LLC April 2009

30 6.2 QA Evaluation Form Questions After identifying the right sections/skills to include in the QA form, the next step is to create a concise list of questions that captures the elements associated with demonstrating competency in each skill. The goal is to write questions that allow QA reviewers to completely and objectively assess agent performance. Once the initial list of questions is developed for each section of the form, have a team of supervisors and QA reviewers test the form by trying it out on a few calls or s. It will typically take at least ten rewrites of the questions before an evaluation form is complete. This exercise helps surface gaps and identifies overlaps and redundancies in questions. Revise the form based on test results, and retest it until the form successfully addresses all requirements. (You will know the form works when there is a place to address every aspect of a call or .) Action Item: Test the QA form with real calls or s. Make sure there is a place on the form to address everything that can happen in a call or . This will surface confusing or ambiguous components of the form DMG Consulting LLC April 2009

31 6.3 Assigning Weights to QA Evaluation Forms Once the form content is finalized, it s time to select a scheme for assigning weights to the questions and/or sections of the evaluation. The simplest approach is to go with a 100 point scale. While there are many scoring methodologies, the most basic way is to assign a point value to each section on the form and distribute the points among the questions in that section. For example, the communications skill section may be assigned a total of 20 points, which are distributed among the four questions in the section. While points can be distributed equally throughout a section or the entire QA evaluation form, we suggest that they be assigned based on the relative importance of each section and each question. Importance may be relative to the channel of communication, customer and/or the business. After preliminary weights are distributed, the form should be tested again by evaluating and scoring actual transactions. Points are generally reconsidered and redistributed during testing. It s also important to consider that in some interactions, not all components of the form are applicable (for example, not every call includes a hold or transfer). The scoring on the forms shown in Section 6.5 was designed to include Not Applicable (N/A) as a consideration; if a particular question on the form is not applicable to the interaction being evaluated, the question is marked N/A. The points are not available to be achieved, nor do they affect the total score. Overall evaluation scores are calculated as follows: Total points achieved Total points available = QM evaluation score For example: 76 points achieved / 84 points available = 90.4% Action Item: When assigning weights to the form, it s also a good idea to identify and define the serious errors that will result in a full and automatic failure of an evaluation form section or an entire evaluation DMG Consulting LLC April 2009

32 6.4 Validation Validation of the metrics associated with each category of the quality assurance form is an important step in developing the quality assurance program. Metrics are typically validated by testing the proposed weights against baseline quality measurements that have already been established. When modifying an existing QA evaluation form, the new form can be validated by using it to reevaluate transactions that had previously been scored using a different QA form. If no quality measurement baseline exists, initial testing sessions should validate QA scores based on agent performance. So, top performers should receive high scores, average performers should be in the mid range, and poor performers should have low scores. Once the point values are validated and coming out as expected, the next step is to determine how QA evaluation scores equate to the performance ratings used by the organization in its annual review process. For example: Excellent: 89% and above Good: 80 88% Average: 71 79% Needs Improvement: 70% and below The scoring ranges above are typical for many contact center QA programs. The ranges should be high enough so that an excellent transaction is clearly identifiable, but not so high so that few agents can attain an excellent rating. As a result, it s often a good idea to start with one range and then as agents get accustomed to the QA program and their quality improves, tighten the ranges. So, for example, when a QA program is first kicked off, excellent may be anything above 85%. However, since most customers are not likely to think that 85% is excellent, the range should be narrowed to anything above either 89% or 90% a few months into the program DMG Consulting LLC April 2009

33 Be sure to communicate to agents exactly what is happening with the ranges starting easy to give them a chance to become accustomed, and then bringing them to a level that will better meet customer expectations so that the staff does not think that management is just giving them a hard time. Be sure to set expectations and communicate clearly at all stages of the program to get the staff s support. Once the final program is rolled out, agents should have a two to three month grace period to become accustomed to the program, criteria and scoring before they are held accountable for QA scores; this will minimize any claims of unfairness within the shop, as the staff will have had an extended period of time to adjust to the program requirements. During this period, QA evaluations and coaching sessions are performed, but are not counted toward agents annual evaluations. Action Item: Review and enhance your QA evaluation forms periodically minimally, every 9 to 12 months to keep them in sync with your business and customers expectations DMG Consulting LLC April 2009

34 6.5 Example QA Evaluation Forms This section includes 5 example QA forms. The first form, Figure 5, is for a financial services customer service contact center. Figure 6 is a form intended for use in an inbound healthcare customer service contact center. Figures 7, 8, 9 depict forms intended for inbound technical support, customer service with up sell/cross sell activities and new order processing, respectively. Use these forms as examples for getting started, but customize them to reflect the needs and priorities of your organization DMG Consulting LLC April 2009

35 Figure 5: Financial Services Customer Service QA Evaluation Form Description: This form is for use by an inbound financial services contact center that addresses customer inquiries, questions or problems on their accounts, and/or services that are provided. Representative name: Call date: Evaluator name: Evaluation date: Greeting (3 points) Evaluation Details Used call greeting as defined in Greeting Policy Coaching Comments: Verification (10 points) Verified caller as defined in Verification Policy prior to releasing account information Verified additional items pertaining to specific caller inquiry Coaching Comments: Product Knowledge (20 points) Accurately identified account, transaction, product or service Point Value 3 Point Value 7 3 Point Value 5 Yes Call Details Account number or customer ID: Caller: Call category: Call type: No Point Totals Yes No N/A Point Totals Yes No N/A Points Available Points Available Points Available Points Achieved Points Achieved Points Achieved Provided complete information or instructions in accordance with established procedure Provided accurate information or instructions in accordance with established procedure 7 8 Point Totals 2009 DMG Consulting LLC April 2009

36 Coaching Comments: Inquiry Resolution (23 points) Point Value Yes No N/A Points Available Points Achieved Accurately understood the nature of the caller s inquiry 5 Effectively/accurately resolved inquiry/ issue in accordance with established procedure Completed fulfillment/referral/ follow-up as promised/required Provided appropriate alternatives relative to customer s need/situation Ensured caller fully understood explanation, process, time frames and/or next steps Coaching Comments: Compliance (8 points) Read mandatory disclosures as required and applicable Coaching Comments: Cross-sell/Up-sell (6 points) Recommended appropriate product/service to meet customer need or extend the relationship Point Value 8 Point Value 3 Point Totals Yes No N/A Point Totals Yes No N/A Points Available Points Available Points Achieved Points Achieved Effectively tied product/service benefit to customer situation/need 3 Point Totals 2009 DMG Consulting LLC April 2009