Denver 311 Performance Audit

Save this PDF as:
Size: px
Start display at page:

Download "Denver 311 Performance Audit"

Transcription

1 Denver 311 Performance Audit August 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor

2 The Auditor of the City and County of Denver is independently elected by the citizens of Denver. He is responsible for examining and evaluating the operations of City agencies for the purpose of ensuring the proper and efficient use of City resources and providing other audit services and information to City Council, the Mayor and the public to improve all aspects of Denver s government. He also chairs the City s Audit Committee. The Audit Committee is chaired by the Auditor and consists of seven members. The Audit Committee assists the Auditor in his oversight responsibilities of the integrity of the City s finances and operations, including the integrity of the City s financial statements. The Audit Committee is structured in a manner that ensures the independent oversight of City operations, thereby enhancing citizen confidence and avoiding any appearance of a conflict of interest. Audit Committee Dennis Gallagher, Chair Maurice Goodgaine Leslie Mitchell Rudolfo Payan Robert Bishop Jeffrey Hart Timothy O Brien, Vice Chair Audit Staff Audrey Donovan, Deputy Director of Audit Services, CIA, CRMA Chris Horton, Internal Audit Supervisor, PhD Marcus Garrett, Lead Internal Auditor, CIA, CGAP, CRMA Anna Hansen, Senior Internal Auditor, CICA You can obtain copies of this report by contacting us at: Office of the Auditor 201 West Colfax Avenue, Department 705 Denver CO, (720) Fax (720) Or download and view an electronic copy by visiting our website at:

3 City and County of Denver 201 West Colfax Avenue, Department 705 Denver, Colorado FAX Dennis J. Gallagher Auditor August 16, 2012 Chuck Fredrick, Chief Information Officer Technology Services City and County of Denver Dear Mr. Fredrick, Attached is the Auditor s Office Audit Services Division s audit report regarding Denver 311. The purpose of the audit was to assess the effectiveness of the Denver 311 contact center. Since the most important consumers of Denver 311 s services are the members of the general public, I m pleased to report that, through the use of two statistically valid survey instruments commissioned by the Auditor s Office, we have learned that 60 percent of Denver residents are aware of Denver 311, and 88 percent of 311 users have been satisfied with their experience. Congratulations to you and the 311 team for such an impressive level of customer satisfaction. However, this silver lining has some clouds, as key performance metrics, including abandoned calls and time on hold, indicate 311 may soon reach a tipping point where customer satisfaction begins to decline. To mitigate this outcome, we suggest increasing efforts to enhance awareness of alternative methods for contacting 311, such as and social media. Additionally, there should be a continued focus on some key performance areas, which have been declining over the past few years. One possible method of addressing this decline is by spending less time trying to resolve calls for some agencies that are better equipped, for various reasons, to address callers themselves. If you have any questions, please call Kip Memmott, Director of Audit Services, at Sincerely, Dennis J. Gallagher Auditor DJG/cnh cc: Honorable Michael Hancock, Mayor Honorable Members of City Council Members of Audit Committee Ms. Janice Sinden, Chief of Staff Ms. Stephanie O Malley, Deputy Chief of Staff Ms. Cary Kennedy, Deputy Mayor, Chief Financial Officer To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

4 Ms. Beth Machann, Controller Mr. Doug Friednash, City Attorney Mr. L. Michael Henry, Staff Director, Board of Ethics Ms. Janna Bergquist, City Council Executive Staff Director Mr. Michael Major, Denver 311 Director To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

5 City and County of Denver Dennis J. Gallagher Auditor 201 West Colfax Avenue, Department 705 Denver, Colorado FAX AUDITOR S REPORT We have completed an audit to assess the effectiveness of Denver 311. This performance audit is authorized pursuant to the City and County of Denver Charter, Article V, Part 2, Section 1, General Powers and Duties of Auditor, and was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The audit contains findings in two primary areas. First, external customers expressed a high level of satisfaction with Denver 311 services. Specifically, 88 percent of 311 users in a survey commissioned by the Auditor s Office reported that they were satisfied with their 311 experience, and 56 percent of these users were very satisfied. However, most users also indicated that they were not aware of the existence of alternative means of contacting 311. Methods such as , Facebook, and Twitter are available as alternatives to a phone call, and increasing awareness of these alternative methods could maintain call volume at a manageable level. However, a downward trend in some key performance metrics suggests that satisfaction rates are at risk. Second, we found that Denver 311 could make some improvements in how it serves internal City agencies. Specifically, utilization of the Customer Relationship Management system could be improved. Additionally, 311 could better codify its partner relationships with City agencies with Memoranda of Understanding rather than the Service Level Agreements that are currently used. We extend our appreciation to the Director of Denver 311 and his staff, as well as the personnel from various City agencies, who assisted and cooperated with us during the audit. Audit Services Division Kip Memmott, MA, CGAP, CRMA Director of Audit Services To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

6 TABLE OF CONTENTS EXECUTIVE SUMMARY 1 INTRODUCTION & BACKGROUND 6 SCOPE 10 OBJECTIVE 10 METHODOLOGY 10 FINDING 13 Denver 311 Receives High Satisfaction Rates from External Users but Should Work to Sustain These Satisfaction Levels and Improve its Partnership with Internal Users 13 RECOMMENDATIONS 23 APPENDICES 25 Appendix A: Interactive Voice Response Survey of Denver Residents 25 Appendix B: Telephone Survey of Denver 311 Users 27 Appendix C: City of Minneapolis 311 Comparative Survey AGENCY RESPONSE 38

7 EXECUTIVE SUMMARY Denver 311 is the City s Comprehensive Source for Non- Emergency Information In July 2006, the City and County of Denver launched the 311 Customer Service Center as a comprehensive one-stop resource for anyone seeking information about nonemergency City services. Denver 311 serves both external and internal customers by providing a quick source of information or call routing for external callers, which assists internal City agencies with their customer service efforts. The use of 311 as a standard abbreviated dialing arrangement for nonemergency information is based on the effectiveness of using the 911 number for emergency situations. In 1997, the Federal Communications Commission (FCC) authorized the creation of 311 as a national number for state and local governments to receive calls regarding non-emergency police, fire, and emergency medical services and other government services. Denver 311 does not address emergency calls, nor is 311 associated with the Denver 911 call center. Denver 911 continues to address emergency calls as well as non-emergency calls that require a response from police, fire, or emergency medical services. 1 Nonemergency calls are made to a standard, non-abbreviated number, and Denver 911 management reports that nonemergency calls are superseded by calls that come in via the 911 (emergency) line. Key Audit Findings 88 percent of 311 users reported satisfaction with their 311 experience. 311 metrics indicate that 311 performance is declining, placing customer satisfaction rates at risk. Additional marketing efforts could increase awareness of alternative ways to contact can improve partnership with internal City agencies, especially with regard to the Customer Relationship Management (CRM) system. City agencies can provide additional information to CRM to help ensure that 311 reports are as useful as possible. Some calls to the non-emergency line involve issues that can be addressed effectively by Denver 311. According to figures provided by the 2012 Denver Budget Book, about 15 percent of all non-emergency calls to Denver 911 are routed to The public safety non-emergency line is P a g e 1 Office of the Auditor

8 Denver 311 Receives High Satisfaction Rates from External Users but Should Work to Sustain These Satisfaction Levels and Improve its Partnership with Internal Users A statistically valid customer service survey commissioned by the Auditor s Office found that 88 percent of Denver 311 users are somewhat satisfied or completely satisfied regarding their overall experience with 311. Specific areas of high satisfaction included timeliness of call answering, resolution of issues, and overall knowledge demonstrated by call center agents. However, a review of internal performance metrics data for key 311 activities, such as abandoned calls and the time to answer calls, revealed that performance measures are trending down, which, if not addressed, will eventually negatively affect the satisfaction rates of Denver 311 users. 2 Further, although external user satisfaction rates are high, Denver 311 can improve its partnership with internal City agencies. A number of City agencies, including Denver 311 partners, are not satisfied with the Customer Relationship Management (CRM) software utilized by 311, including system reports. While CRM is the primary tool used by Denver 311, the CRM system itself is maintained and managed by entities within Technology Services (TS) outside of 311. Citywide enhancements to CRM would allow Denver 311 to better support City agencies as a broker between agencies and TS. Additionally, changes to Service Level Agreements (SLA), which codify the services that 311 and internal agencies provide to each other, may address inconsistencies related to expectations of both Denver 311 and partner agencies. The vast majority of external Denver 311 users are satisfied with the level of service provided A statistically valid telephone survey of self-identified Denver 311 users found that overall users reported a high rate of satisfaction with 311 services. Eighty-eight percent of users reported overall satisfaction with their Denver 311 experience, including 56 percent of the users who reported being completely satisfied with 311 service. 3 In addition to evaluating overall satisfaction with 311, the survey also assessed satisfaction with specific areas. The survey found that 93 percent of external users were satisfied with the courtesy of the 311 call center agents and 86 percent were satisfied with the knowledge of the 311 call center agents. Additionally, 88 percent of external users were satisfied with the timeliness of Denver 311 s response to calls. Finally, 79 percent of Denver 311 users reported that their issues were resolved through their use of 311 and 71 percent of users reported that their issues were resolved with one contact to Calls abandoned are the total number of calls placed into Denver 311 when a call center agent was unable to answer before the caller disconnected. 3 The 88 percent figure comes from responses to a question early in the survey. Following this question the poll probed regarding more specific areas of users 311 experiences. After these questions were asked, the poll returned to a question of overall satisfaction, and found that 92 percent of users were satisfied with their 311 experience. We have chosen to report and discuss the 88 percent figure based on the risk that talking and thinking about 311 might have skewed the overall satisfaction level of 311 users report later in the poll. 4 This figure represents 71 percent of all users surveyed, as opposed to representing only 71 percent of the users who reported that their issue was resolved. City and County of Denver P a g e 2

9 Key metrics indicate that 311 performance is declining thus placing the high satisfaction rates at risk Although Denver 311 receives high satisfaction ratings from external users, its performance metrics indicate that customer service performance is declining. If not addressed, this decline will eventually result in decreased customer satisfaction ratings. Beginning in 2010, performance metric data has demonstrated decreased performance related to calls abandoned and the time to answer calls. Without some action to address these declining service levels, they will eventually negatively impact 311 s impressive customer satisfaction ratings. Due to this downward-trending performance metric data, Denver 311 must consider methods for improving its performance. One option for improving performance in the short-term would be to identify agencies for which first call resolution (FCR) rates are low and changing 311 s level of involvement with those agencies. 5 Low FCR rates for an agency indicate that 311 agents may have difficulty answering questions about that agency. While Denver 311 strives to provide service beyond simply serving as a switchboard for callers, the caller and the agency may benefit from 311 no longer attempting to address inquiries related to the agency. For example, 86 percent of calls to 311 regarding the Public Trustee are forwarded to the Public Trustee s office rather than being resolved by 311. The Public Trustee may be able to take on the additional calls as well, potentially enhancing customer service for the callers. Similarly, other agencies also report very low FCR rates, including Street Maintenance (an 11 percent FCR rate) and Public Health Inspections (a 16 percent FCR rate). Although justifications for the low FCR rates of these agencies may vary, this approach would likely open up more time for call center operators to answer other calls that can be resolved more easily by 311 staff. Additional marketing efforts could help increase awareness of other contact options including social media and improve performance over the long-term As an additional approach to reducing call volume, Denver 311 should identify additional resources for enhanced marketing efforts to increase awareness of the alternatives to phone contact. According to Denver s Chief Information Officer, identifying ways to provide a greater percentage of services through website contacts is part of TS s long-term strategy. Denver 311 has already developed several ways for citizens to contact the City without using the telephone. The primary alternatives are the City s official website, , Twitter, and Facebook, but our survey of Denver residents found that only 37 percent of Denver residents were aware there are a variety of options for accessing Denver 311 services. Denver 311 has a modest marketing budget. In previous years, the marketing budget was $10,000 per year, and in 2012 the budget was reduced to $5,000. However, Denver 311 management expended less than half that amount in previous years. Denver 311 should use its existing resources for marketing to help increase awareness to Denver residents and 311 users of alternative ways of contacting 311. Enhancements to CRM usage would allow Denver 311 to better support internal City agencies Although 311 receives high ratings from external users, we found that 311 could take steps to improve its effectiveness related to internal agencies. Some key 5 First call resolution is a metric used by 311 to indicate its success in resolving callers queries without forwarding the call to another City agency. First call resolution is expressed as a ratio of calls resolved by 311 first to total calls coming into 311. P a g e 3 Office of the Auditor

10 areas for improvement involve the City s CRM system, which is used jointly by 311 and some other City agencies and is maintained by Technology Services (TS). CRM is the primary tool used to manage and track interactions with 311 customers. However, the CRM system does not always allow for the types of management reports some City agencies need. Consequently, some agencies use their own systems alongside CRM, resulting in inefficiencies. For example, one agency keeps historical data in a Microsoft Access database, even though CRM can maintain historical data and synchronize with Access databases to receive the historical information. Representatives of several agencies that work with 311 also identified the need for 311 to work more effectively with the agencies to develop more useful reports, specifically reports that are better tailored to the agency management s needs rather than the standard reports that are provided to all agencies. For example, a representative of the Clerk and Recorder s Office provided to the audit team a report from 311 that contained a description field, but noted that the Office cannot filter reports specific to information contained in the description field. In addition, the Clerk and Recorder s representative noted that knowing the types of calls and age range of records requested in the Denver 311 reports would be very useful. The Clerk and Recorder s Office is working with a third-party vendor to convert hard copy records to digital formatting; knowing the age range of files requested via 311 would help set priority of conversion. Although both Denver 311 and the agencies each have a responsibility to identify areas where additional, tailored information would enhance agency management, 311 is limited in its ability to provide tailored reports without assistance from more sophisticated IT professionals. Although 311 is part of TS, CRM is managed by a group within TS that is separate from 311. Therefore, no one in Denver 311 can make unilateral changes to CRM to create custom reports that would enhance the ability of agency management to make business decisions. Further, some agency representatives indicated in an audit focus group an erroneous belief that custom reports were not available. Therefore, the Director of Denver 311 should inform and periodically remind agencies that custom CRM reports may be requested from TS. The Director should also work with agencies to identify data from CRM that can be used as an effective tool to enhance agency management. As part of the effort to create more useful reports, agencies and 311 can improve on working collaboratively to ensure the most up-to-date information is available in CRM, thus enhancing the quality of management reports 311 provides to agencies. Service Level Agreements are not the best way to codify partnerships between 311 and internal City agencies Denver 311 uses Service Level Agreements (SLAs) to codify agreements between 311 and its partner agencies. Since it is not required, not all City agencies have an SLA agreement in place with 311. According to Denver 311, there are eleven departments comprising a total of fourteen agencies that have signed an SLA with Denver 311; there are four departments comprising a total of six agencies that have received SLAs from 311 but elected not to sign them. SLAs are used to define obligations for each party to the agreement. However, since there is no mechanism in place to hold either 311 or a partner agency accountable for failure to meet the SLA s stated goals, the use of SLAs for this purpose is questionable. City and County of Denver P a g e 4

11 Further, the enhanced spirit of partnership that Denver 311 is attempting to develop with City agencies may be undermined by an unnecessary focus on compliance and enforcement. A different type of document, one that better reflects the collaborative relationship required for Denver 311 and City agencies to have mutual success, is needed. Therefore, Denver 311 should identify a different type of document, such as a Memorandum of Understanding, which emphasizes the mutual goals of 311 and a City agency and a commitment to work collaboratively to reach the stated goals. P a g e 5 Office of the Auditor

12 INTRODUCTION & BACKGROUND National Establishment of 311 The use of 311 as a standard abbreviated dialing arrangement for non-emergency information is based on the effectiveness of using the 911 number for emergency situations. The 911 number, first implemented by AT&T for its Bell Telephone System companies as the nationwide emergency number in the late 1960s, was originally intended to improve the timeliness of emergency services. Over time, however, some local government officials became concerned about the volume of calls their officers were handling. Officials realized that the volume of non-emergency calls to public safety personnel had become an impediment to handling true emergency calls. In 1997, after studying possible solutions to this issue, the Federal Communications Commission (FCC) authorized the creation of 311 as a national number for state and local governments to receive calls regarding non-emergency services and other government services. FCC cited two compelling reasons for creating 311 as opposed to allowing such calls to go to 911 or authorizing the use of a seven- or ten-digit non-emergency number. The first reason was the importance of relieving congested 911 lines. The second reason was the ease of remembering an abbreviated, three-digit number as opposed to a seven- to ten-digit number. 6 Overview of 311 in Denver In July 2006, the City and County of Denver launched the 311 Customer Service Center as a comprehensive one-stop resource for anyone seeking to address non-emergency needs. The 311 Customer Service Center was originally managed by the City s General Services department but in 2007, the Mayor s Office transferred the function to Technology Services (TS). Under TS, Denver 311 was integrated with the City s official website to allow for more effective, user-friendly access to Denver government services and information goals and performance measures Denver 311 has the dual goals of addressing external callers requests for information and providing internal agencies with efficiencies gained by not having to spend as much time and as many resources resolving citizen inquiries. To accomplish these goals and monitor the call center s success, Denver 311 uses various performance measures. Measures that focus on the quality of external customer service include the percent of calls answered within twenty seconds and citizen satisfaction ratings. Measures that relate to internal City agency customer service include the percent of calls resolved during initial contact, also known as first call resolution (FCR), and agency satisfaction ratings. The FCR measure is particularly 6 FCC. First Report and Order and Notice of Proposed Rulemaking (97-51), February 18, 1997, 7 To address external callers needs Denver 311 is available from 7:00 a.m. to 8:00 p.m. on weekdays and from 8:00 a.m. to 5:00 p.m. on weekends. Denver 311 is closed on City holidays and City furlough days. City and County of Denver P a g e 6

13 important to Denver 311, as it illustrates the amount of calls that would have otherwise been handled by an agency. Answering these calls quickly not only saves the agency time, but should also result in better Table 1 Denver 311 Budget service by giving the external Year Budget FTEs customer an answer quickly without 2006 $1,102, a number of call transfers $1,322, budget and staffing As shown in 2008 $1,533, Table 1, Denver 311 has a budget for 2009 $1,650, of approximately $1.6 million $1,611, Since 2007, the first full year that $1,571, operated, its budget has ranged 2012 $1,603, from $1.32 million to $1.65 million. The 311 budget decreased for two Source: City Budget Books consecutive years in 2010 and 2011, before increasing by about $32,000 in The number of full time equivalents (FTEs) within Denver 311 began at nineteen in 2006, but since 2007 the number of FTEs has been relatively steady, ranging from thirtytwo in the period 2008 through 2010 to thirty in 2011 and customer survey data At the end of an inquiry made to Denver 311 through a phone call or an , customers have the option of completing an on-line Customer Satisfaction Survey. Specifically, the survey prompts customers to indicate their method of contacting 311 (e.g., phone, website, ) and the purpose of their inquiry. As shown in Table 2, in 2012, customers most frequently called Denver 311 to inquire about registration renewal, property information, and general questions about vehicle registration. Table 2 Denver 311 Call Topics in 2012 by Frequency Rank # Call Topic #1 Registration Renewal #2 Property Information #3 Vehicle Registration (General) #4 Property and Sales Tax Information #5 Vehicle Titles #6 Property Tax Payment #7 Police Department #8 Recycling Schedule Source: Denver 311 official website: Denver 311 technology To track the types of calls received, the duration of the calls, and to create a work case in relation to each specific inquiry, Denver 311 utilizes a technological tool called the Customer Relationship Management System (CRM). The CRM system is an Oracle-PeopleSoft software application that tracks interactions with callers. Specifically, Denver 311 uses CRM to track call types and create cases. For almost every call a 311 customer service agent takes, the agent creates a P a g e 7 Office of the Auditor

14 corresponding case in CRM, which tracks such information as the caller s question or report, the general area in which the call can be categorized, what the 311 agent tells the caller, and whether the call was resolved by 311 or transferred to another City agency. 8 In conjunction with the CRM system, 311 customer service agents take and track calls through dual-screen desktop computers and Cisco IP phones. One of their computer screens displays the CRM portal, while the second screen is directly connected to the City s website, with Photo 1- Sample Reader Board different tabs that provide quick Source: Google Images access to various agency web pages. This allows the agents to address an external inquiry about the City by accessing information about the City that is stored in the CRM database or available on the City website. In addition to these tools, the Reader Board, mounted on the ceiling of the call center, displays the number of calls waiting to be answered, the number of agents available to take calls, and the number of abandoned calls. 9 In the event that the Cisco phones fail to function properly, the 311 agents workstations are equipped with Nortel phones as a back-up. From call center to contact center Denver 311 is progressively transitioning from the concept of a call center to becoming a more comprehensive contact center. The term contact center means that citizens and other interested parties can contact Denver 311 not only by dialing on their phones, but through other means including the City s official website, , walk-ins to the 311 call center, a smart phone application, and social media, such as Facebook and Twitter. According to 311, changing to a contact center means having a prioritized focus on the customer s experience. Calls have increased annually Since the commencement of Denver 311, the call center has handled an increasing number of calls. By October 1, 2008, less than two and a half years after its launch, Denver 311 had processed one million calls, and by January 2011, 311 had processed two million calls. Thus, the call center handled about 1,100 calls per day from July 2006 to October 2008, and about 1,200 calls a day from October 2008 to January This represents an increase in call volume of approximately 11 percent. Denver 311 has been nationally recognized The Public Technology Institute (PTI) recognized Denver 311 as one of the Citizen-Engaged Communities for PTI 8 If callers disconnect in the middle of a call, a case number may not be issued. 9 Calls abandoned are the total number of calls placed into Denver 311 when a call center agent was unable to answer before the caller disconnected. 10 These figures represent an average number of calls in a full week, Monday through Sunday. According to Denver 311, call volume drops significantly on Saturday and Sunday, which dilutes the average daily call volume. 11 The Public Technology Institute (PTI) provides research and resources for technology executives in local government. PTI recently celebrated its fortieth anniversary and reports almost 150 member cities or counties, including Denver. For more information about PTI, visit its website at City and County of Denver P a g e 8

15 created this program to recognize excellence in multi-channel contact centers and best practices for the use of Citizen Relationship/Records Management systems, 311 services, web portal technology, telephony systems, and mobile communications infrastructure. In addition, Citizen-Engaged Communities designees were recognized for being creative in the face of budget constraints, such as transitioning to more virtual agents, which is a computer generated program that serves in the traditional role of a customer service agent. Denver 311 Service Level Agreements When 311 officially launched in 2006, select City agencies were instructed by the Mayor s Office to work cooperatively with the call center, though not through a formal mandate such as an executive order. As a result, Service Level Agreements (SLA) are used to document the cooperative relationship between 311 and City agencies. Under SLA terms, 311 agrees to: respond to a percentage of phone calls within a specified time frame, resolve a specific percentage of customer inquiries on the first call, and deliver monthly activity reports summarizing CRM data from the preceding month, such as the total number of inquiries, total FCRs, and the percentage of cases still open. In return, the agency assigns an agency liaison and an agency decision maker who are responsible for providing agency information for the CRM database to allow 311 agents to share accurate information with individuals contacting 311. In addition, the agency liaison and decision maker are responsible for updating the call center with new procedures specific to their agency, and handling citizen inquiries related to their agency. According to call center staff, 311 services are most effective for City agencies that have entered into SLAs. For example, these agencies usually have higher FCR rates than City agencies that have not entered into SLAs. Achieving a high FCR rate allows the agency to free up personnel for other key tasks within the department rather than answering calls that could otherwise be handled by 311. Denver 311 serves customers of partner and non-partner agencies There is not an official designation for a partner agency, but Denver 311 distinguishes between partner and non-partner agencies internally while handling calls for both types. Denver 311 defines a partner agency as one that meets two criteria: the City agency s 10 digit number automatically rolls into the 311 center when dialed by a customer; and the City agency uses CRM in some fashion, which includes case management. Partner agencies generally enter into an SLA with Denver 311, but it is not a requirement. Non-partner agencies are identified by Denver 311 as those that do not use a CRM-compatible system or that do not work proactively with 311. A number of non-partner agencies manage their own internal call centers or have staff available to answer calls. Nonetheless, Denver 311 still addresses basic inquiries for external callers about these non-partner agencies In addition to partner and non-partner agencies, there are integrated agencies. These agencies, such as the Denver International Airport (DIA), have their own call centers and even use the same tracking tool, the CRM system. P a g e 9 Office of the Auditor

16 Denver 911 Handles Non-emergency Calls Directly Related to Public Safety Although Denver 311 was instituted to link citizens with the City for non-emergency services and communications, Denver 911 continues to address emergency calls as well as non-emergency calls that require a uniformed response from police, fire, or emergency medical services. 13 According to the 2012 Denver Budget Book, about 85 percent of all non-emergency calls in the City come through the non-emergency line at Denver 911. Non-emergency calls are made to a standard, non-abbreviated number. According to the Director of Denver 911, in 2011, 911 processed approximately 1.1 million phone calls. Of these, about 560,000 were non-emergency calls. Non-emergency calls require a police, fire, or safety response but are not as time sensitive as an emergency call. For example, a stolen bike with no witnesses or suspects may be classified as a 911 non-emergency call. The non-emergency line works based on a phone tree system. The tree allows callers to transfer directly to entities such as police records, the jail, or Denver 311. SCOPE The audit reviewed the Denver 311 Division of Information Technology Services. The audit included a review and analysis of current management processes, laws, revenues and expenditures, policies and procedures, and systems in place, as well as applicable financial data through June OBJECTIVE The purpose of the audit was to determine the effectiveness of Denver 311 at addressing citizen and internal City agency needs. This objective included an assessment of the technology used by Denver 311 and internal City agencies. METHODOLOGY This audit was performed using a variety of traditional audit methodologies. We reviewed various Denver 311 documents including policies and procedures, processes, 311 performance reports, Service Level Agreements, and other key documents. 13 The public safety non-emergency line is City and County of Denver P a g e 10

17 We reviewed internal and external audits completed on 311 programs around the country. We reviewed research on best practices and standards for 311 implementation and management. We evaluated the expenditures, revenues, and performance measures for Denver 311 from the program s inception through June We interviewed executive personnel for Denver 311 including Denver's Chief Information Officer and the Director of 311. In addition, we interviewed other staff within Denver 311, as well as non-311 staff within Technology Services. We conducted walkthroughs and job shadowing for both the Denver 311 call center and 911 emergency call center. Job shadowing entailed auditors listening to calls taken by call center agents and observing their methods of documenting and handling of the calls. We obtained the City of Minneapolis 311 Comparative Survey 2011, which compliled information from thirty-nine 311 centers around the country. We gathered information through interviews and report analysis from the Government Accountability Office (GAO) regarding GAO s 2011 audit of social media use in the federal government. Complementing the methods outlined above, the audit team conducted a focus group comprising representatives from several internal City agencies, most of which were partner agencies of Denver 311. The focus group elicited information about the agencies interactions with 311, including whether 311 had improved agency efficiency, the usefulness of CRM, and other areas in which 311 was performing well or could improve. Finally, the Auditor s Office contracted with a third-party polling firm, RBI Strategies and Research, to perform polling of Denver residents regarding awareness and use of Denver 311. The polling occurred in two phases, first with an assessment of general awareness of 311 among Denver residents and then with an evaluation of the experiences of 311 users. Further information about the methodologies for the two phases of polling is provided below. Some information requested in the surveys was for internal Auditor s Office use and did not relate to our evaluation of Denver 311. Appendices A and B contain all survey data related to Denver 311. Interactive Voice Response survey of Denver residents RBI Strategies and Research conducted an Interactive Voice Response (IVR) survey of 500 Denver residents. Interviews were conducted April 3-10, 2012, by 4Degrees, a market research firm specializing in IVR and robo-call surveys. Respondents were randomly selected from a list of residents of the City and County of Denver. The margin of error for a survey of 577 interviews is +/-4.38 percent at the 95 percent confidence level. The margin of error is higher for subsamples within the full sample. Other sources of error not accounted for by the stated statistical margin of error include, but are not limited to, question wording, question order, refusal to be interviewed, and demographic weighting. The sample was weighted both for P a g e 11 Office of the Auditor

18 ethnicity and gender to more accurately reflect the population of the City and County of Denver as determined by the 2010 United States census. Telephone survey of 311 users RBI Strategies and Research conducted a telephone survey of 501 residents of the City and County of Denver over the age of eighteen who indicated that they had used the Denver 311 system. Interviews were conducted April 9-17, 2012, by Standage Market Research, a market research firm specializing in telephone survey interviewing. Respondents were randomly selected using a random digit dial telephone sample of landline telephones purchased from Survey Sampling International. The margin of error for a survey of 501 interviews is +/-4.38 percent at the 95 percent confidence level. The margin of error is higher for subsamples within the full sample. Other sources of error not accounted for by the stated statistical margin of error include, but are not limited to, question wording, question order, and refusal to be interviewed. City and County of Denver P a g e 12

19 FINDING Denver 311 Receives High Satisfaction Rates from External Users but Should Work to Sustain These Satisfaction Levels and Improve its Partnership with Internal Users A statistically valid customer service survey commissioned by the Auditor s Office found that 88 percent of Denver 311 users were satisfied or highly satisfied regarding their overall experience with 311. Specific areas of high satisfaction included timeliness of call answering, resolution of issues, and overall knowledge demonstrated by call center agents. A review of internal performance metrics data for key 311 activities, such as abandoned calls and the time to answer calls, revealed that performance measures are trending down, which, if not addressed, will eventually negatively affect the satisfaction rates of Denver 311 users. 14 Further, although external user satisfaction rates are high, Denver 311 can improve its partnership with internal City agencies by working to demonstrate the value that 311 can provide to these agencies. A number of City agencies, including Denver 311 partners, are not satisfied with the Customer Relationship Management (CRM) software utilized by 311, including system reports. Enhancements to CRM would allow Denver 311 to better support City agencies. Additionally, changes to Service Level Agreements (SLAs), which codify the services that 311 and internal agencies provide to each other, may address inconsistencies related to expectations of both Denver 311 and partner agencies. Most Citizens Are Aware of Denver 311 and Users Provided High Satisfaction Ratings Two surveys of Denver citizens found that more than half of Denver residents are aware of 311 and that about four out of every ten Denver residents has used 311. Further, one of these surveys found that almost 90 percent of 311 users expressed satisfaction regarding their interaction with 311. The majority of Denver citizens are aware of Denver 311 The Auditor s Office contracted with a third party to administer an Interactive Voice Response (IVR) survey determining Denver residents awareness of Denver 311. The IVR survey polled 500 Denver residents to determine the overall awareness of Denver 311 in the City. This initial survey found that 61 percent of citizens were aware of 311. Of the 61 percent of respondents who were aware of Denver 311, 39 percent had used Denver 311 services. This means that 14 Calls abandoned are the total number of calls placed into Denver 311 when a call center agent was unable to answer before the caller disconnected. P a g e 13 Office of the Auditor

20 approximately 64 percent of Denver citizens who are aware of 311 have also used 311 in the past. 15 The vast majority of users rated 311 services as satisfactory After completion of the IVR survey measuring awareness, a follow-up telephone survey that contacted only Denver 311 users indicated a very high level of satisfaction among these users. Specifically, almost nine in ten users were satisfied with 311 s performance. The survey evaluated overall satisfaction with 311, as well as probing for satisfaction in specific areas. Eighty-eight percent of users reported overall satisfaction with their Denver 311 experience, with 56 percent reporting complete satisfaction and 32 percent reporting that they were somewhat satisfied. 16 Probing further, the survey requested that users rate their satisfaction regarding 311 call center agents courtesy and knowledge, as well as their satisfaction with the timeliness of receiving an answer to their query. The survey found that 93 percent were satisfied with the courtesy of the agents and 86 percent were satisfied with the knowledge of the agents. Additionally, 88 percent were satisfied with the timeliness of Denver 311 s response to questions that came in through either phone calls or . In addition to questions about satisfaction levels, the survey evaluated the extent to which users problems were resolved quickly. Seventy-nine percent of users reported that their issues were resolved through 311 services. Moreover, 71 percent of users reported that their issues were resolved with one contact to Table 3: Denver 311 User Satisfaction As a different way of assessing user satisfaction levels, the survey asked users to rate various aspects of 311 s service on a scale of one to ten, with ten representing the highest level of satisfaction. For qualities such as being treated with respect, wait time, satisfactory issue resolution, timely issue resolution, finding the right person, and using the 311 website, the average satisfaction level ranged from 8.90 out of ten (for being treated with respect) to 7.76 (for the ease with which the right person to talk to could be found). Aspect of 311 Users Experiences Average Satisfaction Rating (out of 10) Respectful treatment from 311 agents 8.90 Issue was satisfactorily resolved 8.12 Resolution timeliness 7.88 Satisfaction with 311 website 7.84 Waiting time on phone, in person, or via Ease of getting to the right person through Source: RBI Strategies and Research survey. 15 The 64 percent of Denver citizens who are aware of and have used 311 is based on the percent of citizens who used Denver 311 services (39 percent) divided by the citizens who were aware of Denver 311 (61 percent). 16 The 88 percent figure comes from responses to a question early in the survey. Following this question the poll probed regarding more specific areas of users 311 experiences. After these questions were asked, the poll returned to a question of overall satisfaction and found that 92 percent of users were satisfied with their 311 experience. We have chosen to report and discuss the 88 percent figure based on the risk that talking and thinking about 311 might have skewed the overall satisfaction level of 311 users report later in the poll. 17 This figure represents 71 percent of all users surveyed, as opposed to representing only 71 percent of the users who reported that their issue was resolved. City and County of Denver P a g e 14

21 See Table 3 for the full listing of average satisfaction scores. Overall, the lowest average satisfaction score was 7.76 out of ten, across a range of key performance measures, which speaks well for the quality of service Denver 311 provides. Finally, the survey evaluated satisfaction by assessing whether users would recommend Denver 311 to friends and whether users would access Denver 311 in the future. Sixtyseven percent of survey respondents would definitely recommend Denver 311 to their friends, relatives, or co-workers. Only 3 percent of users would never recommend Denver 311. Similarly, 73 percent of users would definitely use Denver 311 again in the future. On the other hand, only 2 percent of users reported they would never again use Denver 311. Cost data suggests that 311 offers good value to external users In addition to providing services that elicit high satisfaction ratings from external users, 311 provides its services at relatively low cost to the City $3.03 per call. In 2011, the City of Minneapolis conducted a survey of thirty-nine 311 call centers in a variety of areas. We compared Denver to eight cities that reported cost per call, as Denver does. 18 Cost per call is simply the total calls into the call center divided by the total call center budget. This provided one source of efficiency data with which to evaluate call centers. According to the survey results, Denver s cost per call was $3.03 in The eight cities we selected had a 2010 cost per call ranging from $1.65 to $6.38. While the cost per call data does not provide a true apples to apples comparison, the methodologies are close enough to suggest that Denver s cost per call is relatively low, specifically that it is about $1.00 per call lower than the mean ($4.02 per call) of other call centers that use the cost per call methodology. More information needed to better understand the source of high customer satisfaction Denver 311 should be commended for these high satisfaction ratings. As discussed below, some other performance metrics data suggest that unless action is taken to stem the decline of key service measures these high satisfaction ratings are at risk. While the survey data suggested some reasons for the high overall satisfaction rates, without additional understanding of the sources for users satisfaction, 311 management could inadvertently threaten customer satisfaction by making changes to the current structure of 311. To maintain the high levels of customer service satisfaction, the Director of Denver 311 should gather additional information from 311 users to better understand the specific reasons users are highly satisfied with Denver 311 services and to identify any new areas that could enhance satisfaction. This follow-up survey may be delivered by a professional third-party polling company, by an end-of-contact survey administered through the 311 system, or other cost effective methods as determined by the Director of Denver 311. This follow-up process would ensure that any changes made to Denver 311 are justified and do not negatively impact the high level of customer satisfaction reported by users. Additionally, these survey results can be used to address expansion needs, technological upgrades, and align Denver 311 s organizational initiatives and internal procedures with the needs of Denver 311 users. 18 These cities were Albany, GA; Albuquerque, NM; Arlington, TX; Boston, MA; Houston, TX; Philadelphia, PA; Rochester, NY; and Vancouver, BC (Canada). Some of these cities used total calls, which included abandoned calls, and some used only calls that agents handled. Other cities did not specify either total calls or calls handled. P a g e 15 Office of the Auditor

22 High External User Satisfaction Ratings are at Risk as Denver 311 Performance Measures Show Decreasing Service Levels Although Denver 311 receives very high satisfaction ratings from external users, 311 s internal performance metrics indicate that performance in key customer service areas is declining. These downward-trending metrics could eventually result in an overall decrease in customer satisfaction and suggest that, in some cases, 311 would be wellserved to abandon its effort to resolve calls within 311 prior to transferring a call, which could help improve 311 call center service levels. Performance metrics shows service declining in key areas Denver 311 uses industrystandard call center performance metrics to help assess the level of service it provides to external callers. As shown in Table 4, some key Denver 311 performance measures are trending negatively, such as calls abandoned and the time to answer calls. Specifically, from 2009 to 2011 calls abandoned increased from 44,490 to 144,023. In addition, the time for an agent to answer calls, which is also the time that a caller spent on hold, increased in 2010 and 2011 after declining from 2007 to was the first year since the beginning of 311 that the average time before a caller hangs up was less than the average time to answer a call. This is a concerning development and helps to explain the increasing number of callers abandon their calls. During 2011 the average time spent on a call increased as well, which, given the static number of call center staff, means that longer call times are increasing the wait for callers in the call queue. Table 4: Denver 311 Annual Call Metrics Average Time Average Time Call Calls Calls Average for 311 to Before Citizen Volume Handled Abandoned Length of Call Answer Hangs up , ,418 60,149 2:42 0:39 0: , ,948 55,993 2:38 0:32 0: , ,512 44,490 2:53 0:26 0: , ,080 93,157 3:14 1:07 1: , , ,023 3:43 2:09 1:45 Source: Denver 311 CRM Call Center Metrics January 2007 December Data from a nationwide survey of 311 call centers also indicates that Denver 311 is performing relatively poorly on two important performance measures: average call length and average time to answer. Thirty-nine 311 centers from around the country, including Denver, responded to a comparative survey conducted by the City of Minneapolis in Table 5 compares the performance metrics of the median ten cities relative to Denver based on call volume, and Appendix C provides responses from all thirty-nine call centers Entities that were open twenty-four hours per day were not included in this table, since call volume and calls handled could not be easily compared among call centers with a varying number of working hours. However, efficiency statistics can be more easily compared among these different call centers. City and County of Denver P a g e 16

23 While Table 5 suggests that Denver performs well in comparison to the median call centers in the comparative survey, as shown in Appendix C, only ten of the other thirtytwo cities that report average call length had an average call length longer than Denver s. Further, both Table 5 and Appendix C indicate that Denver has low comparative performance on the time to answer metric. Table 5 shows Denver performing worse than all but two comparison cities, and Appendix C amplifies this finding, showing that twenty-three of the other thirty cities performed better on the average time to answer metric than Denver. Table 5: 311 Call Centers 2010 Comparative Survey Call Calls Average Call Average Time for 311 to Volume Handled Length Answer Greensboro 266, ,344 2:50 0:13 Buffalo 277, ,507 2:10 0:09 Minneapolis 403, ,243 3:03 0:31 Kansas City 411, ,000 4:03 2:20 Vancouver 544, ,904 N/A 0:38 Denver 595, ,823 3:14 1:07 Jacksonville 678, ,242 2:59 1:28 Richmond 811, ,880 5:05 0:32 Philadelphia 1,369, ,000 4:40 0:39 Charlotte 1,860,001 1,635,384 3:41 1:27 Albuquerque 2,175,589 1,964,725 1:38 0:09 Average 853, ,550 3:20 0:49 Source: City of Minneapolis 311 Comparative Survey conducted in Note: Jacksonville s Calls Handled only includes calls handled by agents. Other calls were answered by an automated system. For the purpose of this survey, Denver reported on the last 13 months of CRM call center data for Moving from prioritization of first call resolution for all agencies could increase overall call center efficiency Although 311 management has expressed a desire to enter into agreements with additional partner agencies, based on the decline in service levels and the result of benchmark comparative analysis it does not appear viable for Denver 311 to attempt providing service for additional partner agencies around the City at its current staffing size. Including more partner agencies may unintentionally accelerate the negative trends observed in Denver 311 s performance metrics. An alternative option for 311 is to identify partner agencies for which first call resolution (FCR) rates are low, and begin transferring all related calls directly to these agencies rather than trying to resolve questions using call center agents. Low FCR rates for an agency indicate that 311 agents have difficulty answering questions about that agency. Accordingly, the caller and the agency may benefit from 311 no longer attempting to address inquiries related to the agency. For example, the FCR rate for the Public Trustee within the Clerk and Recorder s Office is 14 percent, compared to an FCR rate of 65 percent for the Elections Division. Although Denver 311 uses FCR as a key performance metric, if 311 routed callers directly to agencies with a low FCR rate, it may expedite 311 s ability to address other calls for P a g e 17 Office of the Auditor

24 which they have more information in a timely manner. This routing process may be accomplished through either an extended call tree option for incoming callers or a process change by 311 to allow 311 agents to immediately transfer customer inquiries related to specific agencies. Denver 311 has already implemented similar process changes for agencies including the Denver Motor Vehicle Division. Further, these process changes may address declining performance metrics at a lower cost than requesting additional staff. Denver 311 should conduct a review of FCR rates to determine which agencies may be the best candidates to receive a direct transfer of all relevant calls. 311 Has Options for Managing Call Volume and Addressing Service Level Decline Unless 311 receives an increase in staffing, which is unlikely in these challenging budget times, it will need to try some alternative approaches to managing call volume, thereby helping to address its declining service levels. First, the relatively high level of awareness that 311 holds among Denver residents has been attained in spite of a low budget for marketing. Therefore, additional marketing is needed to increase the awareness of alternative means for contacting Denver 311, such as , Facebook, and Twitter. In addition, a method of routing inquiries using textual analysis, based on existing technology, could possibly address potential increases in call volume by moving external requests directly to specific agencies. Denver 311 marketing resources have been underutilized Before officially launching Denver 311 in July 2006, the City advertised the program through decals, folded brochures, and public displays. Marketing materials also included 311 magnets, mugs, and t-shirts. In 2007, to continue encouraging Denver residents to approach their local government through the Denver 311 service, the City invested approximately $30,000 in 311 marketing efforts. After this initial marketing campaign, the agency was appropriated an annual marketing budget of approximately $10,000. A review of invoices paid by Denver 311 show that, for 2008 and 2009, 311 only spent an average of $3,000 of this amount, predominately on decals attached to City vehicles, folded brochures, and other display materials. In 2010, due to budget cuts, the 311 marketing budget was reduced from $10,000 to $5,000. Denver 311 management did not use all allotted funds for their marketing initiatives because they wanted to be conservative with new marketing efforts and avoid targeting out-of-city audiences. For instance, Denver 311 elected not to advertise on the radio to avoid reaching an audience external to the City. Other marketing opportunities have been leveraged Denver 311 s marketing budget is not the only source for 311 advertising. A number of neighborhood associations advertise 311 in their monthly newsletters and bulletins provided to constituents. Additionally, in 2011, Denver 311 proactively mailed invitations to the Denver Registered Neighborhood Organizations offering to attend neighborhood meetings and provide tours of Denver 311 to include side-by-side time with 311 call center agents. The survey results confirm the importance of 311 s marketing strategy. Specifically, according to the Auditor s Office RBI survey findings, flyers, media advertisements, and word-of-mouth were the most City and County of Denver P a g e 18

25 effective ways to notify citizens of services provided by Denver 311. Denver 311 should identify additional resources for enhanced marketing efforts. These resources could include additional budget allocations or creative, low-cost linkages with community groups, like the neighborhood registered organizations or other partners. Directing users to other points of contact would assist with managing call volume Due to the high level of satisfaction that Denver 311 users exhibit and their willingness to call back and recommend 311 to others, the volume of calls to Denver 311 will likely continue to increase. However, with no additional personnel to handle these calls, 311 s timeliness in responding to callers may continue to degrade. To avoid this decline, Denver 311 would need to gain additional personnel to handle increasing call volume or identify successful marketing efforts to redirect 311 users to alternative ways of contacting 311, such as the City s official website, , smart phone applications, Facebook, and Twitter. These tools are designed to replace traditional contact-center services and provide additional information through available channels. These tools may also improve consistency in 311 s workload. Because s do not require the same immediacy of response that is required by calls, additional or website inquiries could improve workload distribution, relative to the peaks and valleys associated with incoming call volume. Since an increase in personnel for 311 is unlikely in the current economic environment, Denver 311 should focus its marketing efforts on increasing awareness of the alternative methods for contacting 311. The Auditor s Office s third-party survey of Denver residents found that only 37 percent of Denver residents were aware there were non-telephone options for accessing Denver 311 services. These resources could include additional budget allocations or additional efforts at creative, low-cost linkages with community groups. Some available technology might also assist in managing call volume Although increased marketing is one method for improving 311 performance by moving inquiries to less time-sensitive media such as , some available technology could provide another alternative approach. Specifically, there is technology currently available that reads text-based messages and identifies the best direction to route the message. Denver 311 could use such technology, if it was cost-feasible and accurate, to automatically take reports or social media questions and route the information to the appropriate City agency without using 311 call center employees. The Director of Denver 311 should assess the feasibility, including the cost, accuracy, and reliability, of acquiring technology that automatically routes citizen messages found in s, Facebook, and Twitter to the appropriate City agency. Enhancements to How CRM is Used Would Allow Denver 311 to Better Support Internal City Agencies Although Denver 311 has received high ratings from external users, 311 can improve its interactions with internal City agencies. While CRM is the primary tool used by Denver 311, the CRM system itself is maintained and managed by entities within Technology Services (TS) outside of 311. Although CRM is available Citywide, some City agencies P a g e 19 Office of the Auditor

26 choose not to use CRM based on their belief that CRM does not generate helpful reports or does not automatically interface with their internal databases. Consequently, some City agencies maintain separate databases in addition to or instead of CRM, which results in an unnecessary duplication of time and resources. Denver 311 can only monitor the status of cases for City agencies that actively use CRM. Therefore, Denver 311 and City agencies must work collaboratively, recognizing each agency s respective business needs, to ensure that CRM contains relevant information and that agency management is provided the key information it needs to make critical analyses and decisions. This partnership and overall assessment may include determining the costs associated with upgrading the technological capabilities of CRM or switching to a more compatible software system based on the needs of Denver 311 and City agency users. CRM reports provided to partner agencies can be improved Denver 311 provides to its partner agencies a Monthly Activity Report (MAR) with an overview of metrics such as FCR rates, a synopsis of inquiries and requests related to the agency, and other performance measures. 20 Some partner agencies reported that CRM reports are not tailored to assist them in monitoring the needs of customer inquiries specific to their agency. This lack of data may prevent an agency from making informed business decisions based solely on data from CRM reports. For example, a representative of the Clerk and Recorder s Office provided to the audit team a report from 311 that contained a description field, but noted that the Office cannot filter reports specific to information contained in the description field. In addition, the Clerk and Recorder s representative noted that knowing the types of calls and age range of records requested in the Denver 311 reports would be very useful. The Clerk and Recorder s Office is working with a third-party vendor to convert hard copy records to digital formatting; knowing the age range of files most frequently requested via 311 would help set priority of conversion. Denver 311 and partner agencies may request custom CRM reports using forms available on the City s intranet (DOT), and the requests are addressed on a first-come-first-served basis. 21 TS has a dedicated CRM group that can construct query-based CRM reports, but other technological service needs within the City may take precedence over creating such reports. In the focus group conducted by the Auditor s Office, several agency representatives indicated a lack of knowledge about the ability to request custom reports from TS, instead believing erroneously that custom reports are unattainable. Denver 311 and the agencies each have a responsibility to identify areas where additional or specifically tailored information would enhance partner agencies management. As part of the effort to create more useful reports, agencies and 311 can work more collaboratively to ensure that the most up-to-date information is available in CRM and the MAR. Therefore, the Director of Denver 311 should inform and periodically remind agencies that custom CRM reports may be requested from TS. The Director 20 As provided by each SLA, a partner agency will receive by the 15 th day of the subsequent month a MAR summarizing its CRM data for the preceding monthly reporting period. In addition to presenting a synopsis of the total number of inquiries and requests, raw number and percentage of FCRs, and raw number and percentage of cases that remain open, a MAR will also include an analysis of issues and trends depicted by the CRM data. 21 CRM Report Request forms are available on the City s intranet (DOT) homepage in the field labeled 311 Call Center. Report requests must be mailed directly to the Technology Services CRM Support Team by the agency s 311 liaison. City and County of Denver P a g e 20

27 should also work with agencies to identify data from CRM that can be used as an effective tool to enhance agency management. City agency databases unnecessarily duplicate CRM Several City agencies reported populating their own customer service legacy systems, which were in place prior to entering their partnerships with Denver 311. These systems assisted agencies in creating reports and providing specific data related to their operations and how those operations affected external customers. However, even after partnering with Denver 311, some partner agencies retained their legacy systems. For example, one agency keeps historical data in a Microsoft Access database, even though CRM can maintain historical data and synchronize with Access to receive the historical information. 22 Therefore, some 311 partner agencies enter the same information into both CRM and their legacy systems instead of consolidating information into one system. In addition to the duplication of data and loss of time caused by using multiple systems, some agencies believe that their legacy systems and CRM cannot interface. Further, some City agencies rely solely on the historical data provided by their internal legacy databases, using CRM only to close out cases opened by Denver 311. Many agencies are concerned that using CRM exclusively would end accessibility to the historical data maintained in their legacy systems. However, according to the TS CRM team, CRM has the capability to interface with a number of City systems. Denver 311 should coordinate with City agencies and TS to determine how the historical data captured within partner agencies legacy systems can interface with CRM to reduce duplication, centralize information, and improve reporting functionality. Some third-party databases accessed within the City cannot interface with CRM CRM cannot interface with some of the State of Colorado s databases, which are used by designated agencies within the City. For instance, the Denver Public Library (DPL) does not operate a call center and has an automated phone tree that handles most switchboard functions. In addition, DPL uses a web-based application called Data Access in Real Time (DART), which is provided by the Colorado State Library. This system is not compatible with CRM. Therefore, DPL has declined to enter into a partnership with Denver 311. The Denver County Court provides another example of CRM not interfacing with operating systems used within the City. The Denver County Court's case management system, THEMIS, holds all pertinent information for every matter brought before the Court, but it does not interface with CRM. As such, Denver 311 can only answer basic questions related to Denver County Court, such as the Court s location and hours of operation. Denver 311 personnel are not trained in Court protocol, do not have access to Court records, cannot give legal advice, and may not be able to determine the difference between providing information and advice. In these instances, initial involvement by 311 may add to a caller's frustration or increase hold time before the caller can ultimately reach the Court. 22 According to the City of Denver s Chief Information Officer, Technology Services would prefer to upgrade the agencies legacy systems to an enterprise service tool before integrating the agencies databases with CRM. This process helps better expedite service requests and compatability. P a g e 21 Office of the Auditor

28 The Director of Denver 311 should identify ways to improve the process for callers who need information from or about agencies that do not interface with CRM, such as forwarding calls directly to the designated agency in some cases. Additionally, the Director of Denver 311 should assess the feasibility and cost of adding an upfront queue announcer, which is an automated message informing callers of various options prior to being placed in the hold queue. This information may include current hold times, offering options for call back from a customer service agent at a later time, or other time-sensitive information. This feature may prove especially useful to users and Denver 311 during peak call times. However, as referenced earlier in the Finding, the Director of 311 should base any technological changes on follow-up survey data obtained from users to ensure that 311 s high customer satisfaction ratings among citizens are not negatively impacted. Denver 311 and partner agencies have a dual responsibility to ensure that CRM is up to date Although agency information contained within CRM enables 311 agents to answer caller inquiries, not all partner agencies are populating CRM to the fullest extent. For example, one agency that shared its experiences regarding 311 with us admitted that it did not maintain relevant information about the agency in CRM. Although 311, as the steward of CRM, has a responsibility to maintain CRM accurately and fully, it cannot do so unless agencies also fulfill their responsibility to make updated information available in CRM. This enhanced collaboration between agencies and 311 would have numerous positive effects, including improving the content captured in CRM, which assists Denver 311 in addressing calls on first contact. Further, an expanded CRM database better serves external callers, who would be more likely to get accurate data from their first point of contact with the City. To emphasize to agencies the importance of providing up-to-date CRM data, the Chief Information Officer should coordinate with the Mayor s Office to ensure that City agency representatives are updating CRM as appropriate. Service Level Agreements Are Not the Best Way to Codify Partnerships between 311 and Internal City Agencies Denver 311 uses Service Level Agreements (SLAs) to codify agreements between 311 and its partner agencies. 23 However, since it is not required, not all partner agencies have an SLA agreement in place with 311. According to Denver 311, there are eleven departments comprising a total of fourteen agencies that have signed an SLA with Denver 311; there are four departments comprising a total of six agencies that have received an SLA from 311 but elected not to sign it during the audit period. SLAs are used to define obligations for each party to the agreement. Since there is no mechanism in place to hold either 311 or a partner agency accountable for failure to meet the SLA s stated goals, the effectiveness of using SLAs is questionable. Further, the climate of partnership that Denver 311 is attempting to develop with City agencies may be undermined by an unnecessary focus on compliance and enforcement. A different 23 Denver 311 defines a partner agency as an agency that agrees to work cooperatively with Denver 311. Although non-partner agencies have not entered into an SLA with Denver 311, 311 collects publicly available information related to all City agencies in order to have data in the CRM system for basic call inquires. City and County of Denver P a g e 22

29 type of document, one that better reflects the collaborative relationship that needs to exist for Denver 311 and City agencies to have mutual success, is needed. A Memorandum of Understanding, for example, documents the intent of two parties without assuming legally binding obligations. Therefore, the Director of Denver 311 should identify a different type of document, such as a Memorandum of Understanding, which emphasizes the mutual goals of 311 and a City agency and a commitment to work collaboratively to reach the stated goals. RECOMMENDATIONS 1.1 To maintain the high levels of customer service satisfaction, the Director of Denver 311 should gather additional information from 311 users to better understand the specific reasons users are highly satisfied with Denver 311 services and to identify any new areas that could enhance satisfaction. This follow-up survey may be delivered by a professional third-party polling company, by an end-of-contact survey administered through the 311 system, or other cost effective methods as determined by the Director of Denver Denver 311 should conduct a review of first call resolution rates to determine which agencies may be the best candidates to receive a direct transfer of all relevant calls. 1.3 Denver 311 should identify additional resources for enhanced marketing efforts. These resources could include additional budget allocations or additional efforts at creative, low-cost linkages with community groups. Since an increase in personnel for 311 is unlikely in the current economic environment, Denver 311 should focus its marketing efforts on increasing awareness of the alternative methods for contacting 311, such as , the City s website, Facebook, and Twitter. 1.4 The Director of Denver 311 should assess the feasibility, including the cost, accuracy, and reliability, of acquiring technology that automatically routes citizen messages found in s, Facebook, and Twitter to the appropriate City agency. 1.5 The Director of Denver 311 should inform and periodically remind agencies that custom CRM reports may be requested from Technology Services. The Director should also work with agencies to identify data from CRM that can be used as an effective tool to enhance agency management. 1.6 To fully realize the benefits of CRM, the Chief Information Officer should assess the implementation costs of upgrading the technological capabilities of CRM or switching to a more compatible software system based on the needs of Denver 311 and City agencies. 1.7 The Director of Denver 311 should coordinate with City agencies and Technology Services to determine how the historical data captured within partner agencies legacy systems can interface with CRM to reduce duplication, centralize information, and improve reporting functionality. P a g e 23 Office of the Auditor

30 1.8 The Director of Denver 311 should identify ways to improve the process for callers who need information from or about agencies that do not interface with CRM, such as forwarding calls directly to the designated agency in some cases. 1.9 The Director of Denver 311 should assess the feasibility and cost of adding an upfront queue announcer, which is an automated message informing callers of various options prior to being placed in the hold queue. However, the Director of 311 should base any technological changes on follow-up survey data obtained from users to ensure that 311 s high customer satisfaction ratings among citizens are not negatively impacted To emphasize to agencies the importance of providing up-to-date CRM data, the Chief Information Officer should coordinate with the Mayor s Office to ensure that City agency representatives are updating CRM as appropriate The Director of Denver 311 should identify a different type of document, such as a Memorandum of Understanding, which emphasizes the mutual goals of 311 and a City agency and a commitment to work collaboratively to reach the stated goals. City and County of Denver P a g e 24

31 APPENDICES Appendix A: Interactive Voice Response Survey of Denver Residents P a g e 25 Office of the Auditor

32 City and County of Denver P a g e 26

33 Appendix B: Telephone Survey of Denver 311 Users P a g e 27 Office of the Auditor

34 City and County of Denver P a g e 28

35 P a g e 29 Office of the Auditor

36 City and County of Denver P a g e 30

37 P a g e 31 Office of the Auditor

38 City and County of Denver P a g e 32

39 P a g e 33 Office of the Auditor

40 Appendix C: City of Minneapolis 311 Comparative Survey 2011 Hours of operation Call Volume Calls Handled Average Length of Call (seconds) Average Time for 311 to Answer (seconds) 1 Baltimore N/A N/A 180 sec. 10 sec. 2 Chattanooga M-F N/A N/A N/A N/A 3 Chesapeake N/A N/A N/A N/A 4 Downers Grove N/A N/A N/A N/A 5 Springfield M-F N/A N/A 112 sec. 30 sec. 6 Winston-Salem 24/7 N/A 295, sec. N/A 7 Tempe ,134 6, sec. 21 sec. 8 Anaheim M-F 11,500 11, sec. N/A 9 Albany ,318 23, sec. N/A 10 Hartford M-F 11 Oshawa M-F and Sat. 88,509 N/A N/A N/A 109,100 98, sec. Talk time 27 sec. 12 Hampton , , sec. 33 sec. 13 Riverside M-F and Sat. & Sun. 176, , sec. 18 sec. 14 Virginia Beach ,026 N/A sec. 18 sec. City and County of Denver P a g e 34

41 Hours of operation Call Volume Calls Handled Average Length of Call (seconds) Average Time for 311 to Answer (seconds) 15 Arlington , , sec. 89 sec. 16 Boston 24/7 239, , sec. 226 sec. 17 Indianapolis M-F 18 Columbus M-F 19 Greensboro M-F 20 Buffalo M-F 255, , sec sec. 262, , sec sec. 266, , sec. 13 sec. 277, , sec. 9 sec. 21 Louisville 24/7 297, ,849 N/A 12.5 sec. 22 Minneapolis M-F 23 Kansas City M-F 24 Rochester 24/7 403, , sec. 31 sec. 411, , sec. 140 sec. 470, , sec. 92% in 30 sec. Closed 5 Holidays: Xmas, Thanksgiving, Memorial Day, New Years, Labor Day 25 Vancouver , 365 days/yr 544, ,904 N/A 38 sec. P a g e 35 Office of the Auditor

42 Hours of operation 26 Denver M-F, Sat.- Sun. Call Volume Calls Handled Average Length of Call (seconds) Average Time for 311 to Answer (seconds) 595, , sec. 63 sec. 27 Jacksonville M-F 678, ,242 by agent, others answered by system 179 sec. 88 sec. 28 Richmond M-F 811, , sec. 32 sec. 29 Calgary 24/7 1,190,000 1,004, sec. 68 sec. 30 Austin 24/7 1,200,000 1,188, sec. 8 sec. 31 Toronto 24/7/365 1,313, , sec. 80 sec. 32 Philadelphia M-F 1,369, , sec. 39 sec. 33 Edmonton 24/7 1,677,822 1,599, sec. 15 sec. 34 Charlotte M - Sun. 1,860,001 1,635, sec. 87 sec. 35 Albuquerque M - Sat., Sun. with limited calls (transit calls) 2,175,589 1,964, sec. 9 sec. City and County of Denver P a g e 36

43 Hours of operation 36 Houston M - F, and Sat. - Sun. Call Volume Calls Handled Average Length of Call (seconds) Average Time for 311 to Answer (seconds) 2,262,316 2,134, sec. 36 sec. 37 Washington 24/7/365 2,602,998 2,288, sec. 78 sec. 38 San Francisco 24/7/365 2,651,576 2,311, sec. 29 sec. 39 New York 24/7/ M 9M (over 50% resolved via IVR messaging) 242 sec. 26 sec. Source: City of Minneapolis 311 Comparative Survey P a g e 37 Office of the Auditor

44 AGENCY RESPONSE City and County of Denver P a g e 38

45 P a g e 39 Office of the Auditor

46 City and County of Denver P a g e 40

47 P a g e 41 Office of the Auditor

48 City and County of Denver P a g e 42

Denver 311 Follow up Report

Denver 311 Follow up Report Denver 311 Follow up Report December 2014 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver is independently

More information

Citywide Social Media Usage Follow-up Report

Citywide Social Media Usage Follow-up Report Citywide Social Media Usage Follow-up Report May 2015 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver is

More information

The Department of General Services Contract Administration Follow up Report

The Department of General Services Contract Administration Follow up Report The Department of General Services Contract Administration Follow up Report June 2015 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of

More information

City Attorney s Office: Litigation and Claims Management Follow-up Report

City Attorney s Office: Litigation and Claims Management Follow-up Report City Attorney s Office: Litigation and Claims Management Follow-up Report April 2015 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the

More information

Assessor s Office Performance Audit

Assessor s Office Performance Audit Assessor s Office Performance Audit June 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver is independently

More information

DIA Network Security Management Follow up Report

DIA Network Security Management Follow up Report DIA Network Security Management Follow up Report March 2015 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver

More information

Citywide Identity Management Follow up Report

Citywide Identity Management Follow up Report Citywide Identity Management Follow up Report July 2015 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver

More information

Network Security Management Phases 1 and 2 Follow up Report

Network Security Management Phases 1 and 2 Follow up Report Network Security Management Phases 1 and 2 Follow up Report March 2015 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County

More information

Police Records Management System IT General Controls Follow up Report

Police Records Management System IT General Controls Follow up Report Police Records Management System IT General Controls Follow up Report March 2015 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City

More information

911 Data Center Operations Performance Audit

911 Data Center Operations Performance Audit 911 Data Center Operations Performance Audit June 2010 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver is

More information

FOLLOW-UP REPORT Change Management Practices

FOLLOW-UP REPORT Change Management Practices FOLLOW-UP REPORT Change Management Practices May 2016 Office of the Auditor Audit Services Division City and County of Denver Timothy M. O Brien, CPA The Auditor of the City and County of Denver is independently

More information

DIA Network Device Security Management Performance Audit

DIA Network Device Security Management Performance Audit DIA Network Device Security Management Performance Audit June 2014 Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver is independently

More information

The Department of General Services Contract Administration Performance Audit

The Department of General Services Contract Administration Performance Audit The Department of General Services Contract Administration Performance Audit August 2014 Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County

More information

PeopleSoft IT General Controls

PeopleSoft IT General Controls PeopleSoft IT General Controls Performance Audit December 2009 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of

More information

Network Security Management Phase 1 Performance Audit

Network Security Management Phase 1 Performance Audit Network Security Management Phase 1 Performance Audit March 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of

More information

Denver International Airport Airport Legal Services Section Performance Audit

Denver International Airport Airport Legal Services Section Performance Audit Denver International Airport Airport Legal Services Section Performance Audit July 2014 Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County

More information

City Attorney s Office: Litigation and Claims Management Performance Audit

City Attorney s Office: Litigation and Claims Management Performance Audit City Attorney s Office: Litigation and Claims Management Performance Audit June 2013 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the

More information

Director s Message. Several accomplishments for 2012 include:

Director s Message. Several accomplishments for 2012 include: 311 2012 ANNUAL REPORT Denver 311 provides an informative community link for non-emergency City services and also an accessible information exchange between citizens and local government. Denver 311 accomplishes

More information

Audit Committee. Audit Staff

Audit Committee. Audit Staff The Auditor of the City and County of Denver is independently elected by the citizens of Denver. He is responsible for examining and evaluating the operations of City agencies for the purpose of ensuring

More information

Workers Compensation Program Performance Audit

Workers Compensation Program Performance Audit Workers Compensation Program Performance Audit February 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver

More information

Fixed Assets Management Performance Audit

Fixed Assets Management Performance Audit Fixed Assets Management Performance Audit May 2010 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver is independently

More information

Denver International Airport Planning and Development Division Performance Audit

Denver International Airport Planning and Development Division Performance Audit Denver International Airport Planning and Development Division Performance Audit June 2013 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor

More information

Department of Human Services Performance Audit

Department of Human Services Performance Audit Department of Human Services Performance Audit October 2013 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver

More information

Mobile Devices Performance Audit

Mobile Devices Performance Audit Mobile Devices Performance Audit August 2014 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver is independently

More information

Network Security Management Phase 2 Performance Audit

Network Security Management Phase 2 Performance Audit Network Security Management Phase 2 Performance Audit July 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of

More information

Audit of the City s 311 Call Center

Audit of the City s 311 Call Center Audit of the City s 311 Call Center Report # 2015-04 May, 2015 The 311 Call Center is Unable to Meet Its Current Service Level Goal Additional Technological Enhancements May Help the 311 Call Center Improve

More information

AUDIT REPORT Technology Services IT Service Desk

AUDIT REPORT Technology Services IT Service Desk AUDIT REPORT Technology Services IT Service Desk April 2016 Office of the Auditor Audit Services Division City and County of Denver Timothy M. O Brien, CPA The Auditor of the City and County of Denver

More information

DIA Information Security Management Performance Audit

DIA Information Security Management Performance Audit DIA Information Security Management Performance Audit November 2010 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County

More information

OFFICE OF INTERNAL AUDIT Stan Sewell, City Auditor

OFFICE OF INTERNAL AUDIT Stan Sewell, City Auditor OFFICE OF INTERNAL AUDIT Stan Sewell, City Auditor December 19, 2012 Mayor and City Council City of Chattanooga Chattanooga, TN 37402 RE: IS 311 Call Center Operations, Audit 12-06 Dear Mayor Littlefield

More information

Citywide Identity Management Performance Audit

Citywide Identity Management Performance Audit Citywide Identity Management Performance Audit March 2014 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver

More information

Police Records Management System IT General Controls Performance Audit

Police Records Management System IT General Controls Performance Audit Police Records Management System IT General Controls Performance Audit December 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the

More information

Citywide Records Management Performance Audit

Citywide Records Management Performance Audit Citywide Records Management Performance Audit August 2009 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of Denver

More information

OFFICE OF THE CITY AUDITOR

OFFICE OF THE CITY AUDITOR OFFICE OF THE CITY AUDITOR PERFORMANCE AUDIT OF THE CITIZEN REQUEST MANAGEMENT SYSTEM Thomas M. Taylor, CPA City Auditor Prepared by: Paul T. Garner, CCP, CSP Senior Audit Manager David Henley, CPA Auditor

More information

Enterprise Asset Management Customer Utility Board Charter

Enterprise Asset Management Customer Utility Board Charter Customer Utility Board Charter Executive Summary: The Customer Utility Board (CUB) exists to provide DAS customers with a meaningful voice in the cost, type, quality, and quantity of utility services delivered.

More information

AUDIT REPORT Department of Finance Accounting Services

AUDIT REPORT Department of Finance Accounting Services AUDIT REPORT Department of Finance Accounting Services March 2016 Office of the Auditor Audit Services Division City and County of Denver Timothy M. O Brien, CPA The Auditor of the City and County of Denver

More information

Office of the Independent Monitor Performance Audit

Office of the Independent Monitor Performance Audit Office of the Independent Monitor Performance Audit October 2011 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and County of

More information

Office of the City Manager INFORMATION CALENDAR December 15, 2015

Office of the City Manager INFORMATION CALENDAR December 15, 2015 Office of the City Manager To: From: Honorable Mayor and Members of the City Council Dee Williams-Ridley, Interim City Manager Submitted by: Savita Chaudhary, Director, Information Technology Subject:

More information

Customer Service Performance Report

Customer Service Performance Report Customer Service Performance Report 2009 Pennsylvania Electric & Natural Gas Distribution Companies Pennsylvania Public Utility Commission Bureau of Consumer Services Table of Contents Introduction I.

More information

Interactive Intelligence

Interactive Intelligence Interactive Intelligence Customer Service Experience Study (Wave II) by Joe Staples Chief Marketing Officer Interactive Intelligence, Inc. and Thomas Bailey Content Editor Interactive Intelligence, Inc.

More information

10-15 MEMORIAL HEALTH SYSTEM WORKERS COMPENSATION SELF INSURANCE FUND PUBLIC REPORT. July 22, 2010 CITY OF COLORADO SPRINGS OFFICE OF THE CITY AUDITOR

10-15 MEMORIAL HEALTH SYSTEM WORKERS COMPENSATION SELF INSURANCE FUND PUBLIC REPORT. July 22, 2010 CITY OF COLORADO SPRINGS OFFICE OF THE CITY AUDITOR CITY OF COLORADO SPRINGS OFFICE OF THE CITY AUDITOR 10-15 MEMORIAL HEALTH SYSTEM WORKERS COMPENSATION SELF INSURANCE FUND PUBLIC REPORT July 22, 2010 Denny Nester, MBA CPA CIA CGFM CFE CGAP Interim City

More information

Denver International Airport Fleet Management Program Performance Audit

Denver International Airport Fleet Management Program Performance Audit Denver International Airport Fleet Management Program Performance Audit December 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of

More information

Office of Emergency Management and Homeland Security Performance Audit

Office of Emergency Management and Homeland Security Performance Audit Office of Emergency Management and Homeland Security Performance Audit November 2011 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the

More information

Department of Aviation Revenue Contract Management Performance Audit

Department of Aviation Revenue Contract Management Performance Audit Department of Aviation Revenue Contract Management Performance Audit March 2011 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City

More information

Denver International Airport Facility Management Performance Audit

Denver International Airport Facility Management Performance Audit Denver International Airport Facility Management Performance Audit February 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City

More information

Customer Care Centers

Customer Care Centers Customer Care Centers Audit Report Report Number MS-AR-15-006 June 11, 2015 Highlights Residential and business customers contact the U.S. Postal Service through various toll-free telephone numbers for

More information

Customer Experience Survey Report

Customer Experience Survey Report O F F I C E O F I N S P E C T O R G E N E R A L OFFICE OF INSPECTOR GENERAL EXPORT-IMPORT BANK of the UNITED STATES Customer Experience Survey Report Summary Report Only - Please contact the Office of

More information

INSPECTOR GENERAL UNITED STATES POSTAL SERVICE

INSPECTOR GENERAL UNITED STATES POSTAL SERVICE OFFICE OF INSPECTOR GENERAL UNITED STATES POSTAL SERVICE Trends and Systemic Issues in Defense Contract Audit Agency Audit Work for Fiscal Years 2009-2012 Management Advisory Report Report Number March

More information

OFFICE OF THE AUDITOR

OFFICE OF THE AUDITOR OFFICE OF THE AUDITOR DEPARTMENT OF AVIATION INTERNAL CONTROL REVIEW AND CONTRACT COMPLIANCE AUDIT NOVEMBER 2007 Dennis J. Gallagher Auditor Dennis J. Gallagher Auditor Mr. Turner West, Manager Department

More information

Cable Television Community Needs and Interests Assessment Fort Collins, Colorado. By Constance Ledoux Book, Ph.D.

Cable Television Community Needs and Interests Assessment Fort Collins, Colorado. By Constance Ledoux Book, Ph.D. Cable Television Community Needs and Interests Assessment Fort Collins, Colorado By Constance Ledoux Book, Ph.D. January 2004 CABLE TELEVISION COMMUNITY NEEDS AND INTERESTS SURVEY 1 Executive Summary In

More information

Innovation and Technology Department

Innovation and Technology Department PERFORMANCE AUDIT Innovation and Technology Department IT Inventory Asset Management January 11, 2016 Office of the City Manager Internal Audit Division Cheryl Johannes, Internal Audit Manager PERFORMANCE

More information

Public Safety Camera System Audit

Public Safety Camera System Audit City of Austin AUDIT REPORT A Report to the Austin City Council Mayor Lee Leffingwell Mayor Pro Tem Sheryl Cole Public Safety Camera System Audit February 2013 Council Members Chris Riley Mike Martinez

More information

STATE OF NEW YORK OFFICE OF THE STATE COMPTROLLER 110 STATE STREET ALBANY, NEW YORK 12236

STATE OF NEW YORK OFFICE OF THE STATE COMPTROLLER 110 STATE STREET ALBANY, NEW YORK 12236 THOMAS P. DiNAPOLI COMPTROLLER STATE OF NEW YORK OFFICE OF THE STATE COMPTROLLER 110 STATE STREET ALBANY, NEW YORK 12236 GABRIEL F. DEYO DEPUTY COMPTROLLER DIVISION OF LOCAL GOVERNMENT AND SCHOOL ACCOUNTABILITY

More information

SPECIAL ADVISORY REPORT

SPECIAL ADVISORY REPORT City & County of Denver May 2011 Office of the Auditor Dennis J. Gallagher Auditor SPECIAL ADVISORY REPORT Fleet Management Consolidation SUMMARY OF SERVICE A (SAR) provides information on limited reviews

More information

Office of the Auditor General Performance Audit Report. Statewide Oracle Database Controls Department of Technology, Management, and Budget

Office of the Auditor General Performance Audit Report. Statewide Oracle Database Controls Department of Technology, Management, and Budget Office of the Auditor General Performance Audit Report Statewide Oracle Database Controls Department of Technology, Management, and Budget March 2015 071-0565-14 State of Michigan Auditor General Doug

More information

IT Service Desk Unit Opportunities for Improving Service and Cost-Effectiveness

IT Service Desk Unit Opportunities for Improving Service and Cost-Effectiveness AUDITOR GENERAL S REPORT ACTION REQUIRED IT Service Desk Unit Opportunities for Improving Service and Cost-Effectiveness Date: September 18, 2013 To: From: Wards: Audit Committee Auditor General All Reference

More information

The City of Calgary, 2009 PSC Operational Review Final Report

The City of Calgary, 2009 PSC Operational Review Final Report 1. The City of Calgary, 2009 PSC Operational Review Final Report Prepared by Framework Partners Inc. & Emergency Services Consulting International April 20, 2010 Table of Contents Executive Summary...

More information

CITY OF HOUSTON. Executive Order. Information Technology (IT) Governance

CITY OF HOUSTON. Executive Order. Information Technology (IT) Governance CITY OF HOUSTON Executive Order E.O. No: 1-44 Effective Date: December 20, 2012 1. AUTHORITY 1.1 Article VI, Section 7a, of the City Charter of the City of Houston. 2. PURPOSE 2.1 The City of Houston seeks

More information

Customer Service Performance Report

Customer Service Performance Report Customer Service Performance Report 2010 Pennsylvania Electric & Natural Gas Distribution Companies Pennsylvania Public Utility Commission Bureau of Consumer Services Table of Contents Introduction...

More information

AustinGO: Website Governance and Management Audit

AustinGO: Website Governance and Management Audit City of Austin AUDIT REPORT A Report to the Austin City Council Mayor Lee Leffingwell Mayor Pro Tem Sheryl Cole AustinGO: Website Governance and Management Audit August 2013 Council Members Chris Riley

More information

Audit of the Seattle Police Department s Public Disclosure Process

Audit of the Seattle Police Department s Public Disclosure Process Office of City Auditor 700 Fifth Avenue, Suite 2410 Seattle WA 98124-4729 Ph: 206-233-3801 www.seattle.gov/cityauditor Audit of the Seattle Police Department s Public Disclosure Process 3/16/2015 Seattle

More information

Oregon Employment Department. Unemployment Insurance Call Center Upgrade. May 28, 2010. Submitted by: Troy Rutten, CIO Oregon Employment Department

Oregon Employment Department. Unemployment Insurance Call Center Upgrade. May 28, 2010. Submitted by: Troy Rutten, CIO Oregon Employment Department Oregon Employment Department Unemployment Insurance Call Center Upgrade May 28, 2010 Submitted by: Troy Rutten, CIO Oregon Employment Department Category: Improving State Operations Project Team: Tom Byerley,

More information

Veterans Benefits Administration

Veterans Benefits Administration OFFICE OF AUDITS & EVALUATIONS Veterans Benefits Administration Audit of National Call Centers and the Inquiry Routing and Information System May 13, 2010 09-01968-150 ACRONYMS AND ABBREVIATIONS IRIS OFA&A

More information

Denver International Airport Emergency Preparedness Program Performance Audit

Denver International Airport Emergency Preparedness Program Performance Audit Denver International Airport Emergency Preparedness Program Performance Audit November 2015 Audit Services Division City and County of Denver Timothy M. O Brien, CPA Auditor The Auditor of the City and

More information

Audit of the Office of Emergency Management and Homeland Security: FEMA Urban Area Security Initiative Grant Management 2012 through 2014

Audit of the Office of Emergency Management and Homeland Security: FEMA Urban Area Security Initiative Grant Management 2012 through 2014 Audit of the Office of Emergency Management and Homeland Security: FEMA Urban Area Security Initiative Grant Management 2012 through 2014 MARTIN MATSON City Comptroller STACEY MAZMANIAN Audit Manager City

More information

FOLLOW-UP OF PERSONAL COMPUTER LICENSING REPORT NO. 08-04-107F. City of Albuquerque Office of Internal Audit and Investigations

FOLLOW-UP OF PERSONAL COMPUTER LICENSING REPORT NO. 08-04-107F. City of Albuquerque Office of Internal Audit and Investigations FOLLOW-UP OF PERSONAL COMPUTER LICENSING REPORT NO. City of Albuquerque Office of Internal Audit and Investigations City of Albuquerque Office of Internal Audit and Investigations P.O. BOX 1293 ALBUQUERQUE,

More information

CENTER. Prepared by Audit & Review Division

CENTER. Prepared by Audit & Review Division OPERATIONAL REVIEW OF THE RTA s TRAVEL INFORMATION CENTER Prepared by Audit & Review Division November 6, 2009 OPERATIONAL REVIEW OF THE RTA s TRAVEL INFORMATION CENTER Table of Contents Page Executive

More information

E Source Multichannel Metrics Survey 2012

E Source Multichannel Metrics Survey 2012 E Source Multichannel s Survey 2012 Instructions Fill in only the fields highlighted in blue; the yellow fields will automatically populate with the calculated s. In the "Number" column please use numerals;

More information

Department of Information Technology

Department of Information Technology Lines of Business LOB #132: END USER SERVICES Department of Information Technology Purpose The End User Services LOB in the Department of Information Technology is responsible for providing direct technical

More information

311 2014 Budget Hearing

311 2014 Budget Hearing 311 2014 Budget Hearing Department found on pages E38 - E42 in budget book Presentation to Ways and Means/Budget Committee August 29, 2013 1 311 Minneapolis 311-2014 7am 7pm $3,215,000 Weekend hours $

More information

Public Safety Vehicle Repair Audit

Public Safety Vehicle Repair Audit City of Austin AUDIT REPORT A Report to the Austin City Council Mayor Lee Leffingwell Public Safety Vehicle Repair Audit October 2013 Mayor Pro Tem Sheryl Cole Council Members Chris Riley Mike Martinez

More information

Lori Brooks, City Auditor Susan Edwards, Assistant City Auditor. E-Services Follow-Up Audit March 2015

Lori Brooks, City Auditor Susan Edwards, Assistant City Auditor. E-Services Follow-Up Audit March 2015 Lori Brooks, City Auditor Susan Edwards, Assistant City Auditor E-Services Follow-Up Audit March 2015 City Auditor s Office March 27, 2015 Honorable Mayor and Members of the City Council: I am pleased

More information

MICHIGAN AUDIT REPORT OFFICE OF THE AUDITOR GENERAL. Doug A. Ringler, C.P.A., C.I.A. AUDITOR GENERAL ENTERPRISE DATA WAREHOUSE

MICHIGAN AUDIT REPORT OFFICE OF THE AUDITOR GENERAL. Doug A. Ringler, C.P.A., C.I.A. AUDITOR GENERAL ENTERPRISE DATA WAREHOUSE MICHIGAN OFFICE OF THE AUDITOR GENERAL AUDIT REPORT PERFORMANCE AUDIT OF THE ENTERPRISE DATA WAREHOUSE DEPARTMENT OF TECHNOLOGY, MANAGEMENT, AND BUDGET August 2014 Doug A. Ringler, C.P.A., C.I.A. AUDITOR

More information

Career Service Authority Recruiting Process Performance Audit

Career Service Authority Recruiting Process Performance Audit Career Service Authority Recruiting Process Performance Audit August 2009 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor The Auditor of the City and

More information

Online Channel Office Project Alliance Management Office Report August 2011

Online Channel Office Project Alliance Management Office Report August 2011 Online Channel Office Project Alliance Management Office Report August 2011 1. Overview Service BC was formed in 2004, centralizing service delivery for government to citizens and businesses. The channels

More information

OFFICE OF THE AUDITOR

OFFICE OF THE AUDITOR OFFICE OF THE AUDITOR TIME ACCOUNTING SYSTEMS AND PRACTICES PERFORMANCE AUDIT APRIL 2008 Dennis J. Gallagher Auditor Dennis J. Gallagher Auditor City and County of Denver 201 West Colfax Ave., Dept. 705

More information

STATE OF NORTH CAROLINA

STATE OF NORTH CAROLINA STATE OF NORTH CAROLINA PERFORMANCE AUDIT INFORMATION TECHNOLOGY CONSOLIDATION JANUARY 2013 OFFICE OF THE STATE AUDITOR BETH A. WOOD, CPA STATE AUDITOR PERFORMANCE AUDIT INFORMATION TECHNOLOGY CONSOLIDATION

More information

Reducing Non-Emergency Calls to 9-1-1: An Assessment of Four Approaches to Handling Citizen Calls for Service

Reducing Non-Emergency Calls to 9-1-1: An Assessment of Four Approaches to Handling Citizen Calls for Service Reducing Non-Emergency Calls to 9--: An Assessment of Four Approaches to Handling Citizen Calls for Service I. Introduction The National Institute of Justice (NIJ), with support from the Office of Community

More information

Federal Bureau of Investigation s Integrity and Compliance Program

Federal Bureau of Investigation s Integrity and Compliance Program Evaluation and Inspection Division Federal Bureau of Investigation s Integrity and Compliance Program November 2011 I-2012-001 EXECUTIVE DIGEST In June 2007, the Federal Bureau of Investigation (FBI) established

More information

Business Service Network

Business Service Network Business Service Network Customer Service Audit Report July 9, 2014 Highlights Background The U.S. Postal Service s (BSN) of 300 employees supports nearly 23,000 large customers for service issues, information,

More information

AN AUDIT OF THE CUSTOMER SERVICE CALL CENTER S HANDLING OF SERVICE REQUESTS

AN AUDIT OF THE CUSTOMER SERVICE CALL CENTER S HANDLING OF SERVICE REQUESTS Office of the City Auditor Report to the City Council City of San José AN AUDIT OF THE CUSTOMER SERVICE CALL CENTER S HANDLING OF SERVICE REQUESTS The Call Center Handles Most Service Requests In A Timely

More information

A REPORT BY THE NEW YORK STATE OFFICE OF THE STATE COMPTROLLER

A REPORT BY THE NEW YORK STATE OFFICE OF THE STATE COMPTROLLER A REPORT BY THE NEW YORK STATE OFFICE OF THE STATE COMPTROLLER Alan G. Hevesi COMPTROLLER NEW YORK CITY SCHOOL CONSTRUCTION AUTHORITY IMPLEMENTATION OF THE ENTERPRISE RESOURCE PLANNING SYSTEM 2002-N-6

More information

Austin Fire Department Worker Safety Audit

Austin Fire Department Worker Safety Audit City of Austin AUDIT REPORT A Report to the Austin City Council Mayor Lee Leffingwell Mayor Pro Tem Sheryl Cole Austin Fire Department Worker Safety Audit Council Members Chris Riley Mike Martinez Kathie

More information

E nable the service delivery of our customers

E nable the service delivery of our customers Information Technology Department Vijay Sammeta, Acting Chief Information Officer M I S S I 0 N E nable the service delivery of our customers through the integration of city-wide technology r~soiirc~s

More information

STUDENT LOAN CORPORATION OF NEWFOUNDLAND AND LABRADOR ANNUAL REPORT 2006-07

STUDENT LOAN CORPORATION OF NEWFOUNDLAND AND LABRADOR ANNUAL REPORT 2006-07 STUDENT LOAN CORPORATION OF NEWFOUNDLAND AND LABRADOR ANNUAL REPORT 2006-07 TABLE OF CONTENTS CHAIRPERSON S MESSAGE... i 1.0 OVERVIEW... 1 1.1 MANDATE... 1 1.2 VISION... 1 1.3 MISSION... 2 1.4 LINES OF

More information

What is a Call Center?

What is a Call Center? A Call Center is a centralized office used for the purpose of receiving and transmitting a large volume of requests by telephone. A call center is operated by a company to administer incoming product support

More information

DC-Net Call Center Solutions. Date: 1/7/2011 Version: 2.3

DC-Net Call Center Solutions. Date: 1/7/2011 Version: 2.3 DC-Net Call Center Solutions Date: 1/7/2011 Version: 2.3 Table of Contents Table of Contents...2 1 Overview...3 1.1 Key Features...3 1.2 Key Benefits...4 2 Applications and Architecture...5 2.1 Call Management

More information

Office of Inspector General Evaluation of the Consumer Financial Protection Bureau s Consumer Response Unit

Office of Inspector General Evaluation of the Consumer Financial Protection Bureau s Consumer Response Unit Office of Inspector General Evaluation of the Consumer Financial Protection Bureau s Consumer Response Unit Consumer Financial Protection Bureau September 2012 September 28, 2012 MEMORANDUM TO: FROM: SUBJECT:

More information

TEXAS DEPARTMENT OF MOTOR VEHICLES. 2014 Report on Customer Service June 1, 2014 FY 2014 REPORT ON CUSTOMER SERVICE

TEXAS DEPARTMENT OF MOTOR VEHICLES. 2014 Report on Customer Service June 1, 2014 FY 2014 REPORT ON CUSTOMER SERVICE FY 2014 REPORT ON CUSTOMER SERVICE JUNE 1, 2014 1 TABLE OF CONTENTS INTRODUCTION. 3 STRATEGIC INITIATIVES 4 THIRD-PARTY REVIEW..5 ANALYSIS AND RESULTS. 6 2 Introduction The Texas Department of Motor Vehicles

More information

CUSTOMER GUIDE. Support Services

CUSTOMER GUIDE. Support Services CUSTOMER GUIDE Support Services Table of Contents Nexenta Support Overview... 4 Support Contract Levels... 4 Support terminology... 5 Support Services Provided... 6 Technical Account Manager (TAM)... 6

More information

TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION Improved Toll-Free Telephone Services Should Make It Easier for Taxpayers to Obtain Assistance During a Correspondence Audit August 17, 2012 Reference

More information

June 27, 2001. Ms. Maureen O. Helmer Chairperson Public Service Commission 3 Empire State Plaza Albany, New York 12223-1350 Re: Report 2001-F-7

June 27, 2001. Ms. Maureen O. Helmer Chairperson Public Service Commission 3 Empire State Plaza Albany, New York 12223-1350 Re: Report 2001-F-7 H. CARL McCALL STATE COMPTROLLER A.E. SMITH STATE OFFICE BUILDING ALBANY, NEW YORK 12236 STATE OF NEW YORK OFFICE OF THE STATE COMPTROLLER June 27, 2001 Ms. Maureen O. Helmer Chairperson Public Service

More information

Central Florida Expressway Authority

Central Florida Expressway Authority Central Florida Expressway Authority Back Office Customer Call Center Review May 2015 risks and controls in ways that this report did not and cannot anticipate. Table of Contents Section Page(s) Executive

More information

Program Title: State: Contact Person: Title: Telephone: Agency: e mail: Mailing Address:

Program Title: State: Contact Person: Title: Telephone: Agency: e mail: Mailing Address: Program Title: MassHR Project Contact Person: Paul Dietl Telephone: (617)878 9701 e mail: Paul.Dietl@MassMail.State.MA.US State: Title: Chief Human Resources Officer Agency: Human Recources Division Mailing

More information

O FFICE O F T HE C ITY A UDITOR. 14 04 City of Colorado Springs and Colorado Springs Utilities Claims Reserve and Workers Compensation Funds

O FFICE O F T HE C ITY A UDITOR. 14 04 City of Colorado Springs and Colorado Springs Utilities Claims Reserve and Workers Compensation Funds O FFICE O F T HE C ITY A UDITOR C OLORADO S PRINGS, C OLORADO 14 04 City of Colorado Springs and Colorado Springs Utilities Claims Reserve and Workers Compensation Funds February 2014 O FFICE O F T HE

More information

Request for Proposal: Website Development

Request for Proposal: Website Development Request for Proposal: Website Development REQUEST FOR PROPOSAL Muskegon Community College (MCC) is seeking proposals from qualified website developers to overhaul the design, structure and functions of

More information

1001 Lindsay Street Chattanooga, Tennessee 37402 (423) 643-6200 FAX: (423) 643-6204 E-MAIL: sewell_stan@chattanooga.gov

1001 Lindsay Street Chattanooga, Tennessee 37402 (423) 643-6200 FAX: (423) 643-6204 E-MAIL: sewell_stan@chattanooga.gov May 15, 2014 To: Mayor Andy Berke City Council Members Subject: Post-Audit Review of 311 Call Center (Report #12-06) Dear Mayor Berke and City Council Members: Attached is a summary report on the status

More information

We would like to extend our appreciation to the staff that assisted us throughout this audit. Attachment

We would like to extend our appreciation to the staff that assisted us throughout this audit. Attachment Date: June 25, 2014 To: Brenda S. Fischer, City Manager From: Candace MacLeod, City Auditor Subject: Audit of Glendale Fire Department s Payroll Process The City Auditor s Office has completed an audit

More information

October 21, 2004. Ms. Joan A. Cusack Chairwoman NYS Crime Victims Board 845 Central Avenue, Room 107 Albany, New York 12206-1588

October 21, 2004. Ms. Joan A. Cusack Chairwoman NYS Crime Victims Board 845 Central Avenue, Room 107 Albany, New York 12206-1588 ALAN G. HEVESI COMPTROLLER 110 STATE STREET ALBANY, NEW YORK 12236 STATE OF NEW YORK OFFICE OF THE STATE COMPTROLLER October 21, 2004 Ms. Joan A. Cusack Chairwoman NYS Crime Victims Board 845 Central Avenue,

More information

O FFICE O F T HE C ITY A UDITOR C OLORADO S PRINGS, C OLORADO. 12 15 Memorial Health System Workers Compensation Self Insurance Fund

O FFICE O F T HE C ITY A UDITOR C OLORADO S PRINGS, C OLORADO. 12 15 Memorial Health System Workers Compensation Self Insurance Fund O FFICE O F T HE C ITY A UDITOR C OLORADO S PRINGS, C OLORADO 12 15 Workers Compensation Self Insurance Fund September 2012 O FFICE O F T HE C ITY A UDITOR C OLORADO S PRINGS, C OLORADO 12 15 Workers Compensation

More information

Office of the Auditor General Performance Audit Report. Statewide UNIX Security Controls Department of Technology, Management, and Budget

Office of the Auditor General Performance Audit Report. Statewide UNIX Security Controls Department of Technology, Management, and Budget Office of the Auditor General Performance Audit Report Statewide UNIX Security Controls Department of Technology, Management, and Budget December 2015 State of Michigan Auditor General Doug A. Ringler,

More information