Denver 311 Follow up Report

Transcription

1 Denver 311 Follow up Report December 2014 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor

2 The Auditor of the City and County of Denver is independently elected by the citizens of Denver. He is responsible for examining and evaluating the operations of City agencies for the purpose of ensuring the proper and efficient use of City resources and providing other audit services and information to City Council, the Mayor and the public to improve all aspects of Denver s government. He also chairs the City s Audit Committee. The Audit Committee is chaired by the Auditor and consists of seven members. The Audit Committee assists the Auditor in his oversight responsibilities of the integrity of the City s finances and operations, including the integrity of the City s financial statements. The Audit Committee is structured in a manner that ensures the independent oversight of City operations, thereby enhancing citizen confidence and avoiding any appearance of a conflict of interest. Audit Committee Dennis Gallagher, Chair Maurice Goodgaine Leslie Mitchell Rudolfo Payan Robert Bishop Jeffrey Hart Timothy O Brien, Vice Chair Audit Staff Audrey Donovan, Deputy Director, CIA, CRMA, CGAP Marcus Garrett, Audit Supervisor, CIA, CRMA, CGAP Rudy Lopez, Lead Auditor You can obtain copies of this report by contacting us at: Office of the Auditor 201 West Colfax Avenue, Department 705 Denver CO, (720) Fax (720) Or download and view an electronic copy by visiting our website at:

3 City and County of Denver 201 West Colfax Avenue, Department 705 Denver, Colorado FAX Dennis J. Gallagher Auditor December 23, 2014 Mr. Michael Major, Director Denver 311 City and County of Denver Re: Audit Follow Up Report Dear Mr. Major, In keeping with professional auditing standards and the Audit Services Division s policy, as authorized by D.R.M.C , our Division has a responsibility to monitor and follow up on audit recommendations to ensure audit findings are being addressed and to aid us in planning future audits. This report is to inform you that we have completed our follow up effort for the Denver 311 audit issued August 16, Our review determined that the Denver 311 contact center has adequately implemented some of the recommendations made in the audit report. Despite Denver 311 s efforts, auditors determined that the risk associated with the audit team s initial findings has not been fully mitigated. As a result, the Division may revisit these risk areas in future audits to ensure appropriate corrective action is taken. For your reference, this report includes a Highlights page that provides background and summary information on the original audit and the completed follow up effort. Following the Highlights page is a detailed implementation status update for each recommendation. Although we did not update the status of recommendations that the Division disagreed with, these recommendations are included in the status update section as a reference. This concludes audit follow up work related to this audit. I would like to express our sincere appreciation to you and to Division personnel who assisted us throughout the audit and follow up process. If you have any questions, please feel free to contact me at or Marcus Garrett, Internal Audit Supervisor, at Sincerely, KRM/rl Kip Memmott, MA, CGAP, CRMA Director of Audit Services cc: Honorable Michael Hancock, Mayor To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

4 Honorable Members of City Council Members of Audit Committee Ms. Cary Kennedy, Deputy Mayor, Chief Financial Officer Ms. Janice Sinden, Chief of Staff Mr. David P. Edinger, Chief Performance Officer Ms. Beth Machann, Controller Mr. Scott Martinez, City Attorney Ms. Janna Young, City Council Executive Staff Director Mr. L. Michael Henry, Staff Director, Board of Ethics Mr. Frank Daidone, Chief Informational Officer To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

5 City and County of Denver Office of the Auditor Audit Services Division REPORT HIGHLIGHTS Denver 311 Follow up Report: December 2014 The Denver 311 Division has implemented some recommendations but is still in the process of addressing other recommendations made in the August 2012 audit report. Background In July 2006, the City and County of Denver launched the 311 Customer Service Center as a comprehensive one stop resource for anyone seeking information about nonemergency City services. Denver 311 serves both external and internal customers by providing a quick source of information or call routing for external callers, which assists internal City agencies with their customer service efforts. Denver 311 does not address emergency calls, nor is 311 associated with the Denver 911 call center, which addresses emergency calls as well as non emergency calls that require a response from police, fire, or emergency medical services. Purpose The purpose of the audit was to: Determine the effectiveness of Denver 311 at addressing citizen and internal City agency needs; and Assess the technology used by Denver 311 and internal City agencies. Highlights from Original Audit Finding: Denver 311 Receives High Satisfaction Rates from External Users but Should Work to Sustain These Satisfaction Levels and Improve its Partnership with Internal Users A statistically valid customer service survey commissioned by the Auditor s Office found that 88 percent of Denver 311 users are somewhat satisfied or completely satisfied regarding their overall experience with 311. Specific areas of high satisfaction included timeliness of call answering, resolution of issues, and overall knowledge demonstrated by call center agents. A review of internal performance metrics data for key 311 activities, such as abandoned calls and the time to answer calls, revealed that performance measures are trending down, which, if not addressed, will eventually negatively affect the satisfaction rates of Denver 311 users. Furthermore, it was identified that additional marketing efforts could increase awareness of alternative ways to contact 311, and 311 can improve its partnership with internal City agencies, especially with regard to the Customer Relationship Management (CRM) system. Finally, City agencies can provide additional information to CRM to help ensure that 311 reports are as useful as possible. Findings at Follow up Based on audit follow up work, we determined that five recommendations were implemented (1.1, 1.3, 1.5, 1.9, and 1.10) and four are still in progress (1.4, 1.6, 1.7, and 1.11), which are classified as Agree/Not in accordance with the Audit Service Division s policy and for reporting purposes. The 311 Division originally disagreed with two recommendations (1.2 and 1.8), which did not require additional audit follow up work. For a complete copy of this report, visit Audit Contact Person: Marcus Garrett marcus.garrett@denvergov.org

6

7 Recommendations: Status of Implementation Recommendation Auditee Action Status Finding: Denver 311 Receives High Satisfaction Rates from External Users but Should Work to Sustain These Satisfaction Levels and Improve its Partnership with Internal Users 1.1 To maintain the high levels of customer service satisfaction, the Director of Denver 311 should gather additional information from 311 users to better understand the specific reasons users are highly satisfied with Denver 311 services and to identify any new areas that could enhance satisfaction. This follow up survey may be delivered by a professional third party polling company, by an end of contact survey administered through the 311 system, or other cost effective methods as determined by the Director of Denver Denver 311 should conduct a review of first call resolution rates to determine which agencies may be the best candidates to receive a direct transfer of all relevant calls. In 2013 and 2014 the Denver contact center conducted their own customer service survey. A comparison of the 2012 and 2014 citizen surveys shows an increase in the satisfaction rates in all four categories of courtesy (81 to 85 percent), overall impression (76 to 77 percent), knowledge (73 to 77 percent) and responsiveness (74 to 76 percent). Denver 311 surveys for 2013 and 2014 year to date show that a total of 239 Denver 311 users completed the survey with 64 percent reporting satisfied, very satisfied, or neutral. Denver 311 found that for a majority of the remaining 36 percent that were dissatisfied, their personal experiences with Denver 311 involved a lack of response from the City agency for a service request. Denver 311 believes the positive satisfaction rates are due to the way 311 agents responds to callers. Denver 311 disagreed with this recommendation in the original audit report and therefore did not pursue implementation. Disagree Page 1 Office of the Auditor

8 Recommendations: Status of Implementation Recommendation Auditee Action Status 1.3 Denver 311 should identify additional resources for enhanced marketing efforts. These resources could include additional budget allocations or additional efforts at creative, low cost linkages with community groups. Since an increase in personnel for 311 is unlikely in the current economic environment, Denver 311 should focus its marketing efforts on increasing awareness of the alternative methods for contacting 311, such as , the City s website, Facebook, and Twitter. 1.4 The Director of Denver 311 should assess the feasibility, including the cost, accuracy, and reliability, of acquiring technology that automatically routes citizen messages found in s, Facebook, and Twitter to the appropriate City agency. As part of Denver 311 s Customer Experience initiative, the City and County of Denver website and the 311 mobile applications are being redesigned to encourage more online inquiries. All marketing efforts for these initiatives are on hold until they are ready for launch. Funding remains available for the remainder of 2014, and the 2015 budget also has funds dedicated to marketing activities. Print collateral continues to be the main focus used of current outreach activity. Denver 311 determined that this recommendation could be implemented by acquiring a new application for the agency s Customer Relationship Management (CRM) system. However, due to a long request for proposal (RFP) and product demonstration process of available CRM applications, the new launch date for a new CRM system is now the 2nd Quarter of A statement of work, which explains the work to be done on a particular project, is currently being reviewed by a committed of CRM users and the City Attorney Office. Contract negotiations will follow and development work will begin during the 4th Quarter of An enhanced key word search function and automatic routing will be included with the new CRM. Agree/Not City and County of Denver Page 2

9 Recommendations: Status of Implementation Recommendation Auditee Action Status 1.5 The Director of Denver 311 should inform and periodically remind agencies that custom CRM reports may be requested from Technology Services. The Director should also work with agencies to identify data from CRM that can be used as an effective tool to enhance agency management. 1.6 To fully realize the benefits of CRM, the Chief Information Officer should assess the implementation costs of upgrading the technological capabilities of CRM or switching to a more compatible software system based on the needs of Denver 311 and City agencies. Denver 311 specialists continue to focus on assisting all partner agencies with not only report availability but also web and CRM content. A new CRM application will allow for the creation of new reports tailored to the needs of the individual agency or department rather than the current "one size fits all" report approach. The 311 open data website now has more than 3.5 million cases with more than 63 million bits of data that can be accessed by the public and internal users. Members of the media making Colorado Open Records Act (CORA) requests can now be referred to this site for information, which has cut the number of media CORA requests to zero for YTD Denver 311 is now recording all calls, and requests for these recordings have also increased. Due to a long RFP and product demonstration process of available CRM applications, the launch date for a new CRM has been established for the 2nd Quarter of A statement of work is currently being reviewed. Contract negotiations will follow and development work will begin during the 4th Quarter of An enhanced key word search function and automatic routing will be included with the new CRM. (See Recommendation 1.4 above.) Agree/ Not Page 3 Office of the Auditor

10 Recommendations: Status of Implementation Recommendation Auditee Action Status 1.7 The Director of Denver 311 should coordinate with City agencies and Technology Services to determine how the historical data captured within partner agencies legacy systems can interface with CRM to reduce duplication, centralize information, and improve reporting functionality. 1.8 The Director of Denver 311 should identify ways to improve the process for callers who need information from or about agencies that do not interface with CRM, such as forwarding calls directly to the designated agency in some cases. 1.9 The Director of Denver 311 should assess the feasibility and cost of adding an upfront announcer, which is an automated message informing callers of various options prior to being placed in the hold queue. However, the Director of 311 should base any technological changes on follow up survey data obtained from users to ensure that 311 s high customer satisfaction ratings among citizens are not negatively impacted. The implementation of this recommendation is pending the launch of the new CRM system in Denver 311 disagreed with this recommendation in the original audit report and therefore did not pursue implementation. A queue announcer was added in June 2013 as part of an advanced routing tool called skill basedrouting. The caller now hears an upfront message with expected wait times. Agree/Not Disagree City and County of Denver Page 4

11 Recommendations: Status of Implementation Recommendation Auditee Action Status 1.10 To emphasize to agencies the importance of providing up to date CRM data, the Chief Information Officer should coordinate with the Mayor s Office to ensure that City agency representatives are updating CRM as appropriate The Director of Denver 311 should identify a different type of document, such as a Memorandum of Understanding, which emphasizes the mutual goals of 311 and a City agency and a commitment to work collaboratively to reach the stated goals. Denver 311 reported that CRM case updates by partner agencies have increased in 2014 with agencies updating and closing cases on a more regular basis. The anticipated new CRM application will not only make it easier for agency users to update CRM cases but also possibly encourage additional agencies to begin to use the CRM system to track activity, rather than continuing to use the old legacy systems. The 311 specialists monitor the monthly agency reports to determine which agencies need assistance to identify and improve operations. Memorandums of Understanding will be entered in conjunction with the launch of the new CRM system in Agree/Not Page 5 Office of the Auditor

12 Conclusion While the Division has implemented five of the recommendations made in the Denver 311 audit report, the four others that the agency agreed to implement have yet to be fully implemented. Although we recognize that the Division has made a good faith effort to address the recommendations either fully or partially, auditors determined that the risk associated with the audit team s initial findings have not been fully mitigated. It should be noted that the four recommendations listed as Agree/Not are scheduled to be completed in 2015, pending the launch of the new CRM system. The Audit Services Division may revisit these risk areas in future audits to ensure that appropriate corrective action has been taken. On behalf of the citizens of the City and County of Denver, we thank staff and leadership from the Denver 311 Division for their cooperation during our follow up effort and their dedication to public service. City and County of Denver Page 6