Career Service Authority Recruiting Process Performance Audit

Career Service Authority Recruiting Process Performance Audit August 2009 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor

The Auditor of the City and County of Denver is independently elected by the citizens of Denver. He is responsible for examining and evaluating the operations of City agencies for the purpose of ensuring the proper and efficient use of City resources and providing other audit services and information to City Council, the Mayor and the public to improve all aspects of Denver s government. He also chairs the City s Audit Committee and oversees the City s Comprehensive Annual Financial Report (CAFR) The Audit Committee is chaired by the Auditor and consists of seven members. The Audit Committee assists the Auditor in his oversight responsibilities of the integrity of the City s finances and operations, including the integrity of the City s financial statements. The Audit Committee is structured in a manner that ensures the independent oversight of City operations, thereby enhancing citizen confidence and avoiding any appearance of a conflict of interest. Audit Committee Dennis Gallagher Robert Haddock Charles Husted Timothy O Brien Maurice Goodgaine Jeffrey Hart Bonney Lopez Audit Staff John Carlson, JD, CIA, CICA, Deputy Director Nancy Howe, Internal Audit Supervisor Jacob Claeys, CICA, Lead Internal Auditor Marcus Garrett, CIA, CGAP, Lead Internal Auditor Mary Mutchler, CICA, Senior Internal Auditor Travis Henline, Senior Internal Auditor Emily Gibson, Staff Internal Auditor You can obtain free copies of this report by contacting us at: Office of the Auditor 201 W. Colfax Avenue, Dept. 705 Denver CO, 80202 (720) 913-5000 Fax (720) 913-5026 Or view an electronic copy by visiting our website at: www.denvergov.org/auditor

City and County of Denver Dennis J. Gallagher Auditor 201 West Colfax Ave., Dept. 705 Denver, Colorado 80202 720-913-5000 FAX 720-913-5247 www.denvergov.org/auditor August 20, 2009 Jeff Dolan, Executive Director Career Service Authority City and County of Denver Dear Mr. Dolan: Attached is the Auditor s Office Audit Services Division s report of their audit of the Career Service Authority Recruiting Process for the year ended December 31, 2008. The purpose of the audit was to evaluate the operational efficiency and effectiveness of the City s recruiting process and to determine whether internal controls in place were adequate under the circumstances. Audit work identified opportunities for improvement in all four aspects of the recruiting function examined, including strategic planning, the timeliness of the recruiting function, consistency in recruiting practices, and oversight. The weaknesses identified in CSA s recruiting process inhibit the efficiency and effectiveness of the process and limit accountability. If you have any questions, please call Kip Memmott, Director of Audit Services, at 720-913-5029. Sincerely, Dennis J. Gallagher Auditor DJG/mkm cc: Honorable John Hickenlooper, Mayor Honorable Members of City Council Members of Audit Committee Ms. Kelly Brough, Chief of Staff Mr. Claude Pumilla, Chief Financial Officer Mr. David T. Roberts, Chief Services Officer Mr. David Fine, City Attorney Mr. L.Michael Henry, Staff Director, Board of Ethics Ms. Lauri Dannemiller, City Council Executive Staff Director Ms. Beth Machann, Controller To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

City and County of Denver Dennis J. Gallagher Auditor 201 West Colfax Ave., Dept. 705 Denver, Colorado 80202 720-913-5000, FAX 720-913-5247 www.denvergov.org/auditor AUDITOR S REPORT We have completed an audit of the Career Service Authority s Recruiting Process for the period January 1, 2008 through December 31, 2008. The purpose of the audit was to examine and assess the City s recruiting processes to identify possible inefficiencies and opportunities for improvement. Audit work focused primarily on strategic planning, the timeliness of the recruiting function, consistency in recruiting practices, and adequate oversight as key performance indicators. This performance audit is authorized pursuant to the City and County of Denver Charter, Article V, Part 2, Section 1, General Powers and Duties of Auditor, and was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Audit work identified opportunities to improve all four aspects of the recruiting function examined. Specifically: CSA currently lacks both a clear definition of what the merit system seeks to accomplish and a strategic plan that establishes goals and helps the agency adjust to future developments; CSA s recruiting and hiring process appears to take longer than other jurisdictions; CSA personnel are not consistent in how they document or perform their recruiting activities and weak controls such as poor security over tests create situations that potentially threaten fair and open competition for City positions; and Accountability is hampered by the lack of sound performance measures, limited data, and ineffective tools for both evaluating employees performance and reviewing the timeliness and quality of recruiting efforts. We extend our appreciation to the CSA personnel who assisted and cooperated with us during the audit. Audit Services Division Kip Memmott, MA, CGAP, CICA Director of Audit Services To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

TABLE OF CONTENTS EXECUTIVE SUMMARY 1 Opportunities Exist to Improve the Recruiting Function 1 INTRODUCTION & BACKGROUND 4 Overview of Denver s Career Service Authority 4 Overview of CSA s Recruiting Process 5 CSA Recruiting Uses a Variety of Assessment Methods 8 Equal Employment Opportunity Laws Applicable to the City s Recruiting Process 9 CSA Has Recently Experienced Organizational Change 10 SCOPE 10 OBJECTIVE 10 METHODOLOGY 11 FINDING 12 Opportunities Exist to Improve the Recruiting Function 12 Lack of a Clearly Defined Merit System and a Weak Strategic Plan Hinder CSA s Effectiveness 12 The City s Recruiting and Hiring Process Does Not Produce Timely Results 15 Inconsistencies and Problems Executing CSA s Processes Hinder Objectivity in the City s Recruiting Efforts 17 Accountability Is Hampered by Limited Management Information and Poor Oversight Mechanisms 22 RECOMMENDATIONS 28 APPENDIX A 32 City Positions Not in the Career Service (Charter 9.1.1 E) 32 AGENCY RESPONSE 33

EXECUTIVE SUMMARY Opportunities Exist to Improve the Recruiting Function Audit work identified opportunities to improve all four aspects of the recruiting function examined strategic planning, the timeliness of the recruiting function, consistency in recruiting practices, and oversight. The weaknesses identified in CSA s recruiting process inhibit the efficiency and effectiveness of the process and limit accountability. In summary: CSA currently lacks both a clear definition of what the merit system seeks to accomplish and a strategic plan that establishes goals and helps the agency adjust to future developments; CSA s recruiting and hiring process appears to take longer than other jurisdictions; CSA personnel are not consistent in how they document or perform their recruiting activities, and weak controls such as poor security over tests create situations that potentially threaten fair and open competition for City positions; and Accountability is hampered by the lack of sound performance measures, limited data, and ineffective tools for both evaluating employees performance and reviewing the timeliness and quality of recruiting efforts. Specifically, audit work identified the following areas for improvement. Lack of a Clearly Defined Merit System and a Weak Strategic Plan Hinder CSA s Effectiveness Audit work determined that the components of the City s merit system are not specifically defined. As a result, key stakeholders do not have a consistent understanding of what the merit system should encompass. This lack of clarity has implications for how the recruiting process is carried out. Without a clear understanding and definition of what a merit system means, CSA is limited in its ability to determine whether current recruiting processes reflect merit principles, whether some processes such as testing are even necessary, and whether a merit system is effective or necessary. In addition, audit work found CSA does not have a comprehensive strategic plan. Although CSA does have information typically contained in a strategic plan, it is located in three separate documents. This limits the organization s ability to use the information to plan effectively for the future. The City s Recruiting and Hiring Process Does Not Produce Timely Results A key issue for a recruiting function is whether it succeeds in placing new employees in positions as expeditiously as possible. The most qualified candidates are generally desired by many employers, so as the length of time to hire increases, the opportunity to hire the P a g e 1 Office of the Auditor

most qualified decreases. Audit work found that, relative to other jurisdictions, CSA s recruiting process takes longer. A lengthy recruiting and hiring process can negatively affect overall customer satisfaction. There are steps CSA can take to improve the overall efficiency of the process. For example, an audit sample found the average time to fill a position in 2008 was 79.9 days. Compared against two industry standards, the City s process is slower. CSA is primarily a customer service agency with a variety of customers, including hiring agencies, potential job candidates, as well as current City employees. While most agencies reported they are fairly satisfied with the relationship and service they receive from CSA, several agencies also felt the recruiting process is too complicated or overly stringent. Audit work identified several steps that could be taken to shorten the review process to include: Eliminating or compressing steps; Identifying ways to increase efficiency by the hiring agency; and Shortening the process for certain types of positions. Inconsistencies and Problems Executing CSA s Processes Hinder Objectivity in the City s Recruiting Efforts Audit work found CSA lacks policies and procedures to help prevent discrepancies and potential problems in the recruiting process. Detailed examination of the recruiting process showed numerous inconsistencies in how CSA personnel conducted and carried out various steps of the process. In addition, gaps and lapses in some procedures create situations in which fair and open competition for positions is potentially affected. Rule 3 serves as the primary policies and procedures for Recruiting. Audit work showed that while Rule 3 does provide some guidance, it is inadequate on its own because it is broad and vague. Recruiting has no additional written policies and procedures, and the result is incomplete or inconsistent recruiting practices. Specific examples include: Key recruiting steps are not documented by Recruiters; Recruiting does not have procedures for deciding when to eliminate recruiting steps; and Recruiters are inconsistent in utilizing special and preferred qualifications. Additionally, audit work revealed significant weaknesses in the physical security of test books. Auditors conducted an inventory of the test books stored in CSA s test vault and identified three discrepancies. Accountability Is Hampered by Limited Management Information and Poor Oversight Mechanisms Audit work determined CSA has little ability to monitor and report on its performance. It is generally not collecting the kinds of measures needed to accomplish this task effectively. Further, in several instances when it did report information about its performance in order for the Board to make key decisions about the recruiting process, the information was City and County of Denver P a g e 2

inaccurate. Likewise, key oversight mechanisms, including performance evaluations of employees and quality control reviews of the agency s work are weak. Specifically, audit work determined there are two ways in which the current performance measurement system limits the ability to assess the efficiency and effectiveness of the recruiting function. First, monitoring and collection of performance information is inconsistent. Second, the performance indicators monitored yield little information about efficiency and effectiveness, and the indicators do not align with best practices. The limited focus on such indicators reduces CSA s and the Board s ability to analyze the efficiency and effectiveness of recruiting activities. For example, the cost of recruiting is unknown. CSA does not know how much time Recruiters spend on each recruitment or the total cost to hire an employee. Weaknesses in current performance measures impede CSA s ability to determine whether the City s resources are being wisely and effectively spent. The limited information also reduces CSA s ability to plan strategically for the future. Furthermore, audit work identified several instances of inaccurate reporting. These involved performance information in various CSA reports and the content of presentations to the CSA Board. Reporting must be accurate in order for key stakeholders to hold Recruiting accountable for its performance. Inaccurate information diminishes the ability of stakeholders to evaluate the overall effectiveness of Recruiting. Additionally, accurate information is necessary to determine whether CSA is adhering to the merit system. Audit work found existing quality controls over the recruiting process are vague and largely undocumented. Recruiting has developed a general reporting structure for quality review of Recruiters work, but no way to monitor whether reviews occur. The lack of a well defined and documented quality control process limits CSA s knowledge of performance, their ability to hold individual personnel accountable, and increases the risk of inaccuracies. CSA was specifically established as an independent agency to limit inappropriate political influence, but because of this independence, the CSA Board should actively provide oversight and hold CSA accountable for carrying out the agency s mission and accomplishing its goals. Action is needed on multiple fronts to strengthen the City s recruiting function. In many respects, these issues are interrelated. P a g e 3 Office of the Auditor

INTRODUCTION & BACKGROUND & BACKGROUND Overview of Denver s Career Service Authority The City and County of Denver is one of the largest employers in the Denver metropolitan area. About 68% of the City and County s more than 13,000 employees are part of what is called the Career Service. These employees include general positions such as administrative personnel, laborers, and professionals. 1 Filling these positions with qualified applicants is a sizeable task. In fiscal year 2008, the City and County had 1,769 new hires for positions within the Career Service. The Career Service Authority (CSA) is the primary agency that provides human resource services for Career Service employees. CSA has over 80 authorized positions, including Recruiters, Training Specialists, Classification/Compensation Specialists, and Benefit Specialists. The agency s duties include: Recruiting, examining, and certifying applicants for employment and promotion; Administering the personnel rules that govern appointments, promotions, demotions, transfers, and layoffs; Providing Citywide training; Setting compensation and classification details for City employees; Administering employee benefits; and Managing employee grievances and other employee relations. Established under the City Charter in 1954, CSA's overall mission is to uphold and administer a merit system. Among other things, a merit system allows job applicants to compete for and obtain employment based on their knowledge, skills, and abilities. 2 Merit systems have been utilized by government for decades and were intended to combat the "spoils" system in which government jobs were openly bought and sold. The City Charter establishes CSA as an independent agency that is, an agency removed from the Mayor s direct control. CSA s operations are overseen by a governing board, called the Career Service Authority Board (the Board). The Board is comprised of 1 Examples of positions that are not part of the Career Service include elected officials and uniformed staff of the police and fire departments. Appendix A lists the positions not included in the Career Service. 2 The federal government has a detailed framework for achieving fair and open human resource policies and practices called the Merit System Principles and Prohibited Personnel Practices. The City and County of Denver is not required to abide by these practices. City and County of Denver P a g e 4

five members who are appointed by the Mayor and confirmed by City Council for fiveyear staggered terms. The Board s duties include: Overseeing the Career Service Board Personnel Rules (Rules) that govern employment practices within the Career Service. These Rules address hiring, pay, classification, transfers, dispute resolution, promotions, training, and other personnel matters. The Board may revise or rescind proposed Rule changes at any time provided the process is performed in accordance with the rule-making process specified in the Rules; Overseeing CSA, including appointing CSA s Executive Director; and Overseeing the Career Service Hearing Office, and serving as a quasi-judicial body to decide appeals of decisions of the Career Service Hearings Officers. 3 Overview of CSA s Recruiting Process The Recruiting Section (Recruiting) within CSA s Workforce Management Division is responsible for providing recruitment services for Career Service positions. Rule 3 governs the process for recruiting potential employees. Candidate selection is a key component of the recruiting and hiring process and is instrumental in helping to ensure the City hires the most qualified candidates and upholds principles of a merit system. City agencies play an important role in the hiring process; however, CSA will perform some agency functions upon request. The general process to fill open positions is depicted in Figure 1. 4 As the figure shows, CSA is only one of the City agencies involved in the process. In general, CSA s focus is on advertising the position and establishing a list of the most highly qualified candidates. The hiring agency then decides which of these candidates to hire. The process can be separated into three main phases as follows: Planning and Position Announcement. When a hiring agency has a vacancy, it typically contacts the City s Budget and Management Office to verify available funding. If the Budget and Management Office determines funding is available, it completes an approval to fill form and returns it to the hiring agency, which then contacts CSA Recruiting. Then recruiting preparation begins. This preparation includes a CSA Recruiter working with the hiring agency to determine the recruitment strategy, the methodology for testing applicants, and the posting (job announcement) plan. When complete, Recruiting posts the open position to the City website and any other agreed-upon external websites. Applicant Screening. As individuals apply for an open position, Recruiting reviews every application received and creates what is called an eligible list. This list 3 City and County of Denver Career Service Board Personnel Rules, Rule 2-11A: Duties and Organization of the Board, Effective December 23, 2005, http://denvergov.org/humanresources/rulesandpolicies/tabid/432517/default.aspx (accessed March 20, 2009). 4 The process described here is used for most recruitments, but not all. Other approaches taken by CSA are discussed elsewhere in this report. P a g e 5 Office of the Auditor

includes all applicants who meet the position s minimum qualifications. For these applicants, Recruiting administers an assessment that is, a test designed to further determine applicants relative abilities and ranks applicants in order of their scores. Using these scores, Recruiting creates what is called a certified list (also known as a referred list or referral list), which comprises the top 20 ranked candidates. It then transmits this certified list to the agency. Interview and Selection. The hiring agency determines which candidates from the certified list it wishes to interview. CSA Rules require agencies to interview a minimum of three candidates from the certified list. The hiring agency negotiates salary, makes a verbal offer, conducts background checks, and sends an offer letter. CSA may partner with the hiring agency or perform these tasks as well. The CSA Recruiter verifies that the hiring agency complied with City policies, such as interviewing at least three candidates from the certified list. After the selected candidate begins work, the City Controller s Office completes I-9 paperwork. City and County of Denver P a g e 6

Figure 1: City Recruiting Process P a g e 7 Office of the Auditor

CSA Recruiting Uses a Variety of Assessment Methods CSA Recruiting uses a variety of assessment methods, both in screening applicants qualifications and in developing and administering examinations. When an open position is posted, the position description lists the required minimum qualifications, such as the minimum level of education or relevant experience required. These minimum qualifications are set for each job class by CSA s Classifications Division. Neither Recruiting nor hiring agencies may amend these qualifications under any circumstances. At the same time, agencies may include two additional types of qualifications, as follows: Special qualifications. Rule 3 allows agencies to require additional qualifications beyond the minimum if the skills are necessary to perform the job. For example, an agency seeking to recruit an Administrative Support Assistant may require an additional qualification to be bilingual if, as a regular part of that job s duties, the hired person will be interacting with non-english speakers. CSA refers to these as special qualifications and Recruiters may screen applicants for special qualifications in the same manner they screen for minimum qualifications. Preferred qualifications. These qualifications are not covered in Rule 3 or in any other written CSA policy. They are qualifications that a hiring agency would prefer the ideal candidate to have, but are not necessary to perform the position. For example, the minimum qualifications required for a Senior Accountant are a Bachelor s Degree and two years of accounting experience, but a particular agency may prefer their Senior Accountant have government accounting experience. This could be stated on the job announcement as a preferred qualification. There are no written restrictions regarding when an agency may request preferred qualifications in a recruitment. Additionally, there are no written instructions on how these are to be used in recruiting. CSA administers an examination for every position in the Career Service. These examinations are developed by CSA Recruiters and Test Specialists in consultation with the hiring agency. According to CSA s Test Operations Manual, the following are the most common tests given to Career Service job applicants: Written test questions. These are test booklets typically administered to high volume recruitments or at agency request. Questions originate from the County of San Bernardino, California and are used widely by personnel agencies around the country. Referred to as the Western Region Item Bank, this database contains more than 54,000 test questions covering 375 subject matter categories. Supplemental questions. These are job-specific open-ended questions that are graded using established rating criteria. They are typically part of the job posting, allowing CSA to obtain applicant answers with the application. Generally, this type of exam is developed for professional-level job classifications. City and County of Denver P a g e 8

Performance test. These tests simulate an actual job requirement and are typically administered for Equipment Operator and Heavy Equipment Operator candidates. These examinations are expensive to administer and, therefore, tend to be included in a limited number of exam plans. Skills test. CSA also has programs available to conduct skills based testing on computer applications, such as Microsoft Word, Excel and Access. Other test options may be developed between the Test Specialists and hiring agency based on need. Equal Employment Opportunity Laws Applicable to the City s Recruiting Process As an employer, the City must abide by federal Equal Employment Opportunity (EEO) laws. These laws and regulations include: Title VII of the Civil Rights Act of 1964 - Prohibits discrimination and other unlawful employment practices based on race, color, religion, sex, or national origin; The Americans with Disabilities Act (ADA) - Prohibits discrimination on the basis of disability; The Age Discrimination in Employment Act (ADEA) - Prohibits age discrimination in employment; and The Equal Pay Act (EPA) - Prohibits sex-based wage discrimination as part of the Fair Labor Standards Act. While CSA personnel take many steps to comply with Equal Employment Opportunity Commission (EEOC) regulations, the concept of adverse impact is directly related to issues discussed in this report. Specifically, the EEOC has Uniform Guidelines that incorporate a single set of principles which are designed to assist employers in eliminating employment practices which discriminate on the grounds of race, color, religion, sex, and national origin. These guidelines provide a framework for determining the proper use of tests and other selection procedures. This framework particularly addresses adverse impact which is defined as a substantially different rate of selection in hiring, promotion, or other employment decision which works to the disadvantage of members of a race, sex, or ethnic group. 5 5 Equal Employment Opportunity Commission (EEOC), Uniform Guidelines on Employee Selection Procedures (1978), 43 FR 38295 (August 25, 1978), 29 CFR Part 1607, Section 16B, http://www.uniformguidelines.com/uniformguidelines.html (accessed April 7, 2009). P a g e 9 Office of the Auditor

CSA Has Recently Experienced Organizational Change A new Executive Director joined CSA at the end of 2007 and initiated an extensive reorganization beginning in 2008. This greatly impacted the Recruiting Section as well as CSA s other divisions. Recruiters responsibilities were expanded and most Recruiting support staff was eliminated. Further, CSA hired a new Recruiting Supervisor in mid-2008. A customer service survey completed as part of audit work indicated that since these major changes have taken place, agencies have seen improvement. This audit seeks to identify areas for additional improvement and add to the progress already made by CSA leadership. In 2006, CSA Recruiting began phasing in NEOGOV, an online recruiting management system. Recruiters now use NEOGOV to recruit for all agencies in the City. In 2008, CSA launched NEOGOV Online Hiring Center, an additional software package that allows the hiring agencies to utilize NEOGOV in their part of the recruiting process. SCOPE The scope of the audit was to examine CSA s recruiting processes and practices. Consequently, it focused primarily on the Recruiting Section within CSA's Workforce Management Division. Audit work also included evaluating the CSA Board and the duration of City agencies portion of the hiring process. The audit period extended from January 1, 2008 through December 31, 2008. OBJECTIVE Our objective was to assess the following aspects of the CSA recruiting process: Does Recruiting have clearly articulated goals and a suitable plan for accomplishing them? Is the process producing timely results in terms of screening applicants and filling open positions? Are components of the process consistently implemented, so that the process is objective and fair? Does management have accurate, useful, and timely information that can be used to monitor and improve the process, as well as effective controls for ensuring accountability? City and County of Denver P a g e 10

METHODOLOGY Evidence gathering and analysis techniques used to meet audit objectives included, but were not limited to: Interviewing CSA personnel and personnel with human resources responsibilities at other City agencies; Attending a NEOGOV training course; Attending CSA Board meetings; Documenting relevant CSA policies and practices; Analyzing the City Charter, the Denver Revised Municipal Code, CSA Rules, and federal requirements related to recruiting and hiring; Tracking new job postings for 45 days including both business days and nonworking days; Reviewing job postings, position qualifications, job testing, pre-employment screenings, and CSA employee documentation practices; Benchmarking against other municipalities regarding their recruiting and hiring practices; 6 Attending applicant examinations conducted by CSA; Testing various recruitment-related practices; Collecting and analyzing NEOGOV data; and Evaluating NEOGOV usage and capabilities. 6 Auditors selected the following benchmark jurisdictions due to the similarity of their population and total staff size to Denver s: Long Beach, CA; Portland, OR; Fulton County, GA; and Oklahoma City, OK. Some of these jurisdictions also utilize NEOGOV. P a g e 11 Office of the Auditor

FINDING Opportunities Exist to Improve the Recruiting Function Audit work identified opportunities to improve all four aspects of the recruiting function examined planning, timeliness, consistency, and oversight. The weaknesses identified in CSA s recruiting process inhibit the efficiency and effectiveness of the process and limit accountability. In summary: CSA currently lacks both a clear definition of what the merit system seeks to accomplish and a strategic plan that establishes goals and helps the agency adjust to future developments; CSA s recruiting and hiring process appears to take longer than other jurisdictions, and CSA s ability to monitor its performance is limited by gaps and inconsistencies in the data it collects; CSA personnel are not consistent in how they document or perform their recruiting activities, and weak controls such as poor security over tests create situations that potentially threaten fair and open competition for City positions; and Accountability is hampered by the lack of sound performance measures, limited data, and ineffective tools for both evaluating employees performance and reviewing the timeliness and quality of recruiting efforts. Lack of a Clearly Defined Merit System and a Weak Strategic Plan Hinder CSA s Effectiveness Audit work determined that the components of the City s merit system are not specifically defined. As a result, key stakeholders do not have a consistent understanding of what the merit system should encompass. This lack of clarity has implications for how the recruiting process is carried out. In addition, the absence of a comprehensive strategic plan limits CSA s ability to adjust to future developments. CSA Has Not Clearly Defined the Components of a Merit System Although the principles of the merit system are the foundation of CSA's existence, CSA does not have a specific definition for the City s merit system or a consistent view of what its components should be. This results in differing interpretations of the merit system throughout the City. As part of audit work, auditors asked 16 key personnel to define the merit system, including three in CSA management, all five Board members, and eight Human Resource (HR) representatives from various City agencies. No more than 50% of the respondents gave the same general definition and two did not recognize the term merit system. Table 1 summarizes the interviewees responses. City and County of Denver P a g e 12

Table 1: Merit System Definitions Provided by Survey Respondents Merit System means: No. of Responses % of Responses* Fair and Competitive Hiring 8 50.0% Testing or Ranking to Hire Based on Skills 7 43.8% Pay for Performance or Fair Pay 6 37.5% Consistent/Fair Rules and Employee Treatment 5 31.3% Opposite of Spoils or Political Patronage 2 12.5% Don t Know 2 12.5% *Auditors obtained 16 responses. Multiple answers were accepted so totals do not equal 100%. This lack of clear and consistent focus has implications for how the system is administered and what it accomplishes. For example: Recruiting personnel reported the merit system is the reason testing is required of all candidates. However, developing and administering a test adds time and costs to the recruiting process and is not necessarily needed to conform to merit system principles. Audit work found three of the four benchmarking cities reported adherence to the merit system, and none of these cities require all candidates be tested as part of their recruiting process. In January 2009, the Board approved reducing the minimum amount of time a job announcement can be posted from five to two business days. This might jeopardize the merit principle of fair and open competition. For example, the federal government recommends a minimum posting of five days. Additionally, two of the four benchmark jurisdictions require posting a job announcement for a minimum of five business days, and one requires a minimum posting of 7 business days. 7 Without a clear understanding and definition of what a merit system means, CSA is limited in its ability to determine whether current recruiting processes reflect merit principles, whether some processes such as testing are even necessary, and whether a merit system is effective or necessary. 7 The fourth city requires a minimum posting of 3 days, but typically posts job opportunities for two weeks. P a g e 13 Office of the Auditor

Lack of a Comprehensive Strategic Plan Limits CSA s Ability to Adjust for Future City Demands Audit work found CSA does not have a comprehensive strategic plan. Although CSA does have information typically contained in a strategic plan, it is located in three separate documents. This limits the organization s ability to use the information to plan effectively for the future. A study conducted by the International Public Management Association for Human Resources (IPMA-HR) indicates the absence of a strategic plan reduces a human resource organization s ability to function effectively. The study states, "The failure to have a strategic plan in place suggests that the organization approaches HR management in a fragmented and reactive fashion that, by its very nature, is less effective and more costly than working within a thoughtful design." 8 The study recognizes that developing a strategic plan requires commitment from agency management. According to the study, "If organization leadership does not recognize value in HR, strategic planning may not be a visible priority, and thus, not required or mandated by leadership. Strategic planning requires that HR have the staff, time and energy needed to complete a planning process regardless of how simple or broad the resulting plan." 9 Without a comprehensive strategic plan, CSA is in a less advantageous position to incorporate new ideas or strategies into the City s HR efforts. One strategy currently utilized by the City is branding. Branding, a fairly new concept in the public sector, has been used by private-sector companies as a way to sell products. According to the IPMA-HR, Branding is strategic and often depends on consistency in messaging, communications and standards. This same concept can be applied to public sector organizations seeking to establish themselves as more than just an entity tasked with basic service delivery. Public sector organizations are beginning to recognize the importance of reputation or how the organization is seen through the eyes of a potential employee. Branding is about taking charge of that reputation, defining the benefits of working in a public sector environment and marketing these factors to potential employees. 10 A review of CSA s branding strategy revealed materials to include: Recruiting banners used at job fairs identifying the City logo; Pamphlets on training, organizational development, and job opportunities; Brochures promoting the City and County of Denver as a great place to work; and Full-page color print outs used for executive recruitments. 8 EquaTerra, Recruiting and Staffing in the Public Sector: Results from the IPMA-HR Research Series, International Public Management Association for Human Resources, http://ipma-hr.org/content.cfm?pageid=72 (accessed June 3, 2009). 9 Ibid. 10 Ibid. City and County of Denver P a g e 14

CSA also utilizes electronic formats offering candidates the ability to receive information regarding available positions by e-mail and text messaging. Additionally, CSA advertises on social media sites such as Twitter.com and Facebook.com. CSA uses Twitter and Facebook to promote City job openings. However, although CSA uses a variety of strategies to promote the City, it appears Recruiters may not always be aware of or actively utilizing them. CSA s branding strategy was not mentioned in interviews conducted with Recruiting staff. Furthermore, staff in charge of career fairs and university outreach stated CSA does not have a branding strategy. If Recruiting staff is not consistently utilizing the branding tools available, the City risks losing qualified candidates that are unaware of the benefits of working in the public sector, or who may not have considered the City and County of Denver for employment. The City s Recruiting and Hiring Process Does Not Produce Timely Results A key issue for a recruiting function is whether it succeeds in placing new employees in positions as expeditiously as possible. The most qualified candidates are generally desired by many employers, so as the length of time to hire increases, the opportunity to hire the most qualified decreases. Audit work found that, relative to other jurisdictions, CSA s recruiting process takes longer. A lengthy recruiting and hiring process can negatively affect overall customer satisfaction. There are steps CSA can take to improve the overall efficiency of the process. Available Data Indicate Other Jurisdictions Process Job Applicants More Quickly An audit sample found the average time to fill a position in 2008 was 79.9 days. 11 Compared against two industry standards, the City s process is slower. Comparison with other jurisdictions that use the same recruiting software In 2006, IPMA-HR issued a benchmarking study that surveyed public-sector hiring agencies using NEOGOV, the online recruiting software CSA began using in 2006. 12 This study found the mean number of days reported for notification of vacancy until reports to work (which represents an earlier starting time point and therefore a longer timeframe than was measured in the audit sample) was 49 days, or nearly 31 days shorter than CSA s average time. Comparison with federal standards The federal government has an 80-day standard for its hiring process. While this standard may appear very close to the City s average of 79.9 days, closer comparison shows CSA s process is actually slower in several respects. First, the federal government starts its measurement 11 This is the number of days from when a Recruiter first enters recruitment information in NEOGOV (after step 4 on Figure 1, page 7) and the new employee reports to work. 12 International Public Management Association for Human Resources (IPMA-HR), 2006 Recruitment and Selection Benchmarking, IPMA-HR, http://ipma-hr.org/content.cfm?pageid=42 (accessed January 20, 2009). P a g e 15 Office of the Auditor

earlier in the process the day the request to fill a position is received, and CSA does not measure recruiting preparation time. If auditors had measured CSA s processing time from this earlier point, the average processing time would be greater. Second, the federal hiring time includes 14 days to advertise the job, compared to the minimum of two for CSA, and time for completing an extensive security background check the City does not perform. This indicates other parts of CSA s process are taking much longer. Customer Service is Negatively Impacted by Timeliness Issues Audit work found weaknesses identified in CSA s recruiting processes and practices negatively impact customer satisfaction. CSA is primarily a customer service agency with a variety of customers, including hiring agencies, potential job candidates, as well as current City employees. Audit work included a survey of the City s hiring agencies. While most agencies reported they are fairly satisfied with the relationship and service they receive from CSA, several agencies also felt the recruiting process is too complicated or overly stringent. In a customer satisfaction survey completed by CSA, employees who were less than completely satisfied were asked to offer suggestions for improvement to make the selection process better. Of those who responded, 52% stated they would like to see a faster process. Options Exist for Making the Process More Timely Audit work identified several steps that could be taken to shorten the recruiting process. Eliminating or compressing steps Audit work identified some steps in the recruiting process that are not mandated by law or Rule and could potentially be eliminated. One example is the current requirement that the City s Budget and Management Office (BMO) verifies available funding prior to initiating recruiting activities. However, the hiring agency should be able to determine whether adequate funding is available. Even if this step is retained, there are ways to modify and compress the process. One of the benchmark jurisdictions begins recruiting as soon as the recruiting agency is notified that a position is open, rather than waiting for BMO s approval. This allows the City to begin its advertising and screening efforts while funding is approved. When Recruiting is notified the position can be filled, it is able to send its certification list to the hiring agency within one day. Identifying ways to increase efficiency by the hiring agency Audit testing showed the hiring agency s portion of the hiring process comprised approximately 45-50 days of the 79.9 day average, or about 60% of the total time. While this portion of the process is largely outside of CSA s control, they are currently not monitoring this time. By measuring and addressing this time, CSA could pursue strategies for working with City agencies to decrease it. Shortening the process for certain types of positions CSA does not have formal policies and procedures in place to allow agencies to fill positions when there is an immediate need. For example, two of the four benchmark jurisdictions allow City and County of Denver P a g e 16

individual agencies to hire their own on-call (temporary) employees. Allowing City agencies to bypass CSA by directly hiring or utilizing a temporary agency to fill on-call positions could meet their needs quickly while reducing CSA s workload. Additionally, a 2000/2001 study conducted by IPMA/NASPE identified immediate job offer processes for hard to fill positions as a best practice. 13 Inconsistencies and Problems Executing CSA s Processes Hinder Objectivity in the City s Recruiting Efforts Audit work found CSA lacks policies and procedures to help prevent discrepancies and potential problems in the recruiting process. Detailed examination of the recruiting process showed numerous inconsistencies in how CSA personnel conducted and carried out various steps of the process. In addition, gaps and lapses in some procedures create situations in which fair and open competition for positions is potentially affected. CSA Personnel Execute the Process Inconsistently Rule 3 serves as the primary policies and procedures for Recruiting. Audit work showed that while Rule 3 does provide some guidance, it is inadequate on its own because it is broad and vague. Recruiting has no additional written policies and procedures, and the result is incomplete or inconsistent recruiting practices. Specifically, the following inconsistencies were identified: Recruiting processes are inconsistent. Key Recruiting Steps Not Documented by Recruiters Audit testing indicated CSA Recruiters document key steps inconsistently. Rule 3 does not specifically address which steps are to take place, and when auditors requested written policies and procedures, they found the steps were not written. The Recruiting Supervisor reported the following procedures should occur: When an approval to fill form is required by the Budget and Management Office, Recruiters should attach the completed form in NEOGOV; Recruiters should be using at least two subject matter experts (SMEs), or when an expert from the agency grades supplemental questions asked of applicants. There are no guidelines designating when it is appropriate for Recruiters to grade the questions or when a SME should be used; Recruiters should complete a service level agreement for any recruitment that has not been conducted by that specific Recruiter before. It is not required for highvolume recruitments or for citywide mass recruiting (i.e. Administrative Support Assistant). Any time an agreement is completed, it should be attached in NEOGOV; and 13 International Public Management Association/ National Association for State Personnel Executives, 2000/2001 IPMA/NASPE HR Benchmarking Project: Best Practices, IPMA-HR, http://ipma-hr.org/content.cfm?pageid=380 (accessed January 20, 2009). P a g e 17 Office of the Auditor

During a typical recruitment, Recruiters should notify applicants when they do not meet the minimum qualifications, when they are deemed eligible, when they are referred to the hiring agency and when the position is filled. This is automatically documented in NEOGOV. To determine if these steps were performed, auditors tested two recruitments for each of the ten Recruiters. The testing compared the procedures identified by the Recruiting Supervisor with recruitment data from NEOGOV. In order for management to adequately ensure requirements are fulfilled, each of these recruiting steps should be documented in NEOGOV. As shown in Table 2, documentation was not present the majority of the time. Table 2: Required Recruiting Steps Compared With NEOGOV Data Recruitment Documentation Number of Times Documentation Not Present in NEOGOV Attachment of approval to fill form 8 of 18* Evidence provided that more than one subject matter expert was used 15 of 18** Service Level Agreement Attached 16 of 20 Applicants notified at all appropriate stages of recruitment 17 of 20 *Two recruitments tested were exempt from this requirement. **Two recruitments did not use supplemental questions as a test mechanism. These findings indicate Recruiters are inconsistent when entering information into NEOGOV. Additional audit work found some data from the time to hire sample had to be excluded due to inaccuracy and incompleteness. Specifically, 11 requisitions that auditors tried to verify had to be excluded due to errors found (i.e. the person recorded as hired for that position was actually hired for another position). Furthermore, auditors found variations in how Recruiters recorded recruitments using an existing certified list. In some cases, little to no data was recorded about these particular recruitments which made it difficult to determine who was hired when. NEOGOV has the potential to provide useful information that can be used for monitoring and overseeing the efficiency and effectiveness of the recruiting process, but this potential cannot be tapped if basic information entered into the system is incomplete and inaccurate. No Procedures for Deciding When to Eliminate Recruiting Steps Depending on the circumstances of a recruitment, the recruiting process may be legitimately shortened without violating merit principles. For example, if an agency is hiring for an Administrative City and County of Denver P a g e 18

Support Assistant and a certified list is already available, Recruiters would not need to complete many of the steps in the workflow process. Similarly, if the agency hired the same staff position on many occasions, Recruiters may be able to eliminate the hiring agency strategy meeting. When Recruiters are legitimately able to adjust the recruiting process, agencies may be able to fill their open positions faster. While CSA provided instances when the recruiting process may be shortened, there are no written policies identifying when steps may be eliminated or who has the authority to eliminate certain steps. 14 Inconsistency in Creating Eligible Lists Recruiters could be formulating eligible lists differently because Rule 3 is unclear as to when applicants may be placed on the eligible list. According to Rule 3, an eligible candidate is an applicant for a vacant position who meets the criteria required for placement on a list. 15 Criteria is not defined, which means it is subject to interpretation. For example, the eligible list could include applicants that only meet minimum qualifications, or it could include those who meet minimum qualifications and pass applicable tests. Inconsistency in Utilizing Special and Preferred Qualifications Recruiters gave inconsistent answers as to when special and preferred qualifications can be used in screening, eliminating and ranking applicants. Specifically, only six Recruiters reported they could screen and eliminate applicants based on special qualifications. CSA Rule 3 is clear that Recruiters may screen for special qualifications. Further, six Recruiters reported preferred qualifications could not be used in ranking applicants for the certified list. CSA Rules do not address the use of preferred qualifications. Using special and preferred qualifications to the fullest extent possible is important in order to ensure Recruiters are providing the best customer service to agencies. Recruiters and agencies do not have the ability to amend minimum qualifications. CSA s Classifications Division reported minimum qualifications are broad by design, as they must apply to whole job classes. Further, Classifications reported special and preferred qualifications should be used when possible in order for agencies to get the best qualified candidate for the position. It appears from Recruiter interview responses that these qualifications are being inconsistently or inaccurately used. In the case of preferred qualifications, it is unclear how these relate to the certified list of candidates agencies choose from. Additionally, the Recruiting Supervisor reported to auditors that special qualifications may not be used for some entry-level positions, which contradicts audit work. When special and preferred qualifications are not used to the fullest extent it can increase applicant numbers and workload, eligible lists could be formulated differently, and it could reduce the number of highly qualified candidates. 14 Examples include recurring recruitments and recruiting for parks, seasonal and labor positions. 15 City and County of Denver Career Service Board Personnel Rules, Rule 3-5(C): Selection Definitions, Effective May 4, 2007, http://denvergov.org/humanresources/rulesandpolicies/tabid/432517/default.aspx (accessed June 10, 2009). P a g e 19 Office of the Auditor

Inconsistency in Developing Supplemental Test Questions Recruiters and Test Specialists develop supplemental questions individually and questions may vary from recruiter to recruiter, even for the same job classification. Further, the established CSA review process for supplemental questions is spread throughout several documents and does not appear to be clearly monitored. As a result, questions may not be developed, graded and documented in a consistent manner. Identified Control Weaknesses Potentially Affect Fair and Open Competition for Positions In some aspects of the recruiting process the lack of sufficient policies and procedures creates situations that could jeopardize the fairness and openness of competition for positions. These instances were as follows: Certified Lists Can Remain Open for Lengthy Periods and Be Used for Multiple Recruitments Audit work found that candidates are being hired after they are technically no longer eligible due to a flaw in Rule 3. Rule 3 places an expiration date on the eligible list, but not on the certified list. 16 CSA sends a certified list to the hiring agency and the agency can hire from this list for an undetermined length of time, even when the eligible list may have lapsed. For example, auditors identified a candidate who was made eligible and whose name was sent to the agency via the certified list on August 31, 2007. Because the eligible list expires after 90 days, this candidate technically became ineligible for consideration on November 29, 2007. However, the candidate was hired on May 27, 2008, roughly nine months after being made eligible. The hiring agency was not in violation of Rule 3, since there is no expiration date for the certified list. Contacting candidates when sufficient time has elapsed could negatively impact the City s reputation and compromises its ability to hire quality candidates. The lack of closure on the certified list circumvents the intent to hire in a timely manner. The lack of an expiration date on the certified list allows it to be reused. Although there is an interest in efficiency when using a certified list several times, there needs to be balance. An expiration date on one list and not the other creates imbalance between using an existing certified list and conducting a new recruitment. Audit work showed the practice of reusing certified lists may be extensive. When auditors attempted to review recruitments in September 2008 for time to hire information, only 17 positions could be tested, even though there were approximately 70 requisitions created in that month. This was mainly due to most new hires coming from an existing certified list, meaning a new recruitment was not conducted for the requisition. 17 In other words, only 17 recruitments were conducted (that could be verified) out of 70 requisitions created 16 For an explanation of these two lists, see Overview of CSA s Recruiting Process in the Introduction and Background section of this report. 17 In some cases, auditors had to exclude the requisition, not because the hiring occurred off an existing certified list, but because too much data was missing due to Recruiters entering information into NEOGOV inconsistently. City and County of Denver P a g e 20

that month. This calls into question the balance between conducting full scale recruitments versus using an existing certified list to fill positions. When existing lists are used too much, at some point it will impact the quality of people hired. Weaknesses in Security Over Tests May Compromise Results Audit work revealed significant weaknesses in the physical security of test books. Auditors conducted an inventory of the test books stored in CSA s test vault and identified three discrepancies: Eight of 22 (36%) tests counted did not match the Test Specialists master inventory list; The vault contained test books for three positions that were not on the master inventory list; and The master inventory listed two types of test books that were not in the vault. These discrepancies indicate Recruiters do not consistently use the Test Content Check Out Log or Test Administration Security Control Record to sign out test books from inventory. In addition to limiting CSA s ability to manage test inventory, it is difficult to determine which Recruiter removed test books from the vault. Besides finding discrepancies in the number of test books on hand, auditors also identified other inventory or security weaknesses that further inhibit Recruiting s ability to manage test inventory: Tests that are destroyed or are no longer in use are not accounted for on the master inventory list; CSA Personnel were unable to identify the number of tests originally printed, so no historic (original) inventory numbers were maintained for comparison. The first documented inventory of the test vault was completed by CSA in January 2009; Recruiting has not established a standard number of test books to be printed for each examination, making it more difficult to determine whether the number on hand is complete. For example, auditors counted 49 test books for one test, matching the number on the master inventory list. However, it is more likely a round number, such as 50 books, was originally printed; and A security issue exists related to access to the vault where tests are kept. A key to the vault is available to all Recruiting staff, and Recruiting does not have a process to check the key in and out. Weaknesses in the physical security of test books increase the risk of exam results being compromised. This could potentially allow under or unqualified applicants to pass exams. Because qualified applicants are ranked on the certified list according to test score, certain applicants may appear more qualified than they are, making it more difficult for hiring agencies to determine and hire the best candidate. P a g e 21 Office of the Auditor

These weaknesses are more significant when considering the time needed to take corrective action. Recruiting personnel reported that when a written test is lost or stolen, they develop a new test for that position. The creation of a new test may take between two to six months, depending on the type of questions contained in the test and the time commitments required by other projects in Recruiting. Increased Legal and Public Relations Liability Related to Possible Identity Theft Audit work found supplemental test books for one position contained valid social security numbers. Access to the vault, combined with availability of personal information such as names and social security numbers, could potentially lead to identity theft and a corresponding lack of trust from the clients CSA serves. Pass Points May Unintentionally Screen Out Qualified Candidates Audit work determined Recruiting s use of pass points may increase the risk of similarly qualified candidates being inconsistently removed from further consideration. 18 Specifically, the candidate s test score, regardless of the pass point, determines their rank. However, utilizing a pass point may eliminate statistically similar candidates from further consideration. 19 Furthermore, although adverse impact may not be entirely removed, CSA s current use of fixed pass points may open the City to claims of discrimination. Test Specialists began assigning fixed pass points to each test with the implementation of NEOGOV. The prior automated software allowed Test Specialists to run comprehensive reports that helped them account for and remove adverse impact by adjusting the pass point. However, Test Specialists no longer run these types of reports due to NEOGOV s limited reporting capabilities. Accountability Is Hampered by Limited Management Information and Poor Oversight Mechanisms Audit work determined CSA has little ability to monitor and report on its performance. It is generally not collecting the kinds of measures needed to accomplish this task effectively. Further, in several instances when it did report information about its performance in order for the Board to make key decisions about the recruiting process, the information was inaccurate. Likewise, key oversight mechanisms are weak, including performance evaluations of employees and quality control reviews of the agency s work. Most Available Information is Not Useful in Assessing Efficiency and Effectiveness Performance Indicators Are Inconsistent and of Limited Use Audit work determined there are two ways in which the current performance measurement system limits the ability to assess the efficiency and effectiveness of the recruiting function. First, 18 The minimum score candidates must achieve on the test to continue in the recruitment process. 19 For example, in a test with a pass point of 65 and an expected error rate of 3 points, one candidate s score of 64 is statistically the same as another candidate s score of 67; however, the candidate who scores 64 will be screened out for not passing the test. City and County of Denver P a g e 22

monitoring and collection of performance information is inconsistent. Second, the performance indicators monitored yield little information about efficiency and effectiveness, and the indicators do not align with best practices. CSA s collection of performance information is sporadic. According to the Recruiting Supervisor, performance information is not produced regularly and is mostly prepared for outside meetings or other purposes. This limits Recruiting s ability to conduct trend analyses and identify progress or emerging problems. The lack of consistent performance monitoring creates difficulties when using any of this information in decision making. Moreover, internal accountability is difficult to achieve if information is being collected sporadically and only for outside use. When CSA does collect and report performance information, it tends to focus on agency workload. Specifically, CSA Recruiting has reported on a limited basis the following performance indicators: number of job postings; applications received; applications per posting; testing sessions held; and positions filled. These indicators provide little information about the agency s efficiency or effectiveness. In contrast to the kinds of information CSA collects and reports, best practices research indicates the following types of indicators provide better information about an agency s performance: time to hire (including measuring the time each step takes); cost per hire; time spent per recruitment; volume of applications; turnover rates; completion of probation; and qualified applicants per posting. CSA has reported in the past on two of these indicators volume of applications and time to hire but only on a limited basis. The limited focus on such indicators reduces CSA s and the Board s ability to analyze the efficiency and effectiveness of Recruiting activities. For example, the cost of recruiting is unknown. CSA does not know how much time Recruiters spend on each recruitment or the total cost to hire an employee. Although performance indicators quantifying cost are only a piece of what should be P a g e 23 Office of the Auditor

monitored according to best practices, cost is important in helping to determine Recruiting efficiency and effectiveness. Weaknesses in current performance measures impede CSA s ability to determine whether the City s resources are being wisely and effectively spent. The limited information also reduces CSA s ability to plan strategically for the future. The effectiveness of any plan or steps to improve functions is diminished without information demonstrating how goals will be accomplished. Available Information is Often Incorrect or Reported Inaccurately Audit work identified several instances of inaccurate reporting. These involved performance information in various CSA reports, and the content of presentations to the CSA Board. Issues Identified With Data Submitted in CSA Reports Audit work identified key problems in the creation of two different reports submitted to the City. In its 2008 annual report, CSA reported the average number of days to fill a position was 29.5 days. While audit work suggests this figure represents the time it takes to create a certified list, worded this way, the number appears to represent the time it takes until the new employee reported for work. Beyond the annual report, CSA reports on its success in meeting bonus goals. 20 CSA reported it met its 2008 goals of creating a certified list for non-exempt (hourly) and exempt (salary) positions within 30 and 45 days, respectively. Specifically, CSA reported taking an average of 27.4 days to create a certified list for non-exempt positions and 32.1 days to create a certified list for exempt positions. Auditors attempts to recreate these figures produced slightly higher averages. Auditors attempted to review CSA s calculations but were unable to do so because CSA did not document their methodology for data verification or save the supporting data. Consequently, auditors performed a separate, more detailed analysis of a sample of recruitments conducted during the same period covered in CSA s reports. 21 This analysis indicated a higher average time to create the certified list than CSA reported. Table 3 compares CSA figures to figures calculated by auditors. Specifically, the bonus goal for creating a certification list for non-exempt positions is 30 days. CSA reported an average time of 27.4 days, which qualified for the bonus, while auditors sample showed an average time of 30.5 days, which would not have qualified for the bonus. The discrepancy resulting from this audit work highlights the importance of the data in NEOGOV, and the importance of CSA ensuring data used for reporting is accurate and reproducible. 20 Bonus goals are annual performance measures established by agencies. The City pays bonuses to employees when agencies meet their bonus goals. 21 CSA s figures were for the time period January October 2008. Auditors tested a sample from every other month in the same time period. City and County of Denver P a g e 24

Table 3: Measurement to Fill Positions Measurement CSA Reported Auditor s Office Calculation ANNUAL REPORT: Average Time to fill a Position 29.5 Days 22 79.9 Days BONUS GOALS: Time to create a certification list for non-exempt positions BONUS GOALS: Time to create a certification list for exempt positions 27.4 Days 30.5 Days 32.1 Days 35.4 Days Additionally, audit work revealed the methods CSA used to calculate its averages produced inaccurate and unreliable results. For example, CSA used a pre-programmed NEOGOV report which pulls the first date CSA refers candidates to the agency (the end date of CSA s bonus goal calculation), not the date the actual candidate hired was referred. Audit work identified several instances where the report generated a date much earlier than the hired candidate s actual date referred possibly skewing CSA s average calculations. For example, auditors identified a case where the NEOGOV report pulled July 10, 2008 as the date the candidate was referred to the agency, but upon verification, the candidate hired for this position was actually referred to the agency on September 24, 2008. Therefore, it is possible the date used for calculation in this case was more than two months shorter than the actual date the candidate was hired. The only way auditors were able to obtain 100% reliable data was to verify individual records by comparing the date referred in NEOGOV to the hire date in PeopleSoft. 23 CSA did not consistently take this step in conducting the analysis for its reports and there is no way for auditors to know when CSA verified referred dates. Problems with this report may be eliminated if CSA utilized custom NEOGOV reports. CSA is currently making minimal use of this capability. Overall, accountability could be improved if CSA takes greater advantage of this function. Inaccurate Information Presented to the Board CSA Management presented a large model of the recruiting workflow process to the Board to support a Rule change, but reported to auditors that the flowchart was inaccurate. On January 15, 2009, CSA requested a change to Rule 3-21 regarding the minimum required days for posting job positions. The previous Rule required that job postings appear for a minimum of 5 business 22 This figure may represent the time to create a certified list. However, CSA was unable to provide documentation to substantiate whether it represents time to create a certified list or time to fill. 23 PeopleSoft is the City s human resource and accounting software system. P a g e 25 Office of the Auditor

days, whereas the new Rule reduced that to a minimum of two business days. CSA reported that the Rule change was necessary to limit the number of applications because processing time increases as the volume of applications increases. As CSA stated in this public hearing, a day would count towards the total only if the posting appeared for the full day. For example, if a job was posted on a Monday afternoon, Monday would not count as a day. This requirement would ensure that the posting appeared all day Tuesday and all day Wednesday with the job closing at midnight on Wednesday. Board members approved the change but requested that CSA update them on how many times jobs were posted for less than five days (Quick Posts), which supervisors requested the shorter postings, which positions they were for, and the impact of shortening the time. On March 19, 2009, CSA presented some of the requested information related to the impact of the Rule change. Audit work determined CSA did not report all of the requested information, and presented information that was inaccurate. Specifically: CSA reported three Quick Posts had been made since the January Rule change. However, audit work determined that a total of 12 positions were posted for less than five days as of March 19; and CSA presented the average applicants per position figure which auditors could not successfully recreate. It should be noted that the problems in re-creating this data could be due to a recent archiving project in NEOGOV, but without the original methodology used to create these figures it cannot be verified that this is the case. Reporting must be accurate in order for key stakeholders to hold Recruiting accountable for its performance. Inaccurate information diminishes the ability of stakeholders to evaluate the overall effectiveness of Recruiting. Additionally, accurate information is necessary to determine whether CSA is adhering to the merit system. Performance Evaluations Are Vague and Not Timely Audit work identified several ways in which CSA management s review of staff performance provided limited accountability. First, the document most commonly used to discuss staff performance (the duty matrix) does not adequately define roles, responsibilities, standards, and expectations for Recruiters. The duty matrix contains a general breakdown of duties for which Associate, Senior and Recruiting Specialists are responsible. However, this matrix is deficient in several respects as an accountability tool: The matrix does not contain any definitions or explanation of what each duty entails. For example, the matrix lists both customer service and client management terms that, without further definition, can be interpreted differently and confused easily. It is difficult to hold Recruiters accountable for unclear duty assignments. The matrix lacks an explanation of how it should be applied or how performance will be measured. All Recruiters interviewed had negative comments about the City and County of Denver P a g e 26

duty matrix, and seven of ten Recruiters stated the matrix was not an accurate breakdown of duties. Without clarifying documents, the matrix is an ineffective management tool. A second problem with staff accountability is that while better evaluation tools exist, they are not used to assess most Recruiters performance. The duty matrix was the only standardized Recruiter performance evaluation tool provided to auditors which was used consistently in 2008. A third problem is none of the CSA Recruiting staff received their annual performance evaluations, called Performance Evaluation Plan Report (PEPR), in a timely manner. Specifically, ten of the 13 (77%) Recruiting staff members received a forced evaluation on their most recent PEPR. 24 At the time of testing, the remaining three staff had been employed less than one year, so a formal evaluation was not due. Performance appraisals provide several important benefits for employees, including feedback on performance and areas for performance improvement. They also benefit the employer by helping improve employees understanding of job responsibilities and standards of performance, thereby providing management with a means to hold them accountable. Data obtained from CSA indicated the timeliness of performance evaluations is an issue citywide. It appears the City s framework to hold supervisors accountable for completing PEPRs in a timely manner is inadequate. Audit work found that the State of Colorado has laws in place to hold supervisors accountable when they do not complete evaluations. These include progressive discipline for each year the evaluation is not completed, ranging from suspension (unpaid days off) to Timeliness of performance evaluations is a citywide problem. demotion. 25 The Rules state that supervisors failing to complete PEPRs within 30 days of the due date may be subject to discipline. 26 This rule is vague about what kind of disciplinary actions may be used and leaves this responsibility up to the agencies. The agencies are not required to do anything. While CSA does not have the authority to insist on timely PEPRs citywide, it does have the authority to request a citywide Rule change to address these problems. Quality Control and Oversight Processes Need Strengthening Audit work found existing quality controls over the recruiting process are vague and largely undocumented. Recruiting has developed a general reporting structure for quality review of Recruiters work, but no way to monitor whether reviews occur. Recruiting management was able to list some items that should be checked during quality reviews, such as spelling and content accuracy of job announcements, but could not provide any written documentation of such reviews or the requirements. The lack of 24 A forced evaluation is when City employees do not receive their formal annual evaluation within 30 days of the due date. As a result, the employee is automatically rated as Successful and given the corresponding pay increase. 25 C.R.S. 24-50-104(1)(c.5) (2009). 26 City and County of Denver Career Service Board Personnel Rules, Rule 13-61 (C4): Merit Date, Effective January 1, 2008, http://denvergov.org/humanresources/rulesandpolicies/tabid/432517/default.aspx (accessed June 10, 2009). P a g e 27 Office of the Auditor

a well defined and documented quality control process limits CSA s knowledge of performance, their ability to hold individual personnel accountable, and increases the risk of inaccuracies. Further, audit work found there is a lack of quality controls in the testing process. For example, CSA does not appear to perform consistent quality reviews of exams to ensure testing effectively measures applicant competency. Additionally, although Recruiting had some policies and procedures related to checking written exams out and back in for testing applicants, audit work found recruiting personnel were not adhering to them. The absence of quality controls in the testing process is a significant problem because it increases the risk of litigation related to unfair testing procedures. CSA Governance Can be Enhanced CSA was specifically established as an independent agency to limit inappropriate political influence, but because of this independence, the CSA Board should actively provide oversight and hold CSA accountable for carrying out the agency s mission and accomplishing its goals. Action is needed on multiple fronts to strengthen the City s recruiting function. The preceding sections have detailed a number of issues that need addressing. In many respects, these issues are interrelated. For example, while better accountability mechanisms are needed, these mechanisms will not work well if good performance indicators are not in place. Simply conducting more performance evaluations and quality reviews will accomplish little if there are no indicators that measure what the process is accomplishing. However, even the best performance indicators are of limited use if employees are not consistently entering data into the NEOGOV system or making use of the system s reporting capabilities. RECOMMENDATIONS The following recommendations are designed to help CSA and its Board address issues identified by audit work. Recommendations are grouped to reflect the sequence of topics discussed in the report, but in some cases recommendations made under one heading may help address matters under other headings as well. Establishing Strategic Direction and Planning 1. To better establish strategic direction and planning for the recruiting function, CSA, in consultation with the Board, should take the following steps: Establishing a clear, actionable definition of the merit system and making it a part of agency documents for example, in the purpose statement or the introductory statement of the CSA Rules. The definition of the merit system should be detailed enough to identify when Rules are appropriate and how it specifically applies to all personnel issues; City and County of Denver P a g e 28

Consider increasing the minimum job posting time from two to five business days to align with the federal government and other benchmarking jurisdictions; Developing, in a single document, a strategic plan that includes measurable goals and reportable performance measures that can be tracked over time; and Ensure all Recruiters are aware of materials and tools available to promote CSA s branding strategy. Improving the Timeliness of Recruiting 2. CSA should evaluate the full recruiting and hiring process in an effort to identify steps where efficiencies could be realized. These steps could include: Identifying steps that hiring agencies can take to shorten their portion of the hiring process; Pursuing rule changes that allow for quick hiring, such as authorizing agencies to hire their own on-call positions or utilize temp agencies; and Starting initial recruitment steps at the time the hiring agency decides to fill an open position rather than waiting until the Budget and Management Office determines funding for the position is available. Making Policies and Procedures More Thorough and Consistent 3. To ensure greater consistency in the recruiting process and integrity in data collection, CSA should create written policies and procedures that cover the following: Explaining when Recruiters may eliminate steps in the recruiting process; Specifying the documentation to be attached in NEOGOV to ensure consistent and complete recruiting records; Specifying procedures for developing eligible and certified lists; Specifying when subject matter experts should be used in applicant testing, including how many should be used; Specifying what information should be consistently entered and how to enter it into NEOGOV, including recruitments that utilize an existing certified list; Clarifying when and how it is appropriate to use special or preferred qualifications and promote usage by the hiring agency, including: o o Using special qualifications to establish clear and concise requirements for the open position and help limit the number of applications from candidates that clearly do not meet the qualifications needed for the position. Ranking eligible candidates based on preferred qualifications for the certified list, in conjunction with test scores; and P a g e 29 Office of the Auditor

Clearly documenting the development of supplemental test questions and specifying that supplemental questions are standardized for like job postings, to the degree possible. 4. CSA should seek a Rule change establishing an expiration date for certified lists to ensure that these lists are not used beyond the expiration of the eligible lists. 5. To better ensure the security of test books, CSA should take the following steps: Perform an inventory of the test vault in order to create a baseline for future inventories, establish a set number of test books for each examination, and maintain documentation of test books that are destroyed; Consider replacing manual test books with computer based or video based tests. Video based tests have been shown to produce a more positive response from candidates, and reduce adverse impact; Eliminate pass points on written tests. By eliminating the pass point, each eligible candidate has the opportunity to move forward in the hiring process; and To address test security and pass point issues, CSA could also consider eliminating testing from the recruiting process, or keeping testing for positions only when necessary, such as for positions that may require physical demonstration of a needed skill. Strengthening Accountability 6. To better assess the efficiency and effectiveness of the recruiting function, CSA should develop specific performance measures that coincide with those identified by best practices. At a minimum, these measures should include time to hire, turnover rates, and cost per hire. Other important measures and steps include probation passing rates, time spent per recruitment and analyzing the various components of the recruiting process to identify which parts are taking longer. CSA should then incorporate performance data findings into management practices. 7. To ensure information for performance measures and reports is accurate and consistent over time, CSA should take the following steps: Ensure the methodology used to produce data is documented and can be reproduced. Additionally, backup data behind figures reported publicly should be retained to support such figures; Verify all information before presenting it to the Board or reporting it to City officials; and Measure them consistently over time. 8. To improve personnel performance evaluation, CSA should: City and County of Denver P a g e 30

Develop clear, detailed, and measurable roles and responsibilities for Recruiting personnel. This should include Recruiters as well as HR and Testing Specialists. Duties should be better defined for each level of Recruiter, including expectations for how each duty should be performed and how performance will be measured; and Identify specific disciplinary actions to hold management accountable when PEPRs are not completed in a timely manner. 9. To strengthen monitoring of job processes, CSA should develop quality controls, to include: Documentation on what should be reviewed, who should review it, at what points in the recruiting process reviews should happen, how often it should occur and how it will be monitored; Monitoring compliance with policies and procedures; and A specific quality control process for ensuring data reliability and accuracy. 10. To help analyze and interpret information about the recruiting process, CSA should increase its use of NEOGOV s custom reporting tools. At a minimum, create reports for job posting data and requisition life cycle data to ensure figures reported are accurate and useful. P a g e 31 Office of the Auditor

APPENDIX A City Positions Not in the Career Service (Charter 9.1.1 E) The Career Service shall comprise all employees of the City and their positions except: (i) elected officers; (ii) members of the Mayor's cabinet; (iii) the Director of Excise and Licenses; (iv) up to fifty employees appointed to serve at the pleasure of the Mayor in positions specifically designated or created by the Mayor in any department or agency of the City under the direct control of the Mayor; (v) county court judges and magistrates; (vi) members of the Classified Service of the Police and Fire Departments, the Police Chief if not a member of the Classified Service, and the Undersheriff; (vii) attorneys and part-time employees employed by the District Attorney, other employees of the District Attorney excluded from the Career Service and placed in an alternate merit personnel system pursuant to state law, and up to ten employees appointed to serve at the pleasure of the District Attorney in positions specifically designated or created by the District Attorney in the District Attorney's office; (viii) certified public accountants employed by the Auditor and up to five employees appointed to serve at the pleasure of the Auditor in positions specifically designated or created by the Auditor in the Auditor's Office; (ix) employees of the Denver Art Museum, the Denver Museum of Nature and Science, the Denver Zoological Gardens, and the Denver Botanical Gardens; (x) persons retained on a contractual basis to perform professional or technical services for limited periods of time; (xi) employees of the City Council, Library Commission, Civil Service Commission, Board of Adjustment, and Denver Water; and (xii) any hearing officers and up to two employees in positions specifically designated or created by the Career Service Board, appointed to serve at the pleasure of the Board. (xiii) any employee appointed to serve at the pleasure of the mayor for the purpose of monitoring internal investigations and disciplinary actions in the Department of Safety, and any employees appointed by the monitor to serve at the pleasure of the monitor. The appointment of any monitor by the mayor pursuant to this or any other provision of the charter shall require confirmation by the city council. (xiv) the Director of Elections and no more than one other employee in a position specifically designated or created by the Clerk and Recorder, appointed to serve at the pleasure of the Clerk and Recorder. Any employee of the Denver Election Commission as of July 16, 2007 and formerly excepted from the Career Service pursuant to this section shall retain his or her position as an employee of the Clerk and Recorder if the employee qualifies to retain the position in accordance with the rules of the Career Service Board. City and County of Denver P a g e 32

AGENCY RESPONSE P a g e 33 Office of the Auditor

City and County of Denver P a g e 34

P a g e 35 Office of the Auditor

City and County of Denver P a g e 36

P a g e 37 Office of the Auditor

City and County of Denver P a g e 38

P a g e 39 Office of the Auditor

City and County of Denver P a g e 40

P a g e 41 Office of the Auditor

City and County of Denver P a g e 42

P a g e 43 Office of the Auditor