Public Works Oversight of External Architectural & Engineering Design Work Performance Audit

Transcription

1 Public Works Oversight of External Architectural & Engineering Design Work Performance Audit July 2012 Office of the Auditor Audit Services Division City and County of Denver Dennis J. Gallagher Auditor

2 The Auditor of the City and County of Denver is independently elected by the citizens of Denver. He is responsible for examining and evaluating the operations of City agencies for the purpose of ensuring the proper and efficient use of City resources and providing other audit services and information to City Council, the Mayor and the public to improve all aspects of Denver s government. He also chairs the City s Audit Committee. The Audit Committee is chaired by the Auditor and consists of seven members. The Audit Committee assists the Auditor in his oversight responsibilities of the integrity of the City s finances and operations, including the integrity of the City s financial statements. The Audit Committee is structured in a manner that ensures the independent oversight of City operations, thereby enhancing citizen confidence and avoiding any appearance of a conflict of interest. Audit Committee Dennis Gallagher, Chair Maurice Goodgaine Leslie Mitchell Rudolfo Payan Robert Bishop Jeffrey Hart Timothy O Brien, Co-Chair Audit Staff John Carlson, Deputy Director, JD, MBA, CIA, CRMA Chris Wedor, Internal Audit Supervisor, MBA, CICA Nancy Howe, Lead Internal Auditor, MPA, CICA Manijeh Taherynia, Senior Internal Auditor, CPA, CFE Katja Freeman, Senior Internal Auditor You can obtain copies of this report by contacting us at: Office of the Auditor 201 West Colfax Avenue, Department 705 Denver CO, (720) Fax (720) Or download and view an electronic copy by visiting our website at:

3 City and County of Denver 201 West Colfax Avenue, Department 705 Denver, Colorado FAX Dennis J. Gallagher Auditor July 19, 2012 Jose M Cornejo, P.E., Manager Department of Public Works City and County of Denver Dear Mr. Cornejo: Attached is our performance audit of external architectural and engineering design work practices within the Department of Public Works. The purpose of our audit was to assess Public Works policies and processes related to staffing architectural and engineering (A/E) design work and the impact on the organization and City. Our audit resulted in two findings. The first focuses on the consultant selection process and the second centers on consultant performance measurement. Specifically, Public Works does not have sufficient guidance regarding Selection Committee activities and does not maintain sufficient documentation of associated A/E design activities and communications when selecting external design consultants. Further, Public Works appears not to comprehensively evaluate A/E design project performance. Additionally, my auditors spent significant time researching common practices that may assist in mitigating process gaps that were identified in our audit. I hope that these practices will be implemented to ensure a sound and transparent process with regard to outsourcing City A/E design work. If you have any questions, please contact Kip Memmott, Director of Audit Services, at Sincerely, Dennis J. Gallagher Auditor cc: Honorable Michael Hancock, Mayor Honorable Members of City Council Members of Audit Committee Ms. Cary Kennedy, Deputy Mayor, Chief Financial Officer To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

4 Ms. Janice Sinden, Chief of Staff Ms. Stephanie O Malley, Deputy Chief of Staff Ms. Beth Machann, Controller Mr. Doug Friednash, City Attorney Ms. Janna Bergquist, City Council Executive Staff Director Mr. L. Michael Henry, Staff Director, Board of Ethics To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

5 City and County of Denver Dennis J. Gallagher Auditor 201 West Colfax Avenue, Department 705 Denver, Colorado FAX AUDITOR S REPORT We have completed our audit of Public Works oversight of external architectural and engineering design work. The purpose of the audit was to assess policies and processes related to external architectural and engineering (A/E) design work and its impact on the Department and City. This performance audit is authorized pursuant to the City and County of Denver Charter, Article V, Part 2, Section 1, General Powers and Duties of Auditor, and was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The audit resulted in two findings. The first centered on consultant selection and the second reviewed A/E design performance measurement. Specifically, Public Works does not provide sufficient guidance regarding Selection Committee activities and does not maintain adequate documentation of A/E design consultant selection activities. Further, Public Works does not comprehensively evaluate A/E design project performance. We acknowledge the Public Works Department for assistance during the audit. Kip Memmott, MA, CGAP, CRMA Director of Audit Services Audit Services Division To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

6 TABLE OF CONTENTS EXECUTIVE SUMMARY 1 INTRODUCTION & BACKGROUND 5 SCOPE 11 OBJECTIVES 11 METHODOLOGY 11 FINDING 1 12 Public Works Architectural and Engineering Design Consultant Selection Process Can Be Enhanced to Increase Accountability and Transparency 12 RECOMMENDATIONS 19 FINDING 2 20 Performance Measurement and Additional Practices Will Enhance Public Works Management and Oversight of Architectural and Engineering Design Consultants 20 RECOMMENDATIONS 24 OTHER PERTINENT INFORMATION 25 EXHIBITS 28 Exhibit A Department of Public Works Organizational Chart 28 Exhibit B Department of Public Works Request for Proposal Checklist 29 Exhibit C City and County of Denver General Obligation Bonds Issued To Fund Capital Projects AGENCY RESPONSE 35

7 EXECUTIVE SUMMARY This audit assessed the Department of Public Works (Public Works) policies and processes related to outsourcing architectural and engineering (A/E) design work and the impact of these practices. While our audit did not observe any process control failures, the inherent risk in this area warrants attention. The audit resulted in two findings. The first concerns the consultant selection process and the second assesses consultant performance measurement. Capital Projects Management and the RFP Process Public Works is responsible for the design, construction, operation, maintenance, and repair of public infrastructure in the City and County of Denver including roads, bridges, storm and sanitary sewers, and public buildings. Capital Projects Management (CPM) within Public Works is responsible for the management of the design and construction of capital projects. When A/E design work is conducted in-house, it is performed by the CPM Design Section. When design work is contracted to external parties, CPM manages consultant agreements. City project managers oversee the associated design work. Public Works procures outside A/E design services using a Request for Proposals (RFP) process. The RFP process includes public solicitation for A/E design services, evaluation of proposals by a selection committee, selection of at least three firms for interview, and contract price negotiation with the top ranked firm. FINDING 1: Public Works Architectural and Engineering Design Consultant Selection Process Can Be Enhanced to Increase Accountability and Transparency Each time Public Works solicits for A/E design services through the RFP process, a new Selection Committee (SC) is formed to evaluate submitted proposals and recommend the most qualified applicant to Public Works management. When determining who should serve on the SC, the Project Manager determines which skill sets should be represented, based on project requirements. However, our review revealed that although Public Works has some informal and draft procedures, there are no formal criteria guiding the SC-member selection process. Specific criteria would ensure that SC members have Our review observed a lack of formal criteria for City Selection Committees when retaining consultants for City work. the appropriate knowledge and expertise necessary for recommending a consultant. Once an SC is formed, Public Works does not have detailed procedures or formal criteria to guide SC decisions, such as weighting criteria for ranking firms. Rather, the SC relies on institutional knowledge and verbal instruction from the Project Manager to guide the P a g e 1 Office of the Auditor

8 consultant selection process. This informal process hinders consistency, efficiency, and fairness in the consultant selection process. In addition, Public Works does not maintain adequate documentation of SC activities or communications. With regard to activities, Public Works does not consistently maintain scoring and comment sheets completed by SC members during the consultant evaluation and interview process. Additionally, SC members do not record deliberations and discussions. Retaining this documentation would ensure transparency and would reduce the risk or appearance of impropriety in the selection process. All recommendations made by an SC are subject to approval by Public Works management. However, the recommendations are not communicated directly from the SC to management. Rather, the Director of Capital Projects Management submits a memorandum directly to the Manager of Public Works through the City Engineer to convey the recommendations, and SC members are not included on this communication with the exception of the Project Manager. In the event that Public Works management disagrees with SC recommendations, any resulting override is not documented. The lack of communication in this process further hinders transparency and increases the likelihood that information will Ethics opinions may not provide practical guidance for avoiding the appearance of a conflict of interest. be reflected inaccurately. In addition to the documentation and communication weaknesses, audit work revealed that members of Public Works executive management seek advisory opinions from the Board of Ethics when faced with a potential conflict of interest. While audit work did not find any instances of a violation of the Code of Ethics by Public Works officials, the advisory opinions themselves may not provide practical guidance with regard to the SC process and executive management s involvement therein. Transparency and accountability are essential to maintaining the public trust. Enhanced documentation, as discussed earlier, can help Public Works management avoid the impression of a potential conflict of interest. FINDING 2: Performance Measurement and Additional Practices Will Enhance Public Works Management and Oversight of Architectural and Engineering Design Consultants Our audit revealed that Public Works has few performance measures to evaluate the quality of work performed on capital construction projects. Specifically, the only performance measures used are whether a project was completed on time and within budget. Further, this time and budget information does not address performance on individual phases of the project. City and County of Denver P a g e 2

9 Although measures of time and budget could provide information on consultant performance if broken out by project phase, limiting performance measurement to time and budget is not a comprehensive approach. Specifically, these measures do not measure the quality or efficiency of work performed by a consultant. The State of Arizona s Managing for Results Handbook suggests that additional performance measures should be included to provide a more comprehensive look at consultant performance. 1 The Arizona Handbook incorporates five common performance measure types: inputs, outputs, outcomes, efficiency, and quality. Each category answers a different question, but the measures are best utilized in combination to analyze program outcome results collectively. A/E design consultants frequently submit proposals to the City for new capital projects. However, when a consultant who has previously worked with the City submits a proposal for a new project, Public Works is limited in considering the consultant s previous performance because it does not formally evaluate consultants at the conclusion of each construction project. Without considering previous performance, Public Works risks hiring a consultant with a history of sub-standard performance and thus may not be a strong candidate for future projects. Measuring, tracking, and incorporating past performance into the selection process would Performance measures should include inputs, outputs, outcomes, efficiency and quality. Other Pertinent Information (OPI) Common Practices for Outsourcing Design Work Additional performance measures for consultant performance evaluations should be considered. provide a level of transparency, as Public Works could use this information to support its selection decisions. Additionally, if consultants know their performance is going to be formally evaluated, they may strive harder to achieve success knowing that an evaluation could impact the firm s ability to obtain work on future projects. Audit work included research to identify common practices related to retaining external A/E design work. Auditors also identified additional policies, procedures, and contract requirements that, if adopted by Public Works, may mitigate losses resulting from errors and omissions (E&Os) made by A/E design consultants. The audit team consulted numerous subject matter experts from a wide array of organizations involved in A/E design matters. A Professional Engineer serving as a Director of Real Estate and Construction Advisory Services for a major accounting firm The Department Chair of Construction Engineering Management at the University of Colorado at Boulder Managing for Results Handbook, State of Arizona. P a g e 3 Office of the Auditor

10 Fredrick County, Maryland Maricopa County, Arizona As a result of our research and interviews, we compiled a list of common practices in the areas of change order tracking and categorizing, contract language, cost estimating, and policies adopted by other local governments. These practices present possible contracting solutions that could assist Public Works with strengthening its current processes. These practices are intended to supplement the City and County of Denver s current contracting processes and may help Public Works obtain the best A/E design firms for future capital projects. City and County of Denver P a g e 4

11 INTRODUCTION & BACKGROUND Overview of the Department of Public Works The Department of Public Works (Public Works) provides services that have a significant impact on the quality of life in the City and County of Denver. Public Works is responsible for the design, construction, operation, maintenance, and repair of public infrastructure including roads, bridges, storm and sanitary sewers, and public buildings. Additional services delivered by Public Works include but are not limited to household trash and recycling collection service to 173,000 households, graffiti abatement, parking management, traffic engineering, and oversight of the regulations pertaining to development and use of public rights-of-way. 2 Public Works is the City s second largest General Fund department employing approximately 1,100 personnel and with annual budgeted expenditures of approximately $375.4 million. 3 Public Works was created under the authority of the City and County of Denver Charter. 4 The Charter authorizes Public Works to manage and control the design, planning, construction, and reconstruction of all general improvements for the City and for all City agencies except the Board of Water Commissioners and the Department of Aviation. Public Works can perform these duties using its internal resources, through contracts with external entities, or by other means permitted by the Manager of Public Works and provided by City Council ordinance or resolution. 5 Capital Projects Management Public Works primary responsibilities are carried out by three divisions: Engineering, Operations, and Finance and Administration. The Engineering Division comprises Right-of- Way Services, Traffic Engineering, and Capital Projects Management. 6 Capital Projects Management (CPM) is responsible for the management of the design and construction of City projects that serve to build and maintain the City's infrastructure including buildings, roadways, and wastewater infrastructure. These types of projects are commonly referred to as capital construction projects or just capital projects. The construction work associated with these projects is always outsourced to outside vendors, and, at times, the preceding design work is contracted out as well. When architectural and engineering (A/E) design work is done in-house, it is performed by the Design Section of CPM. When design work is contracted out, CPM manages the agreements with consultants, and City project managers oversee the design work as a part of their overall responsibility to manage capital construction projects. The cost of outsourced design services amounted to approximately $42.8 million for the years City and County of Denver Mayor s 2012 Budget, page Ibid at 537 and City and County of Denver Charter, Article II, Ibid. 6 See Exhibit A for Public Works management organizational chart. P a g e 5 Office of the Auditor

12 through The breakdown of these expenditures by funding sources is shown in Table 1. Table 1 Funding Breakdown for Outsourced Engineering and Design Funding Source Total Amount Outsourced General Fund $10,152,226 Mil Levy $3,901,781 Bond $21,831,746 Grant $1,802,258 Enterprise Fund $5,078,830 Total $42,766,840 Source: Department of Public Works. Amounts do not total exactly due to rounding. Project Controls Office The Project Controls Office (PCO) is a section within Public Works that provides programmatic oversight on all capital- and bond-related projects being undertaken by the City and County of Denver. One of PCO s responsibilities is to perform technical and regulatory review of capital construction projects being managed by Public Works. Review areas include transportation, wastewater, materials engineering, floodplain management, and erosion control. Additionally, PCO s regulatory function acts as the steward of all printed and electronic records of City-owned and -managed facilities. PCO s business function organizes project funding and has regulatory oversight of the administration of project funding, maintains budget and schedule information for the City s capital construction projects and provides reports for project managers and Public Works management. 7 The business function also has shared responsibility for financial oversight of procurement with regard to capital construction projects. 8 Securing Design Consultants for Capital Construction Projects through the Request for Proposals Process As previously discussed, Public Works is responsible for securing contractors and managing the design and construction of the City s capital construction projects. 7 The term schedule in the construction industry refers generally to a timeline for project completion. 8 PCO shares these responsibilities with Public Works Finance and Administration, and the Budget and Management Office within the Department of Finance. City and County of Denver P a g e 6

13 The Capital Construction Project Life Cycle The life cycle of a capital construction project comprises several phases that are summarized in Figure 1. Figure 1 Capital Construction Project Life Cycle Project Definition Project workpapers Project management plan (detail of project budget and funding sources, breakdown of costs and completion dates by project phase, and plan for personnel resources including in-house and outsourced) Engineering Design Design (in-house or outsourced through request for proposals) Conceptual planning and feasibility study Project Design Construction drawings, specifications, cost estimates Construction Procurement Bidding process and contractor selection Construction Construction administration including change orders and contract compliance Construction Contract Closeout Reconciliations Walk-through Occupancy Source: Figure 1 was prepared using capital construction process information from previous audits, and the City and County of Denver s Standard Specifications for Construction, General Contract Conditions 2011 E commonly referred to as the City s Orange Book. Federal, State, and City Guidelines for Professional Services Procurement Public Works process for procuring services from outside consultants has been modeled after various elements from federal, state, and City laws and policies. The Brooks Act, a federal law enacted in October 1972, requires that the federal government announce all requirements for A/E design services and negotiate contracts for such services on the basis of demonstrated competence and qualification for the type of professional P a g e 7 Office of the Auditor

14 services required and at fair and reasonable prices. 9 This qualification-based selection (QBS) process generally does not take price into consideration until the most qualified firm is selected. The QBS process is captured in six basic steps. 1. Public solicitation for A/E design services 2. Submission of a statement of qualifications and supplemental statements of ability to design specific projects for which public announcements were made 3. Evaluation of both the general and project-specific statements 4. Selection of at least three firms for interview 5. Selection of at least three of the most qualified firms based on interviews 6. Contract negotiation with the top-ranked firm The State of Colorado has adopted these federal requirements and made state policy to acquire A/E design services, as well as some other professional services, through a QBS process identical to the federal process. 10 Denver Revised Municipal Code (D.R.M.C.) authorizes the Manager of Public Works to award contracts for the construction, reconstruction, or remodeling of general public improvements through an alternative competitive selection process other than to the lowest, responsive, qualified bidder, if the City's best interests are served by the use of such process. 11 This competitive selection process shall at a minimum include a written solicitation to notify potential qualified contractors. 12 The selection process should include seven components. 1. The nature of the work, including a general description of the scope, anticipated method of project delivery, significant standards and applicable legal and technical requirements, and a description of the anticipated form of the services agreement 2. A written description of the competitive selection procedure to be utilized to award a contract for the project work such as any responsiveness requirements, significant selection procedure milestones, and submission deadlines 3. A description of the minimum qualifications required, including all applicable prequalification rules and regulations, and a discussion of any other requirements 4. A written statement of the criteria and other factors to be used in evaluating each responsive submission, including but not limited to proposed fees, expenses or cost-saving considerations, ability of professional personnel, past performance on similar projects, ability to meet time and budget requirements, and projected workloads of the firm 5. A concept of the project as applicable 9 40 U.S.C C.R.S , et seq. 11 D.R.M.C (a). 12 D.R.M.C (b)(1). City and County of Denver P a g e 8

15 6. A brief narrative of review, evaluation, selection, and the contract negotiation process 7. A written notification of selection, ranking for negotiation, or rejection issued to each entity making a submission in response to the department's solicitation In undertaking any such competitive selection process, the Manager of Public Works has the right to reject any or all submissions. 13 Public Works Request for Proposals (RFP) Process Public Works procures outside A/E design services for capital projects using a process called request for qualifications/request for proposals (RFQ/RFP), or simply and commonly referred to as RFP. The RFP process encompasses multiple steps involving several City agencies. Public Works (management, CPM, PCO, and Contract Administration) City Attorney s Office Office of Economic Development, Division of Small Business Opportunity Auditor s Office (Auditor, Prevailing Wage if applicable) Department of Finance, Risk Management Office Mayor s Office (Mayor, Mayor s Office as City Council liaison) City Council Public Works manages all steps related to the selection of an A/E design consultant. The other agencies that contribute to the RFP process do so in a variety of ways, including through participation in preparation, review, and approval of the RFP documents before advertisement; reviewing proposals for compliance with the disadvantaged business enterprise rules; and reviewing and approving the final contract with the selected A/E design consultant. 14 The comprehensive RFP process starts with the initiation of a project and ends with a Notice to Proceed being granted to the selected A/E design consultant, after the contract is signed and becomes effective. 15 The Public Works RFP process incorporates some of the D.R.M.C. requirements regarding the alternative competitive selection process, as well as some of the federal and state guidance regarding the QBS process described earlier. Specifically, the RFP process includes five major steps. 1. Solicitation for A/E design services through a publicly advertised RFP and receipt of proposals, or statements of qualification 13 D.R.M.C (b) C.F.R. 26 regulates disadvantaged business enterprises participation in construction and design contracts financed in whole or in part by the U.S. Department of Transportation. Recipients of these funds, such as the City and County of Denver, shall ensure compliance with these requirements. Additionally, D.R.M.C., Chapter 28, Articles III and VII, set the requirements related to preventing discrimination against minority and women owned business enterprises and promoting small business enterprises participation in the City s contracts, including but not limited to the agreements for professional design services. 15 See Exhibit B for Public Works RFP Checklist showing the process steps. P a g e 9 Office of the Auditor

16 2. Evaluation of proposals without consideration of price and costs 3. Selection of at least three firms for interview 4. Selection of at least three of the most qualified firms based on interviews 5. Contract price negotiation with the top-ranked firm Public Works utilizes the RFP process to obtain A/E design services for a specific capital project or to secure on-call contracts with A/E consultants, which allow it to use design consultant services on an as-needed basis. Public Works issues task orders or work orders to access on-call services as project needs arise. City and County of Denver P a g e 10

17 SCOPE This audit assessed Public Works policies and processes related to outsourcing design work for capital construction projects. Included within this scope were any evaluations, analyses, policies, or other methodologies used by Public Works to guide its decisions regarding whether it should outsource construction design work or conduct the work inhouse utilizing City employees. OBJECTIVES The audit objectives were to: Assess Public Works controls over the process for selecting A/E design consultants Assess Public Works policies and practices for providing oversight of A/E design consultants Assess Public Works policies and practices for measuring the performance of A/E design consultants, and to determine if this performance information is utilized when selecting consultants for subsequent design contract work METHODOLOGY We utilized several methodologies to achieve the audit objectives. These evidencegathering and analysis techniques included, but were not limited to: Conducing numerous interviews with Public Works management and staff to obtain and understand Public Works policies and practices for outsourcing A/E design work Reviewing Public Works documents related to outsourcing A/E design work for a sample of capital projects Reviewing audits related to outsourcing A/E design work conducted by other government agencies Reviewing legal requirements regarding outsourcing A/E design work, including the City s municipal code and Public Works rules, regulations, and policies Testing Selection Committee activities on a judgmental sample of capital projects Researching best practices related to government outsourcing of A/E design work including but not limited to two interviews with subject matter experts with extensive knowledge of capital project outsourcing and management P a g e 11 Office of the Auditor

18 FINDING 1 Public Works Architectural and Engineering Design Consultant Selection Process Can Be Enhanced to Increase Accountability and Transparency Public Works contracts out a significant amount of the architectural and engineering (A/E) design work associated with its capital projects. The A/E design consultants that are hired by the City to perform the design work are selected through a competitive process that begins with the solicitation of a Request for Proposals (RFP). Public Works has developed a comprehensive checklist and a process flow chart for the RFP process that show the various steps taken within Public Works and by the other City agencies that are involved in preparation, review, and approval of the RFP documents and resulting contract. An evaluation team, referred to as a Selection Committee (SC), is created for each new RFP, and members of the team are responsible for evaluating proposals and ultimately recommending to the Manager of Public Works one consultant to perform the work. While the audit did not find actions that indicated process failures, the inherent risk in these processes warrants attention. Audit work revealed that the A/E design consultant selection process could be improved to enhance accountability and transparency in two areas. First, the criteria for choosing SC members and guidance regarding their activities are inadequate. Second, Public Works retains minimal documentation regarding the SC s activities and communications to management. In addition, audit work found that advisory opinions provided by the Denver Board of Ethics may not offer Public Works executive management with practical guidance to avoid the risk of the appearance of a conflict of interest. Responsibilities of Selection Committee When the need for outside A/E design services on a capital project triggers an RFP, the Project Manager (PM) is responsible for compiling an SC, on which the PM also serves as a member. For each RFP, an SC is formed and then tasked with four major responsibilities. 1. Assisting with weighting and finalizing proposal evaluation criteria, which is mainly adopted from the RFP criteria 2. Evaluating and scoring proposals and creating a shortlist of firms recommended for interview and oral presentation 3. Developing and recommending weighted criteria and questions to evaluate shortlisted firms during interviews and oral presentations 4. Conducting interviews, scoring and ranking firms, and recommending the topranked firm for the contract City and County of Denver P a g e 12

19 All recommendations made by the SC are subject to review and approval by members of Public Works upper management, specifically the Director of Capital Projects Management (CPM), the City Engineer, and the Manager of Public Works. Figure 2 summarizes the process for securing an A/E design consultant using an SC. Figure 2 Public Works Architectural and Engineering Design Consultant Selection Process 1 Project Manager selects and recommends members of Selection Committee and serves as a member 2 Director of Capital Projects Management and City Engineer approve and convey recommendations to Manager of Public Works through a memo 3 Manager of Public Works approves memo Selection Committee: Members individually evaluate and score proposals Consolidates scores and recommends short-listed firms 5 6 Project Manager communicates Selection Committee s recommendations To Director of Capital Projects Management Selection Committee: Members interview and individually score and comment on, and rank short-listed firms Consolidates scores and ranks and recommends contract with top ranked firm Key: # Steps in process Project Manager and Selection Committee activities Public Works management activities Source: Developed by Audit Team using information provided by Public Works. Inadequate Criteria and Guidance for Selection Committee Formation and Activities Audit work determined that Public Works has limited formal criteria for appointing individuals to serve on a new SC. Further, the SC itself lacks formal procedures for use by members when weighting, and at times modifying, selection criteria when evaluating RFP submissions. Despite the lack of formal criteria and guidance, Public Works does have some germane informal and draft procedures, which could be formalized. Such P a g e 13 Office of the Auditor

20 official guidelines could then be used for all new RFPs and followed consistently by future SCs throughout the various stages of the A/E design consultant selection process. Guidance for Appointing Selection Committee Members Is Limited New SC appointments are made for every new RFP that is released to solicit for A/E design consulting services. However, Public Works has not established any practical guidance for the member selection process. The PM assigned to the project, who will occasionally work with his or her supervisor together as a project management team, There is no formal criteria for City Selection Committees to retain consultants for City work. selects individuals to serve on the SC and recommends them to the Director of CPM. However, the PM does not have any established criteria by which to determine which types of individuals should be selected to serve on the SC. Once the PM selects SC members, he or she informs the Director of CPM, who then sends a memorandum through the City Engineer to the Manager of Public Works requesting that he or she approve the recommended SC members. It is in this memorandum where the only formal criterion for selection of SC members is documented. The memorandum includes a general statement indicating that the selection is intended to ensure the diverse interests of the City and Project Stakeholders are upheld in the selection of Consultant. Not only is this statement documented after the selection process has already taken place, but the language is general and does not provide adequate guidance regarding which sections within Public Works, other City agencies, or external stakeholders should be represented on the SC. Specific criteria would ensure that SC members have the appropriate knowledge and expertise necessary for the particular RFP. Public Works Does Not Have Detailed Procedures or Criteria to Guide Selection Committee Decisions Although the criteria published in the RFP serve as the main criteria in the SC evaluation of A/E design proposals, the SC members are often responsible for working with the PM to develop additional evaluation criteria, if needed, and to assign weight to these criteria before they evaluate the proposals. These individuals must evaluate and score the proposals, concur on the final scores, and develop the shortlist of firms that will be recommended for interview and oral presentations. They must also participate in developing the weighted criteria and the questions for the interview with the shortlisted candidates, conduct such interviews, rank the firms, and recommend the tap-ranked firm for the contract to Public Works management. Public Works currently does not have formal procedures to guide these activities. Such official guidelines would help with consistency, efficiency, and fairness in the consultant selection process. City and County of Denver P a g e 14

21 Some Existing Informal Drafts Could Be Formalized Although Public Works does not have formal procedures for SC members to use when evaluating candidates, they did provide the audit team with two draft documents containing detailed consultant selection procedures. The first document is a Microsoft PowerPoint slide deck, RFP Process for Project Managers and Selection Committee Members, September The document provides a detailed list of steps in the RFP process. The steps outlined include but are not limited to proposal initiation, SC formation including number of voting and non-voting members, SC roles and responsibilities, scoring methodology, and shortlist and interview processes. According to Public Works personnel, these slides were presented during a live training of CPM staff in the Fall of 2010 and were posted on the CPM share drive for future use and reference by staff. However, there is no indication that this slide deck is intended to serve as a formal set of procedures. The second document, provided by Public Works towards the end of audit work, titled Department of Public Works Engineering Division Policy Statement, appears to be a draft of proposed guidelines for the RFP process, which is pending management s review and approval. The document contains comprehensive guidelines on the establishment of the SC, weighted selection criteria, proposal review, a scoring system for proposal evaluation, and the interview of the shortlisted consulting firms. 16 These drafts should be revised and formalized for creation of and use by future SCs as official guidance regarding A/E design consultant selection. Public Works management should ensure that these procedures are understood and consistently implemented by PMs and SC members. Inadequate Documentation of Selection Committee Activities and Communications Our review indicated that Public Works does not maintain adequate documentation of SC activities and communications with Public Works management, evident in three areas. Public Works does not maintain adequate First, Public Works does not consistently documentation of Selection maintain documentation of scores and written Committee activities and comments made by individual SC members when evaluating project candidates. Second, communications. Public Works does not maintain documentation of communications between SC members and Public Works management regarding 16 Workload bonus points may be assigned upon approval of the Manager of Public Works. The intent is to encourage a variety of qualified professional service providers to submit proposals for City projects. If applicable, each proposal is initially assigned ten points. The points are then reduced at the rate of one point for every $500,000 of the City s spending authorization with the proposer on any City contract. The reduction excludes unfunded task orders related to on-call professional service contracts, but includes fully executed contract amendments that have been in effect within a twenty-four-month period prior to the date of proposal submittal. P a g e 15 Office of the Auditor

22 recommendations of shortlisted consultants and final recommendations. Rather, SC recommendations are communicated by the Director of CPM directly to other members of Public Works executive management, excluding the members of the SC. Third, these memoranda do not explain the rationale of decisions made by the SC and they are not designed to incorporate the rationale of management when there is disagreement with the SC s recommendations. Inadequate Documentation of Selection Committee Activities Public Works does not maintain documents regarding candidate evaluation activities performed by the SC, including weighted criteria for proposal evaluation, scoring sheets, weighted interview criteria, interview scores and comment sheets, the SC s consolidated scoring sheet, and the minutes of SC deliberations and discussions. The audit team obtained a sample of four projects to evaluate the sufficiency of documentation. 17 Of the sample projects, only one included all of the aforementioned documents. The other three projects were missing all scoring and comment sheets, as well as some documentation of Public Works management approval. These documents are important to the RFP process because they provide transparency and reduce the risk of or appearance of impropriety in the process of selecting A/E design consultants. Additionally, despite the confidential nature of the information contained in some of these documents, they can assist in defending the City in potential cases where consulting firms that were not selected, or any other parties, question or challenge the decision and the integrity of the selection process. Inadequate Documentation of Communication from Selection Committee to Public Works Management As described earlier, the recommendations made by the SC regarding weighted evaluation criteria and consultant recommendations are subject to review and approval by Public Works management, including the Director of CPM, the City Engineer, and the Manager of Public Works. Public Works does not have any documentation indicating direct communications from the SC to Public Works management with regard to the SC s recommendations. The only written communications in this regard are the memoranda sent by the Director of CPM, through the City Engineer, to convey the SC s recommendations to the Manager of Public Works for final approval. Except for the PM, none of the other SC members receives these memoranda. Having SC members included on the distribution of these memoranda would provide a control to ensure that SC decisions are reflected accurately. Further, Public Works practices regarding recommendations run contrary to best practices followed by the federal government, which publishes acquisition and procurement best practice policies and procedures in the Federal Acquisition Regulation (FAR). The FAR states that the SC must prepare a report for the selection authority making its recommendation and include a description of the discussions and evaluations conducted to allow the selection authority to review the considerations upon which the 17 The audit team selected these projects judgmentally for the audit review. City and County of Denver P a g e 16

23 recommendations are based. 18 Without providing this type of detail for consideration by the City Engineer and the Manager of Public Works, decisions are made based on limited information, resulting in a less transparent process. Public Works Does Not Document Management Disagreement with Selection Committee Recommendations Public Works management has a significant role in and approval authority over the A/E design consultant selection process, consistent with D.R.M.C. provisions. 19 However, in the event that Public Works management overrides a recommendation made by the SC, the override is not documented. Rather, the override is verbally communicated to the PM, who then verbally communicates the override to the rest of the SC. According to Public Works personnel, when management disagrees with an SC recommendation, the SC will reconvene to address management s concerns. While the audit did not find actions that indicated control overrides to the process, the inherent risk of an override based on the aforementioned issues indicates that such an occurrence may take place. Best practices recommend that any potential disapproval be documented. For example, FAR states that the final selection decision shall be made by the agency head or a designated authority; however, if the selection authority decides that the recommended firms are not qualified or if the recommendations are considered inadequate for any reason, the selection authority shall record the reasons and return the recommendations for appropriate revisions. 20 Accordingly, Public Works should require and maintain documentation of all management changes to recommendations made by the SC. Board of Ethics Advisory Opinions May Not Provide Practical Guidance to Avoid the Risk of the Appearance of a Conflict of Interest As discussed, Public Works enters into a large number of contracts with private contractors and consultants each year. The value of total annual contracts awarded ranged from $181 million in 2009 to an estimated amount of $75 million in The outsourced A/E design consulting work during the same three-year period amounted to approximately $42.8 million. 22 With such significant expenditures dedicated to outside consultants, the potential risk of conflicts of interest, nepotism, and the perception of impropriety in consultant selection exists inherently with this process. Public Works has 18 The Federal Acquisition Regulation provides guidance for federal procurement of goods and services. Part 36 contains criteria for construction and architect-engineer contracts. Subpart , Evaluation board functions, includes a review of current data files on eligible firms and responses to a public notice concerning the particular project. Boards hold discussions with at least three of the most highly qualified firms regarding concepts and the relative utility of alternative methods of furnishing the required services. Boards prepare a selection report for the agency head or other designated selection authority recommending, in order of preference, at least three firms that are considered to be the most highly qualified to perform the required services. The selection report includes a description of the discussions and evaluation conducted by the board to allow the selection authority to review the considerations upon which the recommendations are based. Id. 19 D.R.M.C FAR , Selection authority. 21 City and County of Denver, Mayor s 2012 Budget, page 544. The Better Denver Bond Program with a voter approved budget of $549.7 million significantly increased Public Works capital project activities in 2009 and Department of Public Works, see Introduction and Background for more information. P a g e 17 Office of the Auditor

24 taken appropriate action by seeking advisory opinions from the Denver Board of Ethics (Board) to diminish such risk. Audit work revealed that, in the last ten years, several members of Public Works executive management have sought advisory opinions from the Board when faced with a potential conflict of interest involving A/E design consulting firms that were going to be considered for City business. In all instances, the Board recommended that the individuals recuse themselves from the process. Ethics opinions may not provide practical guidance for avoiding the appearance of a conflict of interest. The Board s ethics opinions have included a variety of strategies by which executive management could avoid the appearance of a conflict of interest, including disclosing such interest to his or her appointing authority, not acting on the related contract in any way, refraining from attempting to influence the decisions of others involved, and working to ensure that the matter is assigned to someone without conflicting interests. While audit work did not find any instances of a violation of the Code of Ethics by Public Works officials, the advisory opinions themselves may not provide practical guidance with regard to the SC process and executive management s involvement therein. Transparency and accountability are paramount in maintaining the public trust. Enhanced documentation as discussed previously in this finding can help Public Works management avoid the impression of a potential conflict by the public, the media, and competing contractors. City and County of Denver P a g e 18

25 RECOMMENDATIONS 1.1 Selection Committee Guidance - Public Works should develop formal guidelines for appointing Selection Committee members. Such guidance should include a requirement for documentation of any changes to the Selection Committee members during the selection process. 1.2 Selection Committee Procedures - Public Works should develop formal procedures to guide Selection Committee activities, including the selection and interview criteria, proposal reviews and scoring, and interview and ranking of shortlisted firms. 1.3 Selection Committee Documentation - Public Works should maintain documentation of Selection Committee activities. The documentation should include scoring sheets related to proposal reviews, interview ranking and comment sheets completed by the Selection Committee members, minutes of Selection Committee meetings, and communications from the Selection Committee to Public Works management regarding any selection-related matters. 1.4 Selection Committee Management Changes - Public Works should require and maintain documentation of all management changes to recommendations made by the Selection Committee. P a g e 19 Office of the Auditor