Minority/Women Business Enterprise Program Performance Audit

Transcription

1 Minority/Women Business Enterprise Program Performance Audit December 2014 Audit Services Division City and County of Denver Dennis J. Gallagher Auditor

2 The Auditor of the City and County of Denver is independently elected by the citizens of Denver. He is responsible for examining and evaluating the operations of City agencies for the purpose of ensuring the proper and efficient use of City resources and providing other audit services and information to City Council, the Mayor and the public to improve all aspects of Denver s government. He also chairs the City s Audit Committee. The Audit Committee is chaired by the Auditor and consists of seven members. The Audit Committee assists the Auditor in his oversight responsibilities of the integrity of the City s finances and operations, including the integrity of the City s financial statements. The Audit Committee is structured in a manner that ensures the independent oversight of City operations, thereby enhancing citizen confidence and avoiding any appearance of a conflict of interest. Audit Committee Dennis Gallagher, Chair Maurice Goodgaine Leslie Mitchell Rudolfo Payan Robert Bishop Jeffrey Hart Timothy O Brien, Vice-Chair Audit Staff Audrey Donovan, Deputy Director, CIA, CGAP, CRMA Robyn Lamb, Internal Audit Supervisor Nancy Howe, Lead Internal Auditor, MPA, CRMA Anthony Lewin, Senior Internal Auditor Ronald F. Keller, Senior Internal Auditor, CIA, CFE You can obtain copies of this report by contacting us at: 201 West Colfax Avenue, Department 705 Denver CO, (720) Fax (720) Or download and view an electronic copy by visiting our website at: A

3 City and County of Denver 201 West Colfax Avenue, Department 705 Denver, Colorado FAX Dennis J. Gallagher Auditor December 18, 2014 Mr. Paul Washington, Director Office of Economic Development City and County of Denver Dear Mr. Washington: Attached is the Auditor s Office Audit Services Division s report of their audit of the City and County of Denver s Minority/Women Business Enterprise (M/WBE) program. The purpose of the audit was to determine whether the City's M/WBE program is effectively meeting its intended purpose. Throughout their review the audit team discovered that the Department of Small Business Opportunity (DSBO) cannot demonstrate whether the program is effective in meeting its intended purpose. It troubles me that stakeholders, including the public and firms in the construction and professional services industries, do not know whether this program is working. This could negatively impact public perception, not only regarding program effectiveness but about the integrity of the program itself. It is my deepest hope that DSBO will take a more proactive approach to outward reporting as well as developing performance measures. Our expectation is that DSBO will look inward to determine the efficacy of the M/WBE program and communicate those findings to interested parties. The reputation of this great City depends on programs like M/WBE being successful at what they are intended to accomplish for deserving program recipients. If you have any questions, please call Kip Memmott, Director of Audit Services, at Sincerely, Dennis J. Gallagher Auditor DJG/al cc: Honorable Michael Hancock, Mayor Honorable Members of City Council Members of Audit Committee Ms. Cary Kennedy, Deputy Mayor, Chief Financial Officer To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

4 Ms. Janice Sinden, Chief of Staff Mr. David P. Edinger, Chief Performance Officer Ms. Beth Machann, Controller Mr. Scott Martinez, City Attorney Ms. Janna Young, City Council Executive Staff Director Mr. L. Michael Henry, Staff Director, Board of Ethics To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

5 City and County of Denver Dennis J. Gallagher Auditor 201 West Colfax Avenue, Department 705 Denver, Colorado FAX AUDITOR S REPORT We have completed an audit of the City and County of Denver s Minority/Women Business Enterprise (M/WBE) program, administered by the Division of Small Business Opportunity (DSBO) within the Office of Economic Development. The purpose of the audit was to determine whether the City's M/WBE program is effectively meeting its intended purpose. This performance audit is authorized pursuant to the City and County of Denver Charter, Article V, Part 2, Section 1, General Powers and Duties of Auditor, and was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The audit identified weaknesses that inhibit DSBO s ability to demonstrate whether the program is effective in meeting its intended purpose. Program weaknesses were identified in two primary areas. First, audit work revealed that DSBO does not have sufficient program goals or corresponding performance measures in place to enable it to evaluate the success of the M/WBE program. Second, DSBO cannot provide assurance that the M/WBE program is serving its intended purpose. As a result, DSBO cannot demonstrate whether the program is positively impacting M/WBE-certified firms. Audit work determined that only a small percentage of M/WBE firms graduate from the program and that a high percentage of the City s M/WBE contract dollars are awarded to a small and concentrated number of program participants. These are key measures of success that currently cast doubt on the extent to which the program helps M/WBE firms grow by obtaining work on City-funded construction projects. We extend our appreciation to the Office of Economic Development, the Division of Small Business Opportunity, and the personnel who assisted and cooperated with us during the audit. Audit Services Division Kip Memmott, MA, CGAP, CRMA Director of Audit Services To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation.

6 City and County of Denver Audit Services Division REPORT HIGHLIGHTS Minority/Women Business Enterprise Program December 2014 The objective of this audit was to determine whether the City's Minority/Women Business Enterprise program is effectively meeting its intended purpose. Background The City and County of Denver's Minority/Women Business Enterprise (M/WBE) program is administered by the Division of Small Business Opportunity (DSBO) within the Office of Economic Development. The M/WBE program is one of six disadvantaged business programs administered by DSBO. It was developed to address identified disparity in the utilization of M/WBE firms on construction projects. DSBO certifies applicant firms as M/WBE and places certified firms in a directory that helps these firms compete for construction related contracting opportunities funded by the City, including contracting opportunities at DIA. Purpose The purpose of this audit was to assess the effectiveness of the M/WBE program. Auditors reviewed DSBO s methodologies for counting work towards M/WBE goals and other practices that provide assurance on program effectiveness. Highlights The audit found that DSBO cannot demonstrate whether the M/WBE program is effective in meeting its intended purpose, which was apparent in the following areas. First, DSBO has not established sufficient performance measures to ensure that the objective of the M/WBE program is being achieved. Audit work found that most of the M/WBE work is awarded to a small percentage of M/WBE certified firms and few firms have graduated from the program in recent years. Second, DSBO cannot provide assurance that the M/WBE program is serving its intended purpose because DSBO has not produced all required reporting information on the M/WBE program and some of the reporting that has been done is inaccurate. Further, the data maintained by DSBO is not sufficient to make an assessment about the health and quality of the M/WBE program because it is incomplete. Additionally, DSBO should enhance its complaint handling process and use lessons learned from recent complaint investigations to enhance its certification and renewal processes. For a complete copy of this report, visit Or Contact the Auditor s Office at

7 TABLE OF CONTENTS INTRODUCTION & BACKGROUND 1 SCOPE 9 OBJECTIVE 9 METHODOLOGY 9 FINDING 11 The Division of Small Business Opportunity Cannot Demonstrate the Efficacy of the Minority/Women Business Enterprise Program 11 DSBO Does Not Have Sufficient Performance Measures to Evaluate the Success of the M/WBE Program 11 DSBO Cannot Provide Assurance that the M/WBE Program Is Serving its Intended Purpose 18 RECOMMENDATIONS 27 AGENCY RESPONSE 29

8

9 INTRODUCTION & BACKGROUND The Division of Small Business Opportunity Administers Programs, Including the Minority/Women Business Enterprise Program, Intended to Help Small and Disadvantaged Businesses Grow by Providing Access to Government Contracting Opportunities For decades, federal, state, and local governments throughout the nation have implemented programs with the express intent of increasing the number and value of government contracts awarded to minority-owned business enterprises and womenowned business enterprises. 1 Affirmative action programs in the United States have focused on providing opportunity for minorities and women in three main areas: employment, education, and economic opportunity. Government programs have attempted to create and enhance economic opportunities by providing general support for small businesses and targeted support for minority- and women-owned businesses. These government programs are intended to provide assistance to minority and womenowned business enterprise (M/WBE) firms by increasing available opportunities for these firms to compete for government contracts. These programs are intended to help reverse the effects of direct or passive discriminatory practices. Notwithstanding the aggregate efforts of M/WBE programs at the federal, state, and local levels, both minorities and women remain disadvantaged in the marketplace. U.S. Census Bureau data show that minorities and women earn substantially less from their labor both as employees and as entrepreneurs than their non-minority male counterparts. 2 The City s disadvantaged business programs are administered through the Division of Small Business Opportunity (DSBO) within the Office of Economic Development (OED). OED s vision for DSBO is to promote the growth of Denver s small and disadvantaged businesses by providing access to government contracting opportunities. 3 DSBO s stated objective is to create an environment to help small businesses thrive through collaboration between public, private, and non-profit sectors. 4 DSBO administers six disadvantaged business programs, four of which are established in the Denver Revised Municipal Code (D.R.M.C.) and two of which are established in federal law. The four local programs are the Minority/Women Business Enterprise (M/WBE), Small Business Enterprise (SBE), Small Business Enterprise Concession (SBEC), and Emerging Business Enterprise (EBE) programs. The two federal programs are the 1 Research Brief: Taking Affirmative Action to Improve New York State s M/WBE Program, Office of the New York State Comptroller, September Ibid. 3 DSBO s vision statement was provided to auditors by the OED Director. 4 City and County of Denver Mayor s 2014 Budget Book, page 288. Page 1

10 Disadvantaged Business Enterprise (DBE) and Airport Concession Disadvantaged Business Enterprise (ACDBE) programs. 5 Following are descriptions of the M/WBE, SBE, SBEC, EBE, DBE, and ACDBE programs. M/WBE Program This program was created to enable the City to undertake specific activities to encourage utilization of businesses owned by women or minorities in the areas of construction, reconstruction, remodeling, and design. DSBO encourages such utilization by establishing goals for the participation of certified firms on City-funded construction and professional services projects. SBE Program This program was created to encourage City utilization of small businesses in the areas of construction, reconstruction, remodeling, design, and professional construction management. The current program is a Defined Selection Pool (DSP) program, which provides certified SBEs an opportunity to bid and compete as prime contractors on City contracts. Under a DSP program, a percentage of the City s construction and professional design contracts are designated for exclusive bidding by certified SBEs. Specific projects are made available to the defined selection pool, where certified SBE firms bid to perform all or most of the work as a prime contractor or prime subcontractor. City entities that have reserved projects through the SBE program include Public Works, Denver International Airport (DIA), General Services, Parks and Recreation, and Environmental Health. SBEC Program This program is designed to foster the development of small concessions businesses. The program provides concessionaires that do not have the necessary resources an opportunity to conduct business at DIA. EBE Program This program seeks to assist emerging businesses in obtaining City work through construction and professional services contracts. For this program, the City restricts project bids solely to EBEs and provides free technical assistance to participating firms during the bidding process. The assistance provided is designed to enable emerging business enterprises to gain the knowledge, experience, and resources needed to participate in the competitive process for contracts. DBE Program This federal program regulates the participation of disadvantaged businesses in construction and design contracts financed in whole or in part by the U.S. Department of Transportation (USDOT). Specifically, as a condition of receiving federal funds from USDOT, federal law requires recipients, such as the City and County of Denver, to implement a DBE program. The purpose of the DBE program is to maximize government transportation contracting opportunities for minority- and woman-owned businesses, or any other small business owner who can demonstrate social and economic disadvantage. ACDBE Program This federal program awards a percentage of airport concessionaire contracts to disadvantaged businesses in the food and 5 49 C.F.R. 23 and 49 C.F.R. 26. City and County of Denver Page 2

11 beverage, specialty retail, consumer services, and entertainment and amusement industries. 6 DSBO Structure and Responsibilities DSBO is overseen by a Director who is appointed by the Mayor. DSBO employees work at offices located downtown and at DIA. As of June 2014, DSBO employed sixteen full time employees (FTE) with a budget of approximately $1.4 million. This represents an increase in three FTE and approximately $290,000 from DSBO is funded through certification fees and general fund monies. 7 DSBO certifies firms as eligible for the six disadvantaged business programs and monitors these businesses for program compliance through its Certification and Compliance sections. As of October 15, 2014, DSBO s webpage listed 939 firms certified as eligible to participate in one or more program category totaling 1,910 individual certifications. 8 Certification Section DSBO s Certification Section assists disadvantaged businesses in obtaining certification in one or more of the City s programs by ensuring that the businesses have met established eligibility criteria. Once certified, Certification Section personnel place eligible businesses in a directory used by bidders to help them identify certified firms for work as prime contractors or as subcontractors in order to meet disadvantaged business enterprise project goals established for City contracts. This affords disadvantaged business enterprises with an opportunity to bid and work on City contracts, but certification itself does not guarantee receipt of a contract. Employees within the Certification Section also conduct training programs and certification workshops to help participants learn more about the procurement of construction-related contracting opportunities. Further, Certification Section personnel conduct community outreach activities to promote the disadvantaged business programs and contracting opportunities available through the City. Compliance Section Compliance Section personnel monitor construction projects that have M/WBE goals to ensure that prime contractors utilize certified firms listed for participation. The section also enforces adherence to applicable City ordinances, equal employment opportunity regulations, and federal guidelines, and ensures timely payments to certified firms performing work on City contracts. Denver s M/WBE Program Denver s M/WBE program is authorized pursuant to City ordinance and encourages the use of M/WBE firms by establishing goals for participation on City construction and professional services projects. The ordinance includes specific M/WBE program requirements including, but not limited to, reporting and implementing a mentor-protégé program. Businesses certified through Denver s M/WBE program must meet specific 6 The Auditor s Office will be conducting a performance audit of the ACDBE program to be issued in City and County of Denver Mayor s 2015 Proposed Budget Book, page The DSBO website only lists M/WBE, SBE, SBEC, and EBE certifications. Page 3

12 eligibility requirements. Any formal complaints brought forth against M/WBE-certified firms are investigated by DSBO. Figure 1 is a graphical representation of the number of M/WBE program participants as well as the amount of professional services and construction contracting dollars paid to program participants in 2007 through Figure 1 M/WBE Program Participation and Dollars Paid to Participants $100,000, Dollars Paid to M/WBE Participants $80,000,000 $60,000,000 $40,000,000 $20,000, M/WBE Program Participants $ Total Professional Services Total Construction M/WBE Participants Source: Developed by auditors using data from DSBO s 2007 through 2013 Annual Reports. Note: The notable increase in the amount of construction dollars paid to M/WBE participants from 2012 to 2013 is primarily due to the use of M/WBE-certified firms on the Denver International Airport s HTC project. Reporting The ordinance requires that DSBO monitor the M/WBE program and progress towards the annual program goal. The Director of DSBO must report quarterly and annually to the Mayor and City Council on the administration and operations of the program. Mentor-Protégé Program The ordinance requires that DSBO establish, facilitate, implement, and coordinate a mentor-protégé program to encourage majority and minority trade associations to work jointly to connect M/WBE protégés with wellestablished mentor firms. DSBO Complaint Process DSBO has formal procedures to record and track formal complaints and requests for reconsideration of DSBO certification decisions. In addition, DSBO tracks modifications of good faith efforts that are required when a firm does not meet its project goals, which are tracked in conjunction with complaints. Although we describe the records reviewed as complaints, it is more descriptive to refer to this group as a collection of complaints, concerns, or other issues that are important to record and City and County of Denver Page 4

13 track. When a complaint is received, DSBO records it in a tracking log and assigns it to DSBO personnel for follow-up and investigation. Between 2012 and 2014, DSBO received twenty-two complaints. Between 2012 and 2014, there were: Six third-party complaints or Colorado Open Records Act (CORA) requests Nine reconsiderations of certification denials or compliance issues Seven modification of good faith efforts related to goals M/WBE Program Eligibility Criteria As of October 15, 2014, Denver s M/WBE program consisted of 744 businesses certified as M/WBE eligible. To obtain M/WBE certification, both the owner of an applicant firm and the firm itself must meet certain eligibility requirements in the areas of ownership and business operations. Ownership Requirements For M/WBE certification, at least 51 percent business ownership must be held by socially and economically disadvantaged individuals, defined as women and minorities. Federal laws and the D.R.M.C. define a minority individual as one whose lifelong cultural and social affiliation is with one of the following four groups: Black/African-American, Hispanic-American, Native- American, and Asian-American. 9 An eligible owner must also manage and control the daily operations of the firm. This means that he or she must make both routine and major decisions on matters of management, policy, and operations. Ownership alone is not sufficient to establish management and control. The D.R.M.C. also requires that firms be certified only for the specific types of work in which the owner has the ability and expertise to manage and control the firm s operations and work. For example, an engineering firm cannot be certified in architecture unless the owner can demonstrate sufficient ability to manage and control work in the discipline of architecture. Additionally, to be M/WBE certified, an eligible owner s personal net worth cannot exceed $1.32 million, excluding equity in the owner s primary residence and ownership interest in the business enterprise. The City adopted this threshold to mirror the level USDOT applies to its DBE program. Finally, the owner of an applicant firm must be lawfully present in the United States, and cannot be a current official, officer, or employee of the City or a relative of such persons. Business Requirements To obtain M/WBE certification, an applicant firm must also meet specific eligibility requirements. For example, a firm must offer services in construction-related industries. Eligible firms are certified in specific work codes, such as landscaping, trucking, and engineering, as well as various areas of construction, such as masonry or concrete work. However, eligible firms may also 9 A minority may also be an individual subjected to individualized racial or ethnic prejudice or cultural bias because of his or her identity as a member of a group other than the groups established in the ordinance. If an applicant pursues certification under this claim, the D.R.M.C. requires evidence of individual social disadvantage. See D.R.M.C (35) (a) and (b). Page 5

14 provide support services, such as human resources or printing. Certifying firms in specific work codes helps prime contractors identify firms with the specific areas of expertise they need to complete the work under the contract. For example, two firms may be considered construction firms, but one may only perform digging or hauling services while another may be qualified to handle hazardous materials. In addition, firms must not exceed established size standards. Size standards are based on gross annual receipts, the number of employees, or other criteria applicable to the work codes for which the firm is certified or seeks certification. DSBO uses size standards established by the federal Small Business Administration (SBA) as guidelines for its M/WBE program. DSBO uses 50 percent of the applicable SBA professional services size standard and, until recently, used 50 percent of the applicable construction size standard. However, effective February, 18, 2014, DSBO increased the M/WBE program s construction size standard to equal 100 percent of the SBA established size standard. The Finding section of this audit report addresses DSBO management s decision to increase the M/WBE program s construction size standard given the low number of firms that have graduated from the program since Finally, M/WBE-certified firms must be independent and must demonstrate having been in business for a certain period of time. A firm is considered independent when its viability does not depend on its relationship with another business enterprise. The M/WBE program requires the firm to have been actively in business for at least six months. The M/WBE certification process involves DSBO Certification Section personnel reviewing the application form submitted online by an applicant firm for the ownership and business requirements and supplemental documents provided by the applicant to substantiate information represented in the application. DSBO must certify an M/WBE firm in one or more industry codes. DSBO uses nationally recognized commodity codes, based on the North American Industry Classification System (NAICS), to classify M/WBE program participants into industry types based on what a firm represents as its capabilities. 10 Although there are more than 19,000 established NAICS codes, DSBO utilizes only those NAICS codes applicable to the industry types that perform work on City-funded projects. 11 In addition, the SBA uses NAICS codes as a basis for its size standards, which are used by DSBO to determine eligibility of M/WBE program participants. In addition to reviewing a firm s application and supplemental documentation, Certification Section personnel schedule a site visit to verify the provided information and ascertain whether the business is under the daily control of a minority or woman owner. After initial certification, DSBO requires each M/WBE-certified firm to recertify annually that it continues to meet the program s eligibility criteria. The Finding section of this audit 10 NAICS codes originate from the United States Office of Management and Budget (OMB) and are used widely to classify business enterprises for the purpose of collecting, analyzing, and publishing statistical data related to the U.S. economy. 11 This number is as of the 2012 OMB NAICS Codes list located on the U.S. Census website at Census.gov. City and County of Denver Page 6

15 report addresses the certification process in light of our review of DSBO s M/WBE complaint-handling process. M/WBE Program Participation Goals As required by ordinance, the City establishes two types of goals for firms certified under the City s M/WBE program: annual goals and project goals. Annual goals are aspirational goals reflecting the annual aggregate participation of M/WBE-certified firms in City contracts. 12 Currently, the City s M/WBE program has an annual construction goal of 24 percent, meaning that in one year, 24 percent of all City construction project funds will be committed to M/WBE-certified firms. In addition, the City s annual goals for professional services, general services, and goods are 33 percent, 8 percent, and 5 percent, respectively. The annual goals to be awarded to M/WBE firms were established by DSBO based on recommendations from a disparity study. This study evaluates the amount of disparity prevalent in the City and is conducted by an independent thirdparty contractor, who is paid by the City. 13 Project goals, unlike annual goals, are established on a project-by-project basis by one of three goals committees, each of which is responsible for recommending M/WBE goals for different project types: General Construction, Heavy Highway Construction, and Professional Design and Construction Services. Each goals committee is made up of three minority representatives, three women representatives, and three majority firm representatives, all of whom are appointed by the Director of DSBO. The goals committees meet regularly, as required by DSBO rules and regulations, and recommend an individual project goal based on the nature of the work and the availability of certified firms that are qualified to perform the specific scope of work involved in the project. This goal recommendation is then provided to the Director of DSBO for approval. Once approved, the goal is included as a requirement in the project. If the prime contractor does not meet the established M/WBE goal, it may demonstrate that it made a good-faith effort to meet the goal. If the firm does not sufficiently demonstrate to DSBO that it made a good-faith effort, the firm may ultimately be subject to sanctions. DSBO s Program Management Software DSBO utilizes software, B2Gnow (B2G), to manage and track compliance for all of the City s disadvantaged business programs. DSBO also utilizes B2G to report on program performance. B2G houses information on certified firms and incorporates vendor access to make compliance with the disadvantaged business programs more efficient. The Finding section of this audit report addresses the integrity of the data stored in B2G. 12 The M/WBE annual goal is aspirational due to the legal prohibition on quotas for participation or setting aside contracts for certain certified firms. 13 There have been two disparity studies performed for the City, one by NERA Economic Consulting in 2006 and one by MGT of America in Page 7

16 Recent M/WBE Issue Related to Denver International Airport s Hotel and Transit Center Project During 2014, the local media raised questions regarding how the contract work performed by Burgess Services LLC (Burgess), an M/WBE-certified firm, and its subcontractors should be counted towards the M/WBE project goal on the Hotel and Transit Center (HTC) project at DIA. Burgess was awarded a contract related to the HTC project as an M/WBE-certified contractor estimated at $39.6 million. Subsequently, Burgess retained RK Mechanical, Inc. (RK Mechanical), which is not an M/WBE-certified firm, to perform approximately $23 million (58 percent) of the work. The media questioned whether the entire Burgess contract amount would be counted towards the City s M/WBE project goal when a substantial amount of the contracted work would be performed by a non-m/wbe-certified firm. Audit work confirmed that the monies paid to RK Mechanical are not currently being counted towards the M/WBE HTC project goal. In response to the media reports, the Mayor requested that OED conduct an internal investigation relating to the circumstances surrounding Burgess involvement in the HTC project. The investigation included assessing the City s process to certify, manage, and monitor the involvement of M/WBE-certified firms as prime contractors and subcontractors. Based on the investigation, OED submitted to the Mayor five recommendations relating to DSBO protocol and processes. In addition, the report outlined next steps relating to the investigation. Further, OED s internal review concluded that the monies paid to RK Mechanical should not be counted towards the M/WBE project goal; however, both the project s prime contractor and Burgess disagree with this determination. We are encouraged by these efforts to improve and tighten controls and help create a more effective program. 14 Because of the questions raised by the local media, we reviewed the controls that DSBO had in place over this particular contract and identified several weaknesses. We discuss these weaknesses in the Finding section of this report, using examples from the Burgess contract where applicable DSBO Performance Audit Report and 2013 Follow-Up In April 2011, the Auditor s Office issued a performance audit report that assessed DSBO s outreach and certification processes for programs administered by DSBO as well as the effectiveness of those programs. 15 The report included sixteen recommendations, of which DSBO agreed with fourteen. In September 2013, we issued a follow-up report addressing whether DSBO had implemented the 2011 audit s recommendations. 16 We reported that DSBO had not implemented eight of the sixteen recommendations. Our current audit report found weaknesses similar to those identified in 2011 and, therefore, repeats some of the prior report s recommendations. 14 The full report is available on the Office of Economic Development website. 15 Office of Economic Development Division of Small Business Opportunity Performance Audit,, City and County of Denver, April 2011, Auditor's Office website. 16 Office of Economic Development Division of Small Business Opportunity Performance Audit Follow Up Report, Office of the Auditor, City and County of Denver, October 2011, Auditor's Office website. City and County of Denver Page 8

17 SCOPE This audit assessed the effectiveness of the City's Minority/Women Business Enterprise (M/WBE) program administered by the Division of Small Business Opportunity (DSBO) within the Office of Economic Development (OED). Audit work encompassed policies and practices in place in 2013 and prior years, as well as those that took effect on April 1, 2014, under an updated City ordinance. In addition, due to concerns related to M/WBE work on the Hotel and Transit Center (HTC) project at Denver International Airport, we reviewed relevant DSBO controls in the context of this project. The City s contract procurement processes were excluded from audit review. In addition, while M/WBE goals apply to City-funded construction projects and professional services contracts, audit work focused on construction projects rather than on professional services contracts. OBJECTIVE The objective of this engagement was to determine whether the City's M/WBE program is effectively meeting its intended purpose. To address this objective, the audit team sought to determine or evaluate: Graduation criteria and graduation rates in the context of the M/WBE program s mission and objectives DSBO's policies and practices for aligning the mission, objectives, and goals of the M/WBE program DSBO's process for addressing complaints regarding M/WBE-certified firms DSBO's policies and practices concerning counting work towards the M/WBE program s annual and project goals DSBO s measures for evaluating the impact and public outcomes from M/WBE program activities and expenditures The dollar amount of the Burgess Services LLC contract to be counted towards the HTC project s M/WBE goal METHODOLOGY We utilized the following methodologies to achieve the audit objective: Reviewed relevant City ordinances and executive orders to gain an understanding of the City's roles and responsibilities related to the programs for enhancing minority- and woman-owned enterprise utilization in City construction, professional design, and construction services. Specifically, we reviewed Sections through and Sections through of the Denver Revised Municipal Code. Page 9

18 Interviewed DSBO personnel to understand roles and practices Identified and reviewed M/WBE program audits completed by other municipalities to identify risk areas Assessed the implementation status of audit recommendations made in the Auditor s Office 2011 audit report of the Division of Small Business Opportunity 17 Traced the dollar amounts and percentages of M/WBE program participation reported in DSBO s 2010 through 2012 Annual Reports and draft 2013 Annual Report to supporting data Determined how construction, professional design, and construction services work is counted towards the M/WBE project and aspirational goals, and tested a sample of projects Assessed DSBO s M/WBE aspirational goals by trending them and comparing them to determine whether they have been met Evaluated DSBO s use of NAICS codes to categorize M/WBE-certified firms by capability Determined the number of M/WBE-certified firms that have graduated from specific NAICS codes or from the program Determined the number of M/WBE-certified firms that have obtained work on projects with M/WBE project goals Evaluated the goals DSBO has set to demonstrate M/WBE program efficacy Researched common fraud schemes related to M/WBE programs and strategies for mitigating such fraud risks Used work performed by the Auditor s Office Special Investigator, who had reviewed two third-party complaints regarding the M/WBE program to assess DSBO s process for addressing complaints Reviewed the controls established by DSBO to ensure that the Burgess subcontract for work performed on the HTC project was properly counted towards the HTC project s M/WBE goal 17 Office of Economic Development Division of Small Business Opportunity Performance Audit,, City and County of Denver, April 2011, Auditor's Office website. City and County of Denver Page 10

19 FINDING The Division of Small Business Opportunity Cannot Demonstrate the Efficacy of the Minority/Women Business Enterprise Program The Division of Small Business Opportunity s (DSBO s) Minority/Women Business Enterprise (M/WBE) program is intended to help disadvantaged construction and professional services firms grow in size and capacity so they can compete effectively in the larger marketplace. Accordingly, some City-funded projects are required to identify a goal that establishes a certain percentage of the overall work to be completed by M/WBEcertified firms, providing them with dedicated opportunities to work on City projects. To identify available firms, DSBO provides a directory of certified firms, which have met certain eligibility criteria. This disadvantaged business program is premised on the concept that government has a compelling interest in mitigating discrimination and the resulting disparity in the construction and professional services contracting marketplace. What the program is trying to achieve is laudable, and therefore it is critically important that the City be able to demonstrate the effectiveness of the program s efforts. However, we found that DSBO cannot demonstrate whether the program is effective in meeting its intended purpose. Program weaknesses were identified in two primary areas. First, audit work revealed that DSBO does not have sufficient performance measures in place to evaluate the success of the M/WBE program. Second, DSBO cannot provide assurance that the M/WBE program is serving its intended purpose. The absence of reliable and focused program performance data is especially problematic for this type of program from a public perspective. When stakeholders are not informed regarding whether a program is effective, this can negatively impact perceptions and confidence in the integrity of the program. Two specific findings from the audit that do not inspire confidence in the program s efficacy are the fact that very few firms are graduating from the program and that all of the available M/WBE work is being awarded to a small percentage of certified firms. DSBO Does Not Have Sufficient Performance Measures to Evaluate the Success of the M/WBE Program We found that DSBO has not established sufficient program goals to ensure that the objective of the M/WBE program is being achieved. 18 Further, DSBO has not established sufficient performance measures to assess how well the M/WBE program is achieving the few goals that have been established for the program. Finally, two current characteristics of the program that all of the available M/WBE work is being awarded to a small percentage of M/WBE-certified firms and that very few M/WBE-certified firms graduate from the program indicate that DSBO should establish a broader set of goals to clarify what program outcomes are most important. 18 In this section, goal refers to what DSBO aims to achieve through the M/WBE program. This is in addition to the annual aspirational goal and project goals, which are expressed as percentages of M/WBE participation. Page 11

20 M/WBE Performance Measures Do Not Address the Program s Objective Although DSBO has established vision and objective statements for the Division as a whole, these statements are broad by nature, and an overarching statement of the intent of the M/WBE program has not been formalized. Further, DSBO has not established concrete objectives and goals for the M/WBE program that align with the program s overarching mission. Additionally, DSBO has not developed a balanced set of performance measures to provide information on the program s performance and effectiveness. DSBO s vision statement is: Through its operation of the Division of Small Business Opportunity, OED promotes the growth of Denver s small and disadvantaged businesses by providing access to government contracting opportunities. We offer six types of business certifications.with certification, contracting opportunities are available on city and federally-funded projects. 19 DSBO s stated objective is to create an environment to help small businesses thrive through collaboration between public, private, and nonprofit sectors. 20 In addition to these broad guiding principles, the M/WBE program has few goals and performance measures established to carry out the program with more specific direction. The goals that have been established do not consistently align with the stated vision and objective and do not measure program effectiveness. DSBO s only specific goals and performance measures are defined in the Mayor s Budget Book. In the 2014 Budget Book, there are four measures, including the annual goal achievement for prior years, the number of contracts with an M/WBE goal, the number of certified firms, and the average number of days it took to certify a firm as M/WBE eligible. 21 A common thread throughout the stated vision and Performance measures established for the M/WBE program do not address the objective of helping M/WBE firms grow in capacity such that they can compete on non City projects. objective, as well as through the mission of the M/WBE program as articulated by the OED Director, is the idea of providing contracting opportunities to M/WBE firms to help them grow so they can ultimately better compete for work on non-city projects. However, none of DSBO s measures in the Budget Book directly align with the program s stated vision and objective. In addition to the measures not aligning with the program s overall intent, DSBO has not developed a balanced set of performance measures to provide information on the program s performance and effectiveness. For example, although annual goal achievement is a fundamental goal of the M/WBE program, it does not provide information on the health or growth of M/WBE-certified firms. The same concept applies to two other measures: the number of certified firms and how long it takes to complete the certification process. Although these measures provide useful information about the 19 DSBO s vision statement was provided to auditors by the OED Director. 20 City and County of Denver Mayor s 2014 Budget Book, page Ibid. City and County of Denver Page 12

21 program, they indicate a focus on getting firms into the certification program, rather than on the health of the program and whether it is meeting the overall intent. We evaluated DSBO s performance measures using a handbook developed by the State of Arizona. 22 The handbook contains Arizona s strategic planning model, a component of which is performance measure development. The 2011 handbook still serves as that state s guide to strategic planning and performance measure development. According to the handbook, strategic planning begins with the development of an agency s mission, objectives, and goals. Once these are in place, the agency should develop and formally track a balanced set of results-based performance measures to gauge its success in meeting the established program goals. The Arizona Handbook defines five performance measure types. Each category answers a different question, but the measures are best utilized in combination to analyze program results. Table 1 summarizes these five categories. Table 1 Performance Measurement Types Measure Type What Does it Measure? Generic Examples Examples for Certification Programs Inputs The amount of resources needed to provide particular products or services Labor, materials, equipment, supplies, or demand factors such as target populations Number of eligible firms, number of firms seeking certification, number of applications received Outputs The amount of products or services provided Workload measures Number of applications processed Outcomes Whether services are meeting proposed targets Percentage increase or decrease in a specific goal Percentage increase in construction contract dollar amount committed to certified firms Efficiency Productivity Ratio of outputs per unit of time, cost per unit of output Turnaround time, cost per application processed Quality The effectiveness in meeting the expectations of customers and stakeholders Reliability, accuracy, responsiveness Number of errors in program performance reports Source: Arizona Office of Strategic Planning and Budgeting s 2011 Managing for Results Handbook. Note: This information was also presented in our 2011 audit of the Office of Economic Development s Division of Small Business Opportunity. 22 Arizona Office of Strategic Planning and Budgeting, 2011 Managing for Results Handbook. Page 13

22 Government programs should have a clear mission as well as objectives and goals that align with the mission. After a mission, objectives, and goals have been established, the agency should develop a system to define, measure, and track progress. This especially applies to those in which it s difficult to clearly measure success or effectiveness, such as the M/WBE program. During the audit, the OED Director discussed M/WBE goals that DSBO could consider, citing total revenue of certified firms, bonding capacity, and useful work as examples. We recommend that DSBO define and formalize a clear mission and objectives to capture what the M/WBE program is working to achieve. DSBO should then develop goals for the M/WBE program that align with the mission and objective of DSBO and a varied set of performance measures that provide information on progress towards the goals. M/WBE Annual and Project Goals Are Not Linked We found that individual project goals are not established in a manner that will help ensure the annual aspirational M/WBE goal is met and that DSBO does not review or update the annual M/WBE goal. The annual goal is a core measure of success for the M/WBE program. Aligning project goals with the annual goal could help DSBO consistently meet its annual goal. M/WBE project goals are established based on the availability of M/WBE-certified firms to perform the type of work needed for the individual project. The number of certified firms available varies according to the type of work needed. If multiple certified firms can perform the work, the project goal may be set higher than if only a few can perform the work. Although this approach has validity, it does not link to achievement of the annual goal. DSBO s progress towards meeting the annual goal should be considered when a project goal is established to ensure a better chance of achieving the annual goal. When M/WBE goals are being set for individual projects, DSBO does not consider progress toward meeting the annual M/WBE goal. The M/WBE ordinance clearly requires that DSBO consider the annual goal in setting project goals. Specifically, it requires the director to assign a unitary project goal for M/WBE utilization based upon the availability of M/WBE firms to perform the anticipated work and the City's progress towards meeting the annual goal. The ordinance further states that the goal may vary from contract to contract consistent with meeting the appropriate overall annual goals, and requires that DSBO establish a methodology for setting the project goals, which shall consider to the extent applicable, the effect on the annual goal achievement of the varying levels of availability of MBEs and WBEs among industry groupings associated with individual projects. 23 Second, DSBO does not analyze and update the annual goal each year. Once established, DSBO does not assess the annual goals for possible adjustment, and instead used the goals suggested in the disparity studies that were performed in 2006 and D.R.M.C., (a)(1). City and County of Denver Page 14

23 The annual goals have mirrored the goal recommended in the disparity study, and have not been adjusted to reflect changes that may have occurred over the years. 24 DSBO does not update the annual M/WBE goal each year to account for changes in economic conditions and number of available firms. The ordinance governing the City s M/WBE program requires that the DSBO Director establish an annual goal for M/WBE utilization. 25 Further, it requires that, as a basis for the establishment of the annual goal, the director determine the present availability of all M/WBEs doing business in the City and recommend the annual goal to the City Council for its review. 26 The disparity study is one tool available for accomplishing this requirement. Although not prohibited by ordinance, retaining the same annual goal for multiple years does not account for changing industry and economic conditions. Additionally, language in the ordinance indicates that annual participation goals should be flexible to adapt to changing conditions. By keeping the M/WBE annual goal constant over multiple years, DSBO is neither accounting for changes in economic conditions over the life of the study nor the fact that the data in the disparity study regarding availability of eligible firms becomes increasingly dated over time. Specifically, the 2006 disparity study was based on data from 2005, so DSBO s M/WBE annual goals up to 2013 were based on 2005 data. The same situation will apply to the use of the 2013 disparity study: the M/WBE annual goals in 2016 and 2017 will not be based on current data. We noted both of these conditions in our 2011 audit. Specifically, we noted that DSBO does not evaluate and adjust the annual goal, and we discussed the lack of alignment in the annual and project goals. In that audit, we recommended that DSBO assess the annual goals on a yearly basis and determine whether they should be adjusted. At the time of our audit follow-up work, DSBO had not implemented this recommendation, even though DSBO agreed with the recommendation. The fact that the annual goals are still not being consistently met underscores the importance of evaluating the linkage between the annual and project goals. By better linking annual participation goals to project goals, the annual goals will be more meaningful. Therefore, we recommend that DSBO evaluate how it might better align M/WBE project goals and the annual aspirational goal. As it relates to establishing project goals, this should include adding a mechanism for considering the annual goal in the methodology for setting project goals, as required by ordinance. As it relates to reviewing and establishing the annual goal, DSBO should develop a methodology for adjusting the annual aspirational goal based on economic conditions, such as the level of planned City-funded construction, and availability of certified firms. 24 The 2006 disparity study recommended a 22 percent goal for construction and 15 percent for professional services. The 2013 disparity study recommended a 24 percent goal for construction and a 33 percent goal for professional services. 25 D.R.M.C., (a). 26 D.R.M.C., (b). Page 15