REQUEST FOR PROPOSAL FOR THIRD-PARTY EVALUATION SERVICES ISSUED NOVEMBER 25, 2014

Transcription

1 REQUEST FOR PROPOSAL FOR THIRD-PARTY EVALUATION SERVICES ISSUED NOVEMBER 25, 2014 Proposal must be received no later than 3:00 P.M. Pacific Standard Time (PST) December 19, 2014 ABSOLUTELY NO EXCEPTIONS An Equal Opportunity Employer Auxiliary Aids and Services are Available upon Request to Individuals with Disabilities 1

2 THIRD-PARTY EVALUATION SERVICES TABLE OF CONTENTS 1.0 REQUEST FOR PROPOSALS INTRODUCTION Solicitation Estimated Funding Submittal of Proposals RFP Schedule and Timeline Respondents Orientation Notice of Intent to Submit a Proposal Statement of Qualifications (SOQ) RFP Questions Addenda to this RFP Right to Cancel Termination Due to Availability of Funds BACKGROUND INFORMATION Organizational Overview The Workforce Innovation Fund (WIF) WIF Guiding Principles SDWP Governance Roles and Responsibilities of SDWP Roles and Responsibilities of the Contractor PROGRAM DESCRIPTION Breaking Barriers in San Diego WIF Program Design Background Target Population WIF Breaking Barriers in San Diego Evaluation Services Technology System Requirements REQUIRED PROPOSAL FORMAT AND CONTENT Page Limits Formatting Requirements Title Page Table of Contents Executive Summary Proposal Content: Organizational Capacity and Experience (Total 25 Points) Organizational Capacity and Resources (5 Points) Project Experience and Qualifications (15 Points) Project Team (5 Points) Proposal Content: Approach to Evaluation Design and Implementation (Total 45 Points) Approach to Evaluation Research Design (ERD) for RCT Study (15 Points) Approach to Implementation Study (10 Points) Approach to RCT Impact Study (10 Points) Approach to Cost Effectiveness Study (10 Points) Project Work Plan (Total 5 Points) Proposed Budget (Total 10 Points) FINANCIAL REQUIREMENTS Financial Management Structure Budget Information Payment Structure Subcontractors EVALUATION PROCESS AND CONTRACT AWARD 20 2

3 6.1 Compliance Review Evaluation Criteria Selection Appeals Process and Access to Evaluation Information Board Action Contract Award Negotiation/Contract Conflict of Interest Ex-Parte Communication Cooling Off Period Intellectual Property Communications Insurance Contract Terms and Litigation Warranty Funding Warranty Work for Hire Restriction on Disclosure ATTACHMENTS 26 3

4 NOTIFICATION OF REQUEST FOR PROPOSALS FOR THIRD-PARTY EVALUATION SERVICES NOVEMBER 25, REQUEST FOR PROPOSALS INTRODUCTION The San Diego Workforce Partnership, Inc. (SDWP) is issuing this Request for Proposals (RFP) to interested and qualified applicants to provide independent third-party evaluation services in support of SDWP s Breaking Barriers in San Diego project funded through the Department of Labor s (DOL) Workforce Innovation Fund (WIF). On September 29, 2014, SDWP was awarded $6 million in WIF funds through a competitive grants process to provide supported employment services to Temporary Assistance for Needy Families (TANF) participants and other low-income individuals impacted by disabilities. SDWP will be evaluating the impact of this program through a multi-year Randomized Control Trial (RCT) evaluation design. The period of performance for the agreement to be awarded as a result of this RFP is eight (8) months beginning February 1, 2015 and ending September 30, Based on the DOL acceptance and approval of the Evaluation Research Design (ERD) plan, need, performance, and funding availability, this agreement will extend through September 30, 2019 for a total period of performance of fifty-six (56) months. Respondents are invited to submit a proposal in accordance with the instructions indicated in this RFP. This RFP will provide potential Respondents with background information and describe the desired services, guidelines for proposals, and the contractor selection process. 1.1 Solicitation SDWP is seeking to procure the services of a third-party evaluation team that will lead the development of the Evaluation Research Design, Implementation Study, RCT Impact Study, and Cost-Effectiveness Study in collaboration with SDWP and the National Evaluation Coordinator (NEC) selected by the DOL for the WIF projects. SDWP hereby solicits proposals using a competitive bid process to select a contractor to provide thirdparty evaluation services under the DOL WIF program. Eligible applicants include: For-profit organizations; Non-profit organizations; Public agencies; and/or A collaboration of these organizations. A consortium, joint venture, or collaboration of organizations with complementary skills and experience is permitted to respond to this RFP, however, the proposal must clearly demonstrate that all contractual responsibility rests solely with one legal entity and that the proposed arrangement would enable the Respondent to furnish efficient quality for SDWP. This RFP contains the requirements that Respondents must meet in order to submit a responsive proposal. A responsive proposal is one that complies with all format and content requirements of the RFP. This RFP provides information regarding the format in which proposals must be submitted, the 4

5 requirements that must be met to be eligible for consideration, the Respondents responsibilities, and the documents that must be included. SDWP is committed to equal opportunity in its contracting process. 1.2 Estimated Funding Funding for third-party evaluation services resulting from this RFP will not exceed $946,348 over the entire life of the project. Respondents must note funding will not exceed $149,428 during the planning period of this project (October 1, 2014 September 30, 2015). The remaining funds are contingent upon approval of the ERD by the DOL. The not-to-exceed costs of evaluation services are shown below: Period of Performance Total Available Funding February 1, September 30, 2015 $149,428 October 1, September 30, 2019* $796,950 Overall Funding Total $946,378 * Funding from 10/1/2015 9/30/2019 is based on the DOL acceptance and approval of the ERD, need, performance, and funding availability. 1.3 Submittal of Proposals SDWP must receive the proposal(s) marked (Name of Agency) Proposal no later than 3:00 P.M., PST, Friday, December 19, Submit one (1) proposal with original signatures, seven (7) copies, and one (1) electronic PDF copy on a USB flash drive to: Linda Weber, Procurement Advisor 3910 University Avenue, Suite 400 San Diego, CA Late proposals will not be accepted. SDWP will provide a receipt verifying the time and date the proposal is received. If Respondents send their proposals via U.S. Postal Service, UPS, or any other type of delivery service, Respondents are responsible for requesting proof of delivery date and time from their chosen carrier. 1.4 RFP Schedule and Timeline Beginning November 25, 2014, the RFP will be available to download from SDWP s website If you are unable to download the RFP please contact: Linda Weber, Procurement Advisor 3910 University Ave., Suite #400 San Diego, CA breakingbarriersrfp@workforce.org 5

6 Timeline: Activity Date Issue RFP November 25, 2014 Respondents Orientation December 4, 2014 Last Day to Submit Questions December 9, 2014 Last Day Answers Posted on Website December 12, 2014 Notice of Intent to Bid Submission December 12, 2015 (by 5:00 P.M., PST) Required Statement of Qualifications (SOQ) Submission December 19, 2014 (by 3:00 P.M., PST) Proposals Due by 3:00 P.M., PST December 19, 2014 Oral Interviews & Evaluation Complete January 9, 2015 Workforce Investment Board Approval January 15, 2015 San Diego Consortium Policy Board Approval January 23, 2015 Effective Date of Contract February 1, Respondents Orientation SDWP will provide an optional Respondents Orientation to present the RFP requirements and address questions from prospective Respondents. All questions posed during the Respondents Orientation will be compiled and posted on SDWP s website within three (3) business days. No other communication besides Q & A is allowable as per Section 1.8 RFP Questions. The Respondents Orientation will be held on Thursday December 4, 2014 from 11:00 A.M., PST, to 1:00 P.M., PST, at the SDWP office located at 3910 University Avenue, San Diego, CA 92105, on the third floor in the Julie Benedict Community Room. Attendance to the Respondents Orientation is not required to submit a response to this RFP. 1.6 Notice of Intent to Submit a Proposal A Notice of Intent to Submit a Proposal - Attachment A is requested of all organizations planning to submit for the Third-Party Evaluation Services RFP. The Notice of Intent to Submit form is attached and will also be available at the Respondents Orientation. Providing a Notice of Intent to Submit does not commit the Respondent to submit a proposal. All Notices of Intent to Submit are requested on or before 5:00 P.M., PST, Friday, December 12, 2014 to: 6

7 1.7 Statement of Qualifications (SOQ) Jacqueline Collins, Adult Programs Team Manager 3910 University Ave., Suite #400 San Diego, CA SDWP is using a Statement of Qualifications (SOQ) to establish a Respondent s qualifications to bid for workforce development funds. The submission of a SOQ determines an organization s legal, administrative, and fiscal capacity to meet SDWP, state, and federal government requirements. The SOQ places organizations on the Qualified Agencies List (QAL), which allows the organization to be eligible to respond to RFPs and to be awarded a contract with SDWP. Completion of the SOQ is a required step in the process. Organizations that have not submitted the required SOQ documents within the past twelve (12) months must submit a new SOQ. SDWP encourages the participation of Respondents who are certified as small businesses, minorityowned firms, women s business enterprises, Emerging Business Organization, Disabled Veteran Business Enterprise, and Disadvantaged Business Enterprise whenever possible. Such certification must be documented in their SOQ. One (1) original and one (1) copy of all SOQ documents are due no later than 3:00 P.M., PST, Friday, December 19, 2014 to: Attn: Linda Weber, Procurement Advisor 3910 University Avenue, Suite 400 San Diego, CA Detailed information and the necessary forms are available for download from SDWP s website If you have difficulty or problems downloading the document please contact: Linda Weber, Procurement Advisor at (619) or breakingbarriersrfp@workforce.org. 1.8 RFP Questions To ensure a fair and objective evaluation, questions related to the RFP that arise after the Respondents Orientation must be submitted via to: breakingbarriersrfp@workforce.org. Written questions will be accepted through Tuesday, December 9, Questions received after the deadline will not be answered. Written responses to questions received by SDWP will be posted on the website, within three (3) business days of the day the question was submitted. The last date answers will be provided to questions is Friday, December 12, It is the Respondent s responsibility to check the website on a regular basis for updated information and written responses to all questions submitted. NOTE: With the exception of the Respondents Orientation, oral questions will NOT be accepted. Only the responses on the website are considered clarifications to the instructions contained in this RFP. In the event that responses modify any of the terms, conditions, or provisions of this RFP, documentation will be given via a subsequent amendment to the RFP. Respondents are warned that no other individuals are to be contacted in this regard. No other sources of responses or clarification are considered valid. 7

8 Respondents are strictly prohibited from contacting members of SDWP Board of Directors (Board of Directors), Workforce Investment Board (WIB), San Diego Consortium Policy Board (Policy Board), Adult Programs Committee (APC), San Diego Youth Council (Youth Council) or SDWP staff other than as stated in Section 6.7 Conflict of Interest. 1.9 Addenda to this RFP At the discretion of SDWP, if it becomes necessary to revise any part of this RFP, an addendum will be posted on SDWP s website, Respondents are responsible for checking the website frequently to remain informed about the procurement process, receive addenda to the RFP, read responses to questions, and remain updated on other information that may affect this RFP Right to Cancel SDWP reserves the right to delay, amend, reissue, or cancel, all or any part of this RFP at any time without prior notice. SDWP also reserves the right to modify the RFP process and timeline as deemed necessary. This RFP does not commit SDWP to accept any proposal, nor is SDWP responsible for any costs incurred by the Respondents in the preparation of responses to this RFP. SDWP reserves the right to reject any or all proposals, to accept or reject any or all items in the proposal, and to award the contracts in whole or in part as is deemed to be in the best interest of SDWP. SDWP reserves the right to negotiate with any Respondent after proposals are reviewed, if such action is deemed to be in the best interest of SDWP Termination Due to Availability of Funds When funds are not appropriated or otherwise made available by SDWP to support continuation of this contract, this contract shall be cancelled as of the effective date set forth in the termination notice and Contractor shall be reimbursed for the reasonable value of any nonrecurring cost incurred but not yet recovered under this contract. 2.0 BACKGROUND INFORMATION 2.1 Organizational Overview SDWP is a 501(c)(3) tax-exempt organization chartered by the County and the City of San Diego to fund job training programs in the San Diego region. The organization s primary funding is allocated by the Department of Labor under the provisions of the Workforce Investment Act (WIA). SDWP serves as a source of information on the region's current and projected labor market. Research specialists analyze workforce needs and trends in San Diego County, offering jobseekers, employers, educators, trainers, and researchers data on topics such as wages and salaries, occupational projections, employment statistics, and San Diego County s economy. The organization s mission is to empower job seekers to meet the current and future workforce needs of employers in San Diego County. The vision of SDWP is that every business in our region has access to a skilled workforce and every job seeker has access to meaningful employment. Our community will view us as the leader for innovative workforce solutions, which add great value to our region. For additional information, please visit 8

9 2.2 The Workforce Innovation Fund (WIF) The Workforce Innovation Fund (WIF) is one of several Federal grant programs in which grantor agencies fund projects that seek to use evidence-based practices to design interventions that may result in better programmatic outcomes. All WIF projects include an independent third-party evaluation. The WIF supports innovation at the systems and service delivery levels. It is believed that innovation at the systems level where policies, organizational structures, planning processes, performance measurement, procurement, investment priorities, and information management systems reside is necessary to support service delivery strategies that result in better outcomes and lower costs. In support of this goal, the DOL seeks new ways to remove administrative, statutory, and regulatory barriers and enable a closer alignment and integration of agencies and programs with overlapping missions and clients. For additional information, please visit WIF Guiding Principles The overarching goals of DOL s WIF program are to: Deliver services more efficiently and achieve better outcomes, particularly for vulnerable populations; Support both system reforms and innovations that facilitate cooperation across programs and funding streams in the delivery of client-centered services to jobseekers, youth, and employers; and Emphasize building knowledge about effective practices through rigorous evaluation and translating lessons learned into improved labor market outcomes, the ability to bring such practices to scale in other geographic locations and increased cost efficiency in the broader workforce system. A list of reference materials to aid Respondents is available in Reference Materials Attachment B, as well as a glossary of terms available in Glossary of Terms Attachment C. 2.4 SDWP Governance SDWP, under the leadership of the Board of Directors, WIB, and the Policy Board, is responsible for the fiscal and programmatic administration of federal employment and training funds for the City of San Diego and County of San Diego under a Joint Powers Agreement. SDWP Boards provide the framework for administering federal, state, and local funding to provide workforce development activities that increase individuals earnings and skills, and are responsive to business needs. The Policy Board is comprised of two San Diego City Council members, two San Diego County Supervisors, and one business representative appointed by the United Way. The Policy Board oversees funding and policy development under WIA. The Policy Board appoints the members of the WIB. The WIB is a federally mandated board that shares responsibility with the Policy Board for overseeing funding and policy development under the WIA. The board's expertise is in labor market conditions and the development of effective responses to regional economic needs. The Board of Directors represents an array of San Diego s booming industries, including healthcare, life sciences, and technology. This board is responsible for the oversight of SDWP operations. 9

10 There are two other deliberative bodies that oversee SDWP s activities. The Youth Council, a subcommittee of the WIB, is the primary body providing expertise in youth policy and is responsible for oversight of the contracted youth workforce development services and programs. The Adult Programs Committee (APC), also a sub-committee of the WIB, is responsible for oversight of the Adult and Dislocated Worker programs. The APC oversees the operations and performance of the America s Job Centers of California (AJCC) Network in the San Diego Region. For more information on SDWP and its boards please visit SDWP's website at Roles and Responsibilities of SDWP Roles and responsibilities of SDWP include: Administer procurement process to select third-party evaluator; Allocate funds and pay invoices as agreed upon in the contract with third-party evaluator; Monitor third-party evaluator s quality of service, cost effectiveness, overall performance, and report on performance to the Boards; Work with third-party evaluator to provide input, feedback, and access to data and records, as appropriate; Oversee evaluation design to ensure evaluation goals are met and complimentary with program design; Review draft documents and provide timely feedback and approvals, as necessary; Inform contractors of federal and state policies, procedures, and rules that may impact the operations of the program(s), and give assistance as needed to implement them accordingly; Prepare grant plans and/or modifications as required for all funding sources; and Ensure compliance with all rules, regulations, and procedures issued by all funding sources. The roles and responsibilities may be refined and changed as: Federal and State law and requirements are enacted and implemented covering the workforce development system; Regulations and procedures are developed or changed by the DOL; and SDWP s governing boards adopt local direction and procedures. 2.6 Roles and Responsibilities of the Contractor Roles and responsibilities of the contractor include: Attend and lead project kick-off meetings with key SDWP staff and other in-person meetings, as appropriate and necessary for successful project completion; Submit monthly project status reports; Provide a tool for random assignment; Provide training to SDWP and funded partner staff on the details of the evaluation and all activities related to random assignment; Develop and submit draft and final ERD study, final performance data template and final evaluation budget that includes Institutional Review Board (IRB) approval to SDWP and DOL; Develop interim annual and final research reports that include a separate stand alone section for the Implementation Study, Randomized Control Trial Impact Study, and Cost-Effectiveness Study; Provide SDWP technical assistance and support to ensure ERD and all study materials receive final approval from the DOL and IRB; 10

11 Work with SDWP to respond to comments and direction from NEC and DOL to strengthen the evaluation design. SDWP and third-party evaluator will also be expected to participate in WIF NEC technical assistance webinars, discussion forums, and to work with NEC for evaluation technical assistance; Cooperate with any directives from DOL, including collaborating with DOL selected independent evaluators; and Manage fiscal responsibilities associated with payment of the contract. 3.0 PROGRAM DESCRIPTION 3.1 Breaking Barriers in San Diego WIF Program Design Background On September 29, 2014, SDWP was awarded $6 million from the DOL through WIF to replicate and extend a small pilot that brought the evidence-based supported employment model to TANF and other low-income families impacted by disabilities. This pilot, from Ramsey County, Minnesota, served TANF families with disability-related exemptions from the federal Work Participation Rate (WPR) requirement. It was identified as the first program to bring supported employment and the Individual Placement and Support (IPS) models to TANF families impacted by disabilities. 1 The supported employment model was originally developed as an alternative to day treatment centers for individuals with severe mental illness. A number of randomized controlled trials have compared supported employment programs to industry-standard vocational programs. Employment rates were found to be higher for participants receiving supported employment services compared to the control condition. 2 A 2011 meta-analysis found strong support for the IPS model on outcomes of employment attainment and retention. 3 Despite these promising findings, the literature on supported employment primarily addresses its application in the context of severe mental illness. There is a clear need for more research to investigate whether the model is equally promising for other populations, including lowincome families with disabilities. SDWP will expand upon this pilot effort in Ramsey County by implementing a larger randomized controlled trial with approximately 1000 participants. Approximately 500 will be assigned to the treatment group to receive the enhanced intervention while approximately 500 will be assigned to the control group. Preliminary overarching research questions related to this program design include: 1. Are the effects identified in Ramsey County replicable amidst a larger sample size in a different county? 2. How are participants barriers to employment related to success in the program? Does participation significantly alleviate these barriers independent of whether employment is obtained? 3. What types of outcomes can be expected when TANF recipients with a disability exemption receive the IPS model? Are these outcomes consistent across participants or do they vary systematically with any baseline characteristics? 1 Farrell et al. (2013). The TANF/SSI Disability Transition Project: Innovative Strategies for Serving TANF Recipients with Disabilities. OPRE Report Bond, G.R., Drake, R.E., & Becker, D.R. (2008). An update on randomized controlled trials of evidence-based supported employment. Psychiatric Rehabilitation Journal, 31, Campbell, K., Bond, G.R., & Drake, R.E. (2011). Who benefits from supported employment: A meta-analytic study. Schizophrenia Bulletin, 37:2,

12 Additionally, SDWP seeks to better understand the needs and circumstances of TANF or other lowincome families impacted by disabilities. The logic model below summarizes the program s inputs, activities, outputs, anticipated outcomes, and major assumptions used in the development of the program design. SDWP s proposal to DOL can be found in SDWP Proposal Narrative - Attachment D. 12

13 3.2 Target Population The target population for this study is TANF families with disability-related exemptions from the federal Work Participation Rate (WPR) requirement. Qualifying disabilities could include: Mental illness, as diagnosed by a licensed physician, psychological practitioner, or other qualified professional, which prevents the participant from working 20 or more hours per week. Developmental disability, as diagnosed by a licensed physician, psychological practitioner, or other qualified professional, which prevents the participant from working 20 or more hours per week. IQ below 80, assessed by a vocational specialist as severely limiting the participant s ability to obtain or maintain suitable employment. Learning disability (meaning a disorder in one or more of the psychological processes involved in perceiving, understanding, or using language), as diagnosed by a licensed professional who is qualified to assess learning disabilities. Illness, injury, or incapacity that is expected to last more than 30 days and that severely limits the participant s ability to obtain or maintain suitable employment. Another member of the household who is ill or incapacitated. Another adult in the household who has a serious and persistent mental illness, or a child in the household who has a serious emotional disturbance. 3.3 WIF Breaking Barriers in San Diego Evaluation Services The successful respondent to this RFP will lead the planning, design and implementation of a multi-year Randomized Control Trial (RCT) evaluation. The key deliverables/milestones for this engagement are shown in the table below. Breaking Barriers in San Diego: Evaluation Services Key Project Deliverables/Requirements Year 1: February 1, 2015 September 30, Submit Initial Evaluation Research Design (ERD) report to SDWP 5/15/ Final ERD, final performance data template, and final evaluation budget 7/15/ Technical assistance and support getting ERD approved by DOL and IRB 9/1/2015 Years 2-5: October 1, 2015 September 30, 2019* 4 Interim Annual Report #1 ** 9/30/ Interim Annual Report #2** 9/30/ Interim Annual Report #3** 9/30/ Final Report** 9/30/2019 Ongoing Requirements/Deliverables 8 Status Reports and check-in conference calls with SDWP staff Monthly 9 Project kick-off meeting in San Diego and other in person meetings as needed for successful completion of project. Frequency as determined by Project Plan 10 Training and technical assistance on research design to program staff Dates as determined by the Project Plan *Deliverables and related funding contingent on Year 1 performance and acceptance of ERD by DOL (Deliverable #3). **All interim annual and final reports must include, at a minimum, separate stand-alone sections for 1) Implementation study; 2) Randomized Control Trial (RCT) Impact Study; and 3) Cost-Effectiveness Study. 13

14 3.4 Technology System Requirements SDWP has technology system requirements with which the successful Respondent must comply. For technology system requirements see Technology System Requirements - Attachment E provides additional technology systems recommendations. 4.0 REQUIRED PROPOSAL FORMAT AND CONTENT The proposal must clearly demonstrate the Respondent's ability to provide the requested services. As stated in Section 1.1 Solicitation, a responsive proposal is one that complies with the format and content requirements of the RFP. The RFP provides information regarding the format in which proposals must be submitted, the requirements that must be met to be eligible for consideration, the Respondents responsibilities, and the documents to be included. In order to simplify the review process the proposal must be organized as follows: 4.1 Page Limits Respondents must adhere to the following page limits: Title Page (Section 4.3) No limit Table of Contents (Section 4.4) No limit Executive Summary (Section 4.5) three (3) pages Proposal Content (Sections 4.6 through 4.7.4) twenty (20) pages Required Attachments: Organizational Chart (noted in Section 4.6.3) - No limit Required Attachments: Resumes/Curriculum Vitae (C.V.) of key staff (noted in Section 4.6.3) - No limit Project Work Plan (Section 4.8) two (2) pages Financial Management Structure - Attachment K (Section 5.1) six (6) pages Budget Shell - Attachment F (Section 4.9) No limit Budget Narrative - Attachment G (Section 4.9) six (6) pages All other attachments ten (10) pages No other page limits are in effect. SDWP will remove any excess pages from proposals exceeding these limits before the proposals are distributed for review and evaluation. Supplementary items such as job descriptions, work samples, and any other items not specifically asked for in the RFP are subject to the ten (10) page attachment limit. 4.2 Formatting Requirements Respondents must adhere to the following formatting requirements: Font size: 12 point Font: Times New Roman Margins: at least one inch Line spacing: Double-spaced Pages: single-sided Page numbering: Bottom of page Language: English 4.3 Title Page The title page must include: Company name, address, phone number, and federal tax identification number; 14

15 Name of the person authorized to negotiate contracts and make decisions for the organization including the telephone number, fax number, and address; and Authorized signature and submittal date. 4.4 Table of Contents Include a table of contents that shows a clear identification of the material in the proposal by section and by page number. 4.5 Executive Summary The executive summary must include a brief overview of the entire proposal including a summary of the Respondent s understanding of the project, an overview of the Respondent s approach, and a summary of the Respondent s qualifications and experience to carry out the scope of work. The executive summary must not exceed three (3) double-spaced pages. 4.6 Proposal Content: Organizational Capacity and Experience (Total 25 Points) All Respondents must respond to the questions and instructions in Section 4.6 through Questions must be answered in concrete language, using quantifiable measurements whenever possible, and be specific. Responses must not exceed twenty (20) double-spaced pages Organizational Capacity and Resources (5 Points) The Respondent must provide an overview of its organization, including the primary location of the organization, type of organization (for-profit, non-profit, etc.), size of the organization, years in business, history of business, the organization s mission and objectives, major product lines, and any other relevant information that helps provide an overview of the organization. Respondents must describe their administrative and IT support related to this engagement, including data security infrastructure, description of data collection or analysis tools created or used by the research team, and other information describing the organization s ability to successfully achieve all project objectives Project Experience and Qualifications (15 Points) Respondents must provide evidence of the organization s ability to successfully perform the services described in this RFP, including descriptions of past projects completed with a similar scope of work and size and previous publications or other deliverables from similar studies in the content field and/or using similar experimental design Project Team (5 Points) Respondents must provide an organization chart with the names, titles, and time dedicated to this engagement for each key project team member, including the project manager, lead researcher, and any other team members critical to the successful completion of the scope of work described in this RFP. Respondents must include resumes/c.v. of all key personnel and examples of past publications and project experience of key team members providing services similar to those described in this RFP. 4.7 Proposal Content: Approach to Evaluation Design and Implementation (Total 45 Points) In Sections through 4.7.4, Respondents must describe their approach and methodology related to successfully achieving the following deliverables for the WIF third-party evaluation services described in this RFP. For additional guidance on Evaluation Design and Implementation see Evaluation Toolkit for WIF Grantees - Attachment H. 15

16 4.7.1 Approach to Evaluation Research Design (ERD) for RCT Study (15 Points) The primary task in Year 1 for the third-party evaluator is to develop a plan for carrying out the evaluation, called the Evaluation Research Design (ERD). This document will build on SDWP s preliminary research design described in SDWP Evaluation Narrative - Attachment I and will serve as a guide for the evaluator and program staff to the unfolding of various evaluation activities and how they will link to or coordinate with program operations. Key components of the ERD include, but are not limited to, the intervention s purpose and scope, literature review, logic model, key research questions, participants, samples, and units of analysis, the specific evaluation design, data definitions, data security, random assignment process, validity and threats to validity, approach to training program staff on research design,, recruitment and outreach materials, and participant privacy and human subject protections. Respondents must respond to the following related to the ERD: Describe your overall approach to developing and finalizing the ERD for the RCT Study, including the planned process for getting approval from an IRB and DOL by the appropriate project deadlines Describe your approach to refining and finalizing the purpose and scope of the intervention, logic model, literature review, key research questions, and evaluation design based on SDWP s preliminary evaluation design found in SDWP Evaluation Narrative - Attachment I Describe your approach to developing and finalizing the random assignment process for approximately 1000 program participants, half of which will be assigned to the treatment group and half will be assigned to the control group Describe your process/approach to protecting and securing data, participant privacy, and human subject protections Describe how you will partner and communicate with SDWP and other appropriate stakeholders throughout the development of the ERD Describe your approach to training program staff on the ERD Describe your approach to developing the final performance data template Approach to Implementation Study (10 Points) The Implementation Study will document and explain what is happening and why in the design, implementation, administration, operation, services, and outcomes of the Breaking Barriers in San Diego Program Design. This type of study can provide context and information that makes evaluation results more useful for improving program implementation. In addition, findings from implementation research can be used to inform future program development or replication. Respondents must respond to the following items related to the Implementation Study: 16

17 Describe your overall approach to developing and finalizing the Implementation Study component of this scope of work Describe your understanding of the purpose and scope of the Implementation Study Provide a sample of the Implementation Study research questions that could be used for the specified population Describe your approach to data collection and analysis that addresses your Implementation Study research questions Describe potential challenges that can occur in an Implementation Study of similar size and scope and how you would address those challenges Approach to RCT Impact Study (10 Points) The Randomized Controlled Trial (RCT) research design measures the impacts of the intervention or program on individuals or systems. An impact is an estimate of the direction (positive or negative) and magnitude (by how much) of the change in outcomes that can be directly attributed to the intervention. Key elements to consider in the RCT impact study include, but are not limited to, the development and administration of survey and other data collection tools, strategies to mitigate/minimize attrition, how to account for attrition in the analysis, approach to quantitative research, generalized linear models, statistical matching, other statistical techniques to analyze impact, and analysis practice used to investigate the experience of the treatment group. The third-party evaluator will be responsible for collecting follow-up data on both treatment and control group participants. Respondents must respond to the following items related to the RCT Impact Study: Describe your overall approach to developing and finalizing the RCT Impact Study component of this scope of work Describe your understanding of the purpose and scope of the RCT Impact Study Provide a sample of the RCT Impact Study research questions that could be used for the specified population Describe your approach to data collection and analysis that addresses your RCT Impact Study research questions Describe potential challenges that can occur in an RCT Impact Study of similar size and scope and how you would address those challenges Approach to Cost Effectiveness Study (10 Points) The central research question related to the Cost Effectiveness Study is What is the cost, per unit, for the results obtained? This analysis will examine costs in terms of the outcomes described in the logic model, including entered employment, thirty (30) day retention, and six (6) month retention. SDWP will use this study to understand if the service intervention received by the treatment group is a costeffective program that delivers its key outcome at a reasonable cost per outcome (i.e., at a cost that is similar to or lower than comparison group). Respondents must respond to the following items related to the Cost Effectiveness Study: 17

18 Describe your overall approach to developing and finalizing the Cost Effectiveness Study component of this scope of work Describe your understanding of the purpose and scope of the Cost Effectiveness Study Provide a sample of the Cost Effectiveness Study research questions that could be used for the specified population Describe your approach to data collection and analysis that addresses your Cost Effectiveness research questions Describe potential challenges that can occur in a Cost Effectiveness Study of similar size and scope and how you would address those challenges. 4.8 Project Work Plan (Total 5 Points) Respondents must submit a project work plan as an attachment that shows the key phases, activities, and tasks required to successfully complete all project deliverables and carry out this work throughout the life of the project. The work plan must highlight major dependencies, milestones, and a comprehensive project timeline for this engagement through September 30, Proposed Budget (Total 10 Points) With guidance from Section 5.2, Budget Information, Respondents must complete the following Budget Forms to prepare a fifty-six (56) month budget in accordance with Budget Instructions - Attachment J: Financial Management Structure - Attachment K Budget Narrative - Attachment G Budget Shell - Attachment F o Expenditure Plan o Budget Detail o Personnel Detail o Deliverables o Activities Cost Allocation Plan Respondents must adequately complete the forms listed above and describe the reasonableness of their costs. Respondents must explain how Budget Detail costs relate to Deliverable costs and Activity costs for the proposed evaluation services. Respondent must demonstrate their ability to track financial activities for this project. 5.0 FINANCIAL REQUIREMENTS 5.1 Financial Management Structure Respondents are required to describe their financial management structure by completing the form included in the Financial Management Structure -Attachment K. The responses included in this document will serve to evaluate the Respondent s experience managing and accounting for state and federal funds as well as conducting self-monitoring for financial performance and compliance. In the case of a collaborative partnership, this section must also identify the lead agency responsible for the financial activities of the collaborative and the related partner responsibilities. Financial Management Structure -Attachment K must not exceed six (6) pages. 18

19 5.2 Budget Information The period of performance for the contract is eight (8) months beginning February 1, 2015 and ending September 30, Based on the DOL acceptance and approval of the ERD report, need, performance, and funding availability, this agreement will extend through September 30, 2019 for a total period of performance of fifty-six (56) months. Refer to Section 5.3 Payment Structure, for contract payment details. Respondents are required to submit a Budget Shell - Attachment F, (e.g., personnel, non-personnel, contracts, indirect, profit) that covers the entire fifty-six (56) month contract period, a related budget narrative by line-item, and a cost allocation plan. No Respondent should exceed the estimated funding amounts. The overall line-item detail budget should provide a high-level snapshot of the Respondent s total budget. All personnel costs must identify the staff position, annual salary, and percentage of annual time spent on the funding sources for every staff position by program year of the fifty-six (56) month contract. Non-personnel costs should also be further outlined into specific line items (such as supplies, equipment, repairs, and mileage). Subcontractor costs should identify the subcontractor, the service to be provided, and the total cost. Each Respondent is required to submit a Budget Narrative - Attachment G and a cost allocation plan that corresponds to its line-item budget detail. This narrative should justify the need for all costs built into the line-item detail, the methodology used to derive each cost, and the methodology used to allocate the common cost to the WIF project. This narrative must not exceed six (6) pages. For-profit organizations negotiated profit line item will not be negotiated at a rate higher than ten (10) percent of the total project budget. The following factors that may be considered in negotiating profit: a. Contractor effort measured by the complexity of the work to be performed and amount of subcontracting (if approved under the contract terms); b. Contract cost risk; c. The contractor s investment and independent development; d. The quality of the contractor s record of past performance including cost control and past accomplishments; and e. Additional factors as needed. The extent to which a Respondent can meet performance objectives while minimizing indirect and profit costs will be a factor in the evaluation process. Any indirect costs budgeted must be supported by an indirect cost rate agreement with a federal or state cognizant agency. If the Respondent is not required to obtain an indirect rate, then the cost allocation plan must include the methodology describing how common operating costs are distributed to the different funding sources. For guidance on completing the standard budget shell and the budget narrative, please see, Budget Instructions - Attachment J. 5.3 Payment Structure The payment instrument will be a cost reimbursement contract with full recovery of costs contingent on meeting all deliverables. If Contractor fails to meet deliverables negotiated with SDWP, payment forfeiture ranging up to ten (10) percent of the total contract cost will be applied. Proposed deliverables described in the Budget Shell - Attachment F are required to be submitted no later than the due date 19

20 agreed upon. Any revisions made to the proposed deliverables must be negotiated and approved by SDWP. 5.4 Subcontractors The Respondent must describe its plan to subcontract any portion of the services. All procurement transactions must be conducted in a manner to provide, to the maximum extent practical, open and free competition. See Contract General Provisions - Attachment L (Section 29.0 Procurement and Purchases and Section 42.0 Subcontracting) for additional information. If a proposal identifies a specific entity to provide services, SDWP s award does not provide the justification or basis to sole source the services. 6.0 EVALUATION PROCESS AND CONTRACT AWARD 6.1 Compliance Review Upon receipt of proposals, staff will review submitted proposals for completeness and technical compliance with the terms and conditions of the RFP. All proposals must adhere to the required format in Section 4.0 Required Proposal Format and Content, and, in order to be competitive, must include all of the requested information, completed forms, and attachments. Proposals that do not adhere to these requirements will be determined non-responsive and will not be considered for funding. SDWP reserves the right to reject any proposal submitted. SDWP will remove any excess pages from proposals exceeding page limits before the proposals are distributed for evaluation. 6.2 Evaluation Criteria SDWP will put together an RFP scoring panel of community leaders, board members, and outside experts that will score and rank all proposals that meet the compliance requirements described in Section 6.1 Compliance Review. The RFP Scoring Panel will use the criteria described within the RFP and make a recommendation for funding based on the evaluation scores. Respondents who receive seventy (70) or more points on the written portion of the proposal will move on to the oral interview stage of the evaluation. Respondents that qualify for orals will be given fortyfive (45) minutes to highlight the different areas of their proposal and to answer a series of questions from the RFP Scoring Panel. The Respondent may not present any new information during the oral interview. The individual designated as the authorized negotiator, the individual who will serve as the project manager, and other key personnel must be in attendance at the interview. Oral interviews may take place via video conferencing or in person. The Respondent is limited to a presentation team of five. Oral interviews will be worth a total of 15 points. After oral interviews are complete, final evaluation scores will be submitted and the panel will make a recommendation for funding. The table below provides a summary of the evaluation scoring methodology for this RFP. 20

21 Third-Party Evaluation Services: RFP Scoring Methodology Written Proposal Scoring* Section 4.6: Proposal Content: Organizational Capacity and Experience (25 Points Total) 4.6.1: Organizational Capacity and Resources : Project Experience and Qualifications : Project Team 5 Section 4.7: Proposal Content: Approach to Evaluation Design and Implementation (45 Points Total) 4.7.1: Approach to Evaluation Research Design (ERD) for RCT Study : Approach to Implementation Study : Approach to RCT Impact Study : Approach to Cost Effectiveness Study 10 Section 4.8: Project Work Plan (5 Points Total) 4.8 Project Work Plan 5 Section 4.9 Proposed Budget (10 Points Total) 4.9 Proposed Budget 10 Written Proposal Total Available Points 85 Oral Interview Scoring (Respondents that receive 70+ points on written portion will advance to the Oral Interview) Oral Interviews Total Available Points 15 Total Available Points 100 *SDWP will use a weighted scoring methodology to finalize all scores. The RFP Scoring Panel will score each category on a 0-5 point scale. Total points will be converted based on a determined weight as defined by the points and questions per section available. Respondents must have a total average score of 85 points or higher to be recommended for funding. SDWP reserves the right to recommend multiple proposals to governing boards. In the event that the primary Respondent recommended fails to provide services outlined in the agreement and proposal, SDWP may use an alternate Respondent to perform services based on a score of 85 points or higher and approved by the governing board. 6.3 Selection The RFP Scoring Panel will recommend as the successful Respondent the organization whose proposal is determined to best meet the needs of SDWP, based on the evaluation criteria discussed above. The selection of the successful proposal will be based upon information supplied by the Respondent in response to this RFP and upon other information that will be obtained by the RFP Scoring Panel, as deemed necessary. The lowest-cost proposal may not be determined to be the most responsive when all factors of evaluation of the proposal have been considered. SDWP reserves the right to negotiate with any Respondent after proposals are opened, if such action is deemed to be in the best interest of SDWP. 6.4 Appeals Process and Access to Evaluation Information The information below outlines the appeals process and procedure to access evaluation information. An appeal will only be considered valid if there has been a violation of the one of the following criteria: 21