Request for proposal For Data Quality Audit Project Title: Dates, Duration, Days & Hours: Location: Reporting to: Pehchān Data Quality Audit 1 Sep 2011 to 30 Sep 2012 India HIV/AIDS Alliance, Kushal House, 3rd Floor, 39, Nehru Place, New Delhi 110019 Kaushik Biswas, Manager M&E Section I- Introduction I.1 India HIV/AIDS Alliance The India HIV/AIDS Alliance (Alliance India) is a diverse partnership that brings together committed organisations and communities to support sustained responses to HIV in India. Complementing the Indian national programme, we work through capacity building, knowledge sharing, technical support and advocacy. Through our network of partners, we support the delivery of effective, innovative, community-based HIV programmes to key populations affected by the epidemic. Alliance India is implementing several projects through different funding sources that include Pehchān which aims to strengthen MTH (MSM/TG/HIJRA) CBOs to provide specific and sustainable services for these populations, and nurture a policy and social environment in which these communities can live with dignity and improved health. I.2 Pehchān Project Alliance India is the Principal Recipient of a five-year programme funded through Global Fund Round 9. Named Pehchān which in Hindi means identity, recognition or acknowledgement, the initiative will strengthen and build the capacity of community-based organisations (CBOs) to provide HIV prevention programming for men who have sex with men (MSM), transgender and hijras in 17 Indian states. This is the largest Global Fund grant thus far to strengthen HIV prevention among MTH (MSM/TG/HIJRA) communities. The program was launched on 1 October 2010.
By 2015, Pehchān will establish 90 new CBOs and strengthen 110 existing ones. By supporting the development of strong CBOs, Pehchān will address some of the capacity gaps that have often prevented CBOs working with these populations from receiving government funding for much-needed HIV programming. The Pehchān sub recipients (SR) are Humsafar Trust, Maan AIDS Foundation, SIAAP, SAATHII, AIAP and SANGAMA. Together, they aim to reach out to more than 4, 53,000 community members in India. The programme focuses on strengthening HIV prevention among (MSM/TG/HIJRA) MTH and is unique in its design and approach since it is community-driven and focuses on advocacy and policy activities to create an enabling environment that offers easier access to specific services for MTH (MSM/TG/HIJRA). The programme will include comprehensive service packages (including a rapid response mechanism for trauma and violence, life skills, mental health, and referrals for female partners of MSM) over five years and will engage in advocacy to achieve a more favourable policy environment and friendly services for MTH communities at the national, state and community levels. Pehchān recognizes the diverse range of identities in the MTH community and employs a variety of approaches to ensure maximum accessibility for MSM, TGs and hijras respectively, keeping in mind each group s distinct needs for specific approaches and services. The programme will work closely with NACP-III, both to ensure maximum reach and also to improve mapping and tracking mechanisms for MTH in the country. The programme will strengthen CBOs with the goal to implement Targeted Interventions on behalf of the government over the course of the programme to ensure sustainability of services. Section II- Scope of Work Alliance India has a rigorous monitoring & evaluation system in place for all programmes, when it comes to, Global fund programme monitoring the systems and checks are developed and being implemented in 17 states in the country. Quality assurance standards constitute the basic instrument of monitoring performance in a particular service delivery area. These standards can be considered as analogous to indicators, but they often also include a target element, either implicitly or explicitly, within them. Standards may include quantitative aspects but often have a strong focus on quality, hence the term quality standards. Although standards may have an aspirational element, e.g. as a target to be reached, they usually imply a sense of minimum acceptable standards. These standards can also be referred to as benchmarks.
The overarching aim of having service standards is to improve the quality of services provided. These standards can be used in two main ways to achieve this. First, service providers should be trained to deliver the services according to these standards. Second, standards can be used as assessment criteria for monitoring activities, such as site visits, user surveys, etc. Programme monitoring thus indicates whether and to what extent the obtained results correspond to the targets set at the planning stage. If there is overall consistency with the targets and results achieved, then activities will be implemented further according to the plan, with only minor recommendations/corrective actions, as required. If there is any inconsistency from the set targets, corrective actions will be agreed before implementation continues. These corrective actions could include special measures to strengthen the capacity of implementing NGOs in certain areas, or the reallocation of financial resources to another implementing NGO, etc. Alliance India has developed a system for maintaining the highest standard to ensure the quality of data. This is done through different mechanism. A) Sub recipient s (SR) Assessment: Sub-Recipient partners will be assessed in order to determine if they have the minimum capacities to successfully conduct monitoring and evaluating activities within the programme. This assessment is carried out as a part of the capacity assessment process provided by the Grant Agreement. This assessment can be also conducted separately if needed. B) On site data verification visit (OSDV): Monitoring visits are the basic instrument of assuring the quality of data provided by the Implementing NGOs. Monitoring visits will be conducted by the Alliance India staff and by the Sub-Recipient (intermediary) partners of Alliance India. Alliance India has developed an algorithm (both at PR and SR level) for monitoring visits that will aim to regulate the frequency with which the visits are to be carried out depending on the nature of the grant recipients, the services provided by them, etc., as well as the composition of the teams that conduct the visits to different Sub-Recipients and Implementing NGOs. On average each implementing NGO will receive a technical support and supportive monitoring visit within each quarter following the starting date of grant provision or more frequently, as required. C) Technical Assistance in M&E Related Issues: Based on the participatory discussions with SRs and implementing SSRs, PR understands that the capacity building on M&E data collection, reporting and analysis needed to be expanded to local implementing SSRs,
D) External Data Quality Audit: This will get done through external agency to do the data quality audit every quarter on a random basis to maintain the data quality as per the GF norms. India HIV/ AIDS Alliance has developed an M&E framework under GF Rd.-9 programme implemented with 6 SRs (Sub recipient) and 200 CBOs. It is a three tier manual MIS system where in SSRs report to SR and then SR verifies and report he final compiled data to PR. Following diagram explains it in detail. Figure 1: Quarterly progress report flow PR SR 1 (Compilation of data) SR 2 (Compilation of data) SSR 1 SSR 2 SSR 3 SSR 4 About 12 registers maintained at SSR level apart from peer educator weekly and monthly plan and service forms as well as Client Registration Form (CRF). 5 registers maintained at SR level along with supporting descriptive forms and about 3 registers maintained at PR level (Annex E for more details). 1. Purpose & Objectives data quality audit: Alliance India proposes to hire an agency to do the data quality audit in every quarter in some randomly selected CBOs in different states. The agency needs to carry outthe DQA to ensure accurate, reliable, precise, complete and timely data reports. This agency will be selected through contracting procedure. After selection the agency will be given the thorough orientation about the project, M&E system, primary reporting tools, reporting mechanism,
about the donor compliance requirement and the required data quality standards. Then every quarter randomly the certain numbers of SSRs will be selected to go through the data quality audit by the agency. After the field level data auditing the agency will submit a formal report and the necessary strengthening measures will be taken on the basis of the report. Data Quality Audit (DQA) will not only allow program managers and decision makers in the organisation to verify the quality of the reported data, but also will provide periodic information on the underlying data management and reporting systems for, at a minimum, program-level output indicators. Further, DQA system will allow PR to develop action plans to remedy discovered data quality issues. The quality data will help the PR to monitor the data collected by its SRs in order to improve the projects reported results. Objectives: Audit the entire manual and computerised MIS system of the data generated at the selected SSR level and SR level to minimise the possibility of erroneous reporting Provide feedback to SSR / SR /PR based on the audit findings, proposing strengthening measures Methodology conducting the DQA: After reviewing relevant information provided by the PR, the data quality process will be initiated by the agency. Adjustments may be made during the course of implementation to ensure that the system achieves its objectives of supporting the provision of accurate, reliable and timely data within resource constrain. The external data quality audit system consist of five levels of activities to be undertaken by external agency using a bottom up trial audit 1) Minimizing routine data errors: The consolidated report is provided to the agency with disaggregated SSR and SR wise data. Based on which the agency should work with the PR and SR responsible to support for agency data quality audit. This includes advising on minimizing errors from transcription, data entry and compilation level errors. 2) Field validation of data: Prior to visiting a SSR & SR for DQA, agency staff needs to collect all relevant program data or information about the SSR & SR. A number of data quality checks on data items and time period for which data are checked, having an overall indication of the accuracy of data and reinforcing good data practices among staff. 3) Cross-checking to find errors: The cross-checks to be systematically carried out to identify potential data problems include, reported cases vs. total cases recoded in the
register should equal. Cross checking on the storages of all feedback provided through email to the SR & SSR to be cross checked. 4) System for storing data: Data storing system at SR & SSR level are necessary for ensuring that source data are maintained for validation purposes and that data records needed for program monitoring. Ex. The hard copies of data submitted needs to store in protected locations, with in files or folders and in drawer s or closed cabinets, so that multiple papers and registers do not become lost or torn, and so that when DQA carried out, data can be readily found. Also, electronic copies and communication on the reporting needs to be cross checked. 5) Providing feedback: All audit observations and correction measures needs to be shared with SSR / SR in a brief meeting with the SSR / SR team after audit visit. This correction measures will be suggested based on the Pehchān monitoring and evaluation systems of Pehchān project. The Entire auditing function is summarised in following steps: These can be treated as the deliverables from the agency Develop mechanism o Auditing tool o Guideline for auditing o Audit report format o Algorithm for conducting the activity Implement the auditing in the field o Coordinate with Alliance India for the selection of the CBOs every quarter o Agency will work out the visit plan in discussion with the Alliance India o The agency will collect the final report for the last quarter from PR o Agency will visit the SR for the review at SR level o Agency will visit CBO for review at SSR level o Agency will analyse the compiled information from the visit in ā report format along with the checklists to Alliance India Suggest an strengthening measures to address the issues found at the audit In conducting the DQA, the Audit Team will collect and document: (1) evidence related to the review of the program/project s data management and reporting system; and (2) evidence related to data verification. The documentation will include:
Completed protocols and templates included in the DQA Tool Write-ups of observations, interviews, and conversations with key data quality officials at the M&E Unit, at intermediary reporting locations, and at Service Delivery Sites. Preliminary findings and draft Recommendation Notes based on evidence collected in the protocols; Final Audit Report. The Final Audit Report needs to summarize the evidence the Audit Team collected, identify specific audit findings or gaps related to that evidence, and include recommendations to improve data quality. The report will also include the following summary statistics that are calculated from the system assessment and data verification protocols: the data quality audit tools will be finalised by the agency in consultation with Alliance India team. Strength of the Data Management and Reporting System based on a review of the program/project s data collection and reporting system, including responses to questions on how well the system is designed and implemented; Accuracy of Reported Data through the calculation of Verification Factors generated from the trace and verify recounting exercise performed at each level of the reporting system (i.e., the ratio of the recounted value of the indicator to the reported value); and Availability, Completeness and Timeliness of Reports through percentages calculated at the Intermediate Aggregation Level(s) and the M&E Unit. These summary statistics, which are automatically generated in the Excel files, are developed from the system assessment and data verification protocols included in this tool. All follow-up communication with the program/project and the Organization Commissioning the DQA related to the results and recommendations of the Data Quality Audit. Selection of SSR for DQA visit: A total list of SSR with the names will be provided to the agency. Based on the requirement mentioned in the table below the number of SSR will be selected using stratified random sampling for DQA in each quarter
Table 1: Number of SSR to be selected for DQA in each quarter SSR Quarter July Sep 11 Oct-Dec 11 Jan-Mar 12 Apr-June 12 July Sep 12 Oct-Dec 12 Total SSR 13 40 63 87 126 161 (Cumulative) SSR for 5 15 15 20 30 30 DQA (noncumulative) The agency will have to have a data quality audit at SR, if any SSR under the SR is being selected. The project is going to be implemented till Sep 2015. The first phase of the program is till Sep 2012, the data quality audit will be carried out for five quarters starting from Sep 2011 to Sep 2012. Once the project phase II implementation granted to Alliance from the donor the DQA contract can be renewed based on the performance of first five quarters. Ethical considerations The data quality audits must be conducted with the utmost adherence to the ethical standards of the Global Fund, as appropriate, of the Alliance India DQA policy. While the agency audit teams may require access to personal information (e.g., HIV related, identity records etc.) for the purposes of recounting and cross-checking reported results, under no circumstances will any personal information be disclosed in relation to the conduct of the audit or the reporting of findings and recommendations. The Audit team should neither photocopy nor remove documents from sites. In addition, the auditor shall not accept or solicit directly or indirectly anything of economic value as a gift, gratuity, favour, entertainment or loan that is or may appear to be designed to in any manner influence official conduct, particularly from one who has interests that might be substantially affected by the performance or non-performance of the auditor s duty. This provision does not prohibit the acceptance of food and refreshments of insignificant value on infrequent occasions in the ordinary course of a meeting, conference, or other occasion where the auditor is properly in attendance, nor the acceptance of unsolicited promotional material such as pens, calendars, and/or other items of nominal intrinsic value.
Section III Request for Proposal India HIV AIDS Alliance is requesting the DQA agency to submit a detailed proposal as per the outline given below: 3.1 Background 3.2 Methodology 3.3 Implementation plan 3.4 Work plan and time line 3.5 Budget (in attached format Annex B) 3.6 Contact information (Annex D) 3.7 List of Annexes requested with the proposal 1. Detailed rationale/calculations for budget 2. Work plan with Timeline (Format - Annex C) 3. Brief note on relevant past experiences related to the NGOs especially for HIV Intervention 4. Brief bio-data of key resource persons 3.8 Submission of Proposals Agencies who are interested in carrying out DQA are requested to submit their proposal for the same as per the details outlined above. The technical and financial bids should be submitted separately in two sealed envelopes, mentioning Technical Proposal & Financial Proposal in the respective envelopes. Both these envelopes should put in another envelope clearly mentioning the following: Technical & Financial Proposal for Pehchān - Data Quality Audit. Interested agencies meeting the eligibility criteria are requested to submit their technical and financial bids following the guidelines to the below given address on or before the closing date: The Manager: HR & Administration India HIV/AIDS Alliance 2nd Floor, 39, Kushal House, Nehru Place 110 019 Tel : 011 41633081-84 Please also forward your complete proposal as submitted in original to procurement@allianceindia.org. Please note that the proposals received in a sealed envelope will only be considered for technical evaluation.
The bids will be evaluated by the Bid Evaluation Committee constituted by Alliance India. Criteria for selection of an agency will be as follows: Past experience of the agency personnel assigned for the proposed task 25 points Management and composition of team structure for managing the DQA 20 Points Work plan and timeline- 10 points Tools - 20 Adherence to RFP 5 points Budget 20 points Last date for submission of bid documents: 29 th August, 2011 up to 11:30 AM All correspondence / communications regarding the proposal should be made at the above address only.
Annex B Outline for Budget Proposed Budget Narrative Description Unit Unit cost (in Rs.) Amount (in Rs.) A. Direct Cost 1. Human Resource 2. Other Direct Costs Travel Local Transport Office Supplies & Stationary Communication Miscellaneous Expenses Subtotal Other Direct Costs Total Direct Cost Indirect Cost 1. Overhead cost Subtotal Indirect Costs TOTAL (Direct Cost + Indirect Cost) Tax if any Grant Total The grand total should include applicable tax.
Annex - C Timeline for the DQA (to be completed by agency) Activity title Completion on/by 1. 2. 3. 4. 5. 6. Annex-D S.N 1. Name of the Organization 2. Address Contact Information Information 3. Telephone 4. Fax 5. Email Address 6. Two Contact Persons with their position with organization
Annex-E SSR level Recording systems Client registration form Master register PE wise individual MTH compilation weekly/monthly sheets Counselling register (supported with narrative format) Support group meeting attendance register (supported with narrative format) Referral register (supported with referral slips) Adherence support register Crisis management register Condom and lubes distribution register Emergency support register Social entitlement register Advocacy register (supported with narrative format) Meeting register (supported with narrative format) Technical support register SR level Advocacy register Community mobilisation register Training Register Coordination meeting/ CAB Meeting register Technical Support register -SSR M&E checklist (SSR) SSR target entry form PR Level Training register Advocacy register Meeting register M&E Checklist SR SR target entry form