Progress out of Poverty Index

Transcription

1 Progress out of Poverty Index Certification Report: Come To Save (CTS) Bohumukhi Samobay Samity Ltd. This PPI Certification Report provides a summary of the PPI Certification results for: Come To Save (CTS) Onsite Visit: May 14 May 19, 2012 Bankers Without Borders volunteer: Tina Oliphant from Montana West Economic Development

2 Subject: PPI Certification Report Dear Executive Director of Come To Save Grameen Foundation has completed its certification review of CTS s process for using the PPI. The certification review covered intent, data collection, training, processing, and reporting. This certification opinion is based 100% census certification done by CTS in the summer of 2011 with initial planning and training initiated in the Fall of Funding support and technical assistance was provided by Plan International. CTS collected from 23,093 representing all clients of their 10 rural branches located in their working areas of Khanshama and Chirirbandar Upazillas of Dinajpur District. Grameen Foundation conducted the certification review based on the PPI Certification Standards published in November 2010, attached as Appendix A. Grameen Foundation developed these standards through a participatory process that engaged a variety of stakeholders, from MFIs to investors, across the microfinance field. These standards represent the minimum quality standards an institution should have in place to collect and report poverty level information accurately and in a transparent manner. This PPI Certification Report is limited to the process CTS has in place for the PPI as described above in the first paragraph and should not be considered as a certification of a report on any other aspect of CTS and its operations. In our opinion, the process used for PPI data collection at CTS is sound, relevant and reliable and should produce accurate results for internal analysis and dissemination to stakeholders. Based on the review and analysis of CTS documents and processes, interviews with staff and clients, and the onsite visit completed by Tina Oliphant, Grameen Foundation certifies that CTS meets the PPI Certification Standards with a Basic and Advance level of Certification, as summarized below. The following pages provide details on each of these standards according to intent, data collection, training, processing, and reporting. Grameen Foundation thanks you for your time and effort during the certification process and looks forward to working with you in the future. Sincerely, Steve Wright Director, Social Performance Management Center Grameen Foundation swright@grameenfoundation.org 2

3 PPI Certification: Results BASIC* Meets CTS meets all 19 standards. ADVANCED** Meets CTS meets at least one standard within each category TRACKING OVER TIME*** Not Meeting CTS does not meet the three standards. *PPI Users must meet all 19 basic standards to qualify for basic certification. **PPI Users must meet one advanced standard from each category to qualify for advanced certification. ***PPI Users must meet all three tracking over time standards to qualify for tracking over time certification. 3

4 Detailed Certification Results Review INTENT BASIC Meets Meets Basic standards; 1, 2 and 3 ADVANCED Meets Meets Advanced standard 1 Strengths Areas for Development The Executive Director has shown by action his commitment to the value of the PPI process. The Executive Director is very clear on value of the PPI process to CTS clients as a mean to better evaluate the program and product design for the purpose of moving them out of poverty. The Board has actually established a Board Member with documented responsibilities to assess the program and report to the Board. The results thus far have not influenced product design and market research, as the next census survey is slated for Because CTS is in the first cycle (collecting baseline data), there is no comparative data to evaluate feedback on the progress of the clients. CTS has certainly embedded the process of interviewing all new clients into their orientation, so there has been operation shifts to fold the PPI process into their ongoing operations. CTS has interviewed in excess of 1500 new clients since the initial cycle ended. It is their intention to not populate the database with the new client surveys until July 2013, during the next full survey. COLLECTION BASIC TRACKING OVER TIME Strengths Meets Not Meeting Meets Basic standards; 4, 5, 6 and 7 Does not meet Tracking over Time Standard 1 CTS made a strategic decision and commitment to initiate the program by survey of 100% of their clients and then regarding survey in 2 years (July 2013). This required complete organization wide support of this effort. Clients understood the significance and value of the PPI survey and the potential future benefits. Branch mangers were integrated into the system as back up and random check of the interviewers. CTS has interviewed in excess of 1500 new clients since the baseline took place in the summer of It is their intention to not add the new client surveys until July 2013, during the next full survey. Areas for Development In some cases if a client survey results seemed illogical and required further confirmation, the Branch Manager or Loan Officer may receive data from the village social network. While this may provide a good source of information, it could also provide misleading results. PROCESS BASIC Meets Meets Basic standards; 8 and 9 ADVANCED Meets Meets Advanced standards 4, 6,7 and 8

5 Strengths CTS and Plan International, Bangladesh jointly were involved in the processing of data. Data Entry and Reporting was the responsibility of Plan Bangladesh, who has taken over this part of the process for the 1st phase of implementation. Because of the decision to survey 100% of the clients and the aggressive implementation (surveyed within 4 months), Plan Bangladesh used a third party contractor to assure the great amount of data would be entered timely. Plan Bangladesh chose a professional data entry contractor for their quality and skill of work in data entry field. Two key managers of Plan International Bangladesh randomly checked 15-25% of the data entry for quality and accuracy of data entry. 100% of the data was reviewed for completeness of entry. The data is stored in CTS s excel file (separate from the loan system), and other indicators of the clients are included). Scorecards are backed up electronically. Areas for Development CTS PPI data does not integrate into the MIS system. The data is stored into an excel spreadsheet type database, but this is separate from the MIS system. TRAINING BASIC Meets Meets Basic standards; 10 and 11 ADVANCED Meets Meets Advanced Standards 9 and 10 Strengths CTS was trained by Plan Bangladesh. 9 CTS staff members took part in the training. This was a Train the Trainer multi day session. They returned to CTS and with the use of Plan Bangladesh s material including a 30 page-training booklet, they provided training to the all relevant staff including 142 Loan Collectors. The Training Manual addresses the PPI process, purpose and long term goals. CTS has a dedicated in house Training Manager. The monthly manager meeting agenda of CTS always includes any issues or updates of PPI as a means to continue education and awareness. CTS also provided educational meeting in the client community to help them understand the value of the survey process. Plan Bangladesh also provided specific half day training to the data entry contractor. Areas for Development Plan Bangladesh provided all data entry and reporting for this cycle of implementation. CTS have already established a plan to take these roles over in 2013 when the next survey cycle. CTS will need to make sure they have staff and training in place. REPORTING BASIC Meets ADVANCED Meets Meets Basic standards; 12, 13, 14, 15, 16, 17 & 18 Meets Advanced standards; 11, 13, 14, 16 & 18 TRACKING OVER TIME Not Meeting Does not meet Tracking over Time standards 5

6 Strengths CTS has embedded the PPI Process into their system, and, as such, PPI is now an ongoing agenda items in Management and Branch meetings. It is also an agenda item with the Board. Plan Bangladesh provided the reporting function and has created a thorough PPI Report, providing background, the process, statistics on the collection efforts and any rejected scorecards, and a series of graphs and data presentation. The Basic Standards are met in these reports/graphs and narrative, however some graphs illustrate additional detail of minor indicators across multiple MFIs. While this does not present an issue for meeting the Standards, it does reduce usefulness to the individual MFI in these specific reports. Note that some of the Standard Requirements are not relevant as CTS surveys 100% census, including all ongoing Loan Clients. Areas for Development CTS does track client indicators such as loans, cycles some demographic information, but all available tracked social indicators were not included the data base. CTS will be including all social indicators in the next 2013 survey. CTS has not provided the detail of the PPI results at this time to MIX, however they have notified MIX that they are engaging in PPI Reviews. While CTS has not been able to meet some of the Advanced Standards, these are unfortunately an issue of timing: 1. Advanced Standard 12: Poverty levels have not been included in an Annual Report, simply because the Annual Report has not come out in 2012 (YE is June 30). This is just a timing issue and there are plans to include this data. 2. Advance Standard 17: CTS has not tracked any action or policies as they have not established a comparison with their baseline (planned for 2013). 6

7 Onsite Visit Analysis The onsite PPI Certification visit took place May 14 to May 19, 2012 and the final results review meeting on May 19, 2012 with CTS and on May 20, 2012 with Plan International, Bangladesh. The information collected was graciously provided by CTS management and staff during and after the onsite visit. Below is a summary of the areas of strength and those for development. We have also included some ideas for improvements. Overall Institutional Strengths: o o o Leadership Vision and Commitment: Leadership communicated a keen understanding of the value of this program and presented a very strong articulation of the areas of highest value to CTS. Additionally, CTS Leadership has clearly committed to the effort behind the process and leveraged the value of the PPI Program to include tracking the individual social case of all clients in the data base. CTS sees additional value of this program to include the ability to track and respond to beyond the aggregated results, but another set of data to assist clients at their individual level. This was particularly interesting to CTS because of the Extreme Poor and Beggars programs. It is a visionary effort that provides the catalyst behind the daunting work of a 100% census survey for a rural Microfinance organization in its first cycle. Staff Commitment: All Staff understood the PPI process and its value to the mission of CTS. The staff is well disciplined and diligent in this endeavor. Client Commitment: Client in the field also understood the program and could speak to the advantages of their future. Overall Areas for Development and Ideas for Improvement: o o Data Entry and Reporting: Naturally CTS will be manging the Data Entry and Reporting on their own staff time in July 2013, but they have already begun plans to absorb this additional work load in the short time frame. Transition plan - Plan Bangladesh has strongly supported CTS and the latter will start to assume almost all of the critical ppi processes by next collection period, a transition plan should be developed. The plan will provide guidance on key activities and timeline for it's execution. It will likewise outline the responsibilities and technical support that CTS may need as they assume more responsibility. 7

8 o o o MIX: CTS should consider reporting the results of their PPI to MIX. Currently, they only report their participation in the program to MIX. CTS may also consider reporting results to the local government or other agencies as appropriate. Funding: CTS may need to consider funding of the 2013 survey as Plan International, Bangladesh provided support (staff time) for the 2011 surveys. Social Indicators: CTS has collected many social indicators and plans to align these characteristics with the overall loan information and PPI in the next cycle. This will be an important addition to the tracking system to design into the next cycle. 8

9 Overview of PPI Use In 2009 CTS began an analysis of the PPI program in partnership with Plan International Bangladesh. CTS quickly became committed to the program and saw great benefits to the use of PPI for an objective and relevant assessment of their program value and product design. In this manner the PPI will provide important measurement and feedback to amend and more fully develop products to assist in their vision - developing the socio-economic condition of the poor and extreme poor people of the rural areas. PPI Collection Methodology: CTS, in their first PPI cycle, chose a 100% census of their client populations to maximize the relevancy of the study s ongoing value as a means to evaluate and improve product design and its impact on their target population. 142 CTS Loan Officers were dispersed through the work areas and collected 23,097 surveys during the months of January through May of The effort seems daunting, but efficiently completed. Plan International Bangladesh scrubbed data before overseeing the data entry of all 19,079 (after data scrubbing) for all 10 branches. The goal was to establish a baseline to measure poverty levels of the clients for this period and to be used in the future (July, 2013) for tracking changes over time. PPI Results for CTS Poverty Line National Poverty Rate; 2005 HIES Updated 2009 PPI Upper National Poverty Line 37.20% 41.2% Lower National Poverty Line 23.10% 25.6% USAID Extreme Poverty Line 17.90% 19.8% $1.25/Day/2005 PPI Poverty Line 47.50% 50.8% $2.50/Day/2005 PPI Poverty Line 87.50% 76.8% Analysis of 19,079 qualified scored cards. Data collection was from January to May of 2011 (23,097 score cards) 9

10 APPENDICIES Acknowledgements We acknowledge the efforts of Tina Oliphant, the Bankers without Borders volunteer from Montana West Economic Development in conducting the certification and material produced for this report. Appendix A: PPI Standard of Use Appendix B: Onsite visit agenda 10

11 Appendix A BASIC Progress out of Poverty Index (PPI) Standards of Use The BASIC PPI Standards of Use enumerate the minimum standards for appropriate PPI use and reporting. A PPI User fulfilling all of the following required standards is using and reliably producing accurate and transparent PPI results. BASIC Standards INTENT 1 Agreement to the following statement: My organization agrees to use PPI results to assess performance against our social mission as well as for product and service design and improvement. 2 The Board recognizes and approves the use of the PPI. 3 Written (action) plan for PPI use is in place. COLLECTION PROCESS 8 9 TRAINING The PPI is administered inside the client s home (that is, not in a centralized place like the branch office or group meeting). The PPI, as collected by the organization, has no significant deviations from the original country-specific PPI (questions, values and lookup table) as found on the website, Translations of the PPI match the national income/expenditure survey, on which the PPI is based. The latest available version of the PPI (scorecard and lookup table) is in use within one year of release. PPI information (including client scores and lookup tables) is stored in an electronic manner that allows for analysis. A basic system like a spreadsheet is acceptable. A quality control check of the accuracy of PPI collection occurs to verify that (a) the interview actually happened, and (b) the responses recorded by the staff (or enumerator) are reasonably close to those found also be the checker. This quality control check includes checking at least 5% of PPI scorecards collected. The check is performed by someone other than the original staff (or enumerator).

12 10 11 REPORTING Field staff (or enumerators) involved in PPI survey process understand the precise meaning of the PPI questions and responses. Data entry staff (or individuals involved in data entry) are properly instructed on the PPI data entry process. PPI results are reported to management with an appropriate frequency according to the organization's PPI integration plan. For example, upon completion of one time data collection or quarterly for continuous data collection. All reports must indicate what population the results represent. For example: incoming clients, mature clients, rural branches, all regions, all new clients in 2010, all mature clients as of 1/1/2010 etc. A description of how that representativeness is achieved must be provided. 14 PPI results include the number of PPIs analyzed. 15 Computation of the percentage of clients below each poverty line is accurately calculated. 16 PPI results indicate the dates when data collection occurred. 17 PPI results indicate the poverty line(s) used for analysis External reports and communications display confidence level and interval of PPI results, if information is available. External reports and communications indicate the PPI version (or underlying dataset) upon which the results are based (such as, 2004 Annual Poverty Indicators Survey in the Philippines). ADVANCED Progress out of Poverty Index (PPI) Standards of Use The following ADVANCED PPI Standards of Use outline the features of an integrated advanced PPI user. Organizations must meet all the BASIC standards and at least one advanced standard from each of the following categories to be classified as ADVANCED PPI users. ADVANCED Standards INTENT 1 The organization has identified a SPM/PPI "Champion" (or clearly articulated responsibilities for various staff) to manage and monitor the SPM/PPI process. 12

13 2 PPI results have informed a policy or operational shift. 3 PPI results have informed product design decisions and/or market research. PROCESS All staff (or individuals) involved in the PPI process (collection, entry, training, analysis, 4 and reporting) have clear descriptions of each element in the process. The capture of PPI data is integrated into the organization's management information 5 system (MIS). PPI data entry process is checked for accuracy. This check includes at least 5% of PPI 6 data and takes place after the data is entered into the data collection system. This check is performed by someone other than the data encoder. 7 Access to consolidated PPI data in the MIS is restricted to a small number of key staff. A back-up (either physical or electronic) exists for the collected PPI scorecards for back 8 checking. TRAINING 9 10 REPORTING Internal staff capacity exists to train new and current staff on the PPI. Training includes PPI concepts and operational requirements. Staff (or individuals) involved in data analysis are properly trained in accurate PPI analysis and reporting guidelines. 11 Poverty level results are integrated with other available client information in management reports (such as client demographics, product information, and other social indicators). 12 Poverty level results are included in the Annual Report. 13 Poverty level results are communicated regularly to field staff (for example, results are discussed at branch office meetings). 14 Poverty level results are communicated/reported to the board and/or investors. 15 Poverty level results are reported to the MIX (in the Social Performance/Poverty Measurement section) and/or to national associations. 16 An indication is made in external reporting whenever a specific donor or investor funded the use of the PPI within the organization. 13

14 17 Actions and policy changes resulting from PPI data analysis are tracked. 18 Poverty level results are compared with the local poverty rates of the country/region/state when available. TRACKING OVER TIME Progress out of Poverty Index (PPI) Standards of Use The TRACKING OVER TIME PPI Standards of Use outline the basic features of a PPI User using the PPI over time to track changes in poverty levels. TRACKING OVER TIME Standards COLLECTION 1 REPORTING 2 3 Repeat collection of PPI data from the desired population is taking place. If tracking changes in poverty levels, repeat collections occur at least once every three years. Inclusion of drop-out information (drop-out rate or poverty levels of clients that drop out if possible) of clients within the population being tracked over time with the PPI. Regular reporting to management on PPI results and trends (after each data collection or appropriate intervals). 14

15 Appendix B: Onsite Schedule Date Activity Participants May 13, Sun Arrive in Dhaka May 14, Mon Depart from Dhaka To CTS (10 hours) May 15,Tues Introduction Meeting with PPI Project and Senior Officers (Review this process) Md. Aminul Haque, Executive Director Md. Mustafizur Rahman, Consultant Subroto Kumar Das, Manager (Finance Admin) Sardar Md. Mahmudul Alom, Area manager Md. Aftabul Islam, Area manager Md. Rajwanul Islam, Training Officer Md. Shajedur Rahman, Accounts Officer Md. Nasir Uddin, Branch manager Md. Shah Alom, Branch manager Md. Ferdous Alom, Branch manager Krishna Khanta Roy, Branch manager Interview with PPI Project Lead Md. Aminul Haque, Executive Director Md. Mustafizur Rahman, Consultant Interview with PPI Training and analysis person Md. Aminul Haque, Executive Director Md. Mustafizur Rahman, Consultant Subroto Kumar Das, Manager (Finance Admin) Md. Rajwanul Islam, Training Officer May 16, Wed FIELD VISIT Observe two Branches and Center Meeting - Collect PPI from a HH who was a part of the original sample - Collect client story - Collect staff story Meet with Plan International Bangladesh Khamarpara Branch: Branch Manager: Md. Shah Alom Loan Collectors: Morjina Khatun and Mani rani Ray Members: Sayafa Khatun and Fatema Begum Tetulia Branch: Branch Manager: Krishna Khanta Roy Loan Collectors: Putul Ray and Aseyea khatun Members: Mahima Khatun and Hasina Begum Md. Mobarok Hossain, Program Unit Manager, Dinajpur unit Md. Assaduzzaman, Program Co-ordinator (SLP), Dinajpur unit 15

16 May 17, Thurs May 18, Fri May 19, Sat Meet with Project Managers to follow up: Quality Control Reporting Database issues Database indicators Interview with Executive Director - Discuss response to results Departure day; Return to Dhaka (10 hours) Back in Dhaka: Meet with Plan International Bangladesh: Technical assistance and training for CTS Data Entry and Reporting Md. Aminul Haque, Executive Director Md. Mustafizur Rahman, Consultant Subroto Kumar Das, Manager (Finance Admin) Md. Aminul Haque, Executive Director Mohammad Tarequl Hoque, Plan International Bangladesh Muhammad Awais, Plan International Bangladesh M. Rafiqul Islam, Plan International Bangladesh 16