Strengthening the MIS in Social Protection Programs: A Toolkit. Maria Arribas, LCR Cesar Baldeon, ISG May 17, 2007

Transcription

1 Strengthening the MIS in Social Protection Programs: A Toolkit Maria Arribas, LCR Cesar Baldeon, ISG May 17, 2007

2 Purpose, Audience, Methodology A Mitigation tool for Control and Accountability Risks & A framework for the Implementation of an MIS in CCTs/SP Programs A practical tool for TTLs an WBG clients working on SP Programs (with a focus on CCTs) Based on the MIS assessment of Ecuador s CCT; LCR s 2 day CCT review workshop; Visits to Colombia, Chile, and Argentina; Literature Review; and Conversations with several TTLs 2

3 Two Dimensions A look at Control and Accountability Processes in CCTs: An MIS Perspective A framework for the Implementation of an MIS in SP/CCTs Beyond having a Unique Identifier and Cross-checks A more systematic and strategic approach to systems management Useful for both paper-based and automated programs/processes 3

4 Toolkit Components Control & Accountability: MIS Perspective MIS Framework & Operational Risk Framework Case Studies MIS Implementation and Monitoring Checklist Considerations/Recommendations Not an ICT Project Implementation Toolkit! 4

5 MIS for Safety Net Programs Information Providers and Consumers Potential Beneficiaries Government Institutions (Ministries) Service Providers Financial Institutions Civil Society Information flows Program Process Beneficiary Id. Targeting Registry Database Conditions Data Collect Payment Eligible P Control Grievances Registration Validation Verification Payment Processes Graduation Updates Penalization Reconciliation Impact MIS Components IT Expertise & Organization Structure Information Application Quality Information & Communication Technology Infrastructure 5

6 Control & Accountability: An MIS Perspective (1) Information Providers and Consumers Potential Beneficiaries Government Institutions (Ministries) Service Providers Financial Institutions Civil Society Information flows Program Process Beneficiary Id. Targeting Registry Database Conditions Data Collect Payment Eligible P Control Grievances Registration Validation Verification Payment Processes Graduation Updates Penalization Reconciliation Impact MIS Components Functions IT Expertise & Organization Structure Information Application Quality Information & Communication Technology Infrastructure 6

7 Control & Accountability: An MIS Perspective (2) Processes Risks Causes MIS Functions Tools (examples) Beneficiary Identification Inclusion Errors Exclusion Errors Ineffective eligibility criteria Imperfect targeting tool Incoherent Registration Processes Human Errors Fraud Data Quality Data Security Records Separation of Functions Cross-checks Access Control Audit Trails Security Classifications Archiving Strategy Beneficiary Registry Service Interruption Unauthorized access to info Unauthorized changes Creation of false info/transactions Environmental events System breakdown Malicious Acts Human Errors Disaster Recovery Availability Capacity Planning Security Data Warehouse Back ups and redundant systems Downtime management Access Control (logical and physical) Unique identifiers Cross-checks Monitoring of Coresponsibilities Unwarranted penalization Unreliable data Misleading Impact Evaluations Dishonesty Inconsistent collection/recording Human Errors Complex Data Integration Data Governance Database Data Quality Data Architecture Spot checks Audits Error/inconsiste ncies alerts 7

8 Control & Accountability: An MIS Perspective (3) Processes Risks Causes MIS Functions Tools (examples) Payment of Benefits Irregular Payments Inaccurate Payments Interruption of Payments Insufficient Funds Human Errors System Availability Bribery/Dishonesty Service Disruption Data Quality Master Data Systems Availability Mgt Error/inconsiste ncies alerts Service Level Agreements Monitoring & Evaluation Uninformed decision making Reputation Risks Political Risks Lack of Reporting Lack of Transparency Data Quality/Integrity Records Data Availability Data Usability Data warehouse Reporting Tool History Tracking Reports Transparency Case 8

9 Control & Accountability: An MIS Perspective (4) Processes Risks Causes MIS Functions Tools (examples) Institutional Arrangements Lack of interministerial coordination Lack of Enforcement Political Manipulation Enforcement of decentralization arrangements Lack of funding Confusing Roles and Responsibilities Political Volatility Weak Institutions Lack of Political Will Data Governance Org. Structure Service Level Agreements Publication & Transparency Information Agreements Publication on Website Complaint Resolution and Appeals Program Abuse Ping-Ponging between offices Program credibility Misleading Impact Evaluations Missed Opportunity for demand driven improvements Inexistent feedback systems Feedback black-box Broken processes Culture (resignation) Records Process Monitoring Case System Service level monitoring Alerts for systemic appeals (category, geographic, etc.) 9

10 Integrated Framework MIS Components Information Providers and Consumers Potential Beneficiaries Government Institutions (Ministries) Service Providers Financial Institutions Civil Society Information flows Program Process Beneficiary Id. Targeting Registry Database Conditions Data Collect Payment Eligible P Control Grievances Registration Validation Verification Payment Processes Graduation Updates Penalization Reconciliation Impact MIS Components IT Expertise & Organization Structure Information Application Quality Information & Communication Technology Infrastructure 10

11 Why a Framework? Logical Structure Systematic Approach to Risk MIS Framework & Operational Risk Framework Integration among building blocks (and how they constrain one another) Prioritization Cost/Benefit Analysis 11

12 Integrated Framework MIS Components IT expertise and Org. Structure: Organizational Structure Staff Development External Expertise Separation of Duties Information : Data Governance Data Architecture, Analysis & Design Database Data Quality Data Security Master Data Data Warehousing Records Meta Data Application Quality : Quality Planning Quality Control Configuration Change Version Control Release Infrastructure: Capacity Planning Availability Disaster Recovery Security 12

13 IT Expertise and Organization Structure Functions Organizational Structure Staff Development External Expertise Separation of Duties Outputs Inclusion of the IT team in the design of the program Definition of roles & responsibilities in the operation manual Inclusion of the IT team in the operational assessment of the program Assessment of in-house skills and gaps Outsourcing strategy: Identify skill to retain internally Staff Training Plan Outsourcing strategy: identify external expertise required Procurement Plan for external expertise Assessment Process for external parties performance Identification of functions that require separation of duties Definition of roles & responsibilities Evaluation of roles & responsibilities 13

14 Information Quality Functions Data Governance Data Architecture Database Data Quality Data Security Master Data Data Warehousing Records Metadata Outputs Identification of authoritative sources Formalization of institutional arrangements with data custodians Formalization of data governance processes Data Governance Control Business data model: entities, attributes, relationships Logical data model Physical database model Data inventory: data inputs and outputs for each process Data dictionary Beneficiary database Monitoring Plan for database performance Data validation processes Definition/Implementation of data validation rules Data quality evaluation Information security policies Implementation of user profiles (logical security) Implementation Plan of data flows and schedules with data custodians Identification of monitoring & evaluation indicators Definition of reporting requirements (managerial, detailed, ) Training & roll out strategy Definition of Record processes (procedures and standards) Definition of Metadata Requirements Metadata Dictionary 14

15 Application Functions Quality Planning Quality Control Configuration management Change Version Control Release Outputs Definition of Quality processes including: Quality control Change management Version control Configuration management Release management Definition of roles & responsibilities for processes Assessment of Quality processes Definition of functional and acceptance test processes Assessment of quality control processes Systems Change Policy Documentation - List of MIS components included in the final release List of modified and new components after final release Definition of change management processes (change planning, impact, feasibility, cost analysis, documentation, release) Assessment of Change Process Version all components and final release (baseline) Version all components and up-to-date release after final version Monitoring Plan for version control Definition of user and technical documentation requirements Documentation for final release and subsequent changes Training Plan 15

16 Infrastructure Functions Capacity Planning Availability Disaster Recovery Security Outputs Needs Assessment (hardware, software and network requirements) Procurement Plan for Hardware, software and network Monitoring capacity, availability and security Assessment of performance and capacity Re-evaluation of hardware, software and network sizes Definition of availability required (tolerance levels) Definition of Systems architecture Maintenance/Renewal Strategy of hardware, software and network Availability Testing Plan and Schedule Recovery Testing Plan and Schedule Assessment of system s availability Definition of failure and recovery plans Back-ups and disaster recovery strategies Testing Plan for failure and disaster recovery Assessment of disaster recovery plans Definition of physical security for hardware and network Implementation of security monitors and controls Assessment of access monitors and controls 16

17 MIS Operational Risks Framework Risk Info. Software Quality IT Infrastructure Human Capital Interruption: lost, unavailable, unusable Data Governance Data Architecture Database Data Warehouse Quality Control Version Control Change Capacity planning Availability management Disaster Recovery Security Organizational Structure Staff Development External expertise Interception: unauthorized access Data quality Records Quality Control Security Separation of duties Modification: unauthorized changes Data quality Records Quality Control Security Separation of duties Fabrication: false transactions Data quality Records Quality Control Security Separation of duties 17

18 Country Experience (1) Colombia Program Name: Familias en Acción Age: 7 years, since 2000 as part of the RAS (Red de Apoyo Social) Type: Conditional Cash Transfer Counterpart: The Presidency Criteria: provides cash to poor households in rural areas conditional on school attendance of school-aged children and visits to health facilities and participation in nutritional programs for younger children and their mothers Size: as of January 2007 the program has 600,000 beneficiaries, it is expected to grow to 1.5 million beneficiaries within the next 9 months. Program Design and Implementation: Decentralized Program with liaisons on each of the participating municipalities. Most of the decentralized work is paper based The program has not separated functions to avoid conflict of interest Little validation of information with external systems SISBEN, the targeting database, is updated as new municipalities are added to the program SIFA leverages the Presidency's investments for the four MIS components Potential Scalability Problems results of the growth of the program Complaints is a paper based decentralized process. Very difficult to produce process indicators 18

19 Country Experience (2) Argentina Program Name: Jefes y Jefas Age: 6 years, as part of the RAS (Red de Apoyo Social) Type: Workfare Criteria: transferred AR$50 per month to participants. The beneficiaries were to: (i) be unemployed, (ii) be head of a household, (iii) live in a household with at least one minor below the age of 18, or pregnant woman, or handicapped of any age; (iv) work or participate in a training/education activities for 4-6 hours a day. Size: in May 2003: ~2 million reduced January 2006: 1.42 million Program Design & Implementation: Decentralized program where municipalities have access to web base beneficiary registry. Some information continues to be exchanged via excel sheets. The program has separated functions. It includes a production team Beneficiary database is validation with social security (participation in other programs and taxes paid.) Jefes y Jefas MIS leverages the investments of the Ministry's investments for the four MIS components Scalability Program leverages the ministry s central call center for complaints. 19

20 Country Experience (3) Colombia Strengths IT team Skills Systematic Approach to Data Records Defined (?) Robust Infrastructure Economies of Scale Weakness Large Amount of Paper Trail Prompt to Error and lack of traceability Scaling Capacity Not Tested IT Staff Involvement (Structure) Long Payment Cycle (in part due to Manual Processes 60 days) Interoperability/Information Quality Argentina Strengths Scaling Capacity Separation of Functions and Environments Web-based Application Robust Infrastructure Data Warehousing Case System Weakness Application Version Control Real Time Validation with Authoritative Sources Decentralization Challenges 20

21 Considerations/Recommendations Country Context Program Maturity and feedback Paper and Pencil vs. Electronic Automation where it increases efficiency Complementary Systems Procurement Economies of Scale Knowledge Sharing/Collaboration for IT teams Systematic System Reviews (supervision) 21

22 Q & A 22