Evaluation and Monitoring in Programme Management of ESI Funds

Transcription

1 Evaluation and Monitoring in Programme Management of ESI Funds Intensive Workshop March 2017 Brussels EXPERTS Expert at the Government office, at the Ministry of Labour and at the Charles University in Prague; Member of Czech Evaluation Society, Prague Benedict Wauters Deputy Director for Innovation, Evaluation and Methodology, ESF Agency Flanders, Belgium Stanisław Bienias President of IDEA Institute Ltd. and former Head of the National Evaluation Unit of the Polish Ministry of Regional Development, Poland The workshop will provide participants with a deeper understanding of what evaluation can/cannot deliver and therefore mainly focus on the presentation of evaluation/monitoring methods and their limits. The goal of the seminar is to increase your ability to design, manage and monitor the evaluations mainly linked to ESI funds: Learn about the inventory of evaluation methods (process evaluation, impact evaluation and counterfactual impact evaluation) and when/how to apply them Understand what impact evaluation is and what are the different techniques used for its performance Make judgments about the situations (and/or interventions) when impact evaluation is appropriate to use Know which kind of evaluation or decision-making questions is best suited to answer the requirements of the European Commission and can ensure the optimal implementation of your programme Prepare the Terms of Reference for the tender of evaluation services and establish a management plan for cooperation with a sub-contractor

2 Evaluation and Monitoring in Programme Management of ESI Funds General Information Current Challenges for Responsible Authorities Are you aware of the obligations regarding monitoring and evaluation during the management of Operational Programmes and EU-funded projects? Are you able to ensure a high quality of evaluation? Which activities of managing authority have a true added value for applicants and beneficiaries of the individual programmes? What is done unnecessarily in the implementation of the ESI Funds that blocks the administrative capacity? What shall be changed to improve the ESI Funds performance? What is the effect of ESIF funded interventions? Is there an impact on citizens life? Do we deliver results? Evaluation is an essential part of sound management and coordination of assistance from the European Structural and Investment Funds (ESI Funds). The European Union legislation for the programming period puts a stronger emphasis on evaluations than before. Methodology The workshop will provide participants with a deeper understanding of what evaluation can or cannot deliver and therefore mainly focus on the presentation of evaluation and monitoring methods and their limits. The goal of the seminar is to increase your ability to design, manage and monitor the evaluations mainly linked to ESI funds, including practical Workshops. Participants will receive extensive materials: All presentations shown during the training (inclusive online access to all presentations after the training) Confirmations of Participation Selection of corresponding journals published by Lexxion Who should participate? Public officials responsible for the monitoring and evaluation of Operational Programme and EUfunded projects working in: o o o Managing Authorities Certifying Authorities Intermediate Bodies Representatives from ESI Funds authorities in charge of implementation and management of Technical Assistance Evaluation officers in the National Coordination Authorities supervising the management and implementation of ESI Funds in their Member States Applicants and beneficiaries of EU funded projects, i.e.: Private and public companies, associations, research institutes Consultants, auditors and other stakeholders involved in the management of evaluation procedures for policies, programme and projects Why should you participate? Learn about the inventory of evaluation methods (process evaluation, impact evaluation and counterfactual impact evaluation) and when/how to apply them Understand what impact evaluation is and what are the different techniques used for its performance Make judgments about the situations (and/or interventions) when impact evaluation is appropriate to use Know which kind of evaluation or decision-making questions is best suited to answer the requirements of the European Commission and ensure the optimal implementation of your programme Learn which evaluation techniques you should apply to evaluate EU projects and programmes What do you need to take into account when drafting Terms of Reference (ToR) for external evaluators Get informations what does a good evaluation plan require and how can you elaborate your existing evaluation plan in accordance with the ongoing logic framework regulation

3 Evaluation and Monitoring in Programme Management of ESI Funds Programme Day 1 09:00 Registration 09:30 Welcome note by Vladimir Kváča (chair) and Lexxion Session 1: The Importance of Monitoring and Evaluation for Effective OP Management 09:45 Monitoring and Evaluation Roles for the Control System of ESI Programmes The role and significance of Monitoring and Evaluation for Operational Programme Requirements and obligations for bodies involved The role and selection of indicators: Successful measurement of results and its limits Planning and Timing of Activities Using the right Method The role of evaluation plan, Expert at the Government office, at the Ministry of Labour and at the Charles University in Prague; Member of Czech Evaluation Society, Prague 11:00 Coffee Break Session 2: Inventory of Evaluation Methods: Process Evaluation 11:30 Process Evaluation of ESIF Implementation (Evaluation and Technical Assistance) Understanding Technical Assistance role in the Programme Management How process evaluation helps you to target your Technical Assistance Requirements and obligations of the Managing Authority / Intermediary Body Process monitoring measures Case study: Evaluation of project selection pipeline 13:00 Lunch Break 14:00 Practical Considerations for Successful Process Evaluations How to Improve your System How to design project proposals to go quickly through the project selection pipeline? Make sure that beneficiaries understand your call documentation correctly Rejection of project proposals what are the reasons and how to handle them? Session 3: Inventory of Evaluation Methods: Impact Evaluation 15:00 The Role of Impact Evaluation: Introduction What is impact evaluation and how can it support your Operational Programme How to measure results of your projects and programms 15:45 Coffee Break 16:00 Theory Based Impact Evaluation Foundations of causal inference in theory based impact evaluation Realist evaluation and process tracing Contribution analysis and congruence analysis Benedict Wauters, Deputy Director for Innovation, Evaluation and Methodology, ESF Agency Flanders, Belgium 18:00 End of Day 1 Just send us your cases and questions in advance! We will forward them to our speakers, who will include your issues in the programme and answer them during the workshop. All information concerning your questions and cases will be treated confidentially. As the number of participants is limited, participants will benefit from a maximum of exchange and discussion of their individual questions.

4 Evaluation and Monitoring in Programme Management of ESI Funds Programme Day :00 Counterfactual impact evaluation (CIE) What Is It? Random Control Trial as a golden standard for analysis of intervention effects Quasi experimental methods as a substitute of Random Control Trial Difference-in- difference Matching and Propensity Score Matching Stanlislaw Bienias, President of IDEA Institute Ltd. and former Head of the National Evaluation Unit of the Polish Ministry of Regional Development, Poland 10:15 Coffee Break 10:30 Conduct of Counterfactual Impact Evaluation in Practice Discontinuity design Counterfactual impact evaluation when it is applicable? Data requirements precondition for counterfactual impact evaluation What to remember when drafting a Terms of Reference for CIE Ethical issues Stanisław Bienias Session 4: Evaluation of Communication Session 4: Assignement of Evaluation to External Services 13:45 Cooperation with External Evaluation Services: What Needs to be Considered? How to best structure the evaluation process? Which activities could be conducted inhouse and extern? How to manage and control the evaluation projects? 14:30 Writing Terms of Reference and Procurement of Evaluation in ESIF Services Terms of Reference as a step in evaluation design Terms of Reference as a step in procurement of evaluation services ESIF Real-Life ToR examples Price issues Selection Criteria Award Criteria Alternatives to procurement 15:30 Coffee Break 12:00 Evaluation of Communication Measures What needs to be included? Specific aspects of evaluation of communication measures What is the best timing? Tbc 12:45 Lunch 15:15 Writing Terms of Reference and Procurement of Evaluation in ESIF Services Use your new knowledge of methods and write your own Terms of Reference for ESIF evaluation Communication measurements of evaluation reports Tips and tricks in procurement of evaluation services 16:45 Q&A 17:00 End of the Workshop

5 Evaluation and Monitoring in Programme Management of ESI Funds Experts Expert at the Government office, at the Ministry of Labour and at the Charles University in Prague; Member of Czech Evaluation Society, Prague has been working in the field of EU Funds since 2007 holding several positions including Head of the ESF Evaluation Team or Director of the Managing Authority of the OP Technical Assistance. From 2011 to 2013 he served as Director General of the Section for EU Funds at the Ministry of Labour and Social Affairs, Czech Republic. Simultaneously, he was Assistant Professor at the Jan Masaryk Centre of International Studies at the Prague University of Economics (until 2012). is a member of the Czech Evaluation Society and is actively involved in the Community of Practice on Results Based Management in Structural Funds. He strongly supports the principles of evidence based policy in the implementation of the European Structural and Investment Funds in the Czech Republic. Benedict Wauters Deputy Director for Innovation, Evaluation and Methodology, ESF Agency Flanders, Belgium Benedict is an expert in strategy / policy / programme / project formulation, implementation and evaluation as well as organisational development. He has worked for more than 15 years in and for multilateral public institutions (e.g. United Nations, OECD, European Commission), regional and national government departments (e.g. Dutch Ministry of Foreign Affairs, Dutch Ministry of Transport, and the Belgian Regional Governments of Flanders, Brussels and Wallonia), local government (e.g. City of Antwerp) as well as NGOs and public corporations (including Greenpeace, the Flemish port of Ghent, the Dutch National Railway Company etc.). He has been active as a consultant for Deloitte Consulting in Belgium, the Policy Research Corporation in the Netherlands and as an independent expert. In addition, he has been a lecturer at the Rotterdam Business School, teaching strategy, risk management and research methodology in the Master and MBA programmes, while also chairing the school s curriculum board. Currently, he is responsible for innovation, impact evaluation and methodology at the European Social Fund (ESF) department in the Flemish Ministry for Labour and Social Economy (Belgium). In addition, he has led for five years the EU-wide Community of Practice on results based management. In this capacity, he also contributed to the European Commission s new toolbox on quality of public administration. Currently, he is leading the transnational network on public administration and governance on behalf of the EC. At the Haagse Hogeschool in The Hague he lectures on external risk management in the Master programme on risk management. He is also responsible for the skills lab in evaluation for the Master in Public Management at the Antwerp Management School. Stanisław Bienias President of IDEA Institute Ltd. and former Head of the National Evaluation Unit of the Polish Ministry of Regional Development, Poland From 2004 to 2013, Stanisław Bienias worked in the Polish Ministry of Regional Development for the National Evaluation Unit. He became Head of this unit in He was responsible for organising the process of evaluation for the National Development Plan and the National Strategic Reference Framework as well as the negotiation of regulations concerning evaluation, monitoring and indicators for Furthermore, Stanisław Bienias coordinated the work of more than 50 evaluation units located in Operational Programmes and was responsible for building the evaluation culture in the public administration. In his extensive experience, he took part in the preparation of more than 100 evaluation researches and dozens of conferences and evaluation trainings. For the last nine years Stanisław Bienias represented Poland in the DG REGIO Evaluation Network meetings. In June 2013 he founded the non-profit organization Foundation IDEA Institute for Development, Evaluation & Analysis.

6 Evaluation and Monitoring in Programme Management of ESI Funds Registration Venue Management Center Europe - Conference Center Rue de l'aqueduc Ixelles, Brussels Belgium Participation fee YES, I would like to participate for the reduced fee for public authorities EUR 1.090,00 (excl. VAT). YES, I would like to participate for the regular fee of EUR 1.590,00 (excl. VAT). YES, I am EStIF-subscriber (10% discount applicable). Belgian VAT of 21% will be added where applicable. The participation fees include the workshop material, drinks and snacks during the breaks and lunch. Newsletter subscription Please send me information about Structural and Investment Funds issues for free. I may unsubscribe from this service any time. Cancellations Cancellations should be made in writing per post or fax. Please note that the full fee will be charged for cancellations received after 13 March 2017 (included illness or impediment which prevents him from fulfilling his duties). Nomination of a substitute remains possible. A cancellation arising by reason of too small a number of participants is effected no later than two weeks prior to the seminar. In this case the participation fee will be refunded. A requirement for reimbursement of travel and accommodation expenses as well as loss of working hours is not possible, unless such costs result from wantonly negligent or deliberate behaviour on the part of the organiser. For more detailed information, scope and conduct of contract as well as cancellation policy, please check the Terms & Conditions on our website. Please register with: Hella Baumeister Lexxion Publisher Güntzelstr Berlin, Germany Phone: +49 (0) Fax: +49 (0) baumeister@lexxion.de online: Name Position, Department Company Name Address Address Phone, Fax Customer No. of EStIF-Subscription VAT Reg.-No. Date, Signature Upcoming events State Aid on R&D&I 3-4 April 2017 Brussels How to Most Effectively Use Technical Assistance for ESI Funds ? 4-5 May 2017 Budapest Two-day intensive workshop for representatives from public authorities in charge of implementation and management of Technical Assistance for ESI Funds