MID-TERM EVALUATION OF THE EURO-MEDITERRANEAN YOUTH ACTION PROGRAMME - EXECUTIVE SUMMARY

Transcription

1 EXECUTIVE SUMMARY MID-TERM EVALUATION OF THE EURO-MEDITERRANEAN YOUTH ACTION PROGRAMME - EXECUTIVE SUMMARY Background This report presents the findings of the mid-term evaluation of the Euromed-Youth programme, whose pilot phase started in 1999 for a period of two years. The evaluation was undertaken by ECOTEC Research and Consulting for the European Commission and contracted for the period June 2001 to August They key objective of the evaluation was to assess the relevance, efficiency, effectiveness and sustainability of the Euromed-Youth programme, as well as its impact and achievement of the general and specific objectives. The evaluation focused on reviewing, in quantitative and qualitative terms, progress on the current programme (including its strengths, weaknesses and main obstacles faced), on regional and national coordination, on the Mediterranean Youth Forum and to identify possible needs for reorientation for the subsequent implementation phase. The evaluation was informed by briefing meetings with the various Directorates General of the European Commission involved in its implementation, a review of secondary source information, consultations with the National Coordinators (NCs), EC delegations and government representatives as well as with other key actors and stakeholders, and case studies of a range of Euromed-Youth projects. The general aims of Euromed-Youth as specified in the programming document are to facilitate the integration of young people into social and professional life and stimulate the democratisation of the civil society of the Mediterranean partners by stimulating active citizenship within local communities, by promoting active participation of young people, in particular of young women and young people s associations, and by developing the employability of the young people involved. Euromed-Youth is one of the three regional programmes implemented within the third chapter (social, cultural and human affairs) of the Barcelona process, which aims at the development of the Euro-Mediterranean Partnership between the 15 EU Member States and 12 Mediterranean partner countries (Algeria, Cyprus, Egypt, Israel, Jordan, Lebanon, Malta, Morocco, Palestine, Syria, Tunisia and Turkey). The programme is implemented following the same guidelines, rules and criteria as applied for activities with third countries within the YOUTH programme and is centrally managed by the European Commission. Euromed-Youth programme activities concern: Action 1: Youth for Europe exchanges Action 2: European Voluntary Service Action 5: Support measures Programme key statistics The EU planned funding for Euromed-Youth for the period was 9.7 million euro, of which 6 million were to be provided by the MEDA programme and 3.7 million by YOUTH. At the time of the evaluation, figures available 1 on commitments for programme activities and management by the NCs amounted to a total of million euro, i.e. 38% of the total envisaged funding. Updated figures subsequently provided by the EC indicate that, at present, commitments have risen up to 6,612 million euro (i.e. 68% of the total funding). This figure also includes around 500,000 euro for training incurred by SALTO and decentralised activities organised by the NAs, costs not initially foreseen as being an integral part of the Euromed-Youth budget. The activities undertaken within the programme also receive co-financing from the national and local levels, but precise figures on the scale of these contributions are not available. 1 Covering Euromed-Youth activities funded up to the November 2000 deadline for project applications.

2 At the time of the evaluation, the programme had implemented a total of 134 projects and totalling 3,157 participants (updated figures subsequently provided by the EC show that after the two most recent selection rounds the total number of approved projects is 211). The bulk of the million euro of committed programme resources (47.3%) were allocated to Action 1. This action also accounts for the highest number of projects (66 projects or 49.3% of the total number of projects), as well as for the highest number of participants (2,170 or 68.7% of the total number of beneficiaries). With regard to Action 2, this type of activity accounts for a budget share of 9.3%, a project share of 21.6% (29 projects) and a participants share of 2.3% (73 beneficiaries). Action 5 reached a budget share of 24.3%, a project share of 29.1% (39 projects) and a participants share of 29% (914 beneficiaries). An analysis by year shows substantial growth, both in terms of financial and physical programme outputs, between 1999 and 2000 and a further strong increase is noted in In terms of country participation, 23 of the 134 projects were led by countries from the Mediterranean region (Israel, Jordan, Malta, Palestine, Tunisia and Turkey). The remainder of the projects had an EU partner leading organisation, which particularly active countries being France, Germany, the Netherlands and Italy. The country distribution of participants shows a much better balance: 52% of the total number of participants originated from the EU countries, whilst 48% came from the Mediterranean region. The gender distribution of the programme participants was also well balanced (49% female against 51% males), with some variances for example in Action 5, where, especially in the year 2000, there was an imbalance towards a higher share of male participants (67% male against 33% female beneficiaries) and between the partner countries. Relevance of the Euromed-Youth programme Overall, the programme content seems to be relevant for young people living in the Mediterranean partner countries and its design adequately addresses their identified problems and needs. The formal decision to launch the Euromed-Youth programme was in fact preceded by an extensive preparatory and consultation process, involving national and local key actors and stakeholders, which informed overall programme design. Some reservations can be made as to whether the overall age limit of the YOUTH programme (maximum age 25 years) is not too restrictive for youth in the Mediterranean area. With regard to the specific design of Euromed-Youth, more consideration could have been given to the substantial socioeconomic, cultural and political gaps and differences between the EU and the Mediterranean countries as well as among the individual partner countries in the Mediterranean region. As a whole, however, the programme complements well larger programme schemes focusing on youth, i.e. its mother programme YOUTH and the other MEDA programmes. Weakness in the programme design were identified with regard to the internal consistency of the objectives and expected results as stated in the basis document of Euromed-Youth, i.e the financing plan. Also, the monitoring and evaluation indicators proposed in the programme design do not allow for adequate measuring of the extent to which objectives and expected results are being attained. The overall degree of flexibility and adaptability of the Euromed-Youth programme is good in terms of providing (ad-hoc) solutions to problems and issues, but rigid when looking at its formal design and structure. With regard to on-the-ground absorption and implementation capacities in the Mediterranean countries, there is an overall high potential for developing and implementing projects. For various reasons, some countries show a higher effective capacity than others, where the potential to prepare and implement projects needs to be further encouraged and developed. Efficiency review

3 Overall efficiency of the Euromed-Youth programme can be considered as rather low, but there are some positive signs showing improvements between the first and second year of activity. Whilst it is natural that a pilot programme of this nature and geographical coverage encounters problems and obstacles in its start-up phase, changes to the management structures and monitoring / review systems are necessary to allow for a smoother implementation and to avoid a decrease / stagnation of currently growing participation rates. Means, costs and timeframe When assessing the financial and physical outputs of the programme up to the November 2000 project application deadline, overall progress seems to lag behind initial projections. This relates especially to activities under Action 2, the European Voluntary Service, with only 9.2% of the initially anticipated budget allocation, 3.7% of the initially anticipated number of projects and 9.4% of the initially envisaged number of participants achieved to date. Figures on Action 1 and 5 show better results, given that Action 1 has attained 55% of the initially expected number of projects and 50.3% of the initially anticipated number of participants and Action 5 respectively 43.8% and 39.3%. The above actions common budget share is 60.9% of the initially planned budget for these types of activities. In terms of cost-effectiveness / unit costs, the average costs per project, participants, as well as the average number of participants in each project are only slightly higher than projections made on the basis of averages for third country projects implemented prior to the establishment of the Euromed- Youth programme and seem to offer good value for money. With regard to programme management at national level in the Mediterranean countries, from the initially envisaged 1.44 million euro, 0.7 million has been committed through the contracting and the development of the work programmes by the NCs. Progress on the work plans varies greatly between the countries but is overall in line with planned activities. Issues raised with regard to the budget breakdowns included in the work programmes concern its lack of flexibility between the budget headings and underestimation of certain costs, e.g. those related to travel. From the other actors involved in Euromed-Youth, the national agencies (NAs) raised the issue of overall resources available to manage third country projects being too low. Beneficiary organisations raised concerns on accessing local co-financing, lack of own resources and fund-raising capacities, as well as unclarity and inflexibility of the budget structures in the project applications. Management of the programme The centralised management structures and procedures of the programme are considered as both difficult to understand by NCs and beneficiaries and not sufficiently adapted to the Mediterranean countries context. The centralisation / distance of the EC also seems to generate many problems in terms of credibility, visibility, and management of the programme. In addition, there is an overall lack of clear understanding and knowledge in the Mediterranean countries (NC) of the responsibilities of the different Commission services and the TAO. In all partner Mediterranean countries, except Morocco, NCs have been appointed. Delays in issuing contracts and in receiving the first payment have been reported, and there is an inadequacy / lack of human resources in countries where the work plan has been implemented as foreseen or in large countries. The level of technical resources available to NCs seems adequate. Interviews with NCs have shown that cooperation between NC and NAs has been relatively limited and there is a clear need for NCs for a more structured form of cooperation with NAs. The involvement and role of the EC Delegations in the implementation of the programme varies greatly from one country to another, but remains overall relatively low. With regard to national governments, they formally do not play any role in the management of the programme, but have a key input in terms of paying the salaries of the NCs. In most countries, the government s involvement has been higher than anticipated in terms of financial and logistical support. This demonstrates both national government s interest in the programme, but also makes the NCs accountable to their

4 governments, which in turn may introduce some political bias in the work of the NC. The role of regional and local institutions in the programme implementation is also worth stressing, especially in terms of logistical support and relays to disseminate information on the programme. An important and problematic aspect of overall programme management is the selection process, which is centralised for all activities supported under the Euromed-Youth programme. Overall, the process is considered as too long (6-7 months) and not adapted to the reality of youth NGOs in the Mediterranean countries. The use of selection criteria is also perceived as unclear and inconsistent by beneficiaries. Some of the organisations met, which project had been rejected, expressed high discouragement and doubts in relation to future applications. Although with regard to communication and promotion significant efforts have been made at national levels, little action has been taken at EU level and in EU countries. At EU level, the general YOUTH User s Guide is in most cases the first and only EU document sent to potential applicants to a project. Most NCs and programme beneficiaries considered that the information related to Euromed- Youth was scattered throughout the guide, difficult to understand and sometimes ambiguous. Most actors argued for the development of a specific Euromed-Youth User s Guide, preferably in the language of their country. A specific section on the Euromed-Youth programme is included in the general YOUTH website of DG Education and Culture. Only a limited number of actors seem to be aware of the site s existence and it would be useful to have a specific website for the programme. Given that no other information specifically relating to Euromed-Youth are available from the European Commission, there is a need for programme-specific communication and information tools. Information available does not seem to suggest that the EU National Agencies have played a key role in promoting the Euromed-Youth programme in EU countries and that there is a lack of awareness about the programme in many EU associations. The EC is currently undertaking action to tackle these issues. The European Youth Forum (EYF) is active in the promotion of the Euromed-Youth programme and regularly informs its members on developments and opportunities linked to the programme. At national level, approaches and strategies adopted by the National Co-ordinators to promote the Euromed-Youth programme vary. Whilst each NC tend to develop its own information tools in isolation, increased sharing of information and contacts would be welcomed by all NCs. Some national government representatives and EC Delegations play an active role in raising awareness on the Euromed-Youth programme by referring to the programme in policy documents, information brochures, or involving the NC in other activities relating to youth or the EU. Whilst at local level, in the majority of cases, information on the programme is being spread by word of mouth, some youth organisations have organised concrete activities to raise the overall awareness of their members and other potentially interested actors. Intervention methods The three types of measures implemented within Euromed-Youth have been unanimously considered as responding to the needs of young people. The support measures (action 5) are considered, in most cases, a prerequisite for participation to actions 1 and 2. Almost all beneficiaries stressed that this action should be encouraged further and in a few cases, that activities should be more clearly defined. A practical issue with regard to programme participation is getting a visa, which is a major obstacle faced by NCs, youth leaders and young people. Action 1 is considered as very important, in terms of visibility of the programme and multiplier effect. However, it was repeatedly stressed that projects should not be limited to exchanging ideas and experiences but should be followed-up by concrete actions/activities. Careful definition of the projects specific objectives, careful selection of participants and strong involvement of partner

5 organisations and young people as all stages of project development are considered as the main success factors in relation to action 1 and 5. With regard to action 2, the fact that the concept of voluntary service hardly existed in many Mediterranean partner countries and that it is difficult to distinct EVS activities from more traditional forms of development cooperation, as well as project duration, age limitations and cultural differences are inhibiting participation to voluntary service activities. When looking at other intervention methods, such as training and guidance, it becomes evident that overall, this has been insufficient for the NCs. The two training seminars initially planned by the European Commission have not been organised to date. Instead, NCs have been invited to participate to a NA business meeting seminar, which was considered by most NCs to be too advanced. Most of them also participated to a two-week job shadowing in the Youth TAO, which was a useful experience. The need for further job-shadowing in NAs was outlined on several occasions by NCs, as well as more intensive cooperation and joint NA-NC participation in training events in general. The French NA, INJEP, has been designed as one of the 4 Support for Advance Learning and Training Opportunities (SALTO) centres created in September 2000 for the Youth programme, with focus on Euromed cooperation. Whilst it seems that the Commission s objective is that SALTO-INJEP becomes a resource centre for the Euromed-Youth programme, activities undertaken so far show that the emphasis has been put on training. It is suggested that more regular meetings between NAs, NCs and SALTO are organised and that a review and analysis of all Euromed project activities is undertaken (including quality assessment) in order to assess better training needs and to define the conditions for SALTO-INJEP to become a genuine resource centre. Reporting, monitoring and review Monitoring and feedback is insufficient at both programme and project levels. At EC / TAO level, national coordinator and project reports seem to used for administrative purposes only and there are few alternative feedback mechanisms (i.e. visits and meetings) in place. There is a strong lack of a common (but Euromed-Youth specific) data storage system which would allow for comprehensive and systematic monitoring of progress on implementation, results and impact. The above also inhibits the Commission s capacity to review programme progress on an on-going basis and propose redirections if and when deemed necessary. Individual practices of the NCs in terms of monitoring and feedback vary greatly. The general lack of guidance as to how to set up such mechanisms has inhibited the development of a standard comprehensive and systematic model to be used by all NCs and therefore information on progress, results and impact remains mainly anecdotal. In addition, NCs and also NAs, appear to have no access to project reports and are not informed of correspondence / communication between the EC / TAO and the applicants organisations / projects. Effectiveness review The measurement of the Euromed-Youth programme s effectiveness is hindered by the overall lack of internal consistency in the programme design. However, despite the weaknesses identified in its design and taking into account the problems linked to organisation, management and monitoring, the programme has achieved some important qualitative results, especially at local and individual level, but in some cases also nation-wide. Results reached to date have mainly impacted on the more strategic Euromed-Youth objectives, such as the improvement of mutual understanding, dialogue and tolerance; the promotion of youth initiatives and exchanges between the EU and the Mediterranean partners; and the consolidation of youth structures as well as the professionalisation of youth leaders. In addition, the implementation of the Euromed-Youth pilot phase has enabled identification of the programme s key obstacles and weaknesses, as well as its main factors of success. Obstacles and

6 weakness include the delayed launching of the programme in some of the Mediterranean partner countries, issues related to project selection and release of payments, lack of guidance and support to NCs and insufficient experience of youth organisations to manage projects within such an innovative framework. Key success factors include the high relevance of the programme with regard to youth expectations, the high level of motivation of applicant or participating young people and organisations and the programme s multiplier effect on youth exchanges, training and other informal education activities. Both the negative and positive lessons learnt constitute a good basis for further consolidation of the Euromed-Youth programme. Impact of the Euromed-Youth programme The Euromed-Youth programme is characterised by a high level of enthusiasm and interest in the Mediterranean countries and has clearly generated high motivation and new expectations from beneficiaries. At present, impacts of the Euromed-Youth programme are visible at three levels: i.e. young people; local communities; and youth work and policies. They are, however, still rather lowkey, which is mostly due to the recent start of the programme and to significant management problems encountered by NC and youth organisations. Impact on young people Overall, Euromed-Youth activities carried out up to now have mostly contributed to promoting active participation of young people and opening them up to new cultures and ideas. The programme activities were generally not perceived as having a strong and direct impact on the development of employability and on the integration of young people into professional life. Participation in the programme also has a key impact on the development of young people s skills, both linguistic and technical skills linked to project preparation, planning and management. As far as social integration is concerned, the impacts are indirect. For instance, the following elements were the most frequently quoted by young people and youth organisations. reinforcing mutual understanding and awareness as well as intercultural communication; breaking down prejudices both in the EU and in Mediterranean countries and opening dialogue; giving young people a voice and increasing their confidence; showing young Mediterranean people that EU youth also face socio-economic problems such as exclusion and unemployment (Social Communication Centre in Lebanon, Algerian NC, Palestinian NGOs) and encourage them to be active citizens in their countries. Impact on local communities (European and Mediterranean awareness) Evidence on the involvement of local communities in the programme was recurrently mentioned by interviewed organisations and NCs. This occurred for all types of activities (actions 1, 2 and 5) supported by the programme. These include: - bringing the European Union closer to the Mediterranean citizens, - break down isolation of local communities and providing a unique chance for some young people (Palestinians) to get international exposure - enhancing respect and tolerance for EU people and opening the community to the rest of the world. As far as stimulation of active citizenship within local communities is concerned, some projects clearly demonstrate the active involvement of local communities in Euromed-Youth activities. Impact on national youth policies and youth work

7 Together with the specific outcomes linked to the development of youth organisations, impacts of the programme on the developments in youth work and youth policies are tangible and likely to lead to greater impact on civil society in the longer term. Among others, the programme: - helps promoting the concept of youth work and youth activities and distinguish them from sport activities; - brings a new EU or international dimension to existing activities and youth policies; - enhances participating NGOs willingness and capacity to influence youth policies, by creating an arena where youth organisation get a chance to meet, and by raising youth initiatives. Assessment of the visibility of the programme The case studies completed and answered questionnaires confirm that the overall visibility of the Euromed-Youth programme is satisfactory. The visibility of the Euromed-Youth programme to potential beneficiaries, however, tends to be very uneven depending on the actors and countries involved. It is generally high amongst those organisations (and their members) which are part of international networks or with regular international contacts or those organisations which have a higher profile at national and regional levels and are in continuous contact with either the NC or the national governmental bodies. With a few exceptions, access to information on the Euromed-Youth programme is significantly less amongst the smaller, grass-root organisations, especially when located in rural or otherwise isolated areas (e.g. political isolation). The programme s visibility also heavily depends on the size of the country and geographical position. Programme awareness and involvement of EC delegations in Mediterranean countries interviewed varies greatly. This seems to be mainly linked to overall staff and financial resources available to run the representative offices, and the delegations work load and responsibility with regard to other EU programmes and activities in the country. With the exception of Lebanon, all national government representatives interviewed confirmed that they were aware of the Euromed-Youth programme and familiar with its specific objectives and activities. Their involvement and commitment to the programme ranges from providing concrete support to its promotion and dissemination to a more passive position of supervising the National Co-ordinators and dealing with contractual issues. Sustainability / Replicability Sustainability of the Euromed-Youth programme depends on the extent to which its actors and stakeholders consider themselves owners of the programme s objectives and achievements, on whether the programme allowed for capacity building and on various factors conditioning sustainability, such as policy developments, economic and financial issues and socio-cultural factors. In terms of ownership, there is an overall good understanding of the programme objectives and key stakeholders consider that programme achievements make a strong contribution to their own priorities in the field of youth. However, unclarities on some of the programme aspects, such as its overall strategy, roles and responsibilities and lack of communication and information could progressively demotivate the various actors to participate. Overall, there is a correspondence between the Euromed-Youth programme and the national policy priorities in the field of youth in the Mediterranean partner countries, although policy support ranges from very high (with concrete commitments at national and local level in some countries) to more passive forms of support. Another factor influencing the programme s sustainability concerns the institutional and management capacities at EC, national and local levels. In this respect, the highly centralised nature of the programme at EC level, in combination with the current management structures and procedures, seems to be a key obstacle to increasing participation and impacts, thus also affecting sustainability. At national level, NCs would welcome additional assistance, and in some cases, resources to further sustain and expand the programme. It seems unlikely that programme activities such as those implemented under Euromed-Youth can be undertaken solely through self-

8 financing in the countries involved, given that up to date only two examples of self-financed activities have been identified. When looking at socio-cultural factors affecting the programme s sustainability, Euromed-Youth s overall design allowed for flexibility / adaptability to different social and cultural realities. A few aspects, such as the age limit for young participants and the lack of information / application material and communication in any other language than the EU ones could be tackled to further enhance access of beneficiaries and overall sustainability. In terms of know-how and technical support to NCs and organisations, complementary training measures, in particular on voluntary service activities, group work methodologies, project-cycle and financial management and reporting were identified as necessary to increase capacity building and favour continuation of the programme. Conditions to further consolidate the Euromed-Youth programme s sustainability include a more intensive and targeted use of the NCs experience and on-the-ground knowledge, as well as of the project experiences and multiplier effects. Important improvements to be made to reinforce the sustainability of the programme concern simplifying managerial procedures to reduce delays in project selection and release of payments, increasing support in terms of training and information (including the development of an articulated communication strategy and transfer of best practices) and enhance commitment of European stakeholders, in particular the NAs. The Mediterranean Youth Forum Although integrated in the objectives, measures and budget proposed in the Financing Plan, the efficiency, impact and effectiveness of the MYF has been poor. This seems to be mainly due to weaknesses in its design, which lacks clarity and thorough (ex-ante) assessment of absorption and implementation capacities of the actors involved as well as of the potential risks and obstacles related to its establishment. Both the organisations in Malta and in Jordan responsible for running the Forum s secretariat up to now received insufficient support and encountered problems with regard to the release of EC payments. In addition, little initiative from the Mediterranean NGOs, differences in views between the EYF and the MYF actors as well as the six-month rotation principle for the secretariat further reduced the Forum s chances of generating the benefits expected and reaching the target beneficiaries (youth structures at regional and national level and youth leaders). However, the large majority of the Euromed-Youth actors is strongly in favour to continue the development of the Forum, a process which would require, amongst other improvements, significant structural changes, establishment of a permanent secretariat and an increase in (EC and other types) of funding. Recommendations The conclusions and recommendations resulting from the evaluation are built around four key strategic issues, i.e. (a) improvement of coordination and increasing participation; (b) improvement of programme responsiveness and performance; (c) the need for stakeholders to share visions, enhance ownership and awareness; and (d) increasing capacity building. Under these strategic issues, specific recommendations focus on: - The establishment of effective coordination structures; - Enlargement and diversification of potential stakeholders; - Improvement of overall coherence in the programme design; - Improvement and adaptation of selection and contracting procedures; - Enhancement of management performance; - Increasing resources and improving their allocation; - Establishment of an information strategy and related information management system; - Improvement of existing information products and development of new ones; - Increasing and improving initiatives to enhance capacity building.