Evaluation of the impact of the Fp6 in the RTD public system in Spain

Transcription

1 Evaluation of the impact of the Fp6 in the RTD public system in Spain

2

3 Concept and direction of the study: Rocío Castrillo Cancela (coordinator) Carlos Martinez Riera Evelina Santa European Office - Ministry of Science and Innovation Execution: Zabala Innovation Consulting SA Preliminary phases of the study carried out by Deloitte. FINAL Report drawn up by Zabala Innovation Consulting SA Design, layout and printing: Madridcolor I.D.S.L. NIPO: Legal Deposit: M May 2010

4 INDEX INDEX 3 FOREWORD 5 EXECUTIVE SUMMARY 7 1. PARTICIPATION OF SPANISH UNIVERSITIES AND PUBLIC RESEARCH BODIES IN FP METHODOLOGY DESIGN AND STAGES OF THE STUDY STRUCTURE AND CONTENT OF RESEARCHER QUESTIONNAIRE STRUCTURE AND CONTENT OF INTERVIEWS WITH INSTITUTIONAL ACTORS SELECTION OF INTERVIEWEES AMONG UNIVERSITIES AND PUBLIC RESEARCH ENTITIES SELECTION OF WORKSHOPS MAIN RESULTS OF QUESTIONNAIRE TO RESEARCHERS REPRESENTATIVENESS OF SAMPLE OF RESEARCHERS RESPONDING TO QUESTIONNAIRE ANALYSIS OF RESPONSES OBTAINED FP6 IMPACT AT INSTITUTIONAL LEVEL MAIN RESULTS OF INTERVIEWS WITH INSTITUTIONAL REPRESENTATIVES CASE STUDIES RESULTS OF THE OPINION PANELS PANEL ON THE IMPACT OF FP6 ON THE SPANISH PRODUCTION SECTOR PANEL ON THE IMPACT OF FP6 ON THE RESEARCH STRUCTURES RELATED TO THE PROJECT MANAGEMENT OF THE FRAMEWORK PROGRAMME PANEL ON THE IMPACT OF FP6 ON THE INTERNATIONALISATION OF RESEARCH PANEL ON THE IMPACT OF THE FRAMEWORK PROGRAMME ON NATIONAL AND REGIONAL R&D POLICIES CONCLUSIONS AND RECOMMENDATIONS CONCLUSIONS RECOMMENDATIONS 76 I. APPENDIX I. TEMPLATES OF QUESTIONNAIRES AND INTERVIEWS TEMPLATE OF RESEARCHER S QUESTIONNAIRE TEMPLATE OF SCRIPT FOR INSTITUTIONAL INTERVIEW 93 II. APPENDIX II. CHART OF CASE STUDIES IDENTIFIED IN THE INTERVIEWS 107 III. APPENDIX III. INDEX OF INITIALS AND TERMS 111 IV. APPENDIX IV. INDEX OF GRAPHS AND TABLES 113 V. APPENDIX V. GRATITUDE 117 V.1 RESEARCHERS SURVEYED 117 V.2 PERSONS INTERVIEWED AT AN INSTITUTIONAL LEVEL 122 V.3 PERSONS PARTICIPATING IN OPINION PANELS WORK SHOPS 124 Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 3

5 4

6 FOREWORD It is not by chance that this study is being presented during the Spanish Presidency of the EU Council. Spain has supportedthe emphasis put by prior presidencies, particularly that of the Czech Republic, on the need to promote impact assessment of European RTD+I programmes, especially Framework Programmes, and to embrace this commitment throughout its own presidency. One of the overall priorities is to place society at the heart of the RTD+I activities, to root RTD+I strongly at the core of society, to be responsive to its needs and to contribute in facing the grand challenges of the future. Doing so means also accounting for the investment undertaken in RTD+I and hence assessing the impact of this investment on the RTD+I landscape of Spain and its society at large. Thus, it is only by assessing the results of RTD+I investment that we can determine teh degree of fulfilment of these objectives. Since their introduction in 1984, European Framework Programmes for Research and Technological Development (FPs) have played a fundamental part in collaborative multidisciplinary research activities, both within Europe and beyond its borders. Spain, which joined the European Union and, hence, this programme shortly thereafter, has experienced the widescale transformation of its science and technology system during this time. This was due both to domestic efforts and to international influences, which called for a change in the way research was conducted towards a multidisciplinary model undertaken by international research groups, thereby leaving behind the figure of the lone researcher in an ivory tower. The way in which policies are designed has also evolved over the years. It now focuses on open methods of coordination and on the use of evidence to support political decisions. In this context, assessing the impact of public policy is now considered an important tool to analyse the results of R+D+i policies and programmes in terms of their effect on research and innovation systems, the economy and society as a whole. In summary, such assessments deliver a factual basis on which decisions can be made, and contributes to the development, improvement and implementation of appropriate research and innovation policies. In October 2007, the Spanish Ministry of Science and Innovation launched first feasibility study, the general aim of which was to design and implement a method to assess impact of the EU RTD Framework Programme on the public R+D+i sector, and to compare it with that of other EU Member States. Once this first step was completed and an effective methodology had been designed, the study got under way. It took 18 months and involved external consultancy firms, over 600 researchers, and 30 directors of public research centres and universities in Spain. The final results are contained in this report. For the Ministry of Science and Innovation, this initiative represents the cornerstone for future improvements to existing policies to aid the researchers, universities, public research centres and public and private institutions who participate in the EU Framework Programmes for R+D+i. Undoubtedly, it provides a source of information for political deliberations on the Eighth Framework Programme, within the activities of the Spanish Presidency in On a European scale, the Ministry of Science and Innovation, through its European Office, is also working on the CIA4OPM project, the aim of which is to exchange experiences and good practice guidelines for impact assessment of public R+D+i policies, in order to improve coordination of European, state and regional policies. This group is comprised of experts from 15 institutions from 11 different European Member States. Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 5

7 The Ministry, through its National Agency for Assessment and Prospection, also participates in the European Network for R+D Impact Assessment (EUEvalnet), a platform that was set up by the European Commission in 1997 to allow the exchange of information and good practice guidelines on assessment methodologies, research indicators and R+D+i impact measurement. All of the above experiences allow us to join other EU countries in moving forward from a position of leadership towards a culture of assessment of public policy. It is therefore necessary to point out that this exercise has also allowed us to explore and analyse the use of impact assessment techniques, in order to converge towards a strategic assessment system that may, in the long run, become an international benchmark. We realise that this report is only the starting point towards developing, improving and implementing appropriate policies backed up by evidence. It represents a point of departure for future activities that will no doubt allow us to build up a track record in research and development over a longer period of time. This scheme reflects the firm commitment of the Spanish Ministry of Science and Innovation to embark along a path where political decisions are supported by evidence. We believe that this is the most effective way of improving political proposals and ensuring that public resources are put to the best possible use. To quote the Minister of Science and Innovation, one cannot always spend more, but one can always spend more wisely (Barcelona, 28/09/2009). Montserrat Torné i Escasany Director General for International Cooperation and Institutional Relations Secretary of State for Research Spanish Ministry of Science and Innovation 6 Foreword

8 EXECUTIVE SUMMARY AIM OF THE STUDY The overall aim of this study was to make a quantitative and qualitative assessment of the impact of participation in the Sixth EU RTD Framework Programme (FP6) of Spanish universities and public research entities (organismos públicos de investigación - OPIs). It employs several assessment criteria such as consolidation of researchers careers, reinforcement of their research, publications, internationalisation of research projects, cooperation with the corporate sector, improvement in research infrastructure, etc. METHODOLOGY Our study to assess the impact on Spanish universities and OPIs of FP6 was carried out in three stages. GRAPH 1. LEVELS OF ASSESSMENT Level 2 Level 3 Level 3: Economic and social impact Level 1 Level 2: Institutional impact Level 1: Impact on research groups themselves The first stage of the assessment was based on quantitative analyses obtained by means of questionnaires sent to participating researchers. The second stage of assessment focused on more qualitative analyses by means of interviews with institutional representatives (university vice-chancellors and OPI directors). The third stage of assessment took the form of workshops with experts from universities, OPIs and public administrations with a global view of Spain s role in FP6. The workshops allowed us to debate and validate the results of the preceding assessments and to expand our work into those areas where the relationship between actions and their impact is more diffuse. The following diagram sets out the stages to assess the impact of FP6 activities on Spanish public R+D. Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 7

9 GRAPH 2. STAGES OF ASSESSMENT Definition of Assessment Methodology Information Gathering Conclusive Analysis Diagnosis of Current Situation Definition of the Assessment Analysis of Assessability Survey Interviews Workshops Conclusive Analysis Draw Up Final Document Review of available information Determination of scope of assessment based on 7 factors Analysis of information availability Selection of distribution medium and identification of participants Selection of interviewees and scheduling of interviews Select themes and participants Analysis of results Definition of structure and content Comparative study of similar initiatives and personal interviews with experts Ranking of assessment criteria Launch of questionnaire Interviews Organise logistics and execution of workshops Drawing up of conclusions Preparation of draft report, review of interim versions Selection of value ranges and indicators Collection and analysis of data. Conclusions Analysis of information. Conclusions Analysis of information. Conclusions Final conclusions Release of final report Definition of priorities regarding techniques and methods The first step when undertaking the study was to design the assessment methodology. To do so, we analysed twelve existing Framework Programmes (FP) impact assessment studies carried out in other European countries. We analysed the various methodologies that had been used to obtain quantitative and qualitative assessments in these studies, identified them and selected various criteria and indicators to measure the impact of FP6 on researchers from universities and OPIs. During the design stage of the study, we defined the scope of the assessment, based on the following aspects: Geographical scope of assessment, was restricted to five Autonomous Regions (Comunidades Autonomas - CCAA) of Spain which together make up 80% of Spain s participants in FP6: Andalusia, Catalonia, Madrid, the Basque Country and Valencia. FP6 thematic priorities: life sciences, genomics and biotechnology applied to health care; nanotechnology and nanosciences; food quality and safety; sustainable development, global change and ecosystems; and citizens and governance. FP6 instruments: Integrated Projects (IP), Networks of Excellence (NOE), Specific targeted research projects (STREP), Coordinated Actions (CA), Specific Support Actions (SSA). Areas Assessed Assessment criteria for each of the chosen areas Indicators for the assessment criteria The survey was made amongst participants in all FP6 projects that met the chosen criteria. Although the group was initially composed of 827 potential interviewees, the sample was reduced to 647 researchers after filtering out erroneous data and duplicates. sentthe return rate was 51%. The researchers who participated in the study make up a representative sample in terms of distribution both by themes and by FP6 financing instruments. 8 Executive summary

10 Of all the researchers who answered the survey, 85% participated in FP6 projects as partners, while 15% participated as coordinators of European projects. This is fairly representative of the current situation in Spanish universities and OPIs. Researchers Views: For the researchers surveyed, the main impact of participation in FP6 was to increase the knowledge base and to extend interdisciplinary knowledge. The main areas in which the results of the projects exceeded expectations were those relating to development of a collaborative culture, creation of stable research networks, and production of scientific knowledge. The main areas in which the results of the projects fell below the participants expectations were those relating to professionalization of R+D management, and commercial or industrial use of generated knowledge. For 55% of researchers surveyed, the benefits of participating exceeded the costs; for 34% of researchers surveyed, the benefit of participating equalled the cost; and for 11%, the benefits were outweighed by the costs of participation. In other words, 89% of those surveyed said that the benefits of their participation were equal to or greater than the costs thereof. If we compare the views of participants who were partners with those of coordinators, the latter were more optimistic in their assessment of their participation. Scientific Impact: The most common forms of subsequent collaboration with project partners are publication of scientific articles, attendance of conferences, new collaboration projects, and exchange visits by researchers. In 52% of researchers surveyed, their participation in FP6 helped to consolidate their research groups, largely as a result of the scientific excellence generated by the capabilities and abilities acquired in the course of projects. The main beneficiaries of the results of FP6 projects are members of the scientific community, either within the same field or in a different discipline, far outnumbering other possible beneficiaries such as public administrations, the industrial sector or citizens. Knowledge Transfer and Industrial Property: Only 40% of researchers believe that they need to protect the industrial property of the results of FP6 projects, mainly through patents. The other 60% believe that they have not and will not obtain results that would require measures to protect their industrial property rights. Among the researchers, 70% believe that the knowledge obtained or obtainable is transferable. The percentage of transferable knowledge is considerably higher than the percentage of results that require protection, which would appear to indicate that a significant proportion of unprotected knowledge is transferred. Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 9

11 Over half (51%) of researchers surveyed consider that the results obtained or still to be obtained will have a commercial application. The main channel for commercial use is through technical cooperation. The least-used channels for commercial use are licences and new technology-based companies or spin-offs. Complementarity of policies: The value added and the need to finance from the FP, in order to carry out collaborative R+D projects with an international dimension and ambitious in scope, are evident. In 97% of cases, projects would not have been carried out in the same manner had FP financing not been obtained. In 74% of cases, projects would not have been carried out at all without FP financing, and only in 23% would projects have gone ahead with national or regional funding, although with less ambitious objectives and with a more limited scope. Only 25% of the researchers surveyed (participating in the FP6) used the services provided by the National Contact Points that provide support during the preparation of European projects. RESULTS OF INTERVIEWS WITH INSTITUTIONAL REPRESENTATIVES Thirty interviews were held with institutional representatives from 18 universities (those most active in FP6), four OPIs and eight CSIC (Spanish National Research Council) institutes. The trajectory or intensity of involvement by universities and OPIs has gradually increased during the course of the various Framework Programmes. The number of projects in FP6 was lower than in previous projects, but economic returns were higher for several universities because the projects were larger in scope. FP6 fostered focused research, something that is difficult for OPIs with a highly specialised research focus to undertake. OPIs with a highly focused research portfolio found it difficult to fit into the specific call themes in FP6. FP6 had a significant impact in terms of providing universities and OPIs with new channels of access to international environments and affording greater mobility among researchers, enhancing the prestige of the research groups present in Spanish universities. Interviewees often mentioned that collaboration in FP6 had helped to consolidate a hard core within numerous consortia and had resulted in continuing cooperation in FP7. The impact of FP6 in terms of increasing collaboration with other European research groups was greater among the universities interviewed than among the OPIs. This was due to higher participation of the former in FP6, and to the fact that several OPIs already have strong international links established apart from the FPs. Interviewees believe that FP6 has had a limited impact in terms of professionalization of R+D management, stable job creation, collaboration with the private sector, knowledge transfer to the Spanish industrial sector and improvements in R+D infrastructure. The researcher questionnaire revealed that European financing added value, a fact that was confirmed during the institutional interviews. The overwhelming majority indicated that the projects would not have been carried out without FP6 financing, or that they would have had to be scaled down to less ambitious objectives and more limited resources. FP6 has had a significant impact in reinforcing the already established areas of research, but we observed a limited impact with regard to exploring new areas. FP6 has had a significant influence on the strategies and priorities of the universities when defining their international relations and their support structures for R+D management, and it has had a limited influence on the design of education strategies (some impact on postgraduate courses, none 10 Executive summary

12 on undergraduate degree courses). The influence of FP6 on the strategies and priorities of the OPIs interviewed was more limited. Although FP6 placed significant emphasis on the dissemination of scientific knowledge in society, most interviewees were of the opinion that efforts to circulate knowledge from FP projects were not aimed at the public at large. Most of the universities create channels to disseminate the results of research among society through websites or specific publications aimed at circulating knowledge, but such efforts often go unnoticed by society. RESULTS OF WORKSHOPS 1. ON THE IMPACT OF FP6 ON THE PUBLIC SECTOR COOPERATION WITH THE SPANISH INDUSTRIAL SECTOR The main aim of universities or OPIs when they embark on an FP project is not generally to transfer technology to Spanish firms. They have other aims such as internationalising their R+D, greater collaboration or networking. Universities and OPIs use other channels of cooperation with firms, often signing collaboration agreements. It appears that the aim of FP6 is not to foster cooperation between universities and OPIs and private enterprise, so it is not the most appropriate instrument for this. In this regard, knowledge transfer to the productive sector should not be used as an assessment criterion for FP6. 2. ON THE IMPACT OF FP6 ON THE RESEARCH ORGANISATIONS MOST DIRECTLY INVOLVED IN MANAGING THE FP Hiring managers for each project barely has any impact on the professionalization of R+D management. A need is seen for long-term managers who are linked to the institution rather than to a temporary project. However, the current administrative regime of most OPIs makes it very difficult to hire, train and keep R+D project managers. The national Euroscience programme may provide some stimulus but cannot replace co-financing of R+D managers by the universities. As for the FP support structures, workshop participants distinguished two basic tasks: to promote and draw up proposals (prior to winning a project) and to support project management (during and after the project). The optimal structure would be one in which projects were managed in their entirety ( researchers should not be assisted in managing their projects; they should have their projects managed for them ). A lack of advisory assistance regarding industrial property rights was also noted. It was also pointed out that, unlike other European institutions, Spanish institutions have still not developed clear FP participation strategies that will allow them to attend international meetings and speak with a single voice when discussing European R+D programmes and projects. 3. ON THE IMPACT OF FP6 ON THE INTERNATIONALISATION OF RESEARCH Workshop participants maintained that the internationalisation created through participation in FP projects serves as a springboard towards higher levels of internationalisation. However, they also pointed out that few institutions in Spain appear to take advantage of the opportunities associated with, for instance, taking an active role in committee meetings of the European Technology Platforms, joining European expert groups (advisory boards), etc. Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 11

13 Although the repeated presence of a hard core of members from a consortium in successive European R+D projects is often used as a counter-argument against FP collaboration, workshop participants believe that this model is not something negative, if the result of a defined medium-to-long term R+D strategy is continuity, and if the consortium fits in with this strategy in a natural way. FP6 collaboration instruments still offer considerable potential. 4. ON THE IMPACT OF FP6 ON REGIONAL R+D POLICIES Regional R+D programmes generally include FP participation goals based on previous indicators. There are significant differences among CCAA regarding the degree of detail and the breakdown of these goals. FP6 has had an impact on regional R+D policies, especially when defining measures and implementing mechanisms and structures to help regional R+D agents participate in FP6 (regional information structures, promotion and advice on FPs, financial support to prepare proposals, etc.). Furthermore, regional institutions have become involved in the process of drawing up draft work programmes, accompanying Spanish representatives to FP committee meetings, and gathering preliminary information on regional interests. Although European Structural Funds are generally perceived as a mechanism to support positions vis-à-vis the FP, there have also been comments pointing out the disincentive effect of financing from Structural Funds in R+D cooperative projects (Spanish Technology Fund), given that regions who have large sources of financing from these sources do not feel an urgent need to apply for FPs. CONCLUSIONS AND RECOMMENDATIONS Among our summary conclusions, we highlight the following aspects: The Framework Programme (FP) has had a significant influence on aspects such as scientific knowledge creation, and cultural change towards more cooperative, more international research. FP6 has tended to expand the horizons of the Spanish public R+D system. However, FP6 does seem to be of limited use in promoting public-private cooperation and in appropriating and transferring the results to market. The general opinion of FP6 participants is positive, especially that of researchers who participated as coordinators compared to those who participated only as partners. The value added of FP6 becomes evident in comparison with other regional and national programmes. That is to say, researchers and institutions make a clear distinction between programmes, applying for one or another depending on the nature of the proposal in question. At the institutional level, the experience of participants and the results obtained in FP6 clearly point to the need to make progress in establishing and consolidating support structures that will serve the demands, expectations and needs of researchers who wish to participate in the FP. Although the universities and OPIs have gone to considerable lengths to using resources to support participation in the FP, these do not yet seem to be sufficient to fill this demand. 12 Executive summary

14 Based on the above conclusions, we make the following recommendations: Need for adequate, stable staff, not simply to provide support for researchers, but also to manage their participation in the FP. It is essential for researchers to be able to count on this help automatically when they become involved in FP projects, rather than have to seek help actively. Reinforcement of institutional legal support and help in commercialising project results. Institutional commitment to permanently foster participation in the FPs. Support for FPs must be institutional, and must be seen as an inherent task of universities and OPIs. This stability can be ensured by using analytical cost accounting systems on FP projects, which would bring significant financing to universities and OPIs. Strategic reflection by public administrations regarding the type of impact they seek, and the aims of participating in FPs, in order to draw up coherent support policies. Promotion of ongoing work and assessment of the distance to market of FP results in order to provide the necessary support. This requires research results to be assessed, in addition to the financial assessment. Extension of institutional coordination among public research organisations and greater stability, with a view to attending international conferences to discuss European R+D programmes or projects, exchanging information and collective preparation for conferences in order to speak with a single voice. Promotion of FPs as a springboard towards other levels of internationalisation: Technology Platforms, Large Infrastructures, international cooperation with third parties (non-eu), participation in activities of international organisations (e.g., the International Energy Agency), etc. Greater coordination of actions and objectives among national and regional administrations in order to promote greater participation in European research policy. Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 13

15 14 Executive summary

16 1PARTICIPATION OF SPANISH UNIVERSITIES AND PUBLIC RESEARCH BODIES IN FP6 1 According to data published by the Ministry of Science and Innovation through the Spanish Centre for the Development of Industrial Technology (Centro de Desarrollo Tecnológico Industrial - CDTI), annual subsidies to Spain through the Sixth Framework Programme (FP6) amounted to 939.1M, an increase of 35% with respect to the annual average for the Fifth Framework Programme (FP5). However, returns dropped to 6.0% of the total amount awarded ( 0.5 points vis-à-vis FP5), whereas they amounted to 6.6% of returns among EU-25 Member States. Once again, this confirms that research carried out through Framework Programmes continued to be one of the main sources of project financing among Spanish institutions. These data underline the importance of analysing the impact of FP funding on researchers. First of all, in order to help us to remember the situation at the outset of the study, it is worth reviewing some of the ways in which Spanish universities and public research institutions have played a role in FP6. Looking at returns on FP6, broken down by Autonomous Region (AR), the five top spots went to Madrid (35.7%), which nevertheless lost ground vis-à-vis FP5, Catalonia (24.4%), which made significant progress, the Basque Country (12.5%), Andalusia (5.8%), which also increased its share, and Valencia (8.8%), which lost share. Overall, these five CCAA were the recipients of 87.2% of Spanish returns. Spanish universities obtained 27.2% of Spanish returns from FP6, and Public Research Entities (OPIs), 17.5%. Together, these two participant profiles accounted for 44.7% of returns. However, since FP3, Spanish universities have not performed as well as their European peers. Although returns to Spanish universities over the period have increased in absolute terms (millions of Euros), they have diminished substantially in percentage terms compared to returns to EU universities as a whole, as can be seen in the following chart. 1 This summary is based on the document VI Programa Marco de I+D ( ). Análisis y resultados de la participación española ( pdf). Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 15

17 GRAPH 3. FINANCE OBTAINED BY SPANISH UNIVERSITIES IN THE LAST FOUR FRAMEWORK PROGRAMMES FP IN M, AND SHARE OF TOTAL RETURN ON EACH FP % SPANISH UNIVERSITIES/TOTAL EU UNIVERSITIES ,4% 253, ,9 M 150 6,0% 5,9% 6 %ES vs EU ,8 5,0% ,1 2 0 III PM IV PM V PM VI PM 0 Source: The Spanish Centre for the Development of Industrial Technology (CDTI) Overall, 3,959 Spanish institutions presented a total of 11,137 proposals under FP6, of which a majority (48.40%) were from firms, followed by universities (36.13%) and OPIs (17.97%). The following table sets out the number of Spanish proposals that were presented and the number of projects that were approved under FP6, broken down by type of institution. Some proposals may envisage simultaneous participation by more than one type of institution. In this case, this table shows each participation separately. TABLE 1. BREAKDOWN OF PROPOSALS PRESENTED AND PROJECTS APPROVED UNDER FP6 AS PER TYPE OF SPANISH INSTITUTION Type of institution Proposals presented No. Projects led Projects approved No. Projects led Success rate of proposals presented (%) Success rate of projects led (%) Firms 5, % 13.95% of which, SMEs 4, % 12.10% Technology Centres 1, % 14.12% Associations % 16.67% OPIs 2, % 24.64% Universities 4, % 13.31% TOTAL 11,137 2,133 Source: The Spanish Centre for the Development of Industrial Technology (CDTI) 16 Participation of spanish universities and public research bodies in fp6

18 This means that 54.1% of proposals presented under FP6 by Spanish participants came from universities and OPIs. However, universities and OPIs acted as project leaders in only 32.9% of the 2,133 projects that were presented. The success rates on the previous table are for the proposals presented (whether or not as coordinators) as a whole, indicating the percentage of those approved. Nevertheless, it makes more sense to look at success rates for proposals presented as coordinators, given that the institutions involved have a more direct responsibility for the quality of the proposal. Here, Spanish universities obtained a success rate of 13.3% on proposals they led, while OPIs showed 24.63%, technology centres 14.12%, and firms 13.94%. Analysing the breakdown by institution, the following table sets out those that had the most relevant roles in FP6: TABLE 2. PUBLIC RESEARCH ENTITIES AND UNIVERSITIES FROM THE FIVE AUTONOMOUS REGIONS UNDER STUDY, SHOWN BY CUMULATIVE RETURN UNDER FP6. Entidad Activities NO. COORD CSIC (Spanish National Research Council) Universidad Politécnica de Cataluña Universidad Politécnica de Madrid Universidad de Barcelona 82 5 Universidad Pompeu Fabra 49 7 Universidad Politécnica de Valencia 67 8 Universidad Autónoma de Barcelona 59 2 Universidad Autónoma de Madrid 37 0 Universidad de Valencia 46 5 Source: The Spanish Centre for the Development of Industrial Technology (CDTI) Looking at the above ranking within a European context, it should be remembered that of the 7 universities that make up the top 20 European institutions ranked by return on FP6, none is Spanish. The CSIC, however, is one of the top 5 European OPIs, ranked by FP6 return. According to data from CDTI, only 34.6% of projects carried out by Spanish universities and OPIs under FP6 are in conjunction with Spanish firms, applying the knowledge in Spain. Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 17

19 18 Participation of spanish universities and public research bodies in fp6

20 2METHODOLOGY 2.1 DESIGN AND STAGES OF THE STUDY The overall aim of this study was to make a quantitative and qualitative assessment of the impact of the participation in FP6 of Spanish universities and public research entities (OPIs), based on several assessment criteria such as consolidation of researchers careers, reinforcement of their research, publications, internationalisation of research projects, cooperation with the corporate sector, improvement in research infrastructure, etc. The following diagram shows a detailed breakdown of the various stages that were developed to assess the impact of FP6 activities on the national public R+D: GRAPH 4. ASSESSMENT STAGES Definition of Assessment Methodology Information Gathering Conclusive Analysis Diagnosis of Current Situation Definition of the Assessment Analysis of Assessability Survey Interviews Workshops Conclusive Analysis Draw Up Final Document Review of available information Determination of scope of assessment based on 7 factors Analysis of information availability Selection of distribution medium and identification of participants Selection of interviewees and scheduling of interviews Select themes and participants Analysis of results Definition of structure and content Comparative study of similar initiatives and personal interviews with experts Ranking of assessment criteria Launch of questionnaire Interviews Organise logistics and execution of workshops Drawing up of conclusions Preparation of draft report, review of interim versions Selection of value ranges and indicators Collection and analysis of data. Conclusions Analysis of information. Conclusions Analysis of information. Conclusions Final conclusions Release of final report Definition of priorities regarding techniques and methods Evaluation of the impact of the 6th european framework programme of research and development in the public system in Spain 19