ARTEMIS-ED Guidance notes and templates for Project Technical Review involving independent expert(s)

Transcription

1 Guidance notes and templates for Project Technical Review involving independent expert(s) 1

2 CONTENTS 1 INTRODUCTION THE PROJECT REVIEW PROCESS MANDATE OF THE INDEPENDENT EXPERT(S) Objectives Preparation of the review Review material Reporting... 6 Annex I CHECKLIST FOR THE TECHNICAL REVIEW... 7 Annex II - TEMPLATE FOR THE TECHNICAL REVIEW REPORT

3 1 INTRODUCTION The Joint Undertaking is responsible for monitoring the performance of the Project in accordance with its 'Technical Annex'. This will be undertaken through periodic technical reviews. The aim of a technical review is to assess the work carried out under the Project over a certain period and provide recommendations to the Joint Undertaking. Such review may cover scientific, technological and other aspects relating to the proper execution of the Project and the relevant grant agreements (JU Grant Agreement and National Grant Agreements) in line with the common Technical Annex. This document provides guidance on the review process as well as on the content of the report to the Joint Undertaking. 2 THE PROJECT REVIEW PROCESS The Joint Undertaking will initiate a technical review after each reporting period as identified in the Technical Annex of the Project 1. However, the periodicity of the reviews will be at least annual. For Projects lasting two years or more, a first review will be carried out after the first six months unless duly justified by the Joint Undertaking. The periodic technical reviews will allow evaluating the Project reports and deliverables relevant to the period in question. Such reviews may cover scientific, technological and other aspects, including compliance with fundamental ethical principles, relating to the proper execution of the Project and the grant agreements. During the course of the project the Joint Undertaking can also carry out intermediate technical reviews that may not cover the whole of a given reporting period. In this case the review process is similar to the periodic technical reviews, unless otherwise specified by the Joint Undertaking. Reviews may be carried out remotely or involve sessions with Project representatives either at the Joint Undertaking's premises or at the premises of beneficiaries. The reviewing team may have access to the locations and premises where the work is being carried out, and to any document concerning the work. With respect to the Description of the Project (Technical Annex), the review shall objectively assess the following: the degree of fulfilment of the Project work plan for the relevant period and of the related deliverables; the continued relevance of the objectives and breakthrough potential with respect to the scientific and industrial state of the art; 1 In general, the JU may initiate a technical review at any time during the implementation of the Project and up to five years after the end of the Project whenever it considers it necessary. 3

4 the resources planned and utilised in relation to the achieved progress, in a manner consistent with the principles of economy, efficiency and effectiveness; the management procedures and methods of the Project; the beneficiary's contribution and integration within the Project; the expected potential impact in economic, competition and social terms, and the beneficiaries plan for the use and dissemination of Foreground. Reviews shall be deemed to be initiated on the date of receipt by the beneficiary(ies) of the relevant letter sent by the Joint Undertaking. Any such review shall be carried out on a confidential basis. The Joint Undertaking may be assisted in technical reviews by external scientific or technological experts. In that case, the Joint Undertaking will establish contracts with the reviewers in order to ensure their independence and the appropriate level of confidentiality of the reviewed material. The national funding authorities involved in the Project may propose observers in reviews to the Joint Undertaking. Taking into account, if necessary, specific national requirements, the national funding authorities may appoint specific observers for specific projects. The observers will be staff of the national funding authorities or from other public services acting on their behalf. These observers will not take part in the deliberations or the work of the reviewing team and will not perform any technical review for the national funding authority. The observers will have access to all review material under the same confidentiality and nondisclosure basis as the reviewers. The expenses of the observers will be covered by the relevant national funding authority. Prior to the carrying out of the reviewing task, the Joint Undertaking shall communicate to the concerned national funding authorities the dates and venues of the reviews, as well as the names of the reviewers, and to the beneficiaries the identity of the appointed experts acting as reviewers or observers. The beneficiary(ies) shall have the right to refuse the participation of a particular external scientific or technological expert on grounds of commercial confidentiality. The beneficiaries shall make available directly to the Joint Undertaking all detailed information and data that may be requested by it or the external scientific or technological expert with a view to verifying that the Project is being/has been properly implemented and performed in accordance with the provisions of the grant agreements. The consortium shall therefore submit its technical report(s) and deliverables to the Joint Undertaking according to the reporting periods as defined in the Technical Annex and the JU Grant Agreement. Reports submitted to the Joint Undertaking shall be in English. The beneficiaries present at the review should be those involved in the work under review, except if duly justified and provided that the partners present can report on behalf of the missing partners. A report on the outcome of the reviews shall be drawn up. It shall be sent by the Joint Undertaking to the coordinator, who may make observations thereon within one month of receiving it. The Joint Undertaking may decide not to take into account the observations conveyed after that deadline. On the basis of the experts' formal recommendations, the Joint Undertaking will inform the coordinator of its decision: 4

5 to accept or reject the deliverables; to allow the Project to continue without modification of the Technical Annex or with minor modifications; to consider that the Project can only continue with major modifications; to initiate the termination of the JU grant agreement according to Article II. 20; to issue a recovery order regarding all or part of the payments made by the Joint Undertaking and to apply any applicable sanction or initiate judiciary procedures. The technical reports and the results of the technical evaluation carried out by the Joint Undertaking shall be made available by the Joint Undertaking to the concerned national funding authorities 2 after their approval and certification by the Executive Director. The technical evaluation carried out by the Joint Undertaking shall take into account, if necessary, specific national requirements proposed by the national funding authorities that are needed for the national funding authority to accept the cost reimbursement claims of grant beneficiaries. The national funding authorities shall not require any additional technical reports other than those required by the Joint Undertaking. 3 MANDATE OF THE INDEPENDENT EXPERT(S) 3.1 Objectives The reviewer's task is to give external advice to the Joint Undertaking on the Project, with respect to the following issues: 1. the degree of fulfilment of the Project work plan for the relevant period and of the related deliverables; 2. the continued relevance of the objectives and breakthrough potential with respect to the scientific and industrial state of the art; 3. the resources planned and utilised in relation to the achieved progress, in a manner consistent with the principles of economy, efficiency and effectiveness; 4. the management procedures and methods of the Project; 5. the beneficiaries contributions and integration within the Project; 6. the expected potential impact in scientific, technological-, economic, competitive and social terms (where relevant), and the plans for the use and dissemination of results. 2 The national funding authorities that have established national grant agreements with beneficiaries for the realisation of the Project 5

6 A checklist for the review is available in Annex I of this document. The reviewer(s) will also assist the Joint Undertaking by recommending any reorientation that may be required, but the final decision on recommendations and reorientation is taken only by the Joint Undertaking. 3.2 Preparation of the review When a review meeting is scheduled, the expert(s) will read all relevant documents before the meeting and will attend the review meeting. He/she/they will then provide an assessment of the Project based on the written material and information provided at the meeting. In the case of remote review, the assessment will be based on written documents only. 3.3 Review material The documents to be included in the review on the basis of the Technical Annex (contractual Description of Work) should normally include the following: Progress report for the period under review, Deliverables necessary for the assessment of the work, due in this period, according to the deliverable table in the Technical Annex, For a final technical review, the following additional documents should also be part of the material to review: The final publishable Project summary report, The plan for use and dissemination of foreground Reporting At the end of the review exercise, the expert will prepare a report with his/her findings, containing an assessment of the facts as well as suggestions for further actions or changes. A template for the Project review report is included in Annex II of this document. This document has to be completed and returned to the Joint Undertaking within the requested deadline. When more than one expert is involved in the Project review, they might be asked to issue a single consolidated report. 3 The term "Foreground" means information and results arising from the Project, as opposed to "Background" which is information and rights prior to accession to the JU grant agreement 6

7 Annex I CHECKLIST FOR THE TECHNICAL REVIEW 1. Presentation of technical achievements Are project results presented in a clear and concise way? Are supporting arguments, facts and figures provided? Are they sound from a technical/scientific standpoint? Based on the currently available (final / intermediate) results, is the project actually enhancing the state of the art in the specific technical domain? Are the ARTEMIS JU support and the European scale of the project justified by the results achieved? Are they given adequate visibility? 2. Allocation and use of resources Have all partners actually allocated and used the manpower they had committed to the Project according to the Technical Annex? Is there any change in the project's consortium composition? Are the financial resources expected to be entirely utilized? Is there any unjustified mismatch between planned and actual costs? 3. Progress of the work Is today's status in line with the results promised in the Technical Annex? Are the milestones/deliverables reached up to now? Are they of good technical quality? Will later milestones/deliverables be reached? List of milestones/deliverables planned versus actual status Are delays and / or requests for project duration extension expected? Necessary actions on deviations 4. Co-operation between partners within the Project How is project partners co-operation organized? What are the dependencies between the results? Experience of work sharing? Means of result exchange? Limitations to exchange results? Benefits of co-operation? 5. Co-ordination with other Projects Is there any planned/on-going initiative to link this Project to other related Project(s)? What is the information flow between the Project and other related Project(s)? Is there an information flow with other relevant Projects (inside and outside ARTEMIS) 6. Preparation for necessary future modifications Are there plans for changes in the Project? Is a modification of the Technical Annex requested/needed? 7. Competitive situation How do the Project partners position the status of their work/the expected results with respect to the competition? Is world competitiveness achievable? By which mean? Is it possible to commercialize expected results? To what extent? To what extent does the Project take care of IPR? Is there any activities in standardization bodies? 7

8 8. Exploitation/Dissemination Activities What is the status of dissemination and exploitation commitments/plans? Are Project partners willing to exploit project results? How? Are companies outside the Project consortium involved (European/non-European)? What specific results are suitable to be presented at ARTEMIS events? 9. Problem report Are there serious problems which require a decision by the JU? Are there proposals for solutions? 8

9 Annex II - TEMPLATE FOR THE TECHNICAL REVIEW REPORT The template hereafter provides the structure for the technical review report that needs to be prepared by the expert(s) after the review. The template lists a number of questions with 3 possible answers "Yes", "partially", "No". The experts are warmly encouraged to provide factual comments and justifications. In particular, in case "Partially" or "No" box are checked, detailed and quantitative justifications must be provided that describe type and extent of the deviations found and the partner(s) involved. If the expert feels that he/she does not have the competence or the information to answer a question, he/she must declare it in the corresponding sections. 9

10 TECHNICAL REVIEW REPORT JU Grant Agreement number: Project Acronym: Project title: Project starting date: Project duration: Name of the scientific representative of the Project's coordinator and organisation: Period covered by the report, from to.. Date and place of review meeting (if applicable): Name(s) of expert(s): Name of expert drafting the report: Individual report Consolidated report 10

11 1. OVERALL ASSESSMENT a. Executive summary, in particular highlighting the scientific/technical achievements of the Project, its contribution to the State of the Art and its impact: Excellent progress (the Project has fully achieved its objectives and technical goals for the period and has even exceeded expectations). Good progress (the Project has achieved most of its objectives and technical goals for the period with relatively minor deviations). Unsatisfactory progress (the Project has failed to achieve critical objectives and/or is not at all on schedule). b. Overall recommendations (e.g. on overall modifications, corrective actions at WP level, or re-tuning the objectives to optimise the impact or keep up with the State of the Art, or for other reasons, like best use of resources, re-focusing ). 11

12 2. OBJECTIVES and WORKPLAN a. Have the objectives for the period been achieved? In particular, has the Project as a whole been making satisfactory progress in relation to the Technical Annex? b. Has each work package (WP) been making satisfactory progress in relation to the Technical Annex? c. Have planned milestones and deliverables been achieved for the reporting period? DELIVERABLES LIST STATUS No. Title Status (Approved/Rejected) Remarks 12

13 d. Are the objectives for the coming period(s) i) still relevant and ii) still achievable within the time and resources available to the Project? i ii 13

14 3. RESOURCES a. To the best of your estimate, have resources used, i.e. personnel resources and other major cost items, been (i) utilised for achieving the progress, (ii) in a manner consistent with the principle of economy, efficiency and effectiveness. Note that both aspects (i) and (ii) have to be covered in the answer. i ii b. If applicable, please comment on large deviations with respect to the planned resources. 14

15 4. IMPLEMENTATION OF THE PROJECT a. Has the Project management been performed as required? b. Has the collaboration between the beneficiaries been effective? c. Do you identify evidence of underperforming beneficiaries, lack of commitment or change of interest of any beneficiaries? 15

16 5. USE AND DISSEMINATION OF FOREGROUND a. Is there evidence that the Project has/will produce significant scientific, technical, commercial, social, or environmental impacts (where applicable)? b. Is the plan for the use of foreground, including any update, appropriate? Namely, please comment on the plan for the exploitation and use of foreground for the consortium as a whole, or for individual beneficiary or groups of beneficiaries and its progress to date. 16

17 c. Have the beneficiaries disseminated Project results and information adequately (publications, conferences )? d. Are potential users and other stakeholders (outside the consortium) suitably involved (if applicable)? e. Is the consortium interacting in a satisfactory manner with other related ARTEMIS Projects or other R&D national/international programmes, standardisation bodies (if relevant)? 17

18 6. OTHER ISSUES a. Have policy-related and/or regulatory issues been properly handled (if applicable)? b. Have ethical issues been appropriately handled (if applicable)? c. Have safety issues been properly handled (if applicable)? Name (s) of the expert(s): Date: Signature(s): 18