Peer Review College Handbook

Peer Review College Handbook

Table of Contents 1. The AHRC s Peer Review College... 1 1.1 Background of the PRC... 1 2. Becoming a member of the PRC... 4 2.1 Eligibility... 4 2.1.1 Academic reviewers... 4 2.1.2 International reviewers... 4 2.1.3 Strategic reviewers... 5 2.1.4 Knowledge Exchange (KE) reviewers... 5 2.1.5 Non-higher education institution (HEI) reviewers... 6 2.1.6 Technical reviewers... 6 2.2 Nominations... 7 2.3 Appointment... 7 2.4 Induction... 7 3. The review process... 8 3.1 Main stages of the assessment process... 9 3.1.1 Submitting a proposal... 9 3.1.2 Sifting of proposals... 9 3.1.2.1 Proposals are sifted according to the following principles:... 9 3.1.3 Principal Investigator Response ( PI Response )... 10 3.1.4 Panel papers... 10 3.1.5 Grading scale... 11 3.2 Freedom of information and data protection... 13 3.3 Awards... 14 3.3.1 Award announcements... 14 4. Role of the PRC reviewer... 15 4.1 Peer Review College review... 16 4.2 Resubmission... 18 4.3 Interdisciplinary proposals... 18 4.4 Applying to the AHRC... 18 4.5 Monitoring and performance... 19 5. Technical Reviewers... 20 5.1 Grading scale for technical reviews... 20 5.2 Technical reviewer guidance... 22 6. Reviewer guidance... 24

6.1 Value for Money assessment... 24 6.2 Why the AHRC might return a peer review for amendment... 25 6.3 Conflicts of interest... 27 6.3.1 Disclosure... 28 6.3.1.1 When invited to do a review... 28 6.3.1.2 When invited to sit on a panel... 28 6.3.2 Examples of conflicts of interests... 29 6.4 Checklist for an effective review... 30 6.4.1 Preparation for review... 30 6.4.2 Analysis... 30 6.4.3 Delivery... 31 6.5 Public engagement... 31 7. Panels... 32 7.1 Subject panels... 32 7.2 Panel remits and structure... Error! Bookmark not defined. 7.3 Panel membership... 34 7.4 Role of the panellist... 34 7.5 Panel meetings... 35 7.5.1 At the panel meeting... 37 7.5.2 Grading and ranking proposals... 38 7.6 Payment of expenses... 39 8. Joint electronic-submission (Je-S) system... 40 8.1 Accessing and completing a peer review document... 40 Accessing and completing a peer review document:... 40 8.2 Maintaining personal information in Je-S... 41 9. Useful documents... 42 9.1 Non-Employee Expenses Claim Form... 42 9.2 Research Council Travel, Subsistence and Expenses Policy... 42 10. Terms and conditions... 43 10.1 Standards of Service... 43 10.2 Code of practice... 43 10.3 Freedom of information and data protection... 43 10.4 Freedom of Information Peer Review Framework... 45 10.5 Equal opportunities... 46 10.6 Anti-Bribery Policy... 46

11. UK SBS (Shared Business Services) (formerly: RCUK Shared Services Centre (SSC))... 47 Annex A: Main stages of the review process... 48 Annex B: Cross-Research Council proposals... 49 Annex C: Je-S system peer review: frequently-asked questions... 51

Chapter 1: The AHRC s Peer Review College 1. The AHRC s Peer Review College The Peer Review College (PRC) forms a crucial part of the peer review process and has a vital role in ensuring that the AHRC supports research of the highest quality. 1.1 Background of the PRC The AHRC is fundamentally committed to competitive bidding and assessment by process of peer review. Peer Review College members form a key part of this system, which is designed to ensure that peer review is conducted with the utmost attention to fairness and transparency and to the requirements of academic rigour. The Peer Review College was created in autumn 2004 with an initial membership of 460. The College was set up to improve the effectiveness of the peer review process in terms of providing high quality and objective reviews. When the Arts and Humanities Research Board (AHRB) was established in 1998, it set up eight peer review panels to assess research applications. Panellists were selected to serve on the panels on the basis of their high standing as scholars and researchers with the aim of achieving a spread of subject expertise across the membership of each panel. Panellists had a broad knowledge and understanding of the subject areas that fell within their panel s remit. They did not, however, have a detailed knowledge of every specialism with which their panel was concerned. For this reason the AHRC also sought advice from specialist reviewers. During the first six years, this was accomplished by asking applicants to identify external assessors/independent evaluators who would provide the panel with an academic evaluation of an application. Panels were then equipped to determine a final grade for each application, taking account of the information provided by the applicant and the assessors/evaluators. Panellists were also asked to prioritise and rank the applications received to the scheme round. This dual approach to peer review worked well and, importantly, secured the confidence of the sector during the initial years of the AHRB, when applying for funding was a relatively new exercise. After four years, it was considered appropriate to set up a review group to ascertain whether the structure was fit for purpose. The review concluded that the existing approach worked well overall and that there should not be radical change. It did, however, consider that some changes could be made to make the process more effective and efficient and this resulted in the establishment of the College. The most important area concerned the quality of evaluations that the panel received. Under the system where evaluators were nominated by applicants, a high proportion of reviewers gave a Page 1

Chapter 1: The AHRC s Peer Review College high quality and high priority grading (A+), and in many cases provided a personal testimonial or uncritical assessment as opposed to a more objective analysis. This limited the evidence-base for the work of the panels, which then had the difficult task of prioritising proposals in the context of a finite number of awards. In taking account of practice in other subject areas and by other funding bodies, the review group concluded that a college of peer reviewers would be well placed to provide more robust assessments: its members would be independently identified as researchers of high standing who could comment authoritatively on applications for funding, through induction days and written guidance, members of the College would be trained in the requirements of AHRC s schemes and programmes, and College members would be expected to review several proposals in the course of each year, compared to former reviewers who were only approached once or twice over a two-year period. This would enable College members to build up experience and expertise over time and to be well placed to develop the comparative picture of quality that is crucial to the work of panellists. The College also provides other benefits: it provides a base of experienced reviewers from which panellists can be chosen, it provides a means for the AHRC to consult and communicate with the research community, for example in relation to new programmes and schemes, and to receive feedback on its activities, and it broadens the knowledge and experience of, and contact with, the AHRC beyond the panels. This helps universities to engage with the AHRC, to understand the AHRC s priorities and goals, and to understand what makes a good application for funding. In May 2005 a Review of the Peer Review College was undertaken. Despite being conducted at an early stage of the College s development, it was found that the quality of reviews, that are so vital to the work of the AHRC s peer review panels, were proving to be more critical and useful and the profile of grades was more diverse. To ensure that members of the College provide reviews that add value to the peer review process all members are expected to attend an induction event to be trained in the main aspects of the role of a College member. The events aim to equip members in making Page 2

Chapter 1: The AHRC s Peer Review College robust, comparative reviews of the strengths and weaknesses of proposals made to the AHRC s programmes. Since June 2007 the AHRC has been using the Joint electronic-submission system (Je-S) for peer review of proposals. This system means that members of the College can view proposals and complete their review online. Nominated reviews ceased in March 2008. Furthermore, schemes which had allowed for a nominated review had to include an additional assessment from the College. From January 2008, members with technical expertise were recruited onto the College as technical reviewers. These members are called upon to comment specifically on the technical feasibility of research proposals. Also in 2008, the AHRC undertook a review of its decision-making structures, including its assessment procedures. Some of the decisions affecting the College were: From 2009 the standing peer review panels were replaced with non-standing panels with membership convened from the College. For non-standing panels, which receive independent peer reviews, grading and ranking decisions were to be made based upon expert reviews and Principal Investigators responses. Non-standing panels which did not receive independent peer reviews were to assess, grade, and rank applications. To widen membership of the College to cater for the diversity of the AHRC s funding portfolio. To introduce a number of categories of College membership additional to those covering subject expertise. These classifications cover the following areas: Non- HEI, Knowledge Exchange, Strategic Reviewers, International, and Technical. College members can belong to more than one group, depending on experience. Today the College has approximately 1200 members with expertise in research areas across the AHRC s subject domain. A full list of members is provided on the PRC pages of AHRC s website. Page 3

Chapter 2: Becoming a member of the PRC 2. Becoming a member of the PRC 2.1 Eligibility The AHRC categorises its peer reviewers into different groups to improve its ability to match reviewers expertise with proposals, and to enable it to identify reviewers who can assess certain aspects of proposals. Candidates must meet the criteria for academic reviewers to be eligible for membership of the College but they can also nominate themselves for other Groups alongside the Academic Group. 2.1.1 Academic reviewers Academic reviewers are appointed to provide reviews of proposals for academic research and postgraduate training within their area(s) of research expertise. Essential criteria Researchers with an active track record who are held in esteem by their peers. Able to demonstrate potential to review proposals within and on the margins of their main subject specialism or discipline. Experience of leading a research project and having responsibility for the overall management of the research or other activities. Desirable criteria Experience of reviewing proposals for the AHRC and/or other funding bodies. Involvement in activities on a national or international level, for example external examining or national level committees. Experience of supervising PhD students and/or engagement with research student issues. Experience of leadership and responsibility. Experience of managing high levels of academic administration. Experience of collaborative working (within or beyond the academic sector). 2.1.2 International reviewers International reviewers are appointed to provide reviews of research proposals of potentially international significance. The criteria for appointment are the same as for academic reviewers, but nominees for international reviewers must also be able to demonstrate one or more of the following: Page 4

Chapter 2: Becoming a member of the PRC Experience of assessing applications for Research Funding Organisations at International level (e.g. for the National Science Foundation (NSF); or for the Netherlands Organisation for Scientific Research (NWO)). Experience of assessing other types of documentation with an international dimension (e.g. being part of international review panels, refereeing for international academic journals). Experience of international research collaboration (e.g. collaboration with international researchers). 2.1.3 Strategic reviewers Strategic Reviewers are appointed to provide reviews of Fellowship (which from 1 May 2014 is being renamed the Leadership Fellows Scheme) applications and those proposals which involve issues at an organisational level (e.g. Doctoral Training Partnerships or Collaborative Doctoral Partnerships) or high levels of complexity. The criteria for appointment are the same as for academic reviewers, but nominees for strategic reviewers must also be able to demonstrate the following: A good strategic overview of arts and humanities in the United Kingdom. Experience of a senior management or senior leadership role within a higher education institution (e.g. Pro Vice-Chancellor, Dean, Head of School or College, or Research Director). Experience of a leadership role outside the research organisation (e.g. leading a subject organisation or learned society). 2.1.4 Knowledge Exchange (KE) reviewers Knowledge Exchange reviewers are appointed to provide reviews of proposals that are focused on or include elements of knowledge exchange activities. The criteria for appointment are the same as for academic reviewers, but nominees for knowledge exchange reviewers must also be able to demonstrate: A good understanding of the distinctive nature of knowledge exchange in the arts and humanities, and experience of one or more of the following: Participation in any knowledge exchange or impact activity as the result of their research; this can include formal and informal, funded or un-funded, and may have encompassed policy work, Page 5

Chapter 2: Becoming a member of the PRC active dissemination, commercialisation, and/or public engagement activities. Collaboration and/or partnership working (including collaborative studentships) with non-academic partners, including commercial business, public sector, and third sector organisations Reviewing knowledge exchange proposals for Research Councils or other funding bodies. 2.1.5 Non-higher education institution (HEI) reviewers Non-HEI reviewers are appointed to provide reviews on the social, cultural, or economic impact of proposals from a perspective outside the academic community. Reviewers from Non-HEIs or with Non-HEI experience are sought to assess proposals requiring a perspective outside of the academic community. Many reviewers from organisations outside the higher education sector are also part of the other college groups and will also be approached to comment on other aspects of proposals. Non-HEI reviewers will usually be based in a non-academic organisation (e.g. publicsector bodies, businesses, third sector organisations, museums, and galleries) and must have experience of one or more of the following: collaborating with academic researchers, using the results of academic research, or reviewing applications for Research Councils. 2.1.6 Technical reviewers Technical Reviewers assess the appropriateness of the technical methodology outlined in any grant applications where digital outputs or digital technologies are essential to the planned research outcomes. Nominees must have the knowledge and understanding of some of the following activities: Experience of producing digital outputs Use of technical methodologies Selection and use of technical standards and formats, including data development methods Selection and use of software and hardware Page 6

Chapter 2: Becoming a member of the PRC Data acquisition, data processing, analysis and use Preservation of data Sustainability of digital outcomes. It is not necessary for candidates to have extensive expertise under all of these headings. They will need a good knowledge of the issues to be addressed in producing a digital output, specifically that the output will be accessible to its target audience and sustainable for an appropriate period of time that justifies the expenditure required to produce it. 2.2 Nominations From time-to-time the AHRC seeks nominations for new College members. Candidates must be nominated by senior members of staff or representatives within higher education institutions or other organisations, learned societies, or professional associations. Candidates must meet the relevant eligibility criteria. College reviewers are nominated by peers as experts, and are able to comment authoritatively on proposals for AHRC research funding. The profile of the College is intended to reflect the breadth of disciplines and subjects within the AHRC s subject domain. 2.3 Appointment College members are normally appointed for a term of four years and may serve up to two terms. Members are expected to review up to eight proposals to funding schemes or programmes during a 12-month period. This commitment is reduced for anyone acting as a panel member. Members must adhere to the Standards of Service in all College work (please refer to chapter 11.1). 2.4 Induction To ensure that members provide reviews that add value to the peer review process all members should attend an induction day to be trained in the main aspects of the role of a College member. The event provides members with the opportunity to learn about the AHRC peer review process, network with other College members, and participate in a mock panel. We expect members to attend an induction before they are invited to become a panel member. Page 7

Chapter 3: The review process 3. The review process An overview of the review process is provided below and is diagrammatically illustrated in Annex A. For the majority of the AHRC schemes the process operates as follows: Proposals are considered by two or more members of the Council s Peer Review College. Where the proposal requires a Technical Plan, the technical feasibility of the proposal is assessed separately. Applicants provide a response (Principle Investigator Response/Right of Reply) to the anonymised comments of the Peer Review College reviewers and, where applicable, the technical review. Proposals, reviews, technical reviews, and Principle Investigator responses to the reviews are considered by peer reviewers appointed by the Council to its panels. A panel grades and ranks the proposals in order of funding priority. The panels recommendations are considered by the Director of Research and a final decision is taken on the list of awards to be supported. Panel members are expected to make informed judgements of all proposals passed to them for review. It is impossible to achieve coverage within a single panel of the full range of subjects and the wide diversity of proposals submitted to the AHRC. Therefore, under the system outlined above, specialist advice is available via the reviews provided by the Peer Review College reviewers. In framing proposals for peer review, applicants are advised to address as wide a group of peers as possible. There are some exceptions to the peer review process outlined above. For example, for the Research Networking scheme, Peer Review College reviews are obtained, but proposals are not considered by peer review panels. Instead they are moderated by AHRC officers where possible. If an in-house grading decision cannot be made they are passed to a member of the Strategic Reviewers Group within the College. The AHRC also runs Assessment Panels for some schemes where panel members do make assessments of the proposals. The peer review process used for a particular scheme or call will be outlined in the documentation for that scheme or call. Page 8

Chapter 3: The review process 3.1 Main stages of the assessment process 3.1.1 Submitting a proposal Proposals to the majority of our schemes are submitted through the RCUK Joint electronic-submission (Je-S) system. Applicants must complete the appropriate proposal form for the scheme to which they are applying and submit this along with a Case for Support and other supporting information to Je-S. If the scheme has a closing date, this information must be received by the published deadline for the scheme. Late or incomplete proposals will not be accepted by the Je-S system. Applicants indicate up to three subject classifications on their proposal form to identify the research area of the proposal. Where there is more than one panel for the scheme, the primary classification determines the panel to which the proposal will be submitted. 3.1.2 Sifting of proposals Proposals to the following schemes will be quality-sifted by the AHRC: Cross-Council funded responsive-mode schemes Research Grants and Leadership Fellows Standard and Early Careers. Research Networking. 3.1.2.1 Proposals are sifted according to the following principles: The AHRC will reject a proposal if the proposal does not meet the published eligibility criteria; either relating to documentation requirements or where it does not meet the aims or criteria of the scheme to which it has been submitted. The AHRC will sift proposals against quality criteria solely on the basis of information supplied by the AHRC peer review process. The AHRC will not sift on the basis of the information supplied by a non-ahrc peer review process. The AHRC will not sift an application based solely on a technical review. The AHRC will not sift outline proposals submitted to its strategic programmes as this process is conducted by a panel. The AHRC will not sift a proposal if the final funding decision does not fall wholly within a Research Council peer review process. A sifting decision will be made based on the overall confidence levels and grades given by the peer review process. A proposal will be rejected if it receives two or more reviews that give the proposal a non-fundable grade (i.e. grades 1 3). Page 9

Chapter 3: The review process 3.1.3 Principal Investigator Response ( PI Response ) For all schemes for which peer reviews are sought, except the Research Networking scheme, applicants who pass the quality threshold (i.e. receive two or more reviews graded at 4, 5, or 6) will be invited to submit a PI response to the Peer Review College reviews and technical assessments. This allows applicants to correct any factual errors or conceptual misunderstandings, or to respond to any queries highlighted in the comments from the Peer Review College reviewers. It is not intended to be an opportunity for the applicant to change or re-constitute a proposal in light of the comments. Applicants are not obliged to submit a response, but are encouraged to do so as responses from applicants are forwarded to the peer review panel and are taken into account in the ranking of proposals. If a response is not received from the PI within the period stated, then the application will proceed to the peer review panel without it. If a PI considers that a response to a particular review or reviews is not required, they are asked to include a statement to this effect in their response. Points to consider for PI Response: there is no need to repeat reviewer comments the PI should aim to keep language neutral in tone, even if you feel strongly about a reviewer s comments the PI should focus on answering questions and addressing any issues raised by reviewers the PI should aim to be specific in their comments the PI should not re-write the application. 3.1.4 Panel papers Meeting papers are sent to the panel a few weeks before the panel meeting. The papers include a number of documents relating to each proposal: the proposal documentation reviews technical review (if appropriate) Page 10

Chapter 3: The review process principal investigator s response to the reviews (if received) a mark sheet (to record grades and comments), and panel guidance. 3.1.5 Grading scale This grading scale is used for most of the programmes and schemes run by the AHRC, although the descriptors may vary slightly for some schemes such as the Leadership Fellows Scheme and for technical reviews. Any variations to the scale are outlined in the guidance for individual schemes. Page 11

Chapter 3: The review process Grade Descriptor 6 An outstanding proposal that is world-leading in all of the following: scholarship, originality, quality, and significance. It fully meets all the assessment criteria for the scheme and excels in many or all of these. It provides full and consistent evidence and justification for the proposal, and management arrangements are clear and convincing. It should be funded as a matter of the very highest priority. 5 A proposal that is internationally excellent in all of the following: scholarship, originality, quality, and significance. It fully meets or surpasses all the assessment criteria for the scheme. It provides full and consistent evidence and justification for the proposal, and management arrangements are clear and convincing. It should be funded as a matter of priority, but does not merit the very highest priority rating. 4 A very good proposal demonstrating high international standards of scholarship, originality, quality, and significance. It meets all the assessment criteria for the scheme. It provides good evidence and justification for the proposal, and management arrangements are clear and convincing. It is worthy of consideration for funding. 3 A satisfactory proposal in terms of the overall standard of scholarship and quality but which is not internationally competitive and/or which is more limited in terms of originality/innovation, significance and/or its contribution to the research field. It satisfies at least minimum requirements in relation to the assessment criteria for the scheme, provides reasonable evidence and justification for the proposal, and management arrangements are adequate overall. In a competitive context, the proposal is not considered of sufficient priority to recommend for funding. 2 A proposal of inconsistent quality which has some strengths, innovative ideas and/or good components or dimensions, but also has significant weaknesses or flaws in one or more of the following: conceptualisation, design, methodology, and/or management. As a result of the flaws or weaknesses identified, the proposal is not considered to be of fundable quality. A proposal should also be graded 2 if it does not meet all the Page 12

Chapter 3: The review process assessment criteria for the scheme. It is not recommended for funding. 1 A proposal which falls into one or more of the following categories: has unsatisfactory levels of originality, quality, and/or significance, falls significantly short of meeting the assessment criteria for the scheme, contains insufficient evidence and justification for the proposal, displays limited potential to advance the research field, the potential outcomes or outputs do not merit the levels of funding sought, or is unconvincing in terms of its management arrangements or capacity to deliver the proposed activities. It is not suitable for funding. 3.2 Freedom of information and data protection Proposals are submitted to the AHRC in confidence and may contain confidential information and personal data belonging to the applicant (and other researchers named in the proposal). You must not disclose the fact that an applicant has applied to the AHRC for a grant nor the content of the proposal to any other person, without the prior written consent of the AHRC or the applicant. You must not use the information in the grant proposal for any purpose other than providing a review of it to the AHRC. You are asked to bear in mind that your comments will be forwarded to the applicant and their institution and that you should not include any comments that may cause offence. Comments must relate to information in the proposal, they must be evidence-based and be made in relation to the scheme/programme criteria. The reviewer form is anonymised to ensure that your identity is not disclosed to the applicant but please ensure that you do not include comments that would reveal your identity. Full details of how Freedom of Information (FOI) and Data Protection (DP) impact on your role and your obligations to these can be found in chapter 10.3. Page 13

Chapter 3: The review process 3.3 Awards The Director of Research agrees the final list of awards within the budget agreed for each round of a scheme. 3.3.1 Award announcements Once the successful proposals have been approved, it is the role of AHRC Staff to ensure that applicants are notified of the outcome of their proposal. Feedback is provided as appropriate. All peer review that panel members undertake needs to be done in the strictest confidence. This includes all written information that is sent to panel members and all discussion at the panel meeting itself. Page 14

Chapter 4: Role of the PRC reviewer 4. Role of the PRC reviewer This chapter outlines how the PRC reviewer fits into the overall peer review assessment process. College members are invited to submit peer reviews which are used by moderating panels to make decisions on whether applications are of a fundable standard and to rank the proposals. Assessments are made using a pre-defined grading scale. Typically three reviews are required for each proposal. Members of AHRC staff select reviewers on the basis of the research classifications and key words listed in the application. These are chosen from the cross-council classification system, available in Je-S. The same list is used by applicants, to classify their research proposal, and by College members, to describe their research expertise. Using common lists for both reviewers and applicants helps ensure that reviewers receive proposals within their area(s) of expertise. PRC members can help in this process be ensuring that their classifications on Je-S are up to date. The AHRC always endeavours to choose reviewers from within its College membership, but will draw on non-college members from the wider academic and user communities when a suitable College member is not available. Your primary role will be to provide the AHRC with informed reviews of proposals submitted to the AHRC. The Council is seeking expert academic reviews of proposals, not personal testimonials. Reviews that provide an objective analysis of research proposals are key to the assessment process. As a Peer Review College reviewer, you should exercise your knowledge, judgement and expertise to reach clear and soundly based decisions that are fair, objective and evidence-based. The AHRC is committed to equal opportunities. Please ensure therefore that all proposals are reviewed on equal terms. Proposals must be reviewed and graded on their merits, in accordance with the criteria for each scheme or call for funding. You may be invited to serve as a panel member, as a representative of your (broad) subject area. Panels rank and grade proposals to a given scheme based on the aims, objectives and criteria of the scheme, as well as its assessment procedures. Page 15

Chapter 4: Role of the PRC reviewer 4.1 Peer Review College review The AHRC normally contacts two or more Peer Review College members to review each proposal received. If you are selected, you will receive an email request from the Je-S (Joint Electronic Submission) system regarding the proposal that you are being asked to review. The proposal and all the paperwork can be accessed via the Je-S system and you will need to complete and submit your review on the Je-S system. All Peer Review College members need to have a Je-S account. Details about how to do this will be included within the review request email and there is also a set of FAQs about the Je-S system that can be found in Annex C. For almost all of our schemes you will need to use the Je-S system to submit your review. AHRC will not be able to accept your review via email or in hard copy, unless otherwise stipulated by us. If you need any assistance or advice in your use of the system or experience any problems, you should contact the Research Councils Je-S Helpdesk. They can be contacted by telephone on 01793 444164 or by email on JeSHelp@rcuk.ac.uk from 8.30am to 5pm Monday Friday. If you consider that you have been inappropriately matched to a proposal or are unable or unwilling to complete the review, please log in to the Je-S system and decline the review (by selecting Decline to Review under the Document Data menu in the Je-S review form and select SUBMIT ). We ask that any decline notifications are sent as soon as possible and no later than five working days so that an alternative reviewer can be approached immediately. It would be helpful if you could suggest the name of an alternative reviewer but please do not approach the individual yourself. A reviewer can decline to carry out the review at any point. If the reviewer is able to complete the review, but not by the due date given, they should not decline the review immediately but should contact the person named in the Instructions to reviewer section to discuss an extension to the deadline. A reason for declining the review must be given. The options given are: Outside area of expertise Too busy Conflict of Interest Other Page 16

Chapter 4: Role of the PRC reviewer Please note that it is mandatory to provide further details in the text field if other is selected. If you intend to complete the review, it would be helpful if you could let us know. You can do this by emailing the AHRC contact named in the Reviewer Information section of the form. If a review is received after the due date and you have not sought prior approval to submit it late, we may not be able to use it. You will be expected to review up to eight proposals during each 12-month period (normally no more than four per quarter). You will be asked to provide a review, covering the following broad headings: quality and importance people management of the project value for money and appropriateness of resources requested outputs, dissemination and impact overall assessment (grade and overall conclusions on proposal, including strengths and weaknesses). Scheme/Programme specific guidance, including review criteria is available via the Je-S Help Text. This will detail what points you should address under each of the headings. You will also need to assign each proposal a grade, as indicated in the guidance. When composing your review please bear in mind that a copy of your comments will be forwarded anonymised to the applicant. You are asked to keep in mind the guidance within the Freedom of Information and Data Protection section in chapter 10.3. You are also asked to word your comments carefully, to avoid any personal remarks that may cause offence, and to adhere to the AHRC s Equal Opportunity policy. Please ensure that you only provide comments that relate to the information in the proposal and are relevant to the criteria for the scheme in question. You are responsible for keeping your contact details up-to-date. To amend your details, please log in to the Je-S System using your Je-S User Id and password, and then select the "My Details" option to amend the details. Please be aware that some changes, Page 17

Chapter 4: Role of the PRC reviewer including a change to department and/or organisation, will not be displayed until the details have been verified by the Je-S Helpdesk. The system will also prompt you, at regular intervals, to check and re-affirm your details. If you are unable to provide reviews for a given period of time for example, if you are intending to go on sabbatical, are ill or on maternity leave - please log your unavailability in Je-S. You will not be asked to do any reviews during that period. 4.2 Resubmission In line with the AHRC s approach to demand management, unsuccessful applicants will not be permitted to resubmit the same, or substantively similar, proposal to the same scheme. In very particular circumstances the AHRC may, exceptionally, decide to invite the applicant to resubmit the proposal. This will happen only where the panel identifies an application of exceptional potential and can identify specific changes to the application that could significantly enhance its competitiveness. Invited resubmissions will be assessed in the usual way in competition with all other proposals. For general information on resubmission and for information specific to schemes, please see the Research Funding Guide. 4.3 Interdisciplinary proposals The Research Councils have an agreed approach for collaborating on the peer review and funding of research proposals that straddle their remits under their responsive-mode research grants schemes. Guidance on reviewing interdisciplinary and multidisciplinary proposals is detailed in Annex B. If you are asked to review a proposal that crosses Research Council remits it is important that you read this guidance. 4.4 Applying to the AHRC You may apply to the AHRC s research programmes but you will not be permitted to review competing proposals in the round to which you intend to apply or to attend the panel meeting at which your application will be assessed. If you are connected to a proposal which is either awaiting a funding decision or which will be submitted in the next three months, please check whether it was, or will be, directed to the same scheme for which you are being invited to provide a review. If this is the case, please decline the review. Page 18

Chapter 4: Role of the PRC reviewer If you have already agreed to sit on a panel and intend to apply, please notify the staff member responsible for your panel as soon as possible. You will need to stand down from that meeting and will not be able to receive the panel member s fee for that meeting. AHRC Peer Review College reviewers are free to apply for funding from other sources, including the British Academy and other Research Councils. 4.5 Monitoring and performance We expect you to observe the standards contained within the Standards of Service document which you agreed to on becoming a member of the College. We will periodically review data on numbers of requests made, numbers of reviews provided and number of acceptances and declines made. You may be asked to withdraw from the College if you do not comply with the Standards of Service. Page 19

Chapter 5: Technical Reviewers 5. Technical Reviewers A Technical Reviewer is selected from the Peer Review College to provide grades and comments on the technical aspects of proposals which are submitted in a Technical Plan. If you are selected, you will receive an email request from the Joint electronic-submission (Je-S) system regarding the proposal that you are being asked to review. You will be required to complete and submit your assessment via Je-S. The AHRC aims to ask technical reviewers to assess around eight proposals to its funding schemes and programmes during each 12-month period, but on occasions we may ask you to review more. Academic College Members who also have technical reviewer membership will undertake technical reviews in addition to the quota of academic reviews normally undertaken. You will have between three and four weeks to provide the assessment, although the timetable may be shorter if you are not the first person approached. 5.1 Grading scale for technical reviews The grades and definitions you will use are provided in the table below: Page 20

Chapter 5: Technical Reviewers Grade Descriptor 6 Strong Support the technical aspects of the proposal are strong, well thought out and appropriate to the needs of the proposed project. 5 Support with advice the technical aspects of the proposal are sufficient to allow the project to be completed successfully, although there are some issues raised in the review that the PI is advised to bear in mind. 4 Conditional Support in general, the proposal has addressed the technical requirements of the project but there are one or more issues which would need to be addressed before an award is made. Minor issues may be addressed through PI response whereas larger issues may need to be addressed through a conditional grant offer. Please indicate whether the issues are minor or major. 3 Unsatisfactory there are significant concerns about the technical aspects of the proposal. The proposed approach is inappropriate to the nature of the project and would need to be completely revisited. The concerns are of such significance that the project should not be funded as it currently stands. 2 Insufficient information the proposal has not provided enough information to enable a review to be provided. The proposal would need to be resubmitted. 1 Not used The technical assessment will be forwarded to the peer review panels to assist them in making their grading decisions. In addition to providing technical reviews we may ask you to attend at least one panel meeting during your period of appointment to provide technical advice. Please ensure you have read the following sections which provide further information on your role: Conflicts of Interest Decline to Review Equal Opportunities Page 21

Chapter 5: Technical Reviewers Freedom of information and data protection Principal Investigator Response. 5.2 Technical reviewer guidance The purpose of the technical plan of an AHRC grant proposal is to allow the applicants to demonstrate that the proposed project managers have considered and understood the technical issues associated with the research project and are proposing to use appropriate methods, standards and equipment to achieve the aims and objectives of the proposal. You will be asked to provide an assessment covering the following broad headings: Standards and Formats Hardware and Software Data Acquisition, Processing, Analysis and Use Technical Support Preservation of Data Sustainability: Access and Re-use of Digital Outputs Overall Assessment (grade and overall conclusions on proposal, including strengths and weaknesses) For further details on completing each aspect of the technical review please refer to the Je-S help text. You will also find it helpful to refer to the guidance on completing the technical plan provided to applicants within the AHRC Research Funding Guide. The AHRC asks technical reviewers to provide a brief summary of the overall technical feasibility and merit of the proposal as well as any reservations and/or recommendations which are relevant, for example suggestions to enhance the technical feasibility or the merit of the proposal. Please bear in mind the grading you give when formulating your comments. Page 22

Chapter 5: Technical Reviewers In providing the technical review you should concentrate on the information contained within the technical plan. You should, however, make sure that you are aware of the information within the proposal form and case for support which may provide context or detail not outlined in the technical plan alone. Page 23

Chapter 6: Reviewer guidance notes 6. Reviewer guidance 6.1 Value for Money assessment This section provides further advice on completing the Value for Money section and is intended to complement the existing guidance. In considering the Value for Money section, you are essentially being asked two connected questions which should help you decide whether the resources requested are justified and therefore whether the project offers value for money: 1) Will the proposed project be undertaken in an efficient and effective way and is the applicant requesting only resources that are essential to the completion of the project? 2) Do the importance and the quality of the proposed research (and associated outcomes) justify the amount of resource required? In the first question you should consider whether the proposal has demonstrated an understanding of the amount of work to be done. Has the applicant identified the level of staffing (both the amount in FTE and the experience/skills), travel and subsistence and other costs that will be needed in order to achieve successfully the aims of the project? For example, in terms of the work planned for the research assistant (RA), is the amount of work achievable within the timescale, for someone with that level of expertise, and with the level of support and resource described? You should also consider whether the applicant has clearly linked how each resource (e.g. RA, visit to archive, use of materials) will be used in undertaking the project. This information should be provided within the Case for Support and Justification of Resources. In the second question, if you have established that the project looks as though it will be run efficiently and effectively, you are asked to consider whether the importance and significance of the research is appropriate for the amount of resource requested. For example, if the proposed project requires a large number of research assistants, but the output is going to be limited (either in scope or potential audience), then this might not provide good value for money. Finally, we would like to offer some other considerations, in the hope of further clarifying the way in which reviewers should be approaching this part of a review: We are not expecting reviewers to say whether 200 is the correct price for a flight, but we would like them to consider whether all the trips that have been Page 24

Chapter 6: Reviewer guidance notes requested on a proposal are justified as being needed in order to conduct the research. We are not expecting reviewers to comment on whether the researcher should receive a certain salary, rather we would ask reviewers to assess whether there is sufficient work, of the appropriate level, to warrant a researcher being employed for the period which the proposal requests. 6.2 Why the AHRC might return a peer review for amendment The AHRC takes great care to ensure that the reviews completed by members of our Peer Review College are appropriate and of value to both the applicant and our moderating panels. For that reason, if the AHRC feels that there are any elements of a review that do not meet our criteria, we will return them to the reviewer for amendment. The following is a list of common reasons why the AHRC might return a review for amendment. 1. The comments and grades throughout a review contradict the overall grade given by the reviewer e.g. the wider content of the review indicates that a proposal is not of a fundable quality but a fundable overall grade is given. This can sometimes happen when a reviewer is taking the potential of a project into consideration rather than the content of the proposal as it stands. Reviewers should focus on the proposal as it stands and not as it might be if the opportunity to resubmit is available. Revisiting the grading descriptors when awarding a grade can help reviewers to ensure that their comments are consistent with the overall grade given. 2. The comments in a single section of the review don t match the grade given for that section e.g. the content of an individual section indicates that there are serious concerns with that element of the proposal but section is graded Good. This could be because the reviewer has not balanced their comments and only commented on their concerns without highlighting the positive elements of that area of the proposal. Balanced comments, highlighting both strengths and weaknesses, will help the Panel to moderate the various reviews for a proposal and will give the PI the necessary feedback to respond to reviews effectively. 3. Reviewers declaring a confidence level in their ability to provide a review. In asking reviewers to declare their confidence level, the AHRC looks for an indication of a reviewer s expertise within their field rather than their experience of writing peer review per se. If a reviewer doesn t feel that they can confidently review a proposal because it falls outside or only minimally within their field of expertise, they will need to Page 25

Chapter 6: Reviewer guidance notes consider whether they should accept the review request at all. If reviewing a specific area of a multidisciplinary proposal, reviewers should consider their confidence level for the area of the proposal that falls within their field of expertise rather than the entire project. Under these circumstances it is perfectly acceptable to note within a review that the comments and grades relate to a specific area or discipline within the proposal. 4. The confidence levels given by the reviewer are inconsistent with their comments e.g. a reviewer clearly demonstrates an expert understanding of the proposal in question but marks themselves as having a low level of expertise. Most commonly this is because a reviewer is being modest or has limited experience of providing peer review for the AHRC. It is worth remembering that the AHRC aims to approach people with the requisite expertise when seeking peer review. If a reviewer is confident that the proposal falls within their field of expertise and that they can provide a review that meets our criteria then their declared confidence levels should reflect that. 5. The tone and language used by a reviewer is confrontational or emotive. AHRC s moderating panels can only moderate if the peer reviews received are balanced and objective. It s also worth remembering that the PI will have sight of the peer review as part of the right to reply process and it may prove difficult for them to respond effectively if a reviewer s tone or language is subjective or unbalanced. 6. The reviewer identifies themselves within their comments (inadvertently or overtly). While the panel is made aware of the identity of our peer reviewers, applicants are not. Anonymity of peer review is important to ensure that members of the Peer Review College can express their views freely. 7. The reviewer identifies the grade within their comments (inadvertently or overtly). By overtly stating the grade or implying that a particular grade might have been awarded within a review, it can give the PI a sense that the decision has already been made regarding their proposal thus giving them a premature sense of success or failure. This might then affect how they approach the right to reply process and might lead them to question their final outcome if it doesn t reflect the grade implied within a review. 8. If the comments made are speculative or don t relate to the content or context of the proposal. Page 26

Chapter 6: Reviewer guidance notes As noted in point 1, this can happen when a reviewer is looking at the potential of the project rather than the proposal as it stands. Reviewers should focus on the proposal in front of them and not as it could be if changes were to be made. 9. A reviewer s comments are too brief. AHRC s decision making processes rely on expert peer review of the proposals submitted. In order to moderate a proposal effectively, our panels need to have a clear idea of the strengths and weaknesses of a proposal as identified by reviewers. Comments should be justified so that the moderating panel can make an informed decision based solely on the expert peer reviews received. 10. The reviewer has not addressed the areas expected for effective peer review and there is little or no critical examination of the proposal. As noted in point 9, AHRC moderating panels require expert peer review that identifies both strengths and weaknesses in order to moderate a proposal effectively. It is important that research questions, context and methodologies are considered when writing peer review for AHRC. 11. The reviewer makes specific comments regarding AHRC processes and policies. The AHRC welcomes comments and feedback from members of the Peer Review College regarding our processes and policies. However, including this kind of commentary or feedback within a review is unhelpful to the applicant and panel as it has no bearing on the assessment of the proposal in question. Members of the Peer Review College should address any comments or feedback regarding our processes and policies to the AHRC PRC Team. 6.3 Conflicts of interest The Research Councils have adopted a code of practice for all those who assist in the work of the Councils which embraces the "Seven Principles of Public Life" drawn up by the Nolan Committee and endorsed by Parliament. These Principles refer to selflessness, integrity, objectivity, accountability, openness, honesty and leadership. An important aspect of this code is the avoidance of any conflicts between personal interests and the interests of the Research Councils to ensure the probity of the decisions made by the AHRC and its representatives. In the context of peer review of research proposals, a conflict of interest might arise as a result of direct, or indirect, personal, academic, financial or working relationships. The acid test is whether a member of the Page 27

Chapter 6: Reviewer guidance notes public, knowing the facts of the situation, might reasonably think the judgement could be influenced by the potential conflict of interest. 6.3.1 Disclosure Conflicts of interest may occur (1) when you are invited to do a review and (2) when you are invited to sit on a panel. 6.3.1.1 When invited to do a review It is vital that all reviewers, including technical reviewers, are seen to be completely impartial at all stages of the review process. You should not take part in the review of any proposal where a conflict of interest may be construed. AHRC officers endeavour to identify conflicts of interest and will not select you as an assessor if there is a clear conflict. However, not all conflicts are obvious from the information we have available. If you consider you may have a conflict of interest, you must contact the AHRC before proceeding with the review. 6.3.1.2 When invited to sit on a panel It is vital that Panel members are seen to be completely impartial at all stages of the review process. You should not take part in the moderation of any proposal where a conflict of interest could be construed. If you think you might have a conflict, please inform the staff member responsible for your panel at the earliest possible opportunity. You may still serve on the panel but you will be required to leave the room whilst the proposal(s) for which you have a conflict is/are being discussed. You are permitted to be present for the ranking of all proposals and it will be the responsibility of the chair to ensure that such proposals are not discussed again in detail, to ensure fairness and avoid any potential embarrassment. You should not be involved in any way with a proposal prior to its submission or once a decision has been taken, e.g. you should not comment on, or help colleagues in preparing a proposal. If you are in any doubt as to whether you have a conflict of interest, you should consult staff working on your panel. If you are approached by applicants to discuss their proposals in any way whether it be before, during or after the assessment process you should decline. You are free to talk to applicants about the Council s structures, policies and modes of operation, so long as the information is in the public domain (e.g. in the Research Funding Guide or on the website). You must not divulge information about individual awards or application statistics, unless the information is already in the public domain (via press release, Annual Reports, etc.). Page 28

Chapter 6: Reviewer guidance notes 6.3.2 Examples of conflicts of interests A conflict of interest occurs for a reviewer or panel member when you: are a personal friend or a relative of the applicant are intending to submit or have already submitted a proposal to the same round of the competition for which you are being asked to provide a review. For schemes which operate without closing dates, if you have or are intending to submit a proposal within 3 months of when you are being asked to provide a review are directly involved in the work that the applicant proposes to carry out and/or have assisted the applicant with their application for funding are a current member of staff or a Professor Emeritus/Emerita at the same research organisation as the applicant, or at the proposed host institution for Fellowship proposals? If you are at Cambridge or Oxford University you should not review proposals from any college at your institution. Please note that staff employed by a member institution of the University of London or University of Wales may review proposals from other member institutions have collaborated on a research project, or worked closely, with the applicant in the last five years have been employed at the same department as the investigator(s) in the last 12 months were the PhD Supervisor for the investigator(s) are asked to review an application in which your RO is named as a partner institution have a vested interest in the research, for example you are a general editor of the series to which the work that is the subject of the proposal will contribute, or a curator of a gallery where the work will be exhibited have been approached and agreed to be a member of a committee connected with a research project, for example an advisory group or steering committee, you should not - if approached - also act as a reviewer for that project are invited to sit on a panel which will moderate an application that you have reviewed. Please note that, the restrictions which apply to the Principal Investigator, in terms of research organisation, past research organisation, past collaborations and any other types of conflicts of interest as mentioned above, apply equally to the Co- Investigator(s) on an application. Page 29

Chapter 6: Reviewer guidance notes This is not an exhaustive list: if you are in any doubt about whether or not you should assess a proposal due to a possible conflict of interest, please contact the Programmes Coordinator who has approached you for a review. Please also refer to the conflicts of interest section of the AHRC's Code of Practice for members of the AHRC Council, Committees, Panels and Advisory Groups. 6.4 Checklist for an effective review This checklist was collated from College members responses as to what constitutes an effective review. Reviewers may find the checklist a helpful guide for providing a high quality review. 6.4.1 Preparation for review ensure you read the entire proposal thoroughly familiarise yourself with the strategic aims of the AHRC and the aims of the scheme for the proposal you are assessing be aware of the full range of grades and their descriptors at your disposal contact staff at the AHRC if anything is unclear. 6.4.2 Analysis be realistic about your own confidence and expertise. Provide clear evidence of your own expertise in the subject area and state if you re unsure about something always provide evidence to support your observations. Use only the information provided in the application form take into account the information you are being asked to provide under each review form heading. Ensure sufficient detail is provided for each one give a clear assessment of strengths and weaknesses of the proposal and indicate whether these are major or minor concerns provide an evaluation of the risks associated with the project contextualise the proposal that you are reviewing within current work in the field, and comment on its relative importance/significance identify any inconsistencies and contradictions in the proposal identify issues which need clarification by the applicant (PI) in their response Page 30

Chapter 6: Reviewer guidance notes in the case of interdisciplinary applications: do the different disciplines meet up in a coherent way? provide enough information to enable a judgement on the relative quality of this proposal compared to other applications be receptive to new ideas and approaches to thinking within your discipline as well as methodology. 6.4.3 Delivery provide an impartial, objective, fair and analytical assessment of the proposal which you are reviewing avoid overly negative comments and do not include any personal comments make constructive criticism wherever possible, identifying how any issues could be realistically addressed by the PI ensure you are providing an evaluation, not a description of the work proposed ensure that the language you use is clear and jargon-free. Could your review be understood by a non-expert? is your grade justified by, and consistent with, your comments? could a non-expert make a final grading decision based upon your review? 6.5 Public engagement If the project involves public engagement in the research, has consideration been given to ensuring appropriate equality and diversity of engagement? Page 31

Chapter 7: Panels 7. Panels The AHRC convenes non-standing panels from the membership of the College. Panels meet on an ad hoc basis according to the requirements of the scheme for which they provide final grades and ranking. To avoid conflicts of interest College members are not appointed to panels for which they have submitted a review or where they are involved with a proposal that is being considered at the meeting. The AHRC relies on moderating panels to make funding recommendations for most of their schemes, including Research Grants and Leadership Fellows with open deadlines. Moderating Panels assign final grades and rank proposals in order of priority for funding. The rank ordered list agreed by the Panel forms the funding recommendation for AHRC. The role of the panel is to make judgments on the applications on the basis of the feedback from the peer reviewers and PI response. It is not their role to reassess the applications when deciding the final grade. Assessment panels operate differently. It is the role of the members of these panels to assess the application and assign a suggested grade against the scheme criteria, normally prior to the meeting. The panel meets to discuss the pre-assigned grades, agree final grades and rank proposals in order of priority of funding. The rank ordered list agreed by the Panel forms the funding recommendation for AHRC. 7.1 Panels for schemes Research Grants, Leadership Fellows, CDAs and Strategic Initiatives Operate a single panel. Proposals to some schemes such as the Research Networking scheme do not go to panel at all. Further information about the assessment route for these schemes can be obtained from our website. The panels are non-standing and convened on an ad hoc basis from the College membership. Details of the panel membership can be found on our website. Page 32

Chapter 7: Panels Page 33

Chapter 7: Panels 7.3 Panel membership Whenever possible, panel membership is drawn from the College. Not all College Members will have the opportunity to sit on a panel during their term of membership. The following is taken into consideration when determining membership for panel meetings: for each research grants scheme panel, the subject spread and number of applications are predicted and the size and scope of the panel organised accordingly. We aim to get a spread of expertise but we cannot cover all subjects in which we are likely to get applications an equal balance between those that have previously sat on a panel/acted as Chair and those who have not previously sat on an AHRC panel there will only be one panel member from any one institution those who have submitted an application or a review for an application which will be considered at the panel meeting, will not be invited, as these constitute a conflict of interest the AHRC is committed to equal opportunities and aims to ensure an even geographical spread from across the UK and a fair gender balance. 7.4 Role of the panellist As a panellist, you are expected to: Familiarise yourself with the guidelines and assessment criteria for the scheme(s) with which you may be involved Comment on and grade all proposals which are directed to you, except where you have a conflict of interest, also see chapter 6.3 Attend panel meetings to agree final grades and rankings for all proposals, and feedback for applications where applicable Make a note of your opinion of the proposals. If, for some reason and at the last minute, you can t attend we might ask for your notes. Full guidance on what kind of comments you are expected to make will be sent to you along with the applications. Your comments will facilitate discussion at the meeting, and contribute to feedback if applicable. Please ensure that your remarks are carefully Page 34

Chapter 7: Panels worded, as applicants may request access, under the Data Protection Act, to all personal data being held on them by the AHRC. In undertaking the above tasks, you are expected to exercise your knowledge, judgement and expertise to reach clear and soundly based decisions; treat all applications, award reports and assessments as strictly confidential at all times; always be fair and objective; and adhere to the Council s Equality and Diversity Policy. Panel Members are paid a fee of 170 per meeting and Chairs will receive 230 per meeting. This is considered as income from self-employment and is taxable as such. If you are sitting on a moderating panel, you will not be allowed to introduce new criticisms or comments that have not already been raised by the peer reviewers. Full guidance about your role will always be provided when you accept an invitation to sit on a panel. It is impossible to achieve coverage within a single panel of the full range of subjects and the wide diversity of applications submitted to the AHRC, therefore you will be expected to make informed judgements of all proposals assigned to you for review (except where there is a conflict of interest), including those which do not lie within your precise area of subject expertise. Even if you consider that you cannot make informed judgements on matters such as the significance and importance of the research, you may have valuable comments to make on matters such as the feasibility of the methodology, or whether the proposed research would present value for money. You should be guided by your experience as a researcher as well as the expert advice from the peer reviewer. Under the system outlined above, specialist advice will be provided by the assessments from Peer Review College assessors. We advise applicants that, in framing applications for peer review, they should address as wide a group of peers as possible and to consider the widest possible routes for dissemination of their work. If you have been approached to sit on a panel and you know that you will be applying to any AHRC scheme, you are asked to inform the AHRC as soon as possible as you will need to step down. Early notification is important in order to provide the AHRC with enough time to find a replacement. 7.5 Panel meetings The purpose of panel meetings is to consider and reach final agreement on the grading and ranking of proposals, and also to agree broad feedback for applicants, where applicable. The panels recommendations are then presented to the Director of Research Page 35

Chapter 7: Panels who will make the final funding decisions. Panel meetings also provide an opportunity for panellists to raise issues such as the quality of peer review or the potential impact of the research proposed through the applications received to that meeting. If you are invited to act as panel chair, you may particularly wish to focus on those proposals where there is a divergence of views amongst panellists. Your principal role will be: to chair the meeting ensuring that the correct procedures are followed, conclusions and recommendations are agreed, and that the meeting runs to time approve and sign panel meeting minutes if required, approve draft feedback comments provided by staff responsible for your panel moderate the discussion of applications and ensure each application is graded and ranked appropriately ensure that an agreed rank ordered list of proposals is produced help to identify intellectual trends apparent from the proposals assessed help to identify particularly significant or newsworthy projects which might be used in AHRC publicity and other documentation for wider audiences liaise and work with AHRC staff responsible for your panel wherever possible, comply with any deadlines given, to enable staff to conduct panel arrangements as smoothly as possible, for example commenting promptly on draft minutes or feedback comments to unsuccessful applicants. Panellists and chairs are normally appointed for approximately seven weeks before and five weeks after the scheduled panel meeting date. You will be expected to fulfil your agreed role during this period. If for any reason you cannot, you are asked to notify us as soon as possible so that we may find a replacement. For the period of your panel appointment you have the option of making yourself unavailable to undertake reviews by updating your details in Je-S with the period of your unavailability. If you know that you will not be able to attend the meeting, please inform the staff member responsible for your panel as soon as possible. You may still be required to provide detailed comments, which can be taken in to account at the meeting. Page 36

Chapter 7: Panels You are expected to have familiarised yourself with all of the proposals which have been assigned to you as first or second introducer or supporting introducer and have read as many of the other proposals as you can, as this helps the panel s discussions. Taking account of the aims and assessment criteria for the scheme concerned, you will need to have graded and commented in advance on all of the proposals you have been asked to introduce. You should grade the applications independently. If you think that you do not have all the paperwork, or if you are aware of any problems that could be resolved in advance of the meeting, please alert the AHRC staff member responsible for your panel at the earliest opportunity and at the very least before the meeting so that s/he can take the appropriate action. Any feedback given to applicants as part of the review process will normally be released to the applicant in an anonymised form which they are asked to keep confidential. It is important therefore that your comments are coherent, carefully considered and avoid any personal remarks. Your comments and grades will not be used outside the peer review/funding decision making process unless they are subject to specific legal requirements. 7.5.1 At the panel meeting The AHRC convenes the Panel. Before the Panel, every proposal is assigned to two introducers and possibly a supporting introducer. Before the meeting, each introducer will decide an overall grade. The Panel discusses and agrees a grade for the proposal, and ranks it relative to the other proposals. Finally, the Panel reviews its final grades and ranking list which constitutes its funding recommendation. At the panel meeting AHRC staff: minute questions concerning policy, protocols and assessment practices advise on any points of protocol or policy where necessary ensure that proper procedures and protocol guidelines are enforced during the meeting, including adherence to AHRC s commitment to equal opportunities ensure that the panel provides and records appropriate feedback comments for applicants where necessary. Page 37

Chapter 7: Panels It is vital that panel members are seen to be completely impartial at all stages of the process. You should not, therefore, take part in the review of any proposal where a conflict of interest could be construed. If you think you might have a conflict, please inform the staff member responsible for your panel at the earliest possible opportunity. If you are in conflict with a proposal, you will be required to leave the room whilst the proposal is discussed. You are, however, permitted to be present for the ranking of all proposals and it will be the responsibility of the chair to ensure that such proposals are not discussed again in detail, to avoid any potential embarrassment. You should not be involved in any way with a proposal prior to its submission or once a decision has been taken e.g. you should not agree to advise or comment on, or help colleagues in preparing, a proposal. If you are in any doubt as to whether you have a conflict of interest, you should consult staff working with your panel. If you are approached by applicants to discuss their proposals in any way whether it be before, during or after the assessment process you should decline. You are of course free to talk to applicants about the Council s structures, policies and modes of operation, so long as the information is in the public domain (e.g. in the Research Funding Guide or on the website). You must not divulge information about individual awards or application statistics, unless the information is already in the public domain (via press release, Annual Reports, etc.). 7.5.2 Grading and ranking proposals Panels are asked to consider each proposal on its merits and award it a grade. Grading decisions at moderating panels are reached through discussion of the proposals - taking into account the comments of the reviewers and (where applicable) the technical review, the Principal Investigator s response to the reviews, grades and comments of individual members of the panel. At assessment panels grading decisions are reached through discussion of the proposals based on grades and supporting comments of individual members of the panel. Panels will be asked to rank proposals. The number of proposals that need to be ranked will vary according to the scheme, the round and the level of funding available. AHRC staff will be able to advise further at the meeting. When ranking applications where there is a conflict of interest for a member of the panel, the application should not be discussed again in detail. Page 38

Chapter 7: Panels AHRC staff may make adjustments to the costs identified in proposals prior to making awards, acting upon recommendations from panels (for example reducing the amount of staff time, or reducing the overall resource for travel or equipment). Conditions may also be applied by panels before awards can be confirmed. In considering the proposals, you must ensure that your judgment is based solely on the aims and assessment criteria for the scheme, and the information provided to you in the application form, the reviews and the PI s response to these, where received. You should not allow private knowledge of the applicant or the proposed research to influence your judgment nor should you introduce new concerns or criticisms that have not been raised by the reviewers. 7.6 Payment of expenses Expenses can be claimed for AHRC meetings or other events that Members are required to attend and where the AHRC has specified that expenses can be claimed. Expenses for other relevant meetings that Members are invited to attend may be paid but require advance authorisation in writing from an Associate Director. Panel members must complete a Claim Form, available from the relevant staff member, and submit it, signed and with receipts, within 6 months of the meeting/event. Page 39

Chapter 8: Joint electronic-submission (Je-S) system 8. Joint electronic-submission (Je-S) system For all our schemes reviewers will need to use the Je-S system to submit reviews. Please note that the AHRC will not be able to accept a review via email or in hard copy, unless otherwise stipulated by us. 8.1 Accessing and completing a peer review document Below are basic instructions about how to access and complete a peer review document in Je- S. If you need more information please get in touch with the Je-S Helpdesk or the Peer Review College Team. Accessing and completing a peer review document: 1. On the 'Assigned Document Summary' screen, select 'Peer Review'. 2. The first time you review a proposal you will be asked to accept the 'Reviewer Protocols' by marking your decision and selecting 'Save'. 3. You may then be asked to confirm your personal details that are stored on the Je-S database. If your details are correct you can tick the box and click 'Save'. 4. If your details need updating you can make the changes and then confirm (as in point 3, above). 5. You then need to return to the 'Current Documents - Peer Review' screen and select 'Edit' next to the Peer Review Document you are reviewing. The 'Edit' will change to 'View' if the due date for the report has passed, but you will still be able to complete and submit your report. If the date has passed, you may wish to check with the Office that the review is still required. 6. You are now in the 'Document Menu Screen'; under 'Document Data' you can select to view the documents to review. The proposal form, case for support, and any supporting documents will be included for you to view and print. 7. You will see the option to open each attachment individually by clicking on the file name. Alternatively you can open the attachments as one single file or have the documents emailed to you as one single file. 8. To complete the peer review form you select each of the 'Edit' options listed under the 'Document Data' heading; these sections make up your form. The headings may vary depending on the Research Council and type of proposal. Page 40

Chapter 8: Joint electronic-submission (Je-S) system 9. When you have completed your review you can validate ('Validate Document') to check all mandatory sections are complete, and then select the 'Submit Document' option to send your comments back to the Research Council. 10. To see reviews that you have completed and returned to the Research Council you can select the option to 'Show Documents Submitted to Council' from the 'Current Documents' screen. 8.2 Maintaining personal information in Je-S As a user of the Je-S system you are responsible for updating your personal details. It is crucial that you keep your contact details and your classification information up to date as this will enable AHRC to contact you effectively and ensure that you are sent appropriate applications to review. To amend your details please log-in to the Je-S system using your Je-S User ID and password and then select "My Details". Some changes, including a change to Department and/or Organisation, will not be displayed until the details have been verified by the Je-S Helpdesk. For guidance please refer to the System Help notes and the Maintaining Personal Details Tutorial. If you experience any difficulty using Je-S or have any questions please contact the Je-S Helpdesk 01793 444164. Page 41

Chapter 9: Useful documents 9. Useful documents 9.1 Non-Employee Expenses Claim Form Non-Research Council employees must submit their expenses claims using this form. 9.2 Research Council Travel, Subsistence and Expenses Policy The Policy incorporates the policy for travel on Research Council business, including related aspects such as overnight accommodation. The Research Council will reimburse the actual additional costs necessarily incurred by claimants when they are away from home or their normal place of work on Research Council business. Page 42

Chapter 10: Terms and conditions 10. Terms and conditions 10.1 Standards of Service The AHRC relies on reviews from members of its Peer Review College to provide informed assessments of applications as part of the decision-making process. It is imperative that we maintain a high standard in all areas of the process in order to ensure the efficient processing of proposals. Reviews are requested and must be submitted through the Research Councils Joint electronic-submission (Je-S) system. College members are therefore asked to agree to abide by certain standards of service. The AHRC reserves the right to make reasonable changes to these standards of service. Any changes will be effected by a general notice to members of the Peer Review College. 10.2 Code of practice The AHRC has adopted a Code of Practice for members of the AHRC Council, Committees, Panels and Advisory Groups and for those who assist in the work of the Council. In fulfilling your role you should abide by the seven principles of public life that were drawn up by the Committee on Standards in Public Life (the Nolan Committee). More information on the Code of Practice here. 10.3 Freedom of information and data protection This section describes your obligations as a Peer Review College member in relation to confidentiality and information legislation. The following paragraphs provide the context and practical implications. 1. The Freedom of Information Act 2000 (FOI Act) seeks to ensure a culture of openness and accountability in the running of public bodies. Any information the AHRC holds could be requested under the FOI Act, and may need to be released subject to any disclosure exemptions stated in the information legislation. More information on the AHRC Freedom of Information Policy here. 2. The Research Councils must ensure that the peer review process encourages and supports the free and frank exchange of views between specialists, whilst maintaining the highest possible standards of openness and accountability. As such, the Councils use a joint framework (the Peer Review Framework) which outlines which peer review information is routinely disclosed and that which is routinely withheld. The Peer Review Framework can be found on the RCUK web site. Page 43

Chapter 10: Terms and conditions 3. If an individual is identifiable from information in your possession, and is the focus of that information, such information constitutes personal data and is subject to the rules of the Data Protection Act 1998 (DPA). If you hold personal data, such as that found in copies of applications, award decisions, mark lists, consolidated mark sheets or other relevant papers, you must treat it in accordance with the DPA. Personal data must be kept securely, only used for the purpose for which it was supplied, and disposed of securely once it is no longer needed. 4. Any information you hold (including notes you take) relating to AHRC matters could be the subject of a Freedom of Information request. You should be careful how you record your views, in case your views need to be disclosed. It is essential that Peer Review College members operate impartially, honestly and fairly. 5. The AHRC expects that panel members will shred applications, relevant papers and notes after the panel meeting, or leave them at the meeting for the AHRC to dispose of. Any personal data received by email must also be kept securely - copies must not be made, and electronic files, CDs, etc. must be destroyed as soon as possible and in line with the process for the destruction of paper equivalents. 6. If you carry out a peer review using the Je-S system, the first time you are asked to provide an electronic review for the AHRC a screen will show the Reviewer Protocols. These outline the standards the Research Councils require of reviewers in terms of confidentiality and conduct. You must agree to follow these protocols before being presented with any review material. You will be expected to confirm compliance with these protocols each year (not each time a new review is requested). 7. If you receive any requests for information relating to your work for the AHRC, please pass them, immediately, to the AHRC s Information Manager (FOI@ahrc.ac.uk). You are not expected to respond directly to the public, or to provide the public with any information. Section A: Confidentiality and information legislation Peer Review College members agree to treat all applications made to the AHRC confidentially. This duty of confidence covers: the fact that the applicant has applied; and the content of the application. Information about applications must only be disclosed to third parties when the AHRC or the applicant has given consent in writing. Page 44

Chapter 10: Terms and conditions Peer Review College members must not use information provided in a grant application for any purpose other than providing a review or assessment of it to the AHRC. Peer Review College members will destroy the application once a panel meeting has been conducted or a review has been provided (whichever is relevant). The AHRC will not normally disclose panel member comments (or notes) to the applicant, although in some cases it may be necessary. The AHRC will only use comments for carrying out panel meetings or as part of the funding decision process. Comments will only be disclosed to individuals outside of the AHRC if it is required as part of the funding decision process, or is required under the Data Protection Act of the Freedom of Information Act (or any other law or regulation to which the AHRC is or may become subject to). The AHRC will not release your name in connection with any specific comments that are released under the Data Protection Act or the Freedom of Information Act without first obtaining your permission. In such circumstances, the AHRC will ask the applicant to keep your comments confidential and not to release your name in connection with them without first obtaining your permission. However, the names of panel and Peer Review College members are made available on the AHRC s website. The panel s comments on and grading of, applications will be recorded by AHRC staff. The AHRC will not use these minutes or grades, or disclose them to any person or organisation, except: as is necessary to record the decisions of the panel to inform any other person or body within the AHRC, or any other body that may be cofunding the applications, as part of the funding decision process to send to the applicant as part of AHRC feedback - if first agreed by the panel; or as may be required under the Data Protection Act or the Freedom of Information Act (or any other law or regulation to which the AHRC is or may become subject). 10.4 Freedom of Information Peer Review Framework The RCUK Peer Review Framework describes how peer review is used in assessing proposals and making funding decisions. The framework also outlines what information is routinely published relating to proposals and awards, and the approach taken by the Councils in responding to requests for information about the assessment process. Page 45

Chapter 10: Terms and conditions The framework is designed for use by Applicants and Research Organisations, Board/Panel members and external reviewers, members of the public and Research Council staff and can be accessed here. 10.5 Equal opportunities The UK Research Councils are committed to eliminating unlawful discrimination and promoting equality of opportunity and good relations across and between the defined equalities groups in all of their relevant functions. Accordingly no eligible job applicant, funding applicant, employee or external stakeholder including members of the public should receive less favourable treatment on the grounds of: gender, marital status, sexual orientation, gender re-assignment, race, colour, nationality, ethnicity or national origins, religion or similar philosophical belief, spent criminal conviction, age or disability. Equally, all proposals must be assessed on equal terms, regardless of the sex, age and/or ethnicity of the applicant. Proposals must therefore be assessed and graded on their merits, in accordance with the criteria and the aims and objectives set for each scheme or call for funding. 10.6 Anti-Bribery Policy The Arts and Humanities Research Council is committed to the practice of responsible corporate behaviour and to complying with all laws, regulations and other requirements which govern the conduct of our operations. The AHRC is fully committed to instilling a strong anti-corruption culture and is fully committed to compliance with all anti-bribery and anti-corruption legislation including, but not limited to, the Bribery Act 2010 ( the Act ) and ensures that no bribes or other corrupt payments, inducements or similar are made, offered, sought or obtained by us or anyone working on our behalf. The full text of the Anti-Bribery Policy is available here: http://www.ahrc.ac.uk/about-us/policies,-standards,-and-forms/anti-bribery- Policy/Pages/Anti-Bribery-Policy.aspx Page 46

Chapter 11: UK SBS (Shared Business Services) 11. UK SBS (Shared Business Services) The AHRC works with the UK Shared Business Services (SBS) (opens in a new window) to deliver funding activities and uses the Joint Electronic Submission System (Je-S) for electronic grants services. If you have a query, please telephone or email as per the list below: Applications and proposals: Submitting applications and proposals, maintenance requests and general information Email: jeshelp@rcuk.ac.uk Telephone: 01793 444164 Peer review: About peer reviewing, current applications, meetings, and panels Email: enquiries@ahrc.ac.uk Telephone: 01793 416060 Post award: Maintenance, updates to existing awards, all other post-award queries Email: grantspostaward@ssc.rcuk.ac.uk Telephone: 01793 867121 Joint Electronic Submission System (Je-S): There is a dedicated Helpdesk that provides telephone and email support for the Je-S system. They are available between 9am and 5pm, Monday to Friday, and can be contacted by email at jeshelp@rcuk.ac.uk or by telephone on 01793 444164 Page 47

Annex A: Main stages of the review process Annex A: Main stages of the review process Note: This overview is not applicable to all schemes; in some cases aspects of the full process will not be required. Proposals received Peer group? Proposals checked by member of staff Peer Review College (PRC) Proposals reviewed by PRC (number of reviews varies depending on scheme) Proposals which have no prospect of being funded are made unsuccessful Quality sift of proposals by AHRC based on PRC reviews Proposals which pass the quality threshold. Applicant is sent the PRC reviews and invited to respond Panellists individually review, comment on, and assign grades to each proposal Proposals, reviews, and applicant response to PRC reviews are sent to panel Panel meetings: Grades and ranks are decided and feedback for appropriate proposals is agreed Director of Research: Makes the final funding decision Final allocations Successful Unsuccessful Council Page 48

Annex B: Cross-Research Council proposals Annex B: Cross-Research Council proposals The Research Councils' guidance on reviewing interdisciplinary and multidisciplinary proposals, including proposals that straddle Council boundaries, is provided on this page. If you are asked to review a Cross-Council proposal it is important that you read this guidance. 1. The Research Councils have an agreed approach for collaborating on the peer review and funding of research proposals that straddle their remits under their responsivemode research grants schemes which aims to ensure that no gaps develop between the Councils subject domains and to ensure equality of opportunity for proposals at the interface between traditional disciplines, where many major research challenges are located. Further details of the Cross-Council Funding Agreement can be found on the RCUK website. 2. The Research Councils already collaborate on programmes of research with defined remits that cross Council boundaries. However, peer review can often be more difficult for responsive mode proposals, which can be submitted in any research area. Responsive mode is taken to mean unsolicited research proposals received in any area relevant to the research councils remit. For some Research Councils this will include those received in response to highlight notices or priority areas that are processed as if they were unsolicited. 3. This document aims to provide you, as a reviewer, with guidance to help you consider multidisciplinary proposals. This guidance is equally relevant to proposals that cross discipline boundaries within a Council s remit as well as those crossing between remit boundaries. 4. For multidisciplinary proposals, it is unlikely that you will be familiar with all the elements of the programme of research. You may have been approached as a reviewer because of your particular expertise in one element and reviews will also be sought from experts in the remaining aspects. If you only feel confident commenting on particular elements of the proposal, please restrict your comments to these. This will greatly assist the panel in placing your comments in context. 5. You should assign the proposal a grade and respond to those elements on which you consider it appropriate for you to comment. You may additionally want to justify the grade you have given or indicate any reservations you have. Please use the Overall Assessment section of the on-line review form to ensure that the reasons for your grade Page 49

Annex B: Cross-Research Council proposals are clear; but please ensure that you DO NOT disclose the grade you awarded the proposal, as your comments are returned to the applicant. 6. Multidisciplinary research is often to be found at the cutting edge, which is inherently risky. You should not be afraid of recommending innovative, speculative and adventurous proposals. If you think something is risky, it is important to ask yourself what the risks are; is it risky because the outputs are unknown or does the project lack the subject specialism it needs? 7. It is possible that a standard technique or method is being used in a novel way or context. It is not appropriate to lower your grade to reflect this element if it underpins an otherwise exciting piece of research. 8. Do not be tempted to adjust your grade or score downward because you don t think that the research project fits fully within the administering Council s remit. Responsive funding can cross remit boundaries and where this is the case, a co-funding agreement will have been considered by the relevant councils. (Please note that co-funding is not considered appropriate for values of less than 150k). If we have forwarded this application to you, please respond on the assumption that we have accepted it as falling within our remit. 9. Multidisciplinary research may necessitate a researcher moving disciplines. While it is important that you are convinced that the appropriate logistical support is in place (including training where necessary), you should take care to review the project not the applicant(s). 10. Don t forget that your comments may be fed back, anonymously, to the Principal Investigator. Where you consider the research is flawed in approach or contains elements of poor quality, the reasons must be clear, unambiguous and evidence-based. Page 50

Annex C: Je-S system peer review: frequently-asked questions 1) What is the Je-S system? Je-S is the UK Research Councils on-line Joint Electronic Submission system. The system is used for completion and submission of research proposals to the Research Councils, including AHRC, and also for peer review of proposals. 2) Why is AHRC using the Je-S system for peer review? AHRC is committed to full participation in the Je-S system, in order to provide a common interface to all Councils for applicants, research organisations, and reviewers. AHRC provides for proposal submission, financial reporting, status reporting and peer review via the Je-S system. The advantages for both applicants and reviewers are: confidential proposals are more securely handled since the chosen reviewer receives these direct in their personal Je-S account there are no time delays due to postage, particularly for non-uk based reviewers the reviewer can view the full proposal, together with any associated documents before deciding whether they wish to undertake the review the system is used across the UK Research Councils, so reviewers don t have to deal with different systems, processes and forms when reviewing for more than one Council. 3) I ve never used Je-S before and don t have an account. How do I get one? Prior to you being required to do a review for the first time we will have added your details to the Je-S database in order to create an account. When you are notified of your first review request, the email you will be sent will include a link to activate your account. In order to do this, you will need to enter a username and password and some prompts and responses in case you forget your log-in details. When your account has been activated, you will first be presented with the Peer Review Protocols and you will have to Accept these in order to be able view AHRC proposals. To access the review forms and associated proposals select the Documents option and then the Peer Review option. 4) When I first tried to access the peer review documents I ve been sent, I got to a screen called Reviewer Protocols and couldn t get any further. What is this? Page 51

The Reviewer Protocols outline the Research Councils expectations regarding issues such as confidentiality of the material contained in the review documents and conflicts of interest. Reviewers must agree to abide by these protocols before being presented with any review material. Once you have indicated your agreement to these protocols, you will not be asked to do this for further reviews. You will, however, be asked to reaffirm your agreement once a year. AHRC also expects these protocols to be followed by panel members and in respect of any other activity College members may be engaged with on AHRC s behalf. 5) I d like to see the proposal before I decide whether to do the review or not. How can I do this? You may view the full proposal prior to making a decision about whether to complete the review. You will need to log in to Je-S, navigate to the Peer Review area, select Open and you will be presented with the Document Menu screen and you should select Documents to Review from the menu. 6) Can I get a printed copy of the review form, so that I can see all the questions I need to answer before I begin making any responses in the system? It is possible to print a blank copy of the review form. Open the review form and select the Document Actions button, then select Print Document. If you then choose include blank sections under the print options, you can create a printout of the form that includes all questions. We ask you to treat all printed documents securely and shred them once you have completed your review. 7) Do I need to answer all the questions on the form? You need to answer questions in all Edit sections of the form if you are completing the Review. If you are declining the review, please complete Decline to Review as soon as possible. There are maximum character limits (including spacing and punctuation) for each question but no minimum limits. If you are unsure whether you have missed a question or questions, choosing Validate Document at any time will list any questions still to be answered. Once you have completed the review, you must select the Submit Document option in order for AHRC to receive your response(s). 8) What information should I give in response to each question on the form? Are any guidance notes available? Page 52

Within Je-S there is context-sensitive Helptext containing AHRC s guidance notes for completing the form. If you select Help at the top-right corner of the screen, while in any section of the review form, the appropriate guidance will appear. It is important that you take account of the Scheme under which the proposal has been submitted, as there will be schemespecific guidance in many cases. The Scheme may be found in the banner at the top of each screen. 9) I m not sure what to do next. Who can I contact for help? If you need any assistance or advice in your use of the system or experience any problems, you should contact the Je-S Helpdesk. The Helpdesk can be contacted by telephone on 01793 444164 or by email on JeSHelp@rcuk.ac.uk from 8.30am to 5pm Monday Friday. 10) I m not sure that I m eligible to complete the review. Whom can I contact for advice? If you need advice on your eligibility to carry out the review or have any questions regarding information contained in the proposal or on the review process, you should contact the AHRC officer whose details appear in the Instructions to Reviewer section of the form. 11) I don t want to complete the review. What should I do? It would be very helpful if you inform us as soon as possible that you are unable or unwilling to complete the review, so that we may approach an alternative reviewer. You should do this by selecting Decline to Review under the Document Data menu in the Je-S review form. It is important, once you have provided the information requested in this section, that you select the Decline to Review button: this is necessary for your response to be forwarded to the AHRC. 12) I have elected to decline the review request and have been asked to suggest an alternative reviewer. Do I need to do this? This is optional; however, it would be very helpful if you could suggest an alternative. 13) I am happy to do the review and will submit it by the due date. Do I need to tell you this? Page 53

It would be helpful if you could give us an indication that you are intending to complete the review. You can do this by emailing the AHRC contact named in the Reviewer Information section of the form. Page 54