Practical Seminar Research Evaluation & Assessing Research Quality Indicators Impact Integrity 15 th 16 th September 2016, Berlin With Experts from: Brunel University London, Planning and Policy Department, United Kingdom Delft University of Technology, Strategic Development Office, the Netherlands University of Bologna, Research and Technology Area Division (ARIC), Italy Health Research Board, Post-Award Management and Evaluation, Ireland European Commission Joint Research Center, Corporate Planning, Reporting and Evaluation, Policy Support Coordination University of Leicester, Research Services Manager, Library, United Kingdom Aalto University, Leadership Support Services, Finland Brochure 1
PRACTICAL SEMINAR Research Evaluation & Assessing Research Quality Indicators Impact Integrity Benchmarking Effectively comparing research As research results are becoming increasingly important for international rankings and external funding, it is crucial to find a way to compare research. How can research results be compared across different subject areas? Where to find the right counterparts to benchmark the institution s research performance? Which tools are available to assess research quality? Furthermore, aspects such as research integrity and ethics in research complement the wide spectrum of quality in research and represent further challenges in the daily work of staff responsible. Evaluating research is challenging Assessing the quality of research comprises many aspects that need to be taken into consideration. Quite a large number of research outcomes can be measured numerically. However, this also leaves room for debate: are number of publications, citations or doctoral students sufficient indicators for research assessment? Which other criteria are useful to measure research? The most difficult question remains how to demonstrate the actual impact of research on society. From an institutional point of view it is therefore essential to establish a quality management system that can fulfil external demands. Furthermore, it is necessary to fully involve all stakeholders concerned. Take the chance to assess the topic of quality management in research from institutional organisation to benchmarking research results at our Practical Seminar. Exchange and network with peers and our experts and benefit from lessons learned and shared experiences. Who is this seminar for? Directors, Heads of Departments and Units, Managers, Officers and other members of staff responsible for Research evaluation Research and transfer support Research governance Monitoring of research results Research and scientific coordination Research integrity Quality management in research From the following departments: Research and Development Innovation and Valorisation Rector s Office Vice Rector for Research, Quality, Development, Strategy Quality Unit/Office/Department Strategy and Development Planning and Controlling Evaluation and Monitoring Internal Audit Dean s Offices on Faculty/College/ School Level University Library In: Higher Education Institutes (HEIs) Universities Research Institutes University Hospitals Thought-provoking impulse on the assessment of research quality. Interesting exchange and valuable insights on ensuring research integrity. 2
What will you learn at this practical seminar? How to effectively operate a research evaluation routine in your institution How to address issues of research misconduct in your organisation? What needs to be considered to address demands for research integrity? How can responsibilities and priorities best be communicated to stakeholders involved? Which indicators are most useful to report on research? What needs to be considered when dealing with peer review in research evaluation? What are the benefits of this method? What are specific challenges for measuring research results and outputs? In which way can research evaluation be integrated into a wider reporting system? How can available data and statistics be used best to monitor results? What needs to be considered in regards to data management and open access? In which way can benchmarking be used to compare research and to assess research quality? How to assess research impact on society and how can those results be made visible In which way can available data be used to plan for future investments? What are the key quantitative and qualitative methodologies in research evaluation and impact assessment? Your benefits Improve your research evaluation skills and ensure the performance of your institution Support your institution s development and competitiveness Learn how to best cope with new data management challenges Learn how to assess and monitor research outputs most effectively and how to best make use of them Exchange and network with colleagues from all over Europe Receive best practice insights from experienced speakers Erasmus Staff Mobility Staff Training (STT) Did you know that university staff can get financial support for travel and accommodation costs for vocational training through the Erasmus Staff Mobility Staff Training programme? See here for details. Very informative, good discussions and a nice environment for a difficult topic. 3
PROGRAMME DAY 1 Research Evaluation & Assessing Research Quality Indicators Impact Integrity 8.30-9.00 Registration and Handout of Seminar Material 9.00-9.05 Opening Remarks from the European Academy for Taxes, Economics & Law 9.05-9.30 Welcome Note from the Chair and Round of Introductions Dr René Haverslag, Policy Advisor, Strategic Development Office, Delft University of Technology, the Netherlands 9.30-10.15 Research Evaluation and QM for Research at Delft University of Technology Pros and cons of the various aggregation levels for research assessment Utilising a full range of assessment methods: Self-assessments, international benchmarking and peer review Aligning the QM s purpose and goals with those of the institution (or vice-versa) Responsibilities and commitment of actors, motivating the actors QM-routine/evaluation and monitoring cycle Challenges and possibilities for assessing quality of research Accountability and follow up of assessments Dr René Haverslag, Policy Advisor, Strategic Development Office, Delft University of Technology, the Netherlands 10.15-10.30 10.30-11.00 Coffee Break and Networking Opportunity 11.00-11.45 Successful Internal Communication on Evaluations with all Stakeholders Lessons learned from the Introduction of a New Quality Assurance System How to motivate university stakeholders to participate in audits and evaluations a participative approach Stakeholders and their roles Communication challenges with different stakeholders Means of communication How to communicate and utilise evaluation results Dr Luciana Sacchetti, Quality and International Academic Rankings Officer, Research and Technology Area Division (ARIC), University of Bologna, Italy 12.00-12.45 Benchmarking: How to Compare Research Outputs Overview of bibliometric and related methods for research evaluation Analysing performance and setting benchmarks The need for clarity around counting conventions and units of analysis Data normalisation for comparative bibliometric analysis Understanding the indicators: Context is crucial Mini-case study: The h-index Mini-case study: Journal impact factors Dr Ian Rowlands, Research Services Manager & Bibliometrican, Library, University of Leicester, United Kingdom 12.45-13.00 13.00-14.15 Lunch Break and Networking Opportunity 14.15-15.00 Research Indicators: Intellectual and Data Management Challenges A rich and evolving landscape of publication and research indicators Indicators as reputation systems Managing information about information effectively Indicators and the continuing importance of peer review Mini-case studies: Practical applications of bibliometrics at the University of Leicester Complex data, simple messages for busy people: The need for strong communication skills Dr Ian Rowlands, Research Services Manager & Bibliometrican, Library, University of Leicester, United Kingdom 15.00-15.15 15.15-15.45 Coffee Break and Networking Opportunity 11.45-12.00 4
Interactive Session 15.45-17.30 Control and Measurement of Research Results Using Key Performance Indicators In this interactive workshop participants improve their skills in working with key performance indicators. They work in groups on different scenarios to assess where to use which indicators. Pro s and Con s: What do we actually want to measure? What can be measured? Measures and indicators to drive behaviour How to set effective and useful indicators Which metrics to use (conventional versus alternative metrics)? How to make the best use of indicators Should indicators be published? Interesting exchange and valuable insights on ensuring research integrity. And their achievements? How to use indicators to demonstrate research outputs Excursion: How to measure results when indicators are not applicable? The different groups then present their results. In case of any questions our expert is ready to give advice. Dr Rosa Scoble, Director Planning, Planning and Policy Department, Brunel University London, United Kingdom 17.30 End of Day One 17.30 Get-Together and Networking Opportunity All seminar participants are cordially invited to an informal evening get-together. 5
PROGRAMME DAY 2 Research Evaluation & Assessing Research Quality Indicators Impact Integrity 9.00-9.15 Welcome Note from the Chair Dr Maura Hiney, Head of Post-Award Management and Evaluation, Health Research Board, Ireland Interactive Session 9.15-11.15 Addressing Research Integrity Understanding the issues and implementing local governance of research integrity. - What is research integrity? - Research misconduct, research integrity s evil twin, explored - A look at the data on incidence of misconduct - Some of the myths about research misconduct - Seven reasons why research integrity is so important - Some recent survey data on research integrity processes from around Europe - Case study on research misconduct Facilitated group discussion to include: - What process for managing an allegation of misconduct do you currently have in your institution? - What evidence do you need to collect and how would you preserve the chain of evidence? - What sanctions are proportionate and available to you legally? Feedback and summary Dr Maura Hiney, Head of Post-Award Management and Evaluation, Health Research Board, Ireland 11.15-11.45 Coffee Break and Networking Opportunity Case Study 11.45-12.30 Assessing the Impact of the Scientific Policy Support Process Dimensions and time scales of policy impact Multi-criteria appraisal systems JRC s annual internal evaluation Evaluation and knowledge management Evaluation as part of a wider reporting system Evaluation in the context of the strategic programming and planning cycle of the Commission 12.30-12.45 12.45-14.00 Lunch Break and Networking Opportunity Case Study 14.00-14.45 Research Evaluation: The Perspective of a Health Research Funding Agency Why are funding agencies interested in evaluation data? What can we measure when we measure health research? What are the tools of evaluation for a funding agency? What does evaluation data really tell us as a funding agency? How can we use outcomes and impacts data to plan for future investment? Dr Maura Hiney, Head of Post-Award Management and Evaluation, Health Research Board, Ireland 14.45-15.00 15.00-15.30 Coffee Break and Networking Opportunity 15.30-16.15 Assessing Research Impact Challenges and Opportunities What do we mean by research impact? Measuring impact Examples of impact through systematic assessment What challenges and opportunities does the need for quantification pose? Dr Antti Saaristo, Development Manager, Leadership Support Services, Aalto University, Finland 16.15-16.30 16.30 End of Seminar and Handout of Certificates 6 Dr Vera Calenbuhr, Planning and Programming Officer, Corporate Planning, Reporting and Evaluation, Policy Support Coordination, European Commission Joint Research Center
SPEAKER Research Evaluation & Assessing Research Quality Dr René Haverslag Policy Advisor, Strategic Development Office, Delft University of Technology, the Netherlands Dr René Haverslag is a Policy Advisor in the corporate office for Strategic Development at Delft University of Technology. In this function, he is responsible for the central coordination of research assessments and the follow up of the outcomes. His further responsibilities include gaining a better understanding of societal impact via indicators and in general improving the availability and use of indicators and other types of (management)information at TU Delft. Before joining TU Delft, Dr René Haverslag contributed to the development of the Dutch Standard Evaluation Protocol for research assessments as a Policy Advisor at the Association of Universities in the Netherlands (VSNU). He started his career in strategic development of HEIs as an advisor for the Graduate School of Life Sciences at Utrecht University. Dr Luciana Sacchetti Quality and International Academic Rankings Officer, Research and Technology Area Division (ARIC), University of Bologna, Italy Dr Luciana Sacchetti is a Quality and International Academic Rankings Officer in the Research and Technology Transfer Division (ARIC) at the University of Bologna. Her main field of activity is to give support to the thirty-three research departments in implementing a quality management system focused on research processes. In addition, she is also tasked with planning and implementing proactive actions to improve the institutional positioning of the University of Bologna in both international and national university rankings and league tables. Dr Luciana Sacchetti is qualified as auditor for both ISO 9001 and the Common Assessment Framework (CAF). Dr Vera Calenbuhr Planning and Programming Officer, Corporate Planning, Reporting and Evaluation, Policy Support Coordination, European Commission Joint Research Center Dr Vera Calenbuhr currently coordinates the corporate planning, reporting and evaluation activities at the JRC s Directorate for Policy Support Coordination in Brussels. Prior to this, she managed the European Science and Technology Observatory (ESTO) as well as various projects in relation to environmental policy, sustainable development and impact assessment at the Joint Research Centre s Institute for Prospective Technological Studies (JRC-IPTS) in Seville. Vera Calenbuhr also teaches at the University of Basel on European Environmental Policy as well as on the Economic Dimension of Sustainable development. Vera Calenbuhr has a Diploma in Chemistry from the University of Cologne and a PhD in Natural Sciences from the Free University of Brussels. Her academic career concentrated on modelling and simulations in complex systems theory and self-organisation. Dr Rosa Scoble Director Planning, Planning and Policy Department, Brunel University London, United Kingdom Dr Rosa Scoble is currently Director of Planning, Planning and Policy Department at Brunel University. The department is responsible for supporting the university research strategy, the management and review of the academic planning cycle (including student number planning), management of risk, setting and monitoring of KPIs and benchmarking. The office also provides analysis of sector policies and business intelligence. She was responsible for the submission to REF2014 and to RAE2008. Rosa Scoble has an extensive knowledge of research information management and is involved in a number of Jisc funded projects. From 2002 to 2007 she has lead a team at Brunel that compiled the annual Guardian University league tables. 7
SPEAKER Research Evaluation & Assessing Research Quality Dr Maura Hiney Head of Post-Award Management and Evaluation, Health Research Board, Ireland Dr Maura Hiney is Head of Post-Award Management and Evaluation at the Health Research Board Ireland. The unit supports the management and monitoring of the HRB s portfolio of active awards and evaluation of HRB-funded research, in particular the post-hoc evaluation of research awards, and works with the CEO and board to identify evaluation data that will best allow planning for future investment. The unit also provides a policy development function and contributes to national and European strategic and policy initiatives. From 2000 to 2007 Maura Hiney was Head of Research Support Services at NUI Galway, which included development of research funding opportunities, researchrelevant policies and a research outputs database for the university. Maura Hiney has a particular interest in issues of research integrity (RI) in health research and published a review of the topic in 2015. She chairs the Science Europe Working Group on RI, is a member of the ALLEA Permanent Working Group on Science and Ethics, which is currently revising the European Code of Conduct for Research, the European Advisory Committee for the 5th Worlds Congress on Research Integrity and the Advisory Board for the EU PRINTEGER Project. She has acted as an external expert to the European Commission on Research Integrity and Open Science and participated in the preparation of the Council of Ministers Conclusions on Research Integrity in December 2015. Dr Ian Rowlands Research Services Manager & Bibliometrican, Library, University of Leicester, United Kingdom Dr Ian Rowlands works at the University of Leicester, a research-intensive institution in the English Midlands. He and his team are responsible for developing specialised library and information services to support the needs of research staff and students: including bibliometrics services, open access to research outputs and data, and training in research evaluation. The team manages an open access repository (Leicester Research Archive), and administers open access payments on behalf of the seven UK Research Councils and the major UK biomedical charities. Current open access projects include the digitisation of PhD theses back to 1924, and the launch of a new open access journals platform for local content. The team also provides bespoke bibliometrics services to the University, supporting grant capture, recruitment and selection of senior staff, and supporting preparations for the national Research Excellence Framework exercise. Ian Rowlands holds a PhD in Information Science and spent eighteen years teaching and researching in information science and journal publishing at City University and University College London before moving to Leicester in 2012. Dr Antti Saaristo Development Manager, Leadership Support Services, Aalto University, Finland Dr Antti Saaristo is Development Manager at Aalto University s Leadership Support Services (LSS). LSS supports the top university management in strategic and quality management, operating environment analysis, public affairs and international partnerships, and provides legal services for administration. Antti Saaristo was the project manager of Aalto University s international Research Assessment Exercise in 2009. Since then, the research assessment model created at Aalto University has been adopted by a number of Finnish universities. After the 2009 exercise, Antti Saaristo has participated in the construction of the research quality management and evaluation system of Aalto University, including the planning of the university s next large research assessment exercise, scheduled to 2018. Prior to joining Aalto University, Antti Saaristo was a visiting scholar at UC Berkeley and a postdoctoral researcher at the University of Helsinki. 8
ORGANISATIONAL MATTERS Research Evaluation & Assessing Research Quality Date of Event 15 th 16 th September 2016 Booking Number S-1347 Event Language The event language will be English. Event Price Early booking price valid until the 1 st of July 2016: 1.189,- Euro excl. German VAT (19%) Normal price from the 2 nd of July 2016: 1.389,- Euro excl. German VAT (19%) The above price covers the following: Admission to the seminar Hand-out documents Seminar certificate, if seminar fully attended Soft drinks and coffee / tea on both event days Lunch on both event days Upon request you can receive a digital version of the seminar documents after the event for 60,- Euro excl. German VAT (19%) in addition to the seminar. BOOKING E-mail: booking@euroacad.eu Phone: +49 (0)30 80 20 80 230 Fax: +49 (0)30 80 20 80 250 For online booking please visit our website: www.euroacad.eu Contact European Academy for Taxes, Economics & Law at Potsdamer Platz, Entrance Leipziger Platz 9, 10117 Berlin, Germany Phone: +49 (0)30 80 20 80 230 Fax: +49 (0)30 80 20 80 250 E-mail: info@euroacad.eu Internet: www.euroacad.eu Your contact persons for the programme: Regina Lüning, M. Sc. econ. Head of Marketing and Sales Phone: +49 (0)30 80 20 80 246 Fax: +49 (0)30 80 20 80 259 E-mail: regina.luening@euroacad.eu Paul Hafellner Conference Manager Phone: +49 (0) 30 80 20 80 2 1324 Fax: +49 (0) 30 80 20 80 2 2250 E-mail: paul.hafellner@euroacad.eu (Programme is subject to alterations) Event Location Adina Apartment Hotel Berlin Checkpoint Charlie Krausenstraße 35-36 10117 Berlin, Germany Phone: +49 (0)30 200 767 555 Fax: +49 (0)30 200 767 599 E-mail: aber@adina.eu Please contact the hotel directly and refer to the European Academy for Taxes, Economics & Law if you wish to benefit from a limited number of available rooms. Of course you can always look for an alternative hotel accommodation. 9
BOOKING E-mail: booking@euroacad.eu Phone: +49 (0)30 802080-20 Fax: +49 (0)30 802080-22 250 For online booking please visit our website: www.euroacad.eu Booking Booking Number: S-1347 (DM) 15 th 16 th September 2016, Berlin Herewith we register the following persons for the Practical Seminar: Research Evaluation & Assessing Research Quality. Delegate 1 First name Last name Your organisation Department Unit Job position Tel. Fax E-Mail I hereby order the digital version of the seminar documents (fee required) in addition to the seminar. Street Postcode / City Country Delegate 2 First name Last name Your organisation Department Unit Job position Phone Fax E-Mail I hereby order the digital version of the seminar documents (fee required) in addition to the seminar. Street Postcode / City Country Invoice Organisation First name Last name Your Organisation Department Unit Street Postcode / City Country Phone Fax Job Position E-mail With my signature I confirm my registration and accept the General Terms and Conditions as legally binding. I herewith agree to receive further information from the European Academy for Taxes, Economics & Law In case of registration of more than one delegate do you prefer: Single invoice? Collective invoice? NOTE Only Valid with Signature and Stamp. 10 Place, Date Authorised Signature and Stamp European Academy for Taxes, Economics & Law Brauner Klingenberg GmbH Leipziger Platz 9 10117 Berlin Phone +49 (0)30 80 20 80 20 Fax +49 (0)30 80 20 80 22 259
Terms & Conditions for Conferences, Seminars and other Training Courses 1. Area of Application The following terms and conditions settle the contractual relationship between conference participants and the European Academy for Taxes, Economics & Law Brauner Klingenberg GmbH [referred to as European Academy for Taxes, Economics & Law in the following]. Differing terms and conditions, as well as, other settlements and/or regulations have no validity. 2. Registration / Confirmation of Application A registration can be made via internet, mail, fax, or email. The registration is considered granted and legally binding if not rejected by the European Academy for Taxes, Economics & Law in writing within seven (7) days after receipt of registration. The registration will be supplemented by a booking confirmation via email. Partial bookings are only valid for seminars designed in modules. 3. Service The course fee covers the fee per participant and course in net, subject to current German VAT. It includes training course documents as per course description, a lunch meal/ snack and refreshments during breaks, as well as, a participation certificate. The European Academy for Taxes, Economics & Law has the right to change speakers/instructors and to modify the course program if and where necessary while maintaining the overall nature of the course. All registered participants will be notified in case of a course cancellation due to force majeure, due to speakers preventions, due to troubles at the chosen location or due to a low registration rate. Course cancellation notification due to a low registration rate is issued no later than two (2) weeks before the course date. Course fees are reimbursed in the cases listed above; however, reimbursement for travel expenses or work absenteeism is only granted in cases of intention or gross negligence by the European Academy for Taxes, Economics & Law. Any reimbursement of travel expenses are to be considered as an exceptional goodwill gesture and form no future general obligation. In case of disturbances and/or interruptions, the European Academy for Taxes, Economics & Law commits itself to solve or limit any problems that might occur in order to maintain and continue the course as planned. 4. Payment Date and Payment, Default of Payment Payment of the course fee is payable immediately upon receipt of invoice. Where payment is not received or lacking clear assignment to a participant prior to commencement of the course, the European Academy for Taxes, Economics & Law may refuse the relevant participant s participation in that course. The course fee, however, is still due immediately and can be claimed as part of a dunning procedure or legal action. In accordance with BGB 247 (1), in case of default of payment within the stipulated time period, default interest on arrears of at least 5% above the ECB base rate is due and payable. The European Academy for Taxes, Economics & Law can claim higher damage for delay if and where proven. Equally, the participant may prove that a damage has not occurred or has had less effect than estimated by the European Academy for Taxes, Economics & Law. Payment shall be made by cashless bank transfer; cash or cheques will not be accepted. The European Academy for Taxes, Economics & Law is not liable for any loss of means of payment. The participant may only offset such claims against the European Academy for Taxes, Economics & Law s as are undisputed, legally recognized or recognized in writing by the European Academy for Taxes, Economics & Law. The right of retention is only acceptable in accordance with a counterclaim based on the same contract. 5. Cancellation Cancellations need to be issued in writing. Cancellation by the participant will be subject to cancellation charges as follows: 30 days or more prior to commencement of the course: service charge of 80,00 net, subject to current German VAT, payable immediately, course fee will be reimbursed, two (2) weeks to 30 days prior to commencement of the course: 50% of course fee net, subject to current German VAT, payable immediately, non-attendance or cancellation less than two (2) weeks prior to commencement of the course: 100% of course fee net, subject to current Germany VAT, payable immediately The European Academy for Taxes, Economics & Law gladly accepts without additional costs a substitute participant nominated in case of a cancellation if the substitute participant is registered at least three (3) days prior to the commencement of the course. Neither cancellation of a specific module/part of the course or substitution per module/per day is possible. 6. Copyright Seminar/course documents are protected by property rights and may not be duplicated, processed, amended, circulated or published in any other way without the written consent of the European Academy for Taxes, Economics & Law. The European Academy for Taxes, Economics & Law reserves all rights. 7. Liability All seminars and courses are prepared and presented by qualified speakers and instructors. The European Academy for Taxes, Economics & Law accepts no liability for the upto-dateness, correctness and completeness of the seminar documentation, as well as, presentation of the seminar. 8. Applicable Law, Place of Jurisdiction, Place of Performance All cases shall be governed and construed in accordance with German law to the exclusion of the UN Sales Convention. As far as legally admissible, place of performance and place of exclusive jurisdiction shall be Berlin, Germany. 9. Data Protection The European Academy for Taxes, Economics & Law protects personal data by taking appropriate protection measures. For the purpose of optimization of the product and service portfolio and according to the regulations of the data privacy laws, it stores and processes person-specific data on the training participants. Hence, all European Academy for Taxes, Economics & Law website hits are registered. All personal data will, in accordance with the law, be used for documentation requests, placed orders or other enquiries in order to send information out by post. The European Academy for Taxes, Economics & Law will, in accordance with the law, inform participants by email about special offers that resemble previously booked seminars. If and where personal data needs to be transferred to countries lacking appropriate data protection schemes, the European Academy for Taxes, Economics & Law shall grant alternative adequate protection. Furthermore, the European Academy for Taxes, Economics & Law will use personal data as far as participants have granted respective permission. When collecting personal data, the European Academy for Taxes, Economics & Law will always ask for permission regarding email information about offers. The participant may, at any time, express their objection to data collection for the purpose of advertisement or address via email or fax. Any data provided to the European Academy for Taxes, Economics & Law will be processed for reservations and bookings, as well as, for information about other seminars. Names and company names will be published in a participants list and forwarded to the mailing company. 11