this agreement and the specific definitions may lead to misinterpretations.

Transcription

1 (VSNU) Scientific Research: Agreed Definitions June 2011 Introduction Disclaimer: The original agreement is written in Dutch. This document is merely a translation of the original document. Due to cultural differences the translation of this agreement and the specific definitions may lead to misinterpretations. Adjustments of this agreement will be done in the original Dutch version and then translated. This document lays down the scientific research definition agreements. These definitions are the result of close consultation with university participants, have been evaluated by means of a pilot and were approved by the General Council of the Association of Universities in the Netherlands (VSNU) on 11 June 2010 (AB VSNU meeting 207). The definition agreements relate to research input and output. The research input is determined on the basis of scientific staff under contract, broken down by HOOP discipline, job category and funding. Specific definitions have been made for this purpose. The agreements define four main types of research output, each with various forms of publications. The classification criterion used to determine to which main type the research output should be allocated is the target audience (see notes). This document contains specific definitions pertaining to each publication type. The definition agreements and their formats are designed to allow all scientific disciplines to submit relevant research output. Where specific research output is not relevant, the agreements determine that it shall not be submitted. Each year, the universities will provide data in accordance with these agreements. These data are end data (from 1-01 to 31-12) and always concern the year x-1. It has been agreed that universities are allowed to make corrections on the year (year x-2) preceding the last query. The universities will provide their data in the appropriate format. Once the data have been processed and analysed by the VSNU office, they will be returned for a verification of their accuracy. After such verification has been given, the university data will be incorporated into the national database. The national database is used in its totality for benchmarking between universities. Information for the ministry and other parties is provided via a separate publication reporting on (for the time being, exclusively) the distribution and main categories of scientific and professional publications. The definition agreements are in line with the information requested in the Standard Evaluation Protocol (SEP), elaborated to fit the present purpose. The use of data from the annual request for research input and output ensures that validated data are used that are furthermore comparable to data from other universities and disciplines. However, as the research input and output data are based on standard populations (input) and basic agreements on publications, research reviews will always need to be accompanied by explanations/observations on the quantitative data and, in some cases, additional domain-specific quantitative information as well. Key elements of the definition agreements 1 of 19

2 (VSNU) Scientific Research: Agreed Definitions June 2011 The definition agreements are based on the following core assumptions: 1. Research output is aimed at three distinct audiences: a. Researchers b. Interested professionals c. Broader public 2. The agreements cover only the bibliographically traceable part of the research output. Note: this does not include submitted but as yet unpublished publications. 3. The agreements apply to output in both print and non-print form (electronic publications, audiovisual media, computer programs). 4. Research publications are categorised into four main types based on purpose and audience: a. scientific publications, b. professional publications, c. popular publications, d. other results. 5. In the case of journal articles and conference papers in the main category of scientific publication a distinction is made based on whether the publication is evaluated by independent content experts (referee system). 6. Each main category distinguishes several different types of publications. In some cases, a publication type is classed under two or more main categories. When deciding to which main category a publication should be awarded, the description of the main category prevails. 7. The aggregation level at which the data is requested is the HOOP (Higher Education and Research Plan) categories 1. Not all main categories and/or publication forms are relevant for all HOOP categories. If the main categories and/or publication types are not applicable, the fields are left blank. 8. Publications are related to programmes and projects. 9. Publications are related to authors and organisational units. 10. Publications, programmes and projects are classed according to ISN and NABS. 1HOOP categories: Agriculture & the Natural Environment; Science ; Engineering & Technology; Health; Economics; Law; Behaviour & Society; Language & Culture; Education; Multisectoral 2 of 19

3 (VSNU) Scientific Research: Agreed Definitions June 2011 Re. 1 and 5: The four categories of Scientific publications, Professional publications, Popular publications and Other results Publications are classed as scientific publications, professional publications, popular publications or other results based on the target group. The definitions assume that classification cannot be based on the 'scientific character but only on the target group to which the publication is addressed. After all, the scientific character' is in principle the same as and independent of the target group at which the publication on the research results is aimed. Academic research generates sound, reliable knowledge that is made permanently available and accessible through publication. (See also Re. 3.) Quality criteria do not play any role in the breakdown into four categories. The reasons for this are as follows: (1) quality criteria cannot be applied at the time of the data entry; (2) quality criteria cannot be based on a formal classification, but only on considerations of content; (3) there is no reason to assume that research-oriented publications are necessarily of higher quality than publications aimed at other target groups; (4) it is impossible to capture all conceivable quality characteristics, such as size, innovation, relevance, status of the author, status of the publication medium etc., for all disciplines and all assessment situations in standard definition agreements. Re. 3: Accessibility Public accessibility is considered an important feature of research publications. In this context, publications that are not publicly available are a contradiction in terms. Publications are only public if they are traceable and permanently available in other words, bibliographically traceable. A television broadcast is public, but in principle is not traceable and available after the time of broadcast unless special steps are taken such as archiving the recordings. The same applies to electronic publications. In addition to considerations of 'availability' and traceability, accessibility must also provide for 'verifiability': where stored data is also used for quantitative analysis, there must be safeguards against incorrect input. The risk of incorrect input can immediately be reduced by attributing as little independent interest as possible to counts of publication totals. A count should never serve as anything more than a general indicator and in the case of careful analyses must always be accompanied by other information and safeguards for proper interpretation, such as rebuttal. Re. 6: Referee System The classification into refereed and non-refereed journal articles and conference papers within the main category of scientific publication uses an objectively recordable feature to allow meaningful categorisation or selection without implying differences in quality. A referee system means a publication is evaluated by independent content experts. Re. 8 and 9: Double counting Double counting has been a problem for years because it is often difficult to allocate publications with multiple authors or contributions by several organisational units to only one author or unit, as the guidelines dictate. Such allocation is not strictly necessary to avoid double counting. If two authors have contributed to an article, it is not double counting if the article appears in their both their lists of publications. In the list of publications of the faculty of both authors, that publication should occur only once, but it is not necessary for 3 of 19

4 (VSNU) Scientific Research: Agreed Definitions June 2011 the publication to be attributed to only one author! The same applies to contributions by two faculties to one publication: while the publication may only be listed once in the university overview, it need not be assigned to just one faculty. Where aggregation is applied, it is normal for the totals to drop. The definition agreements state that the registration of publications should be done in a way that enables the elimination of duplicates where required for a particular overview. A practical solution when making statements is to indicate when a title, author and publication medium are identical. When aggregating it is necessary to eliminate duplicates. Re. 10: ISN and NABS 2 Since it is sometimes necessary to organise data into content categories, it has been agreed that the ISN and NABS codes will continue to be used as they have been for several years now. These codes are assigned to Publications, Projects and Programmes. In some cases the ISN-NABS codes may derive from those used by the faculty. To ensure continuity with previous years, these classifications have not been replaced (or supplemented with) other content formats, such as those used by libraries or citation indexes. However, a correspondence table has been drawn up for ISN and NABS codes and HOOP disciplines, on the one hand, and the discipline format for VSNU research evaluations, on the other (see Annex). 2 ISN: International Standard Nomenclature, drawn up by UNESCO NABS: Nomenclature pour Analyse des Budgets Scientifiques, prepared by EUROSTAT (Statistical Office of the European Community) 4 of 19

5 Format for input of employed WP 3 HOOP field Per job category WP1 (FTE) 1. Professors 2. Senior University Lecturers 3. University Lecturers 4. Researchers WP2 (FTE) WP3 (FTE) Total WP 5. Doctoral candidates a. Doctoral candidates b. Contracted doctoral candidates Definitions: input Academic staff Definition Explanation Employed scientific staff All scientific staff who have an employment contract with the relevant university in line with the WOPI 4 definition of staff complement All staff that hold a permanent or temporary post at and/or have a permanent or temporary employment contract with the relevant university for a specified and recorded number of hours, in other words staff for whom the Executive Board of the university in question acts as an employer. The following categories must be excluded from the counts: volunteers; student interns; student assistants and two-year trainee research assistants; unpaid posts (therefore also zero-time appointments); staff who only receive (occasional) payments on a reimbursement basis under agreements for the performance of individual services, master or on-call contracts; see the Collective Labour Agreement of the Dutch Universities, annex to the CLA: 9, item 1); house officers and trainee specialists; 3 Scientific staff (wetenschappelijk personeel) 4 University Personel Information (Wetenschappelijk Onderwijs Personeelsinformatie) 5 of 19

6 Academic staff Definition Explanation staff who are employed by another competent authority and who have been seconded to the university (e.g. agency workers, staff of cleaning companies to which cleaning work has been outsourced, employees under the Sheltered Employment Act [Wet Sociale Werkvoorziening], in other words staff for whom the university itself does not act as an employer); former staff to whom the university still makes payments (e.g. redundancy pay, payments as a substitute for unemployment benefit etc.); professors occupying an endowed chair, appointed to this position by a foundation or legal entity other than the university, are only included in counts insofar as they also have an independent employment contract with the relevant university that relates to another post (e.g. as senior university lecturer) and then only in that capacity and not as a professor. 1. Professors (UFO 5 : 0105; and ) 2. Senior University Lecturers (UFO: 0106) 3. University Lecturers (UFO: 0107) 4. Researchers (UFO: 0108) No. of FTEs research input in tenths in the post of 'Professor' (UFO), regardless of the type of chair and type of teaching and research remit, in the reporting year (x). No. of FTEs of research input in tenths in the post of 'Senior University Lecturer' (UFO) in the reporting year (x). No. of FTEs of research input in tenths in the post of 'University Lecturer' (UFO) in the reporting year (x). No. of FTEs of research input in tenths in the post of 'Researcher' (UFO) in the reporting year (x). This refers to 'Professors' employed by the university. SEP: full professor (tenured staff) This refers to 'Senior University Lecturers' employed by the university. SEP: Associate Professor, also senior lecturer (UHD) or senior researcher (tenured staff) This refers to 'University Lecturers' employed by the university. SEP: Assistant Professor also lecturer (UD) or researcher (tenured staff) This refers to 'Researchers' employed by the university. Post-docs also fall within this category because they are classed as holding the post of researcher within the UFO and as having a temporary employment relationship. SEP: non-tenured staff 5 University Job Classification System (Universitair Functie Ordeningssysteem) 6 of 19

7 Academic staff Definition Explanation 5. a. Doctoral candidates (UFO: 0110) No. of FTEs of research input in tenths in the post of 'Doctoral Candidate' (UFO) in the reporting year (x). Doctoral candidates: employees who have an employment contract with the university or an University Medical centre with an agreement to pursue a doctorate. SEP: PhD students 5. b. Contracted doctoral candidates Note: not employed No. of FTEs of research input in tenths of Contracted Doctoral Candidates in the reporting year (x). Contracted doctoral candidates: persons who do not have an employment contract with the university under which they are supervised in their pursuit of a doctorate, but rather an agreement to carry out doctoral research or to pursue a doctorate as their primary task. These doctoral candidates do not have a UFO code (they are not employees), but are indeed relevant for some universities when determining input. If they are disregarded in terms of input, this produces strange figures in relation to the output. The input of these doctoral candidates must be presented separately as a sub-group within the main group of doctoral candidates. 7 of 19

8 Definitions: flow of funds Flow of funds Definition Explanation FTEs 1 st flow of funds (WP 1) No. of FTEs of research input in tenths in the reporting year (x) as funded by central government grants and tuition fees. The 1 st flow of funds refers to central government grants and tuition fees. The definition differs from SEP in this sense, but brings this in line with the practical situation. SEP: direct funding: funds provided directly by the higher authority for research and exploitation FTEs 2 nd flow of funds (WP 2) FTEs 3 rd flow of funds (WP 3) No. of FTEs of research input in tenths in the reporting year (x) as financed with funds that come directly from the KNAW 6 and NWO 7. No. of FTEs of research input in tenths in the reporting year (x) as financed with funds that come directly from financial backers other than in the 1 st and 2 nd flow of funds. ESF 8 falls under the 3 rd flow of funds. 6 Royal Netherlands Academy of Arts and Sciences (Koninklijke Nederlandse Akademie van Wetenschappen) 7 Netherlands Organisation for Scientific Research (Nederlandse Organisatie voor Wetenschappelijk Onderzoek) 8 European Science Foundation 8 of 19

9 Format for output HOOP field 1. Scientific publication HOOP field 3. Popular publication a. Refereed article in a journal a. Book b. Non-refereed article in a journal b. Volume/ Book chapter c. Book d. Volume c. Contribution to daily newspaper/weekly magazine or journal e. Dissertation 1 Dissertation 1 HOOP field 4. Other results 2 Dissertation 2 a. Patent f. Refereed convention contribution b. Book review g. Non-refereed convention contribution c. Book editing HOOP field 2. Professional publication a. Article in a journal b. Book c. Volume/ Book chapter d. Annotation d. Journal editing e. Inaugural lecture f. Abstract g. Design h. Television or radio appearance i. Miscellaneous e. Conference proceeding f. Protocol g. Report 9 of 19

10 Definitions: output Definition 1. Scientific publication A publication by (a member or members of) staff relating to the results of scientific research aimed at the forum of researchers, published in the reporting year (x). Explanation A focus on the target group of researchers means that a publication has specific features such as an emphasis on the research process, justification of methodology and reference to previous research results. These features can vary per discipline. In principle, a scientific publication relates to the results of the author s own scientific research designed to contribute towards increasing scientific knowledge. Exceptions to this rule include publications that relate to research performed by others but still clearly contribute towards increasing scientific knowledge (instead of solely disseminating this knowledge), such as a review article. a) Refereed article in a journal b) Non-refereed article in a journal An article that has been subject to a critical, independent review by one or more experts in the field, called referees, who are responsible for determining whether the topic of the article falls within the scope of the journal and who establish the originality, quality of the research, clarity of presentation etc. An article that has not been subject to a critical review as referred to under a. If it can be argued that the journal has an extensive editorial board ( editorial staff), with members who are considered experts in their field and who can independently express their views (who effectively function as referees), then an article can be regarded as refereed. If it is unclear whether an article is refereed in accordance with the above description, or if there are doubts as to the independence of the referee process, the article is classed as publication type b. c) Book This does not include book editing, which is classed under the main category of Other results. If the editor of a book has also written one or more chapters of the book him or herself, this publication is classed under d) volume/ book chapter. d) Volume/ Book chapter Part of a book (chapter). 10 of 19

11 Definition e) Dissertation Publication on the basis of which the title of Doctor has been conferred at the university in the reporting year (x). Explanation A distinction is made between two types of publication within this category: 1. the conferral of a doctorate at the candidate s own institution, internally prepared; 2. the conferral of a doctorate at the candidate s own institution, externally prepared. f) Refereed Conference proceedings g) Non-refereed convention contribution A full version of an article as part of a collection of scientific articles published in the context of a convention. The article has been subject to a critical, independent review by one or more experts in the field, called referees, who are responsible for determining whether the topic of the article falls within the scope of the journal and who establish the originality, quality of the research, clarity of presentation etc. A full version of an article as part of a collection of scientific articles published in the context of a convention. The article is not subject to a critical review as referred to under f. The previously used publication type 'conferral of a doctorate at another institution, internally prepared' has been dropped as these publications cannot be allocated to the university in question. If the above distinction cannot be made, the dissertations are aggregated at the level of dissertation. In Metis this category includes both bundled articles and proceedings. If it can be argued that the article has been reviewed by an extensive editorial board ( editorial staff), with members who are considered experts in their field and who can independently express their views (who effectively function as referees), then an article can be regarded as refereed. If it is unclear whether an article is refereed in accordance with the above description, or if there are doubts as to the independence of the referee process, the article is classed as publication type g. In Metis this category includes both bundled articles and proceedings. 11 of 19

12 Definition 2. Professional publication A publication by (a member or members of) staff relating to the results of scientific research aimed at professionally interested parties, published in the reporting year (x). a) Article in a journal Explanation The term professionally interested parties refers to those persons who are interested in the research results based on the practice of their profession (excluding the occupational group of scientific researchers). A focus on the target group of professionally interested parties means that a publication has specific features such as an emphasis on the use of the research results in the practice of certain professions, the use of professional jargon and the professional experience of the practitioners. Professional publications are designed not so much to increase, but instead to disseminate scientific knowledge and to promote the application of existing knowledge. Some examples of professional fields: medicine law agriculture chemical industry pharmacy accountancy management consultancy politics and policy education social welfare b) Book c) Volume/ Book chapter 12 of 19

13 Definition d) Annotation A legally explanatory comment on a decision handed down by a court or arbitral tribunal in the reporting year (x). e) Conference proceedings A full version of an article as part of a collection of scientific articles published in the context of a convention. The article is not subject to a critical review as referred to under 1f. Explanation According to a decision by the DOO Law, Annotations fall within the category of professional publications. In the report entitled Oordelen over rechten ( Judging Law Research ) issued by the Committee for Preparations for the Assessment of Law Research (Commissie Voorbereiding Onderzoeksbeoordeling Rechtsgeleerdheid), Oct. 2005, on p.34, paragraph 2, the first sentence states: Unless there is good reason to decide otherwise in exceptional cases, the committee therefore stipulates that annotations must generally be classed as professional publications. In Metis this publication type includes both bundled articles and proceedings. f) Protocol Instructions describing how to act in specific situations. g) Report Reporting on research almost always by commission. Reports are classed as professional publications as they are a product aimed at the target group of professionally interested parties. This does not include confidential reports, for reasons of traceability. 13 of 19

14 Definition 3. Popular publication A publication by (a member or members of) staff relating to the results of scientific research and aimed at a broad public, published in the reporting year (x). a) Book Explanation A publication that reports on the results of scientific research and targets educated laypeople. Explanation: A focus on the target group of educated laypeople means that a publication has specific features such as general, understandable language and no need for prior knowledge. Example: articles in daily newspapers and weekly magazines, encyclopaedia entries. b) Volume/ Book chapter c) Contribution to daily newspaper/weekly magazine or journal A submission or interview printed in an international, national or regional daily newspaper/weekly magazine or journal. Newspapers, magazines and journals. 14 of 19

15 4. Other results Definition Explanation a) Patents An exclusive right to make or sell an industrial product that is granted by the government or competent authority in the reporting year (x). b) Book review A book review is a description, critical analysis and evaluation of the quality, intention and significance of a book, focusing on its objective, content and authority. Based on a patent granted in accordance with SEP requirements. The body that has granted the patent is not relevant at this time. All existing patents will be grouped together until such time as clear patent registration procedures are in place and there is agreement regarding the desired indicator. Please do note that patents must appear in the registers with at least the following details: inventor(s) (=author), title, requested/granted, requesting organisation (under Remarks ), priority number and letter code (=patent number) and date. These details ensure traceability. Book reviews are classed under the main category of Other results regardless of whether they concern a scientific, professional or other type of publication. c) Book editing Editing of a book. Book editing is classed under the main category of Other results. If the editor of a book has also written one or more chapters of the book him or herself, this publication is classed under d) volume. This is because editing is an activity and not a type of publication. However, as it is one often named by scholars, the option to submit it has been included, albeit under this main category. d) Journal editing Editing of a journal. Journal editing is classed under the main category of Other results. This is because editing is an activity and not a type of publication. However, as it is one often named by scholars, the option to submit it has been included, albeit under this main category. e) Inaugural lecture A public speech made on the occasion when a newly appointed professor officially takes up his or her 15 of 19

16 Definition post. f) Abstract An abstract is a concise independent publication that describes a future or existing more extensive publication. g) Design A design is a description (projection or model) of the real thing as planned or produced. A design can comprise a textual description, a Computer Aided Design model, a drawing, a pattern, a scale model etc. h) Television or radio appearance An interview or contribution broadcast in an international, national or regional television or radio programme. i) Miscellaneous This type of publication is intended to permit the addition of publications that are deemed to be of real value and that cannot be classed under one of the other publication types. Explanation It is becoming increasingly common for researchers to write abstracts. This is due in part to the growth in online journals and online presentations of long articles, as well as to the rise in submissions for research grants, book proposals, volumes and conference contributions. This does not include abstracts that are included in articles and books (summaries). Designs are classed under the main category Other results as no clear definition exists as yet and because they are often not bibliographically traceable. Once there is more clarity, this type of publication can be classed under professional publications. Universities are expressly requested to specify which types of publications are being included when submitting them in order to make it possible to determine at a later stage whether further distinction is needed when requesting publication types. 16 of 19

17 Annex: Correspondence tables for HOOP, VSNU, ISN and NABS codes HOOP VSNU ISN NABS ECON BDK (bedrijfskunde) 53 (economic sciences) N08 (economie) (5311 organization and management of enter-prises) ECON ECO (economie en econometrie) 53 (economic sciences) N08 G&M BKP (bestuurskunde, politicologie, 59 (political sciences) N08 (gedrag- en communicatiewetenschappen) maatschappij) G&M LER (lerarenopleiding) 58 (pedagogy) N08 (5803 teacher training and employment) G&M POK (pedagogiek, onderwijskunde) 58 ( pedagogy) N08 G&M PSY (psychologie) 61 (psychology) N08 G&M RUI (ruimtelijke wetenschappen, w.o. geografie en milieukunde) (maatschappelijke structuren en relaties) 54 (geography) N01 (exploratie en exploitatie van het aardse milieu, N02 (infrastructuur en ruimtelijke ordening), N03 (milieubeheer en milieuzorg) 63 (sociology), N08 51 (cultural anthropology) G&M SCW (sociologie, culturele antropologie) GEZO GNK (geneeskunde) 32 (medical sciences) N04 (bescherming en bevordering van de (gezondheid) menselijke gezondheid) LAND DGZ 24 (life sciences), N06 (landbouwproductie en -technologie) (landbouw) 31 (agricultural sciences) NATU (maar zie ook TECH) (diergeneeskunde, zoötechniek (plantenteelt en biotechnologie vallen onder HOOP:NATU en VSNU:BIO!) AAR (aardwetenschappen) 25 (earth and space sciences) N01 (exploratie en exploitatie van het aardse milieu) NATU BIO (biologie, plantenteelt, biotechnologie met primair biologische vraagstelling) 24 (life sciences), 31 (agricultural sciences) N06 17 of 19

18 Annex: Correspondence tables for HOOP, VSNU, ISN and NABS codes HOOP VSNU ISN NABS NATU FAR (farmacie) 32 (medical sciences) (3208/3209 pharmacodynamics, pharmacology) N04 NATU NSK (natuurkunde en sterrenkunde) 21 (physics), N10 (maar zie ook 21 (astronomy, astrophysics) TECH) NATU (maar zie ook TECH) NATU (maar zie ook TECH) RECH (recht) T&C (taal en cultuur) SCH WIN (scheikunde, chemische technologie, biotechnologie (primair procestechnologische vraagstelling)) ((ook technische) wiskunde en (ook technische) informatica) 23 (chemistry), 33(03) (chemical technology and engineering) 12 (mathematics), (3307 electronic technology, 3304 computer technology) RGL (rechtsgeleerdheid) 56 (juridical sciences and law) N08 HIS (geschiedenis, archeologie, preen protohistorie) N07 N07 55 (history) N10 T&C GGL (godgeleerdheid) 73 (theology and religion) N10 T&C LET (taal-en 57 (linguistics), N10 literatuurwetenschappen, kunstgeschiedenis, muziekwetenschap) 62 (sciences of arts and letters) T&C WSB (wijsbegeerte) 11 (logic), N10 71 (ethics), 72 (philosophy) TECH AAR (aardwetenschappen) 25 (earth and space sciences) N01 (zie ook NATU) TECH BWK (bouwkunde) 62 (6201 architecture, komt bij 33, technological sciences, niet voor) N10, N07 (bescherming en bevordering van de menselijke gezondheid) (niet in te delen onderzoek), N09 (exploratie en exploitatie van de ruimte) (industriële productie en technologie) 18 of 19

19 Annex: Correspondence tables for HOOP, VSNU, ISN and NABS codes HOOP VSNU ISN NABS TECH CTG (civiele techniek, geodesie) 25 (earth and space sciences) (2504 geodesy), 33 (3305 construction technology inc!. civil engineering) N07 TECH ELE (elektrotechniek) 33 (technological sciences) N07 (3306 electrical technology, 3307 electronic technology) TECH INO (industrieel ontwerpen) 33 N07 (industriële productie en technologie) TECH LRT (lucht-en ruimtevaarttechniek) 33 (3301 aeronautical technology, 3324 space technology) N09 (exploratie en exploitatie van de ruimte) TECH NSK (technische natuurkunde) 33 N07 (zie ook NATU) TECH (zie ook NATU) SCH (technische scheikunde) 33 N07 TECH WIN (technische wiskunde, technische 33 N07, N10 (zie ook NATU) informatica) TECH WTB (werktuigbouwkunde) 33 N07 19 of 19