PhD Student Evaluation

F A C U L T Y O F H E A L T H A N D M E D I C A L S C I E N C E S U N I V E R S I T Y O F C O P E N H A G E N PhD Student Evaluation Evaluation of the PhD Training Programme at the Graduate School of Health and Medical Sciences PhD Candidates July-December 2012 Niels Hoffmann Haahr Pia Lassen Evaluation Unit, Student Services Faculty of Health and Medical Sciences March 2013

Contents INTRODUCTION... 3 RESPONDENTS... 3 METHOD... 3 DATA PRESENTATION... 4 1. PHD ENROLMENT... 5 THE PROJECT AND ITS DESCRIPTION... 5 ACADEMIC VALUE OF PERIODIC EVALUATIONS (HALF-YEAR REPORTS)... 7 2. SUPERVISION... 8 OVERALL SATISFACTION WITH SUPERVISION... 8 NUMBER OF SUPERVISORS... 9 WHO IS CONSIDERED THE PRIMARY SUPERVISOR?... 10 SATISFACTION WITH THE PRIMARY SUPERVISOR... 10 SATISFACTION WITH CO-SUPERVISORS... 11 PHD SUPERVISION WOULD BE BETTER IF... 11 RECOMMENDATIONS FROM RESPONDENTS ON HOW TO IMPROVE SUPERVISION... 12 3. STUDY ENVIRONMENT... 13 SATISFACTION WITH ACADEMIC AND SOCIAL ENVIRONMENT... 13 THE PHD COORDINATORS... 14 4. PHD COURSES... 15 SATISFACTION WITH GENERIC AND ACADEMIC COURSES... 15 PARTICIPATION IN PHD COURSES ABROAD... 16 5. PARTICIPATION IN OTHER SCIENTIFIC ENVIRONMENTS... 17 6. TEACHING AND DISSEMINATION... 18 TEACHING ACTIVITIES... 18 DISSEMINATION ACTIVITIES... 19 7. GRADUATE PROGRAMMES... 22 SATISFACTION WITH THE ACTIVITIES OF THE GRADUATE PROGRAMME... 23 8. FRAMEWORK FOR THE PHD TRAINING PROGRAMME... 24 PHYSICAL ENVIRONMENT AND TECHNICAL ASSISTANCE... 24 CONTACT TO THE DEPARTMENT... 25 CONTACT TO THE PHD ADMINISTRATION... 25 SUMMARY AND RECOMMENDATIONS... 27 RESULTS AND RECOMMENDATIONS TO THE GRADUATE SCHOOL... 28 2

Introduction This is the third report presenting data submitted by PhD students to evaluate their PhD studies at the Graduate School of Health and Medical Sciences, University of Copenhagen. The objective of the evaluation is to provide qualified input to the Head of the Graduate School and the heads of Graduate Programmes on the PhD students experiences to ensure that informed decisions can be made when adjustments and changes are needed. The evaluation targets central elements of the PhD studies described in the PhD plan. Emphasis is on elements that are essential to the quality assurance of the programmes. The report presents data from July-December 2012. It will be published on the Graduate School's web: http://healthsciences.ku.dk/phd/current/schoolstructure/evaluation/ Respondents In the period from 1 July to 31 December 2012, a total of 115 students submitted their PhD thesis at the Faculty of Health and Medical Sciences. All of these received the questionnaire shortly after handing in their thesis 1. Up to and including 2012, the questionnaire was sent to PhD students enrolled at the former Faculty of Health Sciences only. Therefore, the results herein do not cover students from the previous Faculty of Pharmaceutical Sciences and Faculty of LIFE Sciences. As from January 2013, the evaluation questionnaire is sent to all PhD students from the units which have now merged into the Faculty of Health and Medical Sciences 2, once they have handed in their thesis for assessment. Number of students the questionnaire was sent to 115 Undeliverable mails 5 = potential respondents 110 Questionnaires completed 84 Response rate 76.4% The 76.4% response rate is satisfactory, and we consider the results representative for this group of respondents. Method The evaluation was designed as a web-based survey consisting of closed-ended questions supplemented by a number of open-ended questions to afford students the opportunity to elaborate on their answers. The Evaluation Unit at the Faculty 3 handled the continuous submission of invitations to complete the questionnaire along with the subsequent collection, analysis and dissemination of data. The Graduate School of Health and Medical Sciences has defined the parameters used in the evaluation, and the 1 Students who submitted a PhD thesis without previous enrolment as a PhD students (in pursuance of the PhD Order) are not included in this evaluation, since they have not participated in the same activities as standard PhD students. 2 In future, the survey will cover students from the former Faculty of Pharmaceutical Sciences, the former Faculty of Health Sciences and the veterinary departments at the former Faculty of LIFE Sciences. 3 The Evaluation Unit at the Faculty of Health and Medical Sciences assists with the development of quality assurance measures for the programmes of the Faculty. These activities are performed in connection with the running evaluation of the programmes, courses and major evaluations of the study and training programmes. Furthermore, the Evaluation Unit provides sparring for the Dean s office, heads of study, study boards and heads of department to follow-up on results and with a view to developing the quality of training and study programmes. 3

questionnaire has been designed and revised as a joint effort of the Graduate School and the Evaluation Unit. The questionnaire was revised and amended twice. The more recent revision process came into effect in the questionnaires sent out in the present collection period (July-December 2012). From previously submitted answers we identified the answers that needed a clearer wording or additional explanatory comments or which overlapped with other questions and were therefore superfluous. The revision of the questionnaire has resolved the overwhelming majority of the previously observed issues with imprecise interpretation of the questions included in the two previous reports. Specifically, the questionnaire is sent at the beginning of every month to all PhD students who have handed in their thesis during the previous month. The Evaluation Unit draws lists of relevant respondents from the University of Copenhagen s database. The Graduate School administration is responsible for updating the database with relevant information (PhD students names, private email addresses, date for thesis submission, and affiliations to departments and graduate programmes). As from July 2012, information on respondents' department and graduate programme affiliation has formed part of the questionnaire. This information was included in the survey at the request of the Graduate School and graduate programmes to meet the need to analyse and present results according to these variables. The background is an intention to employ results in specific quality assurance efforts. However, to safeguard the anonymity of respondents, it was agreed that no results will be published if they are based on the answers of less than five respondents. Data presentation In this report, quantitative data are mainly presented as percentages of the total number of respondents who answered the item in question. Further, and where relevant, data are divided into the categories of clinical and pre-clinical PhD students. This is a potentially interesting division as these categories of PhD students enjoy different terms of employment. It therefore seems reasonable to assume that some of the issues and trends studied may vary according to these categories Additional segmentation to single out the results from individual departments or graduate programmes was not done in this report due to the limited number of respondents, i.e. partly to avoid reporting results of limited significance, partly to safeguard the respondents' anonymity. In order to ensure user friendliness and obtain reliable data, the survey contains conditional logic as well as a number of questions that are not mandatory. The response rate therefore varies, and the response rate is clearly indicated (n=x) by each question or illustration. The qualitative data (i.e. the students comments) are used to exemplify and elaborate on the quantitative results. In the cases in which qualitative statements make reference to specific persons or places, these were not included in the report. We hope you enjoy reading the report! The Evaluation Unit, Student Services Faculty of Health and Medical Sciences, University of Copenhagen Niels Hoffmann Haahr Pia Lassen 4

1. PhD enrolment To enrol at a PhD training programme, students must have a project, a supervisor and sufficient funding. This section presents enrolment data related to the Graduate School; specifically: The degree to which the PhD students were enrolled on the basis of a pre-defined project announced as a scholarship, or rather participated actively in the development of the project and in the efforts to achieve funding for the project. The project and its description Half of the respondents' (49%) PhD projects were not defined in advance in a call for applications for a PhD grant. Figure 1.1: Was your PhD project defined in advance in a call for applications for PhD grants? Furthermore, a total of 6% answered Don't know, while the remaining 45% stated that their PhD project had, indeed, been defined in advance. N=84 The 84 completed questionnaires were equally distributed between clinical PhD students (51%) and pre-clinical PhD students (49%). Figure 1.2: Was your PhD project defined in advance in a call for applications for PhD grants? Subgroups clinical and pre-clinical. The figure to the right presents the distribution of answers to the above question by clinical (purple) vs. preclinical (green) respondents. As shown, there is no significant difference between the answers of the clinical and pre-clinical PhD students ( Don't know answers are not presented in the figure and therefore N = 79). N=79 Furthermore, it is interesting for the Graduate School to acquire knowledge about the degree to which PhD level research builds directly on research carried out by the student as part of his/her pre-graduate training, and the degree to which the PhD student was involved in the wording of the project either in cooperation with the primary supervisor or others. These data are presented on the next page. 5

For 12.3%, their PhD project was a continuation of research carried out at the pre-graduate level ( to a very high extent or to a high extent ). Figure 1.3: To which extent was you PhD project a continuation of research that you began on your undergraduate programme? 71.6% replied that their PhD project was not at all or only to a low extent a continuation of research initiated as an undergraduate student. N=81 The figure below presents the answers to three questions concerning the project and the project description. Figure 1.4: Assessment of influence on project description and discussion of the project description. The first question invited respondents to assess their influence on the project description: 55% answered to a high extent or to a very high extent. 12% answered that this was only the case to a low extent. The project description was discussed to a high extent or to a very high extent with the main supervisor in 74% of the cases. Only 3.7% stated that they had at no point discussed their project description with their main supervisor. 67% experienced that they had the opportunity to a high extent or to a very high extent to continually engage in discussions about their project with others in addition to the main supervisor and project supervisors. Furthermore, a total of 13.4% answered not at all or to a low extent to the question on the opportunity to continually discuss the project. N=82 6

Academic value of periodic evaluations (half-year reports) The half-yearly report which is currently mandatory for all PhD students at the Faculty of Health and Medical Sciences may serve as an important project management tool for the PhD students. The objective of the half-year reports is to assess if their project is making good progress on the basis of a dialogue between the student, the main supervisor and any co-supervisors. On the basis of the half-year reports, the student and supervisor(s) prepare a statement to the Graduate School on the current status of the PhD project. Half-year reports are thus a tool which potentially has considerable impact on the quality assurance of the PhD training, if used actively to achieve such an end. However, results show that the academic value of half-year reports was assessed to be limited. Thus, two of every three students (67%) answered that the value of half-year reports was limited og non-existent. Only 11% stated that the academic value of halfyear reports was high or very high. Figure 1.5: Please rate the academic value of the periodic evaluations (half-year reports) N=82 The two below comments are examples which characterize the general view on half-year reports: I do not think the half-year evaluation had any value for me. I have also always wondered if anyone actually reads them. I do acknowledge that there is a point to the supervisors signing a statement that everything is going well. "Half-year evaluations fail to provide real insight into the state of the PhD project. What would be the consequence if the supervisor were to state that the project is not progressing satisfactorily? Why can the student not state that progress is unsatisfactory, and what would the consequence be, if he/she did?" 7

2. Supervision Apart from being a training programme, a PhD programme is also a form of apprenticeship. The supervision received by the individual PhD student and the working relation with supervisor(s) is therefore of great importance. The students perception of the supervision they receive is a parameter which the Graduate School is required to report to the Research Training Council 4 and to the Management Team 5 at the University of Copenhagen. Overall satisfaction with supervision 82.9% were overall very satisfied or satisfied with the supervision they received, whereas 1 out of 10 were neutral. Figure 2.1: All in all, how satisfied were you with the supervision you received during your PhD? Overall, 6.1% were dissatisfied with the supervision ( dissatisfied or very dissatisfied ). N=82 Comparison of the replies of clinical and pre-clinical candidates reveals that there was no real difference between the distributions of replies between the two groups (the don t know reply is not included in the illustration, thus n=81). Figure2.2: All in all, how satisfied were you with the supervision you received during your PhD? Subgroups: Clinical and pre-clinical. N=81 4 The Research Training Council is a forum for exchange of experience among all Graduate Schools at the University of Copenhagen. 5 The Management Team consists of Rector, the University Director and the Deans of all faculties at the University of Copenhagen. The Management Team is responsible for overall strategic decisions affecting more than one faculty. 8

Number of supervisors PhD students at the Faculty of Health and Medical Sciences are required to have a main supervisor, preferably also a project supervisor, and may additionally have one or more co-supervisors affiliated to their PhD training programme. The main supervisor carries the primary responsibility for the PhD student s study. The main supervisor is an assisting professor or a professor at a faculty department. Half of the PhD students had three or more supervisors, whereas 30% had two supervisors. Figure 2.3: How many supervisors did you have, including your primary supervisor? 21% only had their primary supervisor attached to the PhD project. N=82 The majority of students who replied that they had only one supervisor were pre-clinical candidates. Figure 2.4: How many supervisors did you have, including your primary supervisor? Subgroups: Clinical and pre-clinical. Conversely, among the students who had three or more supervisors, the majority (63%) were clinical candidates. N=82 9

Who is considered the primary supervisor? Almost one out of four PhD students perceived the project supervisor to be the most important supervisor. For the majority of students (68.3 %), the faculty supervisor was considered the more important supervisor on the PhD project. Figure 2.5: Who was the primary/most important supervisor for you during your PhD project? N=82 Satisfaction with the primary supervisor Figure 2.6: Assessment of supervision received from primary supervisor, various parameters: N=82 Between 68% and 79% were satisfied or very satisfied with the supervisor whom they considered their primary supervisor (illustrated by the blue and red bars above). Conversely, between 5% and 13% were dissatisfied or very dissatisfied. The highest degree of dissatisfaction (13.4%) was associated with guidance on preparing the dissertation. 10

Satisfaction with co-supervisors As stated above, 69% of the PhD students had a co-supervisor. These respondents were asked about their overall level of satisfaction with co-supervision. Less than 5% indicated that they were very dissatisfied or dissatisfied with co-supervision. Figure 2.7: All in all, how satisfied were you with your co-supervisor(s)? Subgroup: Students with co-supervisors A total of 76.9% of the students were very satisfied or satisfied. PhD supervision would be better if In the questionnaire the PhD students are encouraged to reflect on the statement My supervision would have been better if. A total of 32 persons made comments. As part of the preparations for the October 2012 evaluation report 6, the PhD students' statements were analysed and divided into a total of four main themes. No new themes were identified in the current data material. This confirms the relevance and importance of the previously identified themes (each theme is illustrated through one or a few recent respondent quotes): 1) A need for more project management and other forms of follow-up/assessment during PhD studies "In order to minimize the risk of misunderstandings and unmet expectations, it s good to agree in advance on the contents etc. of supervision, e.g. how often you will meet, what should be prepared, and what the expectations are". 2) A need for good cooperation with a group of supervisors (My supervision would have been better if) the supervisors had more time and commitment to the project, and the supervisors were more like a research group and more interested in the project. My supervision would have been much better if the faculty supervisor had been more involved, since the project supervisor was not that much into my work. N=65 6 Student evaluation of the PhD Training Programme at Copenhagen Graduate School of Health and Medical Sciences. January- June 2012. Prepared by the Evaluation Unit, Education and Student Services, Faculty of Health and Medical Sciences. This report is available together with the present report at the Graduate School website: http://healthsciences.ku.dk/phd/current/schoolstructure/evaluation/ 11

3) The supervisor should contribute more to the training course and try to further the student s future career rather than just focus on research-related issues If they (the supervisors) share with me their opinion about my work, and give feedback more often, so I can improve my skills and I could be more efficient. 4) A wish to limit dependence on the supervisor (My supervision would have been better) if I got more freedom in making scientific decisions and plans in the period after I was halfway through my PhD. (My supervision would have been better) if the course had been more structured (...) and if an external supervisor had been available to ensure feedback from a person who wasn't mixed up in the process and therefore had the perspective to stop hopeless attempts at an earlier point." In the interest of fairness, it should be mentioned that this question specifically addressed suggestions to improvements relative to the supervision provided. Correspondingly, it deserves mention that, as described previously, the overwhelming majority of PhD students were satisfied with the supervision they received. The answers from the respondents of this and the previous surveys may be summarised in a series of recommendations from PhD students on how to improve the overall supervision quality. Recommendations from respondents on how to improve supervision A minimum of two supervisors in total should be assigned to each PhD student The group of supervisors should be clear about the division of responsibilities between them (who is responsible for what in relation to the PhD student and the PhD project?). Supervisors should receive supervisor training The supervisors' obligations towards their PhD students should be made transparent Primary supervisors should not be responsible for too many PhD students at the same time Mutual expectations between supervisors and PhD students should be clarified (e.g. by drafting a written agreement) Supervision sessions should be more systematic and held regularly Meetings should be held with all supervisors to discuss the project once or twice a year. 12

3. Study environment Establishing some form of safety net with a view to guaranteeing an expedient PhD study environment has been a cornerstone in the establishment of the Graduate School at the Faculty of Health and Medical Sciences. With approx. 1,500 enrolled PhD students, it is not possible for the Graduate School s Head and the PhD administration to uphold a personal relation with the individual PhD student. Consequently, efforts have been made to set up a structure that ensures a solid framework for researcher training, including possibilities for the PhD students to create and uphold academic and social networks. This section focuses on the PhD student s daily research environment and on the study environment that the Faculty seeks to maintain via the departments. Nine out of ten shared an office with other PhD students or other scientific staff. Figure 3.1: Describe you scientific workplace (tick one box) 10% had their own office. N=80 Satisfaction with academic and social environment Figure 3.2: Degree of satisfaction with academic and social environments N=81 79% expressed satisfaction with the academic environment. With regard to the social environment at the workplace, 83% answered that they were satisfied or very satisfied. 13

The PhD coordinators The PhD coordinators are primarily scientific staff who serve as spokespersons for the PhD students and ambassadors to the Graduate School. Coordinators may therefore serve as a link between the PhD student, the main supervisor, the department head and the Graduate School. One of the tasks of the coordinators is to contribute to a good working and social environment for PhD students at every department. First, the PhD students were asked if they had contact with the PhD coordinator at their department. Figure 3.3: Have you had contact with the PhD coordinator at your department? The majority (59.3%) had not had contact with the PhD coordinator. One in five (21.0%, n=17) replied yes to the question. N=81 The considerable share of students who answer don t know or no to the question demonstrate that knowledge of the coordinator scheme is not widespread among the PhD students who handed in their thesis in the final semester of 2012. The qualitative statements confirm this, as some respondents clearly mistake PhD coordinators for employees of the Phd Administration (see Section 8). Consequently, some of the positive answers are probably erroneous, as some respondents may have been commenting on PhD administrators when answering. The degree of satisfaction recorded with coordinators should therefore be interpreted with caution. The overwhelming majority of the PhD students who answered the question (n=18) stated being satisfied or very satisfied with their contact to coordinators. Figure 3.4: How satisfied were you with your contact with the PhD coordinator? N=18 14

4. PhD courses During their three years of studies, PhD students are required to participate in course activities corresponding to a total of approximately 30 ECTS. The PhD student can choose to participate in a range of specialized (academic) courses, and they can participate in various generic courses such as statistics, data analysis, project management and academic writing. The Graduate School runs a wide range of courses every year and students may also participate in other courses held in Denmark and abroad if these courses serve to earn them credits under the PhD plan. Satisfaction with generic and academic courses The overall level of satisfaction with PhD courses offered was similar for generic and academic PhD courses. Approximately 70% were very satisfied or satisfied. About one in ten was dissatisfied or very dissatisfied with the courses offered. Figure 4.1: Degree of satisfaction with generic courses. N=81 Figure 4.2: Degree of satisfaction with academic courses. N=81 15

Respondents were given the opportunity to offer additional comments regarding PhD courses. A total of 13 persons did so. Many of them described the courses in which they had participated in Denmark and abroad. However, one respondent stated that there was a need for more non-clinical PhD courses: I mainly attended methodological courses - social epidemiology, scale validation, etc. As to the academic courses, it would have been nice with more courses about psychosocial issues and courses (not clinically oriented) on the period of child and adolescence and their health and behaviour. Another respondent also called for more PhD courses, but failed to specify which: I had a previous MSc in clinical epidemiology, so the generic courses were not relevant. In general I think the University of Copenhagen offers too few academic [PhD] courses The only comments provided on any specific course concerned the course "basic statistics" - a 13-week statistics course which is "a must" for many PhD students at the Faculty of Health and Medical Sciences. One respondent stated that it was too difficult to ensure a seat on the course. Another PhD student was unsatisfied with the course, and noted that it seemed too advanced for many of the participants' needs. As a consequence, these students had not achieved the needed understanding of statistics despite the duration of the course: I only had one 'bad' course, but overall I was extremely satisfied with the academic contents and set-up. However, the 13-week statistic course was not useful at all, and I think all PhD students felt the same way. All PhD students fear statistics, but using statisticians as teachers was not ideal; the majority of teachers could not communicate very well; it was too advanced. ( ) the head teacher was good, though. Many of my PhD colleagues had to buy additional statistic courses in order to get a basic understanding. Participation in PhD courses abroad Half of all PhD students in this survey (49.4%) attended one or more PhD courses abroad. 16

5. Participation in other scientific environments All PhD students are expected to visit another scientific environment (than the one at which they are enrolled) for a period of time. This is stated in the PhD plan and is also a statutory demand in pursuance of the PhD Order. To the question: As part of your PhD, did you stay in another scientific environment (in Denmark or abroad)?, only slightly less than half (46%) answered affirmatively. Figure 5.1: As part of your PhD, did you stay in another scientific environment (in Denmark or abroad)? The questionnaire defines Stay in another scientific environment as a period of at least one week s duration, meaning that conferences, courses, etc., do not count as such stays. N=81 Among the clinical PhD students, 42.9% responded that they had stayed in another scientific environment in Denmark or abroad. Among pre-clinical PhD students, the corresponding share was 48.7%. Figure 5.2: As part of your PhD, did you stay in another scientific environment (in Denmark or abroad)? Subgroups: Clinical and pre-clinical N=81 The PhD students who had, indeed, stayed in another scientific environment were asked to specify the duration of such stay. Figure 5.3: How long was your total stay in another scientific environment? 40.5% answered 1-2 weeks. However, nearly every third PhD student (29.7%) had stayed in another scientific environment for more than six months. N=37 17

6. Teaching and dissemination As part of the PhD study, the PhD student shall learn how to disseminate knowledge orally as well as in writing. Dissemination may be in the form of publication of research results in periodicals or presentations/posters at conferences or symposia. Additionally, PhD students are encouraged to accept teaching activities as part of their PhD study. Teaching activities Respondents were asked if they had engaged in teaching activities during their PhD studies. Figure 6.1: Did you teach during your PhD programme? Two out of every three (67%) replied that they had engaged in teaching activities. Among the clinical PhD students, 64% answered Yes, and the corresponding share among pre-clinical PhD students was 69%. N=81 The students who had been teaching as part of their PhD were then asked to estimate how much time they spent on teaching activities. Figure 6.2: How much of your PhD education did you spend on teaching activities (please estimate)? About half (55.3%) had spent 1-5% of their time teaching, while 36.2% had spent 6-10%. None of the respondents estimated that they had spent more than 20% of their time teaching. N=47 In the final part of the section with questions on teaching, the PhD students who had engaged in teaching activities were invited to assess their satisfaction with the planning and supervision of the teaching activities, and the relevance and outcome of such activities. 18

Figure 6.3: Degree of satisfaction with the relevance and benefit of participation in teaching activities. N=54 The highest degree of satisfaction was observed for relevance of the teaching activities and the student's own academic benefit from the teaching activities. To these questions, 64% and 67% respectively were satisfied ( satisfied or very satisfied ). Furthermore, 63% were satisfied or very satisfied with their degree of influence on the planning of the training activities. The lower satisfaction scores were observed in relation to the supervision/feedback the PhD students had experienced in connection with the teaching activities. Here, 23% were dissatisfied or very dissatisfied. A large group (42%) remained neutral on this issue, while 23% stated being satisfied or very satisfied with the supervision/feedback provided. Dissemination activities As shown in Figure 6.4 on the following page, the respondents participated in several types of dissemination activities during their PhD studies. The purple column shows the share of clinical PhD students who stated that they disseminated their research in the manner specified. Correspondingly, the green columns show the same shares for preclinical PhD students. The options of "Conference/congress (oral/poster) and "Scientific communication (oral/written)" were selected by the highest share of students. In both cases, the dissemination forms were used more frequently by clinical than by pre-clinical PhD students. Conversely, the pre-clinical PhD students were more likely to disseminate their project results at seminars held in their own scientific environments (87.7% versus 74.4% in the group of clinical PhD students). 19

Figure 6.4: Where and how have you communicated the results of your project? Subgroups: Clinical and pre-clinical N=81 Publication of articles was important to the overwhelming majority of students (98.8%). Only one out of 81 answered no to the question: Was publishing or submitting articles to peer review important to you while you were on the programme? The amount of time spent on dissemination activities varied considerably among the PhD students. Figure 6.5: How much of your PhD programme have you spent on dissemination activities? 10% spent more than half of their time on dissemination. At the other end of the scale, 22% of the PhD students spent only 1-10% of their time on these activities. N=80 20

In general, the PhD students were satisfied with the dissemination activities in which they were involved during their PhD studies. As shown below, 92.6% were satisfied or very satisfied with the academic outcome of their dissemination activities. In terms of relevance of the dissemination activities, the equivalent level of satisfaction was 90.1%. When invited to state their satisfaction with the supervision/feedback you received on your dissemination activities, the degree of satisfaction was slightly lower, as 77.8% stated being satisfied or very satisfied, whereas a large group answered neutral (21%). Generally, only very few students were dissatisfied with their participation in dissemination activities. To each of the four questions, 0-2.5% stated that they were dissatisfied or very dissatisfied. Figure 6.6: Degree of satisfaction with various aspects of dissemination activities. N=81 One student elaborated on his/her answer concerning dissemination activities as follows: I have attended many conferences/seminars. At my institution, you can only attend a conference if you're presenting your work. Therefore, I always presented something and that has added up to about 10-15 presentations, which is VERY useful and it has made me able to communicate (and understand) layman s view of my work. I believe it is an indispensable part of being a researcher that you can communicate your research, and it only comes through practicing. 21

7. Graduate programmes The graduate programmes are responsible for offering very specific research courses which are primarily relevant to the PhD students affiliated with the individual programme. Furthermore, the objective is to organize activities which help students to strengthen their networks as these activities are relevant to everyone participating in the programme, whether they are PhD students, postdocs, assistant researchers or more senior researchers. Several of the graduate schools are cross-institutional and have formed alliances and networks with other similar programmes in Denmark and abroad. A list of graduate programmes can be found at the Graduate School website: http://healthsciences.ku.dk/phd/ In the present survey, 63% of the respondents indicated that they were affiliated with a graduate programme, as shown in this illustration. This number is expected to increase given the fact that, as from 1 January 2011, all PhD students at the Faculty of Health and Medical Sciences have been required to participate in a graduate programme as from their enrolment. Previously, participation was voluntary, although encouraged. Figure 7.1: Were you affiliated with a graduate programme during your PhD? N=81 This illustration shows that a majority of the students who answered that they did not know whether they were affiliated with a graduate programme (the bar to the far right) were from a clinical department. However, the numbers are small and should be interpreted with caution. Figure 7.2: Were you affiliated with a graduate programme during your PhD? Subgroups: Clinical and pre-clinical. N=81 22

Satisfaction with the activities of the graduate programme The respondents who were affiliated with a graduate programme were subsequently asked questions regarding their level of satisfaction with the graduate programme (see illustration below). Approximately half the PhD students were satisfied ( satisfied or very satisfied ) with the courses as well as the seminars and annual meetings arranged in the context of the graduate programme. The satisfaction level was somewhat lower when assessing graduate programme activities in terms of their value to build and maintain academic (34.6% satisfied) and social (30.8% satisfied) networks. It is worth noting that across the questions, a relatively high number of respondents answered don t know when assessing graduate programme activities. This probably reflects that they have not participated in the graduate programme activities, or that they have not identified certain courses and/or activities as graduate programme activities. Figure 7.3: Degree of satisfaction with the activities of the graduate programme. N=52 In the final part of the graduate programme evaluation, the PhD students were asked if affiliation to a programme had been important to their project, their future career and in general. The answers (summarized below) generally show that a relatively large share of the PhD students (40-47%) found that graduate programmes are of limited importance in this respect. Figure 7.4: How important was affiliation with a graduate programme to you? N=52 23

8. Framework for the PhD training programme The objective of the PhD training programme is for the PhD student to execute a research project under supervision while participating in courses and dissemination activities. The framework surrounding the PhD training programme, e.g. the physical facilities, IT facilities and administrative support may, however, be of considerable importance to the students' perception of their studies and their ability to focus on their research project. Physical environment and technical assistance The students were asked to rate their satisfaction with the following five parameters: Figure 8.1: Assessment of the physical context and IT facilities. N=81 As shown in the figure above, the degree of satisfaction ( satisfied or very satisfied ) was approx. 70-75% with regard to workspace and offices, access to technical assistance and access to resources and financing. Satisfaction with access to experimental facilities was 67% (however, as many as 12% answered don't know to this question, possibly because experimental facilities were irrelevant to their project). The lowest satisfaction score in the section about physical environment, etc., concerned the IT facilities made available to students. To this question, a total of 65% answered that they were satisfied or very satisfied, whereas 16% were dissatisfied or very dissatisfied. With regard to satisfaction with the IT facilities, the below comments made by two PhD students show that the conditions offered to students may vary considerable from one workplace to the next: During the transition to SUND IT, I experienced a lot of problems. Both with e-mails and programs. A lot of time went into solving these problems - some remain unsolved. I have not been impressed with the helpfulness. When you work at Rigshospitalet, there is an excellent IT department, so there was always someone to help you with IT things. 24

Contact to the department Respondents were asked to assess their contact to the Faculty department at which they were affiliated. This contact mainly concerns issues regarding employment and economic conditions as well as the physical context of their research project. As not all PhD students have had contact with their department, they were initially asked if they had had any such contact during the course of their PhD. Only 57% students replied that they had been in contact with the department or department administration. Among pre-clinical students, 61.5% had contact with the department, whereas 52.4% of the clinical students had departmental contact. The students who had contact to the department were then asked to assess their satisfaction, as illustrated here. Figure 8.2: How do you rate your contact with the department/department administration? 68% were satisfied or very satisfied with their contact to the department. 14% were dissatisfied ( dissatisfied or very dissatisfied ). N=50 Contact to the PhD administration In the final part of the survey, respondents were asked to assess their contact to the PhD administration. This contact mainly concerns any study administrative matters associated with the PhD studies, including enrolment, course activities and submission of theses. 88% of the respondents answered that they had, indeed, had contact with the PhD administration during their PhD studies (i.e. the Head of the Graduate School and/or the staff in the PhD administration). These respondents (N=71) were invited to rate their satisfaction with their contact with the PhD administration. See Figure 8.3 below. 25

Figure 8.3: Assessment of contact to the PhD administration. N=71 Typically, the individual PhD student will not have met the head of the Graduate School unless significant problems needed to be addressed during their studies, e.g. conflicts involving the main supervisor and the student. In line herewith 63% answer don t know/not relevant to this question. The number of students who selected one of the remaining options of the question is so low that is not reasonable to draw conclusions on the question adressing contact with the Head of the Graduate School. 50.7% stated that the contact with their contact person at the PhD administration was good or satisfactory ; 32.4%, however, rated such contact as being less satisfactory or not satisfactory. Much the same trend was seen for contact with the PhD administration in general. In the elaborative comments, 17 respondents noted some of the issues observed in their contact with the PhD administration. Several respondents commented that they had noted improvements in the service and communication during the final part of their PhD studies. The recurring trends in the respondents' comments were as follows (in order of priority): 1) Very long response times for e-mail requests made to the PhD Administration. In some cases, a response was never sent. One PhD student stated the following: Early in my enrolment the PhD office seemed to function poorly, and I experienced long waits on response to emails. It functions much better after the changes that have been made. 2) Varying service levels depending on the person handling your query. Several respondents noted improvements after the administration started team-work initiatives. One respondent summarized the two above points as follows: It was very hard to get hold of my contact person. Much better now with teams. Response time has been extremely long in some cases. Lately better. One PhD student stated the following: Very person-dependent: Some persons' [PhD administrators ] answers were rapid and relevant. Others simply did not answer, or only got back to you after weeks of latency. A few times information/mails were not forwarded because the "person who had received it was on sick-leave and the remaining staff were not allowed to open "personal mail"! All in all: A very instable quality which created a hightened level of uncertainty. I sometimes felt that I needed to insist by calling and sending reminders repeatedly. 3) Several of the respondents called for a receipt/confirmation mail when the administration had received their half-year evaluation, for example. 4) The webpage was characterized as inefficient by some students. Other comments, however, state that considerable improvements have also been seen in this regard. All qualitative comments were sent to the Head of the PhD administration. 26

Summary and recommendations This report presents the results of the Graduate School's final evaluation of the July-December 2012 period. It is based on the answers to a questionnaire which was sent to all PhD students from the Faculty of Health and Medical Sciences who handed in their thesis for assessment during the period. Up to and including the 2012 calendar year, the questionnaire was sent to PhD students enrolled at the former Faculty of Health Sciences only. The results herein therefore do not cover students from the previous Faculty of Pharmaceutical Sciences and Faculty of LIFE Sciences. As from January 2013, the evaluation questionnaire is sent to all PhD students from the units, which have now been merged into Faculty of Health and Medical Sciences 7, once they have handed in their thesis for assessment. The questionnaire was sent by mail and contained a link which respondents followed to fill in their answers electronically. The survey achieved a response rate of 76.4% (84 of 110 potential respondents answered the questionnaire). This is the third half-year evaluation report summarizing the Phd students' perception of their PhD studies. The report was prepared by the Evaluation Unit, which also collected and analysed the data. The Evaluation Unit is in continuous contact with the Graduate School to optimize the survey's design and revise the questionnaire, etc. Furthermore, the Evaluation Unit remains at the disposal of the Graduate School including the PhD Study Board for a more detailed presentation and discussion of any evaluation results and recommendations. The evaluation reports are processed by the PhD Study Board which, in pursuance of the PhD Order and the University Act, makes any statements on evaluation of PhD training and supervision. The report is published on the Graduate School's web: http://healthsciences.ku.dk/phd/current/schoolstructure/evaluation/ As from July 2012, information on respondents' department and graduate programme affiliation has formed part of the questionnaire. This information was included in the survey on request from the Graduate School and graduate programmes to meet their need to analyse and present results according to these variables. The background is an intention to employ results in specific quality assurance efforts. However, to safeguard the anonymity of the respondents, it was agreed that no results will be published if based on the answers of less than five respondents. On the basis of the respondents' departmental affiliation, the preparation of the present report included a supplemental analysis in which answers from the clinical and the pre-clinical field were analysed separately 8. Additional segmentation to single out the results from individual departments or graduate programmes was not done in this report due to the limited number of respondents, i.e. partly to avoid reporting results of limited significance, partly to safeguard respondents' anonymity. 7 I.e. the former Faculty of Pharmaceutical Sciences, the former Faculty of Health Sciences and the veterinary departments at the former Faculty of LIFE Sciences. 8 This distribution was made on the basis of the students' department affiliation as every student is affiliated to the department where his or her main supervisor is employed. The clinical departments comprise the five departments which were subsequently merged to form a single joint Department of Clinical Medicine : Department of Orthopaedics and Internal Medicine; Department of Surgery and Internal Medicine; Department of Neurology, Psychiatry and Sensory Sciences; Department of Gynaecology, Obstetrics and Paediatrics; Department of Diagnostic Sciences. The remaining departments are defined as preclinical (see list of departments at the Faculty of Health and Medical Sciences here: http://introduction.ku.dk/organisation/departments/ 27

As from now, it will be possible to pool data on selected variables from a longer period than six months. This will make it possible to analyse results according to graduate programmes. Naturally, such data extractions are only possible if the relevant questions have not been re-worded for a longer period of time. The following pages present a summary of the primary results as bullet points (the left-hand column). In the right-hand column, recommendations are made to the Graduate School, where relevant. The recommendations are based on data from the current evaluation, taking into account any trends observed with respect to the previously published PhD student evaluation reports. They describe the answers made by PhD students who handed in their PhD thesis in the January 2011 - June 2012 period 9. Numbers in parentheses refer to page of the report on which data are presented. Results and recommendations to the Graduate School Results in brief 1. PhD enrolment Approx. half of the PhD students were enrolled to complete a pre-defined project (5). Equal distribution between clinical and pre-clinical PhD students. 12% stated that their PhD project was a continuation of research carried out at their pre-graduate level (6). Three out of four stated having had the opportunity to continuously discuss their project description with their main supervisor (6). The written half-year evaluations was assessed as being of limited academic value (7). Recommendations Most of the results in the PhD enrolment section are in line with the results achieved in the two previously published evaluations. However, approx. 25% of those previously asked stated that their PhD project was a continuation of research made as pre-graduate students. In the present survey, only half as many (12%) based their project on pre-graduate research efforts. It would be interesting to study this trend further in connection with strategies on PhD training programme recruitment. In later years, the Graduate School has increased its intake of and the activities surrounding research-year students 10, which in this context is a very positive trend. Half-year evaluations should be changed and rethought to increase their value for PhD students. The Graduate School's ongoing efforts to implement new evaluation and supervision methods should therefore be maintained and implemented as soon as possible 11. 9 Report 1 (only available in Danish): Student evaluation of the PhD Training Programme at Copenhagen Graduate School of Health Sciences. January-December 2011 (March 2012) and report 2: Student evaluation of the PhD Training Programme at Copenhagen Graduate School of Health and Medical Sciences. January-June 2012 (October 2012). Both reports were authored by the Evaluation Unit, Education and Student Services, Faculty of Health and Medical Sciences, University of Copenhagen. 10 In Danish forskningsårsstuderende, see: http://healthsciences.ku.dk/phd/studentresearch/ 11 According to preliminary statements from the Graduate School, an initiative is being prepared that will include routine status/evaluation meetings at which the student presents the status of the project and receives feedback from supervisors and from an external assessor. This model will allegedly include a requirement that every student shall have a minimum of one cosupervisor in addition to their main supervisor. The objective is an evaluation model with a higher supervision/feedback value. 28

Results in brief 2. Supervision Overall, 82,9% were satisfied with the supervision provided. 6,1% were dissatisfied with their supervision (8). Half of the PhD students, including the majority of clinical PhD students had a minimum of three supervisors. 21% only had a single supervisor, and of these the majority were preclinical PhD students (9). One in every four considered a project supervisor the more important supervisor, while 68% perceived their main supervisor as the more important supervisor (9). On seven essential quality variables, satisfaction with supervision ranged from 68% to 79% (10). A total of 5% to 13% were dissatisfied. Particularly, some PhD students experienced lacking supervision on the drafting of their thesis. 3. Study environment Nine out of ten PhD students shared an office with one or more younger researchers (13). 79% and 83%, respectively, were satisfied with the academic and social environments at the workplace (13). Only 5-7% were dissatisfied. The knowledge of the PhD coordinator scheme was limited, but those who had been in contact Recommendations Supervision quality is pivotal to achieving successful PhD study courses. It is therefore encouraging to see that the overall satisfaction with supervision at the Faculty of Health and Medical Sciences is relatively high 12. The qualitative input from respondents indicates that focussing on frequent supervisor feedback, a minimum of two supervisors and clear agreements between students and supervisors will be instrumental in ensuring a sustained, high level of satisfaction. If the above-mentioned plans are implemented by the Graduate School (cf. note 11), this will probably help to ensure satisfaction. Of the 32 answers to the statement my supervision would have been better if, none provide recommendations for what they themselves might have done to optimize the supervision results. Thus matching of expectations and reciprocity of the relation between supervisor and student may also be important issues to keep in mind. Furthermore, it is necessary to keep in mind that even though only 6.1% were dissatisfied with the supervision received, this corresponds to approx. 90 dissatisfied PhD students at the combined Faculty of Health and Medical Sciences (which has a total of 1,500 PhD students), a considerable number given the importance of supervision for the quality of the PhD studies as a whole. The answers in this section, including the degree of satisfaction with the academic and social environments, are in line with the answers collected in the two previous evaluations. In this study, slightly fewer students than previously stated having had contact with the 12 Due to a revision of the questionnaire, it is not possible to compare this level of satisfaction directly with the previous evaluations (see footnote 9). Previously, the questionnaire made a distinction between satisfaction with main supervisor and project supervisor/other supervisor. 29

Results in brief with coordinators were generally satisfied (14). 4. PhD courses Overall satisfaction with the PhD courses in which the respondents participated was approx. 70% for academic as well as generic courses (15). Half of the PhD students had participated in one or more PhD courses abroad (16). 5. Participation in other scientific environments Less than half of the PhD students (46%) answered that they had stayed in another scientific environment in Denmark or internationally during their PhD studies (17). Among those PhD students who were affiliated Recommendations PhD coordinators (21% versus approx. 25% in the previous evaluation). The above figures should be interpreted with caution as knowledge of the PhD coordinator scheme remains limited (nearly as many answer "don't know" when asked if they had had contact with their coordinator). Several other comments made in the present and previous evaluations indicate that some PhD students find that there is, indeed, a need for the type of services that coordinators can provide (e.g. practical counselling, contact to other PhD students and young researchers and sparring in case of problems between the student and his/her supervisor. It is, however, decisive that as much as possible is done to ensure that the PhD students become aware of the coordinator scheme, and that those appointed coordinators understand their tasks and roles and are highly motivated. Since November 2011, the Graduate School has been introducing new PhD students to the coordinator scheme in connection with a mandatory introduction course. This is a step in the right direction and presumably increases students' awareness of the scheme. Additionally, however, it is essential to improve the presence and visibility of coordinators in the PhD students' day-to-day work - which is where questions arise, in some cases years after the introduction course. This "participation in other scientific environments" may, naturally, be completed in many manners in relation to the specific research environment in which the PhD student is embedded. Nevertheless, efforts should be made 30

Results in brief with a pre-clinical department, the probability of having spent time in another scientific environment was slightly higher than among clinical students. Recommendations to ensure that all PhD students stay in another scientific environment during their PhD studies 13. 6. Teaching and dissemination About two thirds stated that they were engaged in teaching activities during their PhD studies. More than half of the students who served as teachers had spent 1-5% of their time on teaching-related activities (18). The majority of the PhD students saw their teaching activities as being relevant and assessed the academic benefit accociated herewith as being high. Nevertheless, nearly one in every four PhD students felt that supervision/feedback on such activities was lacking (19). The results of this part of the evaluation are very much in line with those obtained during the first half year of 2012. As previously described, the degree of satisfaction among students with their teaching and dissemination activities was relatively high, but supervisors and the Graduate School should note that a considerable share of the students lacked supervision/feedback in relation to the these activities. The PhD students participated in a wide range of dissemination activities (19). There is, however, considerable difference in the amount of time the students assess having spent on dissemination activities. One in every ten stated having spent more than half of their time on dissemination, whereas one in every three answered that they had dedicated 11-20% of their time to such activities (20). Generally, the degree of satisfaction with the benefits achieved from the PhD students' dissemination activities was high. About 90% expressed satisfaction with the benefit achieved, while only 0-2.5% were dissatisfied. As observed in the previous sections, satisfaction with supervision/feedback was lower (78%) than that observed for the remaining parameters (21). 13 As stipulated in the PhD order and the internal guidelines for the PhD programme at the Faculty of Health and Medical Sciences (see the Graduate School website: http://healthsciences.ku.dk/phd/current/rules/) 31

Results in brief 7. Graduate programmes Only 63% stated being affiliated with a graduate programme. Of these, approx. half stated that they were satisfied with the courses and seminars offered under their graduate programme (22). 8. Framework for the PhD training programme 70-75% were satisfied with their workspace and office, their access to technical assistance and their access to resources and financing. 65% were satisfied with the IT facilities (24). Only about half (46.5%) of the PhD students participating in this survey were satisfied with their contact to the PhD Administration (25). Among the dissatisfied students, the primary issues mentioned were excessive response times, a varying service level and lack of response/confirmation of receipt in connection with queries to the administration. However, in the comments many noted that they had observed improvements during the latter part of their studies with regard to both contact, service and website. Recommendations The knowledge of the graduate programmes remains somewhat limited among PhD students who conclude their studies. As noted above, only two in every three students stated that they have been affiliated with a programme. This partly reflects that the students were enrolled at a time when affiliation with a graduate programme and the former research schools was voluntary. As from 1 January 2011, graduate programme affiliation has been mandatory from enrolment. Regardless of this fact, and as previously recommended, efforts should be made to increase the visibility of the graduate programmes and to assist their development towards becoming professional platforms which the students may make use of to identify relevant professional networks, courses and activities. As in previous evaluations, among these topics the average degree of satisfaction is lowest with regards to IT facilities. However, this issue was commented on much less frequently than in previous evaluations. As is the case at Faculty of Health and Medical Sciences in general, this issue deserves sustained attention. The satisfaction with the PhD Administration in the present evaluation (46.5%) is substantially lower than that of the evaluation completed six months ago (71.6%), but largely in line with the level observed in the evaluation based on answers from students who concluded their PhD studies in 2011. We find it hard to explain this marked difference in satisfaction. It is a positive sign that several respondents "of their own will" stated that they had perceived improvements during the final part of their PhD studies. Still, the issues stated to the left naturally deserve much attention in connection with future development of the administration and the service provided to PhD students at the Faculty of Health and Medical Sciences. 32