QANU, June Assessment of the Master s Degree Programme Public Policy and Human Development at Maastricht University

Transcription

1 QANU, June 2010 Assessment of the Master s Degree Programme Public Policy and Human Development at Maastricht University

2 Quality Assurance Netherlands Universities (QANU) Catharijnesingel 56 P.O Box RA Utrecht The Netherlands Phone: Fax: Internet: QANU Text and numerical material from this publication may be reproduced in print, by photocopying or by any other means with the permission of QANU if the source is mentioned. 2 QANU / Public Policy and Human Development, Maastricht University

3 Table of Contents Foreword 5 Preface 7 Part I General Part 9 1. Structure of the report Task and composition of the assessment committee Working method of the assessment committee Domain Specific Requirements 17 Part II Programme Report Report on the master s programme in Public Policy and Human Development 21 Appendices 49 Appendix A: Elaborated learning outcomes 51 Appendix B: Curricula Vitae of the committee members 53 Appendix C: Programme of the site visit at Maastricht University 55 QANU / Public Policy and Human Development, Maastricht University 3

4 4 QANU / Public Policy and Human Development, Maastricht University

5 Foreword This report describes the findings of the Public Policy and Human Development assessment committee for the master s programme Public Policy and Human Development of Maastricht University. The report is part of the quality assessment of university bachelor s and master s programmes in the Netherlands. The purpose of this report is to present a reliable picture of the results of the degree programme, to give feedback to the internal quality assurance of the programme, and to serve as the basis for accreditation of this programme by the Accreditation Organisation of the Netherlands and Flanders (NVAO). Quality Assurance Netherlands Universities (QANU) aims to ensure independent, unbiased, critically constructive assessments using standardised quality criteria, while taking specific circumstances into account. The QANU Public Policy and Human Development assessment committee has fulfilled its task in Maastricht with great dedication. The programme has been evaluated in a thorough and careful manner. We expect that the judgements and recommendations will be carefully considered by the programme organisation and the Board of the University. We thank the chairman and members of the assessment committee for their willingness to participate in this assessment and for the dedication with which they carried out their task. We also thank the staff of the department concerned for their efforts and for their cooperation during the assessment. Quality Assurance Netherlands Universities Mr. Chris J. Peels Director Dr. Jan G.F. Veldhuis Chairman of the Board QANU / Public Policy and Human Development, Maastricht University 5

6 6 QANU / Public Policy and Human Development, Maastricht University

7 Preface On April 12 to , the Public Policy and Human Development assessment committee paid a visit to the University of Maastricht in the context of her task to evaluate the master s programme in Public Policy and Human Development. To evaluate the programme in Maastricht, the committee carefully read the self-evaluation report and many underlying documents, and talked to faculty, staff and students during the site visit. The committee greatly appreciates the openness she encountered, both in the written documents as well as in the many interviews. On the basis of this, the committee was able to form a well-reasoned opinion of the various aspects of the programme, as summarized in this report. Two key elements in the committee s assessment are the scientific nature and job orientated outcomes of a programme that should be clear from all of the programme s elements. In particular, the thesis should reflect this, as it is the pièce de résistance of a student. In this thesis, the science taught and job oriented competences during the whole programme should result in an (applied) scientific piece of work. The committee has formed its opinion on these crucial aspects on the basis of a careful evaluation of available theses, the procedures that guide the thesis writing process as well all other programme elements leading to this end product. In all, the committee considers the programme to be, indeed, of a (applied) scientific nature. On behalf of the committee members, I would like to thank all of those involved in the preparation and execution of this assessment procedure for their contributions and support, both at the Maastricht University and at QANU. Without their effort and their willingness to respond constructively to the many requests from the committee, we could not have carried out the work as smoothly and pleasantly as we did. Also, I gratefully acknowledge the contributions of the other committee members. In a very pleasant and creative atmosphere, we have been able to work through the whole process to produce this report. In this context, the support of Nikki Verseput deserves a special note of appreciation. Without her, we would have been nowhere. Paul Verweel Chair Public Policy and Human Development assessment committee QANU / Public Policy and Human Development, Maastricht University 7

8 8 QANU / Public Policy and Human Development, Maastricht University

9 PART I: GENERAL PART QANU / Public Policy and Human Development, Maastricht University 9

10 10 QANU / Public Policy and Human Development, Maastricht University

11 1. Structure of the report In this document, the Public Policy and Human Development assessment committee reports its findings. The report consists of two parts: a general part (Part I) and a programme part (Part II). The general part summarises the tasks, composition, input documentation and working methods of the assessment committee. This part of the report also contains the domainspecific requirements that were used by the assessment committee. The programme part describes the evaluation and assessment of the master s programme in Public Policy and Human Development at Maastricht University. This programme part is structured in accordance with the accreditation criteria of NVAO (Accreditation Organisation of the Netherlands and Flanders). QANU / Public Policy and Human Development, Maastricht University 11

12 12 QANU / Public Policy and Human Development, Maastricht University

13 2. Task and composition of the assessment committee 2.1. Task of the assessment committee The task of the assessment committee is to evaluate and assess the master s programme in Public Policy and Human Development at Maastricht University according to the accreditation criteria set by NVAO. Based on and in accordance with these criteria, the assessment committee is expected to assess different aspects of quality of the programme, based on the information provided by the programme in the self-evaluation report and on discussions during the site visit. The assessment report contains implicit recommendations by the committee; however, the emphasis lies on the assessment and justification of basic quality. The assessment committee has been requested to assess the master s programme Public Policy and Human Development (CROHO number 60328) 2.2. Constitution of the committee The assessment committee consists of a chairman and two members. Appendix B lists short descriptions of the curricula vitae of the committee members. Chair Professor P. Verweel, professor, researcher and head of department and director of education for the Utrecht School of Governance, of the Faculty of law, economics and governance of Utrecht University. Members Professor A. Barrientos, Senior Research Fellow van het Brooks World Poverty Institute, University of Manchester, en Senior Researcher van het Chronic Poverty Research Centre; Mr. B. Paulussen BSc, student of the the Utrecht School of Governance (bachelor s programme, Utrecht University. All members of the assessment committee signed a declaration of independence as required by the QANU protocol to ensure that the committee members judge without bias, personal preference or personal interest, and the judgement is made without undue influence from the institute, the programme or other stakeholders. Originally, QANU had the intention to install a committee of four members to assess the programme. On 12 April 2010, during the inaugural meeting in which the committee was installed, it became clear that the participation of one of the intended committee members would lead to a possible conflict of interest, because he had recently been involved with the programme as a guest lecturer. QANU did not know about this prior involvement until the inaugural meeting of the committee. Consequently, QANU decided during the meeting that the intended committee member was not in a position to sign the required declaration of independence. After consulting both the committee and the management of the programme, QANU determined that he could only participate in the site visit as an advisor, and carefully made sure that he did not actively contribute when the committee decided on their assessment of the programme at the end of the site visit. The project leader of the assessment was N.M. Verseput MSc, QANU staff member. The site visit took place on 12 and 13 April The programme of the site visit is included in appendix C. QANU / Public Policy and Human Development, Maastricht University 13

14 14 QANU / Public Policy and Human Development, Maastricht University

15 3. Working method of the assessment committee 3.1. Introduction The assessment committee was constituted formally on 12 April During this inaugural meeting the assessment committee discussed its task and the working methods. Furthermore, the assessment committee discussed the proposal for domain-specific requirements. This proposal was adjusted and subsequently instituted as the Domain-Specific Requirements, provided in chapter Preparatory phase After receiving the self-evaluation report, the project leader checked the quality and completeness of the information provided. After approval, the self-evaluation report was forwarded to the assessment committee. During the initial meeting at the start of site visit, the assessment committee discussed their findings. In addition to the self-evaluation report, the committee members each read two theses for the programme that is assessed. This led to the assessment of a total of 8 theses for the entire programme. When considered necessary, committee members could read additional theses during the site visit. Selection of the theses was done at random by the project leader and the members of the committee. Since the assessment committee has to evaluate programmes leading to a scientific degree (MSc), specific attention was given to the scientific level of the theses, the requirements, carefulness of judgement by the reviewer of the programme and the assessment procedure used. After all, in a thesis the student has to show evidence of the required qualifications to earn a degree. Within the committee a specific allocation of tasks was agreed upon, based on its expertise and composition. It should be emphasized that although specific tasks are assigned, the entire assessment committee remains responsible for the judgements and the final report Site visit Before the site visit the project leader created a programme for the interviews. The draft programme was discussed with the chair of the assessment committee and the coordinator of the programme. During the site visit, interviews are held with a representative of the faculty board, Programme Director, Educational Committee, Examination Board, alumni, study advisor and other supporting staff. Furthermore, a selection of both students and lecturers was interviewed. During the site visit the committee received additional information, for example study books and reports from the meetings of the Educational Committee. This information was studied during the site visit. A consultation hour was scheduled to give students and staff of the programmes the opportunity to talk to the assessment committee. No requests were received for the consultation hour. The assessment committee used a significant part of the final day of a site visit to discuss the assessment of the programme and to prepare a preliminary presentation of the findings. The site visit concluded with a presentation by the chairman in which the preliminary findings were provided. The presentation consisted of a general assessment and several specific findings and impressions of the programme. QANU / Public Policy and Human Development, Maastricht University 15

16 3.4. Scores of the standards The assessments are performed in accordance with NVAO s accreditation framework. The scale for the scores of the standards prescribed by NVAO was adopted; each standard is scored on a four-point scale (unsatisfactory, satisfactory, good, and excellent), themes are scored on a two-point scale (satisfactory, unsatisfactory). The assessment committee adopted the standard decision rules provided by QANU. These are: Unsatisfactory, which means that the level for this facet is below the basic standard of quality; Satisfactory, which means that the level meets the best basic standards of quality; Good, which means that a quality level is attained that exceeds the basic standards of quality and is the result of a well-considered policy; Excellent, which means that a quality level is attained that is very good in all aspects and meets international benchmarking. It is an example of international best practice. The default assessment is satisfactory, i.e. the programme complies adequately with the criteria. The assessment committee feels that despite critical remarks, the score satisfactory can be given to a specific standard. In those situations, the critical remarks will be accompanied by positive observations. When the assessment committee observes a good national practice, the judgment will be good. When both a good practice and a critical remark are observed by the committee, a weighed average score is given. In the rare case that the assessment committee decides to grant an excellent score, it aims to signal a best international practice that deserves to be copied within the academic community Reporting After the site visit the project leader writes a draft report based on the findings of the committee. The draft is first read and commented upon by the committee members. The draft report is then sent to the faculty involved to check for factual irregularities. Any comments of the faculty are discussed with the chair of the assessment committee and, if necessary, with the other committee members. After that, the report becomes official. 16 QANU / Public Policy and Human Development, Maastricht University

17 4. Domain Specific Requirements The programme decided to write its own Domain Specific Requirements. The programme management is of opinion that the programme fits these Domain Specific Requirements. General requirements Because policy challenges are not neatly bundled into academic subjects, the typical master s programme Public Policy (MPP) is interdisciplinary. It draws upon economics, political science, applied statistics, sociology and other areas to provide students with the theoretical grounding and hands-on tools needed to address a vast range of public concerns like social policy problems, environmental problems, trade and globalization questions and challenges in migration and health. None of these problems can adequately be addressed without understanding the economic and financial contexts, the political processes needed to come to decisions and the social-, historical- and cultural setting in which the chosen policy has to be designed, implemented and monitored. None of these problems can be analyzed without an analysis of the empirical data at hand and without understanding the evidence which is presented by the data and by other analysts. The contemporary policy analyst needs a wide array of analytical frameworks for understanding and analyzing policy and the contexts that give rise to policy options. In order to analyze these policy options skills fundamental to assessing the policy process and policy environment as well as a theoretical and practical understanding of how policy-related organizations operate and are constructed, are needed. In addition, policy analysts need to grasp fully the economic consequences of policy interventions both in terms of their financing needs as well in their potential microeconomic and macroeconomic impact. Performing careful policy analyses is seen as a crucial element of good governance. Writing logically and convincingly, using quantitative and qualitative research methods, making effective oral presentations, distinguishing advocacy from analysis and developing the strategic and moral qualities required of effective policy professionals, are considered to be important skills valuable in many settings from national and local government to research institutes, think tanks consulting firms, community-action groups, and direct-service providers. The typical MPP programme emphasizes practical and applied dimensions of policy-making and implementation and pays attention to develop skills in: Defining policy issues to make them more intelligible to officials in the public or private sector; Providing a broader perspective for assessing policy alternatives; Examining techniques for developing policy options and evaluating their social consequences; Developing strategies for the successful implementation of public policies once they have been adopted. More specifically, the courses are designed to enable the graduates to achieve the following: Skill in written communication and in verbal reporting; An understanding of political institutions and processes, strategies, and skills associated with policy creation and adoption; QANU / Public Policy and Human Development, Maastricht University 17

18 Knowledge of the organizational and bureaucratic structures involved in programme development and implementation; Skill in application of economic analysis to questions of economic trade-off and policy choice and efficiency; Familiarity with cost-benefit analysis and other applications of quantitative analysis and modeling, as well as the use of statistical software; An understanding of social science methodologies for dealing with problems of data collection, analysis, and programme evaluation. There are very few master s programmes in Public Policy Analysis in Europe. In the United States, however, MPP programmes have a long tradition. The oldest programme has been installed at the Maxwell School for Public Policy at Syracuse University (1937). The Association for Public Policy Analysis and Management (APPAM) is the professional association of the universities and schools offering MPP programmes and of research institutes and other professional organizations active in the field of public policy. Although APPAM is too small (90 members approximately 70 of which are schools or universities) to have a formal accreditation organization, the members are in close contact with each other to discuss the curricula and other academic issues during a yearly meeting of the academic directors/deans in the spring and a yearly conference in the fall. During these meetings course syllabi and teaching methods and -technology are exchanged and discussed. Through informal and formal contacts, through exchange of lecturers and students and through peer pressure, APPAM sets the standards for the academic quality of the programmes of their members. The Maastricht programme Several academic members of APPAM advised the Maastricht University at the time of designing the MPP programme in 2004, the standards of the APPAM where closely monitored by the Maastricht Graduate School of Governance (School), which offers the programme, when implementing its MPP programme. The School is an active APPAM member and organized the first European APPAM conference in February The School benchmarks the content of its MPP programme with similar programmes at, among others The Woodrow Wilson School at Princeton, The Kennedy School at Harvard, the L.B. Johnson School at Houston Texas, the Ford School at Michigan, the School for Public Policy at George Mason University and the Richard & Rhoda Goldman School at Berkeley. There are substantial similarities in the way the MPP programmes are designed across APPAM members. The main elements are found in all the programmes, but the specializations offered, differ, reflecting the availability of specialized staff with certain colleagues. The Maastricht MPP programme has five specialization options: Social Protection Financing/Design, Globalization, Trade and Development, Migration studies, Sustainable Development and Food, Health and Development. The specializations are chosen on the one hand to reflect the research capacity of the Maastricht Faculty and thus the available expertise at the university and on the other hand to respond to important contemporary policy challenges, in turn reflected by the interests of students. The Maastricht MPP programme is unique in its specialization in Social Protection Financing and Design. It is most probably the only MPP with this focus in the world. 18 QANU / Public Policy and Human Development, Maastricht University

19 PART II: PROGRAMME REPORT QANU / Public Policy and Human Development, Maastricht University 19

20 20 QANU / Public Policy and Human Development, Maastricht University

21 5. Report on the master s programme in Public Policy and Human Development offered by Maastricht University Administrative data Master s programme in Public Policy and Human Development: Name of the programme: Public Policy and Human Development CROHO number: Level: master Orientation: academic Number of credits: 60 EC Degree: Master of Science Mode(s) of study: full-time Location(s): Maastricht Expiration of accreditation: 2 May 2011 The site visit of the Public Policy and Human Development assessment committee (from here on refered to as committee ) to the Faculty of Humanities and Sciences (Maastricht Graduate School of Governance) of Maastricht University took place on 12 and 13 April Structure and organization of the faculty The master s programme in Public Policy and Human Development (MPP) is offered by the Maastricht Graduate School of Governance (from here on refered to as School ) at Maastricht University. The School has a limited number of activities including the MPP programme, PhD programmes, a portfolio of professional training programmes, research projects (commissioned) and policy analysis-relevant advisory services. To provide the MPP programme and implement other activities, the School works together with the Maastricht University School of Business and Economics, the Faculty of Law, the Faculty of Arts and Social Sciences, the Faculty of Health, Medicine and Life Sciences, and the International Centre for Integrative Assessment and Sustainable Development. Most of the teaching is done by staff members of Maastricht University, while approximately 30% of the teaching is delivered by guest lecturers of other universities and research institutes and by professionals in the field of public policy analysis. The MPP programme is managed by the Programme Director, who is responsible for strategic decisions and daily implementation of the programme. Recruitment, admission, course administration, exam administration and alumni relations are organized by the Students Affairs Office of the School and the School s central administration The assessment framework 1.1. Aims and objectives S1: Subject-/discipline-specific requirements The learning outcomes of the programme correspond with the requirements set by professional colleagues, both nationally and internationally and the relevant domain concerned (subject/discipline and/or professional practice). QANU / Public Policy and Human Development, Maastricht University 21

22 Description According to the self-evaluation report, the MPP programme offers a solid basis in the theories and methods used in public policy analysis to students with a background in one of the social sciences. The programme aims to equip them with the tools necessary to analyse actual policy problems and familiarize students with the current research practice in the field. Seven basic learning outcomes are distinguished. Appendix A provides an overview in which these learning outcomes are elaborated and subdivided into learning outcomes regarding knowledge and skills. The learning outcomes are: 1. To give students a thorough understanding of the academic literature in the field of policy analysis and in the field of their specialization; 2. To teach students the methods and tools to analyse and interpret empirical data and actual policy analyses; 3. To teach students to apply social science theories and methods to analyse policy decisions and outcomes; 4. To foster the necessary attitude and skills to conduct independent academic research; 5. To enhance the genuine interest in the public sector of students; 6. To widen the scope of students beyond their own cultural and national background to enable them to use their knowledge and skills in an international setting; 7. To provide students with the knowledge and the skills to be generalists in their intellectual scope and specialists in the field of policy analysis with an open attitude towards differences in points of view and in interests. The self-evaluation report states that the MPP programme is focused on public policy analysis, and thus differs from master s programmes in public administration emphasizing policy processes. There are very few master s programmes focused on public policy analysis in Europe. In the United States, however, MPP programmes have a long tradition. The oldest programme was established at the Maxwell School for Public Policy at Syracuse University in The Association for Public Policy Analysis and Management (APPAM) is the professional association of the universities and schools offering MPP programmes and sets the standards for the academic quality of its members programmes. These standards were closely examined by the School when implementing the MPP programme. Assessment The committee studied the learning outcomes defined by the MPP programme, and ascertained that they correspond to the requirements set by professional colleagues and by the concerned field. The committee also compared the programme with similar academic programmes referred to in the domain-specific requirements, e.g. with the Maxwell School for Public Policy at Syracuse University, and with domain-specific requirements, and found that the learning outcomes of the programme are in line with accepted domain-specific requirements. The committee concluded that the programme certainly meets the criteria concerning this standard. The committee states that the focus of the MPP programme is quite unique, and a fruitful combination of the public policy and human development fields. There is no programme in the Netherlands that integrates the different disciplines the way the MPP programme does. For this reason, the domain-specific requirements are internationally focussed. The committee noted that the domain-specific requirements are set out very clearly and match the learning outcomes of the programme. The committee would like to point out that there could be a stronger focus on human development within the domain-specific requirements. 22 QANU / Public Policy and Human Development, Maastricht University

23 Overall, the committee finds it very useful to have this programme in the Netherlands and believes the programme has a great future. In addition, the committee concluded that the learning outcomes of the programme highly correspond with the requirements set by professional colleagues, and the relevant domain concerned. The committee is impressed by the statements and ambition of the MPP programme, and assesses the learning outcomes as both promising and challenging. They are multidisciplinary and offer students a broad perspective, which the committee finds relevant taking into account the requirements set by professional colleagues. Alumni confirmed this statement during the interviews and stressed the value of the knowledge of different disciplines they acquired during the programme. Master s programme in Public Policy and Human Development: the committee assesses this standard as good. S2: Bachelor and master level The learning outcomes of the programme correspond with the general, internationally accepted descriptions of a Bachelor s qualification or a Master s qualification. Description The MPP programme meets the internationally accepted requirements for an academic master s programme as specified by the Dublin descriptors, according to the self-evaluation report. Table 1 relates the Dublin descriptors and the elaborated learning outcomes which are specified in Appendix A. Dublin descriptor Learning outcomes Knowledge and understanding K1, K2, K3, K4, S1, S2, S3 Applying knowledge and understanding K5, K6, S4, S5, S6 Making judgments K7, S7 Communication S8, S9 Learning skills S10, S11, S12 Table 1. Learning outcomes linked to Dublin descriptors Assessment The committee verified the relationship between the learning outcomes and the Dublin descriptors, which are considered to be general, internationally accepted descriptions of a master s programme. The committee observed that all Dublin descriptors are reflected in the learning outcomes. The committee therefore concluded that the learning outcomes correspond with general, internationally accepted descriptions of a master s programme. The committee established that the Dublin descriptor for knowledge and understanding is reflected in, for example, learning outcome K2, an advanced knowledge of theories relevant for policy analysis and for the chosen specialization. The Dublin descriptor for applying knowledge and understanding is reflected in, for example, learning outcome S4, capable of analyzing policy options using the existing approaches. The Dublin descriptor for making judgements is reflected in learning outcomes K7, understanding the link between theories and methods and the moral philosophical ideas, and S7, capable of assessing, quantitatively as well as qualitatively, the consequences of different policy options. The fourth Dublin descriptor, communication, is reflected in learning outcome S8, capable of reporting in English both orally and in writing and learning outcome S9, capable of presenting their results to both academic audience and to wider audiences;. The fifth Dublin descriptor, learning skills, is, for example, reflected in learning outcome S12, capable of learning new theories and methods and applying them outside their usual field. In addition, the committee QANU / Public Policy and Human Development, Maastricht University 23

24 states that the various learning outcomes clearly reveal that graduates of the MPP programme acquire knowledge, understanding, skills and attitudes at an advanced level that is typical for a master s programme, among them the learning outcomes which state that students learn to be capable of recognizing links between different theoretical frameworks and stemming from various disciplines; (S3), and to be capable of analyzing policy options using the existing approaches (S4). The committee concluded that the programme meets the criteria concerning this standard. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S3: Academic orientation The learning outcomes of the programme correspond with the following descriptions of a Bachelor s and a Master s qualification: The learning outcomes are derived from requirements set by the scientific discipline, the international scientific practice and, for programmes to which this applies, the practice in the relevant professional field. An academic bachelor (WO-bachelor) has the qualifications that allow access to at least one further programme at academic master's level (WO-master) and the option to enter the labour market. An academic master (WO-master) has the qualifications to conduct independent research or to solve multidisciplinary and interdisciplinary questions in a professional field for which academic higher education is required or useful. Description The self-evaluation report mentions that the learning outcomes are derived on the one hand from what is defined as the discipline of policy analysis and on the other hand from what the requirements are in the professional field. According to the self-evaluation report, graduates of the MPP programme have attained the general qualifications typical of academic training and suitable for the practice in the professional field. At the end of the programme, students will be able to conduct independent academic research and are ready to be trained further as academic researchers in PhD programmes. During the site visit it became clear that the programme would like to have a two-year research programme, but this is currently not possible. Along with academic research, students will be trained as well-qualified, internationally oriented professionals who can recognise, analyse and solve policy problems as well as provide advice, based on a thorough academic understanding of the underlying theories and methods in the field of policy analysis and in the area of their specialization. Graduates typically find a job as a researcher, a policy advisor or a consultant in the national public sector, in the private sector, in NGOs, or in international organizations. Assessment The committee has examined the learning outcomes from the perspective of the required academic orientation. It noted that the learning outcomes of the MPP programme correspond sufficiently to the requirements of the academic discipline, of its practitioners, and of professional communities (see standard 1). It assessed both the professional and academic learning outcomes and established during the site visit that graduates of the programme can start their careers as professionals or as PhD students. The committee concluded that the learning outcomes are explicitly of an academic nature and level. This is evident from the described learning outcomes and table 1, in which the Dublin descriptors are related to these learning outcomes. In addition, the committee interviewed alumni who chose to obtain a doctorate. The committee is impressed by the percentage of graduates who choose to take a doctoral degree, and established that graduates feel well equipped in terms of research skills. Although the committee regrets the fact that 24 QANU / Public Policy and Human Development, Maastricht University

25 the programme does not have a two-year research programme, it is convinced that the programme is able to provide students with sufficient research competencies within one year. The committee confirmed that the programme provides graduates with a solid foundation that qualifies them for a career in the public and private sector as well as for an academic career in the field of public policy analysis. Based on interviews and questionnaires aimed at the experiences of alumni, the committee concluded that graduates of the programme are well prepared for a professional career and find jobs easily. This leads the committee to conclude that the programme meets the criteria specified for this standard. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. Assessment of the theme Aims and objectives The committee comes to an overall assessment of the theme Aims and objectives on the basis of its assessments of the separate standards. In the case of the master s programme, it assesses this theme as satisfactory Curriculum Description of the curriculum of the programme The one-year MPP programme consists of two semesters of studies and a master s thesis. In the first semester, all courses are compulsory. The focus is on the theories and practice of governance and policy analysis. The basic knowledge and skills related to policy analysis are introduced. In the second semester, the students choose one of the specializations and write a master s thesis. The specializations include Social Protection Financing and Design; Globalisation, Trade and Development; Sustainable Development; Migration Studies; and Food, Health and Development. Each specialization has four specialization-specific compulsory courses. Students also have the possibility to start the programme in February (see standard 6). All courses are taught on a full-time basis during four weeks, approximately coinciding with a month. Examinations are made at the end of each course. Courses are offered in this condensed format to allow more opportunities to invite academic and professional colleagues from abroad to contribute to the teaching of the courses as a guest lecturer by restricting the length of their stay in Maastricht. The first semester of the curriculum is summarized in tables 2a ( ) and 2b ( ) and the second semester of the curriculum is summarized in table 2c. September October November December January Governance, Politics, Policy and Practice Public Economics and Empirical Analysis Public Policy Analysis Risk and Uncertainty or The Economics of the Welfare State Table 2a. Curriculum of the MPP programme, semester 1, Research methods and preparation for master s thesis In response to remarks from the students and the staff about an uneven spread of the workload for the programme during the current academic year, the structure of the first semester has been reconsidered. Students comments stress that they would prefer to have an extended period to study, and apply methods and techniques during the whole semester, rather than only working on it during concentrated periods. The lecturers involved share this QANU / Public Policy and Human Development, Maastricht University 25

26 view. From the next academic year onwards, the first semester will be organised as spelled out in table 2b, allowing the time devoted to policy analytical skills to be spread over the entire semester. September October November December January Governance, Politics, Policy and Practice Public Economics Applied Public Policy Analysis and Policy Implementation Risk and Uncertainty or The Economics of the Welfare State Statistics, econometrics and methods and techniques of empirical policy analysis Table 2b. Curriculum of the MPP programme, semester 1, Ethics, Political Philosophy and Presentation Skills and preparation for master s thesis February March April May June July-August Social Protection Financing/Design (SPF/SPD) SPF Financing Social Protection/ SPD Financing Social Protection SPF Social Budgeting/ SPD Social Budgeting SPF Actuarial Practice in Social Protection/ SPD Poverty and Inequality Master s thesis work and presentation skills SPF Quantitative Techniques in Healthcare Financing/ SPD Comparative Analysis of Social Policy Design SPF/SPD Thesis Writing + internships, capstone projects Globalisation, Trade and Development International Economics The Law and Policy of the WTO Sustainable Development Introduction Governance for to Sustainable Sustainable Development: Development Concepts and Practice Migration Studies Introduction Migration and to Migration Remittance Studies Effects Food, Health and Development Economic Food Policy Analysis of Analysis Healthcare Systems International Intellectual Property Law and Policy Sustainability in a Globalizing World Applied Migration Studies Assessing Needs and Understanding Environmental Risk Factors Master s thesis work and presentation skills Master s thesis work and presentation skills Master s thesis work and presentation skills Master s thesis work and presentation skills International Development Law Integrated Assessment Comparative Migration policy Quantitative Techniques in Healthcare Financing Master s thesis and Research Skills Table 2c. Curriculum of the MPP programme by specialization, semester 2, Thesis Writing + Thesis Writing+ Thesis Writing+ Thesis Writing+ S4: Requirements for academic orientation The proposed curriculum meets the following criteria for an academic orientation: The students develop their knowledge through the interaction between education and research within the relevant disciplines The curriculum corresponds with current developments in the relevant discipline(s) by verifiable links with current scientific theories The programme ensures the development of competences in the field of research Where appropriate, the curriculum has verifiable links with the current relevant professional practice. 26 QANU / Public Policy and Human Development, Maastricht University

27 Description According to the self-evaluation report, students develop their knowledge through the interaction between education and research. This interaction is for example ensured by the research experience of the teaching staff and the master s thesis. The interaction also ensures the correspondence between the curriculum and current developments in relevant disciplines. The core courses of the first semester cover the basic disciplines underlying the field of policy analysis, public economics, political science, econometrics, probability theory and statistics, applied research methods for policy design and evaluation. The courses in each of the specializations cover the core disciplines within the specific specialization. During the next academic year new specializations may be proposed. Another interesting development that is considered is the possibility of cooperating with the Master School of Management. This cooperation also may lead to the inauguration of new specializations in the future. In addition, the self-evaluation report states that the programme ensures the development of competencies in the field of research. In the first semester, research skills are systematically developed in the empirical analysis course, in the policy analysis course and in the skills training course. The methods, techniques and practice of policy analysis are treated in these courses. In the second semester, students are guided extensively in the research work for their master s thesis. The professional practice is referred to by the nature of the assignments, by the discussion sessions during the courses, and by the nature of the proposed capstone projects, internships and thesis research assignments. In capstone projects, the students cooperate with professionals in doing their research for their thesis. Assessment The committee was assessed that the programme meets the requirements for an academic level and orientation. It finds that the interaction between teaching and research is adequately ensured, and that the curriculum corresponds with current developments in the field. The programme combines the transfer of academic knowledge through teaching and research in the relevant disciplines of the field. The committee established that the coordinators of courses and specializations are professors, associate professors or assistant professors. The committee confirmed that lecturers actively engage students in their research, so current scientific developments and theories, and research concerning the different disciplines permeate the curriculum. During the interview with management, the potential cooperation between the School and the Master School of Management, and the idea of new specializations were discussed. The committee agrees with the benefits of this cooperation, and values this active attitude. Still, the committee wants to draw attention to the fact that the School should be aware of its boundaries regarding growth (see standard 8). The committee is of the opinion that the development of scientific research skills is adequately addressed within the curriculum in ways other than the interaction between teaching and research. Courses stimulate students to think academically and prepare them for writing their thesis. The committee established that because of the multidisciplinary focus, students of the MPP programme will have different backgrounds (see standard 8) and therefore will have different competencies in the field of research at the start of the programme. For example, students with a background in economics might have an advantage over other students concerning methodology. The committee appreciates the fact that the programme is aware of the differences between enrolling students and that lecturers provide extra courses on methodology to students who perform poorly. The committee feels that the differences between students could also be compensated by putting more emphasis on technological analyses and research methodology during the first semester, to ensure that all students have the same level when finishing the programme. QANU / Public Policy and Human Development, Maastricht University 27

28 The committee noted that the programme ensures contact with the professional field. Case studies are integrated within courses to provide interaction with practice. Also, the possibility of an internship stimulates contact between students and the professional field. The committee finds that both students and lecturers are satisfied with the balance between the academic orientation, and the level of the programme and its linkage to the professional field. Overall, the committee is of the opinion that the programme adequately meets the criteria for the standard. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S5: Correspondence between the aims and objectives and the curriculum The curriculum is an adequate realisation of the learning outcomes of the programme and this regards the level, the orientation and the subject-/discipline-specific requirements. The learning outcomes are adequately transferred into the educational goals of the curriculum or parts thereof. The contents of the curriculum ensure the students achievement of the learning outcomes. Description The self-evaluation report points out that the seven learning outcomes of the MPP programme (see standard 1) are reflected in the curriculum requirements. Learning outcome 1, academic understanding of policy analysis and specialization, is translated in the curriculum by focussing the coursework on the core subjects, such as public economics, political science, and risk management. Journal articles and the latest textbooks and handbooks are used as basic study material. Learning outcomes 2 and 3, skills and methods to perform policy analysis, are trained, discussed and applied specifically in the empirical analysis, public policy analysis and research methods courses. Their application is central to the master s thesis and the various assignments that are required throughout the other courses. Learning outcome 4, attitudes and skills to perform independent academic research, is trained in all courses, especially when doing assignments and during the master s thesis. Genuine interest in public policy and the public cause, learning outcome 5, is a feature that the students display when they choose to engage in this programme, and is encouraged as much as possible through the admission requirements (see standard 8). The course work during the MPP programme is intended to foster and enhance this interest in public policy. For example, the students interest is nurtured by lecturers during the courses. Demos, the study association, plays an essential role in implementing learning outcome 6, widening the scope beyond one's own culture and applying skills in an international setting. The student body and staff members in the MPP programme are very international. Many courses and lecturers (guest) refer to experiences and policy applications stemming from many countries or have a genuine international (comparative) approach. The MPP programme also offers its students the opportunity to engage in research activities in various countries and with international organisations. Learning outcome 7 refers to the intention to train and form generalists in scope and specialists in their field. The MPP programme aims to train intellectuals technocratic insights. Being well trained in methods and techniques, graduates ideally understand how the policy interventions they are analyzing fit within the history and the views, as explicitly and implicitly formulated within the culture of the country they operate in. Although there is no course to train intellectuals, nor is there a standardized test to assess its outcome, this does not mean that it is unimportant. The MPP programme management plans to increase the attention paid to political philosophy and policy implementation from the next academic year onwards, pointing out these elements to students in their future work. 28 QANU / Public Policy and Human Development, Maastricht University

29 Assessment The committee studied the correspondence between the learning outcomes and the curriculum and established that the self-evaluation report reveals how the individual courses are systematically cross-matched with the learning outcomes. Learning outcome 1 is crossmatched to different courses during both the first semester and the second semester. Learning outcomes 2 and 3 are linked to the empirical analysis, public policy analysis and research methods courses and the master s thesis. Learning outcome 4 is trained in all courses, but mainly through the assignments and the master s thesis. Learning outcome 5 is ensured through the admission requirements and stimulated by different courses and lectures. Learning outcome 6 is fostered by the international composition of the students and the fact that many courses and lecturers (guest) refer to experiences and policy applications. Learning outcome 7 is not specifically related to the courses and assessments within the programme, but receives attention when pointing out elements in their future work to students. During the site visit, the committee gained insight into the way the learning outcomes are translated within the courses by studying syllabi. The committee values the way the syllabi are organized; they provided sufficient insights regarding the relation between the learning outcomes and the courses, on the basis of which the committee concluded that the learning outcomes are translated adequately within the courses. The response of the alumni strengthened this conclusion. The committee realizes that it takes a lot of effort to fulfil all the learning outcomes of such a multidisciplinary programme, and is impressed by the way the programme manages to guarantee that students achieve them. Master s programme in Public Policy and Human Development: the committee assesses this standard as good. S6: Consistency of the curriculum The contents of the curriculum are internally consistent. Description According to the self-evaluation report, the internal consistency is the primary responsibility of the Programme Director. The Programme Director uses the input of both students and lecturers plus the regular evaluation of courses to review the consistency of the programme. The first semester consists of four courses and a skills training period. The objective for this semester is to cover general aspects of governance and to provide general theoretical and research tools that graduates will need for policy analyses outside their area of specialization, in order to make sure that they are sufficiently trained as generalists. For the second semester, students choose an area of specialization. Each student then follows four courses that belong to the specialization. The consistency of the specializations is arranged by the coordinators. They are responsible for making sure that the four courses in the specializations fit together and that unnecessary overlap is avoided. Coordinators use the input of students and discussions with colleagues and guest lecturers to review the internal consistency of the courses. The self-evaluation report mentions that the peculiarity of starting the semester in February is an issue that has been monitored closely. The intake in February was initiated because the demand for it was and still is considerable. While it is logical for students first to study a general introduction in policy analysis and later follow specialization courses, the practice over the last few years has shown that almost all students cope well with the reverse order. This is also due to the fact that the knowledge and skills learned in the specialization are not just applications of the first semester, but have a body of knowledge of their own which can be QANU / Public Policy and Human Development, Maastricht University 29

30 studied independently. In order to facilitate this group of students, the programme for the February cohort is adjusted so that some basic policy analysis elements are introduced in May, covering the main elements of the first semester. Moreover, significant efforts are made to structure the thesis work of the February cohort along similar lines to the thesis work of the other students. This hopefully will lead to more students of the February cohort graduating within the twelve month period. Assessment The committee studied the internal coherence of the programme. In conclusion, it found that there is substantive and logical consistency, and therefore assesses that the programme meets the criteria concerning this standard. It feels that the second semester builds logically on the first semester. Alignment between course contents is covered by the coordinators for each course, who are, according to the committee, well aware of the relationship between their course and other courses. This awareness is realized by developing courses in dialogue with guest lecturers and checking periodically together with the Programme Director whether the contents of the different courses are well aligned. According to the committee, this organization makes it possible to involve guest lecturers within the programme, without threatening its coherence. The committee established that because the semesters are connected but do not necessarily build on each other, the programme also provides the possibility of taking the second semester first, which allows students to start in February (see standard 8). The committee appreciates the fact that the programme is well aware of the obstacles this brings and is actively working on solutions. The committee noted that the different specializations within the second semester of the programme are well thought out and grounded. The committee appreciates the benefits of the freedom of choice students have and the fact that students also have the possibility to switch when they feel they have chosen the wrong specialization. Nevertheless, the committee wants to point out that this switch of specialization could decrease the coherence of the programme. The committee determined that the coherence of the programme is safeguarded closely, because of the high degree of commitment and the small-scale organization, in which lecturers and students meet regularly, also on an informal basis. The committee wants to point out that when the number of students grows (see standard 8), the School could lose the benefits of a small-scale organization and may for example have to increase its formal control and procedures on coherence. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S7: Workload The curriculum can be successfully completed within the set time, as certain programme-related factors that may be an impediment in view of study progress are eliminated where possible. Description The MPP programme is a one-year programme, comprising 60 EC. The study load is 4 or 5 EC per course and 18 EC for the master s thesis and the relevant skills. The number of hours of classes per week varies per course in the range of 12 to 16 contact hours. The number of hours that students report having actually actively studied per course is approximately 25 to 30 hours on average per week, apart from the contact hours. On average, the workload for the students is approximately 40 hours per week. In order to avoid delays in graduating, all exams are offered twice per year on preset days. Students who fail a course twice but succeed 30 QANU / Public Policy and Human Development, Maastricht University

31 in passing all other exams can request the exam commission for a third chance to sit the exam for the failed course during the same academic year. Until last year, students did not complain about the level of the workload, although they reported that it represented a full working week. During the first semester of the current academic year ( ), students complained about a high workload in the second and the fourth course. This was especially due to the fact that the quantitative analysis part of these courses had been changed in This problem was discussed with the students and the staff and has led to new arrangements for the organization of the first semester (see table 2b). The adjusted first semester aims to spread the workload more evenly on the one hand and to strengthen the learning environment for the training of the statistical and quantitative analysis skills of students on the other. Assessment The committee studied whether the curriculum of the MPP programme can be successfully completed within the time nominally set and whether programme-related factors that may impede study progress are eliminated where possible, and concluded that the programme fulfils the criteria specified for the relevant standard. According to the committee, the programme could be studied in 40 hours a week, but also offers the possibility to put more time in than needed by offering students optional literature. Each course of the programme is provided within one month. The committee firstly considered this a risk but discovered during the site visit that this innovative organization of courses seems manageable and is appreciated by the students. The committee established that the School is able to adjust specific parts of the curriculum when students have complaints about the workload. As a result of the site visit, the committee concluded that students assess the workload of the programme as quite high, but doable. The committee established that students are committed to and enthusiastic about the MPP programme and are willing to work hard and to put extra efforts into it. This remarkable motivation can partly be explained by their international background, which motivates them to complete the programme within the set time and return to their country. In addition, the School also succeeded in creating a community of very motivated and committed students, who are guided well and closely followed by a committed staff (see standard 16). Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S8: Admission requirements The structure and contents of the intended curriculum are in line with the qualifications of the incoming students: Academic bachelor s programme (WO-bachelor): VWO (pre-university education), propaedeutic certificate from a hogeschool (HBO)or similar qualifications, as demonstrated in the admission process Master s programme (WO-master): a bachelor's degree and possibly a selection (with a view on the contents of the discipline). Description According to the self-evaluation report, recruitment information activities are organized for interested prospective students. During the academic year several meetings (open days) take place. During these meetings information about the programme structure and content is presented. Furthermore, information about application procedures and practical information about studying in Maastricht is given as well. For prospective students who are not able to visit Maastricht, the programme is presented at international recruitment fairs. The website provides a virtual tour and additional information via an online avatar, which can be followed QANU / Public Policy and Human Development, Maastricht University 31

32 up by phone, mail or Skype. Online courses in Economics, Mathematics/Statistics, Governance and Political Sciences are offered to admitted students. The courses aim at refreshing the prior knowledge of the incoming students in these disciplines. The MPP programme is selective; all students apply for the programme and have to be admitted by the Board of Admission. To be admitted, students have to possess a bachelor s or master s degree from a relevant academic field such as economics, international relations, political science, law, social sciences or business administration. In addition, the MPP programme requires basic knowledge of statistics and mathematics, economics, social sciences, political science and/or law, and proficiency in English. Apart from compliance with the formal requirements, the admissions board will take into consideration a letter of motivation, an essay related to one of the specializations, two letters of recommendation by current or former professors or employers, and a curriculum vitae. The self-evaluation report provides an overview of the number of applicants and percentage of applicants admitted to the MPP programme and of the MPP students of the September 2009 cohort by nationality (region) and previous degree. Table 3 provides an overview of the absolute numbers of the MPP students by cohort and specialization. Social Protection Financing/ Design Growth, Trade and Development Sustainable Development Migration Studies Feb Sept Feb Sept Feb Sept Feb Sept Feb Table 3. Absolute numbers of MPP students by cohort and specialization Food, Health and Development Assessment The committee examined whether the qualifications of the incoming students match the contents of the programme. The committee studied the admission requirements and concluded that the qualifications of the incoming students are in line with the structure and contents of the intended curriculum and that at least one bachelor s degree gives direct admission to the programme without restrictions. The committee established that students have the possibility to start the programme in September or February, and is impressed by the way this possibility is realized, taking into account the experiences of the students and the coherence of the programme (see standard 6). The committee noted that the programme strives to recruit international students and appreciates the fact that the percentage of international students is quite high. Of the total number of students who started in September 2009 (61), 13 are Dutch, 34 are from Europe, and 14 are non-european. The committee noted that international students are well taken care of when they start the programme, partly by Demos, the study association, and partly by initiatives of the School, like cultural dinners. As noted before, the committee praises the School for the way it succeeds in creating a community to which all students feel committed, although the groups are quite large. 32 QANU / Public Policy and Human Development, Maastricht University

33 Besides the differing backgrounds (nationalities), students of the MPP programme also come from different disciplines. During the site visit, the committee noted that students feel that the School is doing a great job in bringing students of different disciplines and nationalities to the same level, through the way the selection takes place and the fact that class sizes are kept small (see standard 10). The committee agrees with this and finds that the programme tries to create streams in which the four disciplines are represented equally, which succeeds reasonably according to the committee. As amplified under standard 10, the committee advises the School to safeguard this equal representation, and also to translate it to the composition of groups within specializations and classes. In this way, differences between students related to their background are minimized, for example the advantage students with an economic background have concerning methodology (see standard 4). The committee is satisfied with the average annual inflow of students. At the moment, the programme, staff, facilities and quality assurance are perfectly tuned to this. Despite the fact that the committee feels the situation is currently more than satisfactory, it would like to point out potential problems in the future. The desired and expected growth in numbers could have consequences for example regarding guidance, coherence and quality assurance. During the site visit, the committee established that when the number of students grows, the School will focus on maintaining small groups by starting new programmes and specializations instead of enlarging groups and will attract more core and support staff (and it would like to encourage this) (see standard 12). In addition, the committee advises formalizing some procedures. Master s programme in Public Policy and Human Development: the committee assesses this standard as good. S9: Credits The programme meets the legal requirements regarding the range of credits: Academic bachelor s programme (WO-bachelor): 180 credits Academic master s programme (WO-master): a minimum of 60 credits. Description The curriculum of the master s programme in Public Policy and Human Development comprises 90 EC and complies with the formal requirements with respect to the size of the curriculum. Assessment The master s programme in Public Policy and Human Development complies with the formal requirements with respect to the number of credits. S10: Coherence of structure and contents The educational concept is in line with the aims and objectives. The study methods correspond with this educational concept. Description The self-evaluation report states that the educational concept derives from Problem-Based Learning (PBL). At the heart of PBL lies the idea that successful learning depends on the empowerment of learners to conduct research, to review the literature, to integrate academic theory with professional practice, and to develop the necessary skills to approach a defined problem situation. Critical to the success of PBL is the selection of course materials that engage students in thinking, analysis, and reviewing of what has been learned when working on problems, cases, or research assignments. On top of that, it is crucial that teachers (tutors) guide the learning process, provide structure to the students work on projects, cases, or QANU / Public Policy and Human Development, Maastricht University 33

34 assignments, and conduct a thorough debriefing and review process of what has been learned. The programme leaves the lecturers a considerable amount of freedom to organize their course the way they deem best. The diversity in the nature of the course content explains the wide range of teaching methods used in the MPP programme. Some courses are very technical. In that case, the most effective way of working is to give an introduction, followed by a discussion and a small assignment for the students to work on, and at the end to discuss the elaboration of the assignments in the entire group. Other courses are more theoretical, requiring a lot of preparatory reading and thorough discussions around problems in small classes. The self-evaluation report indicates that the number of students in the MPP programme is relatively modest, and class sizes are typically small, particularly in the specialization courses (10 to 15 students). During the first semester, the number of students taking a single course is bigger, since they all follow more or less the same programme. In one case (course 3) the students are given two options with a maximum number of students. In all the other courses, lectures are given to the entire group, and exercises, discussions, presentations and the like are organized in smaller groups. Assessment The committee studied the coherence of the structure and contents of the programme and concluded that it fulfils the criteria specified for the relevant standard. The committee established that the study approach fostered by the programme enables students to achieve the learning outcomes, and the study methods correspond with this educational concept. The committee concluded that, at the course level, there is a link between the learning outcomes, the implicit educational concept and the teaching methods. The committee feels that allowing the lecturers to organize their course is quite a valuable aspect, so the course will fit the intended learning outcomes and the teaching materials. Despite the fact that there may be differences between courses, the overall didactical approach, PBL, is reflected in all of them. The committee firstly felt like this approach is outdated, but concluded that it seems to work quite well and has a lot of advantages, especially taking into account the multidisciplinary and international character of the programme. Students are motivated to integrate their knowledge on different disciplines while solving cases. The number of contact hours and the concept of small groups also contribute to the integration of knowledge of students from different backgrounds and enable the students to get to know and support each other. The committee advises making sure that the composition of groups within specializations and classes also enhances integration of the different backgrounds. The committee noted that students value highly the number of contact hours, the PBL approach and working in groups. This method allows students to ask for feedback and discuss the material with each other and the lecturers. The committee would like to state again their appreciation of the motivated and enthusiastic attitude of both students and lecturers. Master s programme in Public Policy and Human Development: the committee assesses this standard as good. S11: Learning assessment By means of evaluations, tests and examinations, the students are assessed in an adequate and for them insightful way to determine whether they have achieved the learning outcomes of the programme or parts thereof. 34 QANU / Public Policy and Human Development, Maastricht University

35 Description According to the self-evaluation report, the method of examination and assessment is aligned with the didactic structuring of the programme, the learning outcomes, and the requirements per course. Therefore, a standardized testing system for the MPP programme is not in place. In most cases though, students have to pass a written or oral examination at the end of a course, but it is also possible for the final examination of a course to consist of writing a paper. Moreover, students are often required to write papers and/or give presentations during a course. The precise requirements that students need to fulfil in order to pass a course are described in detail in the course manual and are known before a course starts. It is believed that consistency in the feedback that students are being given optimises the effect of education. The self-evaluation report mentions that as a result of some of the teaching staff being specialists from outside Maastricht University, they may sometimes employ different standards and habits. A significant portion of the US-affiliated colleagues, for example, are used to relative grading with more generous grading traditions than the common Dutch practice. The differences are not always understood by the students, and efforts are being made to explain these differences better. The MPP programme also faces logistics challenges as many members of the teaching staff visit for relatively short periods. Organizing feedback on assignments and exams to the students after the teacher has left the School is sometimes a challenge. The MPP programme is paying close attention to this issue in the current and subsequent academic years by using videoconferencing and online feedback sessions for the students. The Examination Board (EB) is responsible for the organization and the quality of the exams. It has the formal responsibility to make decisions on the objections of students to interpretations or applications of the Teaching and Examination Regulations (OER) with respect to e.g. grading of exams, regulations for the successful completion of courses, rules for good order during exams, complaints from teachers concerning fraud by students. Furthermore, the EB handles individual requests for exemptions in the study programme (e.g. exam exemptions) and issues the diplomas. Assessment The committee examined the learning assessment procedure. From the OER and from information gained during the site visit, it noted that students are assessed adequately and concluded that the programme meets the criteria concerning this standard. The committee established that a reasonable mix of evaluations, tests and examinations is used, such as papers and exams. Although the committee initially had its doubts about the aim to include multiple examinations within one course because of the short duration of each course, it concluded that this has worked out well. The committee verified that students are positive about examinations and feel that they do receive feedback during the course on their progress. Students also have the possibility to ask the lecturer for feedback and discuss their grade at the end of a course. The committee values the fact that students can ask for coaching if they need to resit an examination. In addition, the committee confirmed that students can ask for the template in which the expected answers are specified. The committee feels that in combination with the feedback during the course, students are provided with good insights about the examination and grading methods. The committee noted that the EB functions as a group. It meets every six weeks and deals with complaints from lecturers and students. During the site visit, the EB stated that it mainly QANU / Public Policy and Human Development, Maastricht University 35

36 implements the OER in a reactive manner. Also, the committee feels that the EB mainly reacts to complaints concerning exams and grading. The committee advises the EB to become more proactive by initiating decisions. Finally, the committee wants to point out to the EB that its role will be changing in the future (legally required), making it formally responsible for the quality of the contents of the examinations. The committee advises the EB to consider the consequences of this for its role. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. Assessment of the theme Curriculum The committee comes to an overall assessment of the theme Curriculum on the basis of its assessments of the separate standards. In the case of the master s programme, it assesses this theme as satisfactory Staff S12: Requirements for academic orientation The programme meets the following criteria for the deployment of staff for a programme with an academic orientation: Teaching is principally provided by researchers who contribute to the development of the subject/discipline. Description To deliver its MPP programme and to implement its other activities, the School works together with the Maastricht University School of Business and Economics, the Faculty of Law, the Faculty of Arts and Social Sciences, the Faculty of Health, Medicine and Life Sciences and the International Centre for Integrative Assessment and Sustainable Development. Most of the teaching, 70%, is done by staff members of Maastricht University. Approximately 30% of the teaching is delivered by staff members of other universities and research institutes, and by top professionals in the field of public policy analysis. Each course has a coordinator who is usually a senior staff member (professor, associate professor or assistant professor) at one of the faculties of Maastricht University and is a respected researchers in his/her field of expertise. The self-evaluation report provides a list of the lecturers and their affiliation in the MPP programme and a list of guest lecturers in the MPP programme. The self-evaluation report states that the fact that most of the staff is affiliated outside the School is both an opportunity and a challenge. The School has the opportunity to attract real specialists by recruiting among a wide range of scholars and colleagues. The School is convinced that the relatively modest drawbacks of the formula are largely compensated for by the big gain in content and quality brought by the teaching staff. Assessment The committee studied the requirements for the academic orientation of staff members, and concluded that the MPP programme meets the criteria expected of a scientific degree programme. The academic staff originates from both the Netherlands and abroad. The committee recognises the staff s scientific quality, national and international academic reputation and teaching experiences. A majority of the teaching staff holds a PhD and publishes in highly ranked journals. The committee noted that recent developments from scientific research, as well as those from the professional field, are brought into the courses. The committee established that the School has contacts with various foreign universities, with other universities in the Netherlands, and with other faculties of Maastricht University, via formal and informal networks. It finds that the programme is principally offered by 36 QANU / Public Policy and Human Development, Maastricht University

37 researchers who contribute to the development of their discipline as well as to the other disciplines of the MPP programme. The committee also concluded that there is a lot of cross-communication, which contributes to the view lecturers have of integration of disciplines and the way they deal with students. The committee assesses that lecturers have positive experiences regarding this exchange of knowledge of different disciplines. The committee wants to make the School aware of the position of the core staff. This is quite small, four persons. The other lecturers are derived from other faculties and universities. The committee established that, as a consequence of this division, deployment for the core staff is limited. According to the committee, the core staff does not have enough possibilities to focus on research and to grow in academic orientation. During the site visit, the committee noted that the School is aware of this consequence and is committed to fostering the development of core faculty by attracting more core staff and support staff. The committee applauds this move. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S13: Quantity of staff Sufficient staff are deployed to realise the desired quality of the programme. Description The MPP programme deploys 101 lecturers and guest lecturers in a single year. Their contributions range from one to several weeks of teaching per year. The student-staff ratio is difficult to interpret since, contrary to classical programmes and schools wherein most of the lecturing is done by full-time staff, most of the teaching staff is only at the school temporarily. Therefore, the student-staff ratio would be high by all standards since many staff members are involved for relatively short periods of time. Calculating the student-staff ratio on the basis of the permanent academic staff would be highly misleading since they are only responsible for a very small part of the teaching and supervision done within and on behalf of the MPP programme. Almost by definition the MPP programme deploys enough staff to deliver the programme. Assessment The committee studied the quantity of staff for the MPP programme. It concluded that a sufficient number of staff members are present to assure the desired quality of the programme. For both the committee and the School, it is hard to judge the adequacy of the student-staff ratio. During the site visit, the committee noted that, in general, both students and staff members affirm that there is an adequate student-staff ratio for the first semester and all of the specializations. It has established that the staff members find the size of their teaching load acceptable and it has not received any information which indicates that the amount of staff time available for master s programme is not sufficient. Students noted that the lecturers are always available to them, and contact occurs regularly. In addition, the small size of the groups seems to be guaranteed adequately by the School. The committee is positive about the fact that the School is focusing on a new programme and new specializations as a consequence of the expected growth (see standard 4). This will prevent increased class sizes. As stated before, the committee is of the opinion that the core staff of the MPP programme should be strengthened. This will result in a better balance between core staff and staff from other faculties. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. QANU / Public Policy and Human Development, Maastricht University 37

38 S14: Quality of staff The staff deployed are sufficiently qualified to ensure that the aims and objectives regarding the content, didactics and organisation of the programme are achieved. Description According to the self-evaluation report, the entire staff is highly qualified to teach in their fields. The didactic quality of the teaching staff is closely monitored by the evaluation system (see standard 17). Most lecturers receive an excellent score from the students, on average 7.9 on a 10-point scale. In those incidental cases where the students appreciation is far below average, action is undertaken by the Programme Director which leads to agreements on the improvements to be made during the next academic year or to the replacement of the lecturer. The School is very serious in taking the teaching qualifications and experience of academic staff into consideration when recruiting new staff or seeking new guest lecturers. With student numbers growing rapidly, the School has chosen in the current academic year to involve last year s PhD fellows in the tutoring of students in some courses. This has led to the request by some of the PhD fellows for training courses in didactics. This demand will be met, ensuring that also the PhD fellows who are involved in teaching the students perform to the required level. The School will use the BSK/SKO training (Basic Qualification Courses for Academic Teaching) organized by the university for this purpose. Assessment The committee assessed the quality of the staff at the MPP programme and concluded that they are adequately qualified to ensure that the learning outcomes regarding content, didactical quality and organisation of the programme are achieved. The committee verified that the didactical quality of the staff members is an important matter at Maastricht University and that facilities are present to foster it. This was confirmed by the outcome of the course evaluations. Course evaluations and evaluation meetings are means by which the quality of the lecturers is verified. The committee observed that the programme undertakes action which leads to agreements on improvements, if these evaluations are below average. Staff that originally worked for another faculty have their annual evaluation meeting with an executive of their own faculty. The committee established that this makes it harder for the School to monitor the quality of those lecturers. On the other hand, it is easier for the School to replace external lecturers who perform poorly. Another advantage that comes with the appointment of staff from different disciplines, according to the committee, lies in the fact that the lecturers work a lot with peers of different disciplines, which provides them with new insights and motivation. During the site visit the committee established that many contacts between lecturers are organized by the School, for example at PhD defences. The committee was impressed by the commitment of the lecturers, especially taking into account that they are mostly from other faculties and universities. It established that all lecturers are closely involved with the students and are highly motivated to deliver a great job. This was confirmed by the students, who are generally positive about the quality and educational competencies of their lecturers. The committee noted that part of this commitment is realized by the organization of the programme. Because the courses are taught in four-week modules, lecturers have to be well prepared beforehand and heavily involved during the course. In addition, the committee observed that the Programme Director regularly meets with lecturers from other faculties, to keep in touch with their ideas and wishes. The committee is convinced of the fact that the School wants to ensure that lecturers of other faculties also feel part of the School and contribute to the achievement of the aims and objectives of the programme. 38 QANU / Public Policy and Human Development, Maastricht University

39 The committee established that the didactical skills of new teaching staff are trained if they choose to obtain the Basic Qualification Courses for Academic Teaching training (BKO). The committee advises the School to consider making this training mandatory for both new and established staff members. This will also contribute to the possibilities of career development for the core staff of the MPP programme. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. Assessment of the theme Staff The committee comes to an overall assessment of the theme Staff on the basis of its assessments of the separate standards. In the case of the master s programme, it assesses this theme as satisfactory Services S15: Facilities Housing and facilities are adequate to achieve the learning outcomes. Description The self-evaluation report describes how the School building was specially renovated and equipped to house a graduate school. It has three lecture halls, for a capacity of 56 students each, and five seminar rooms, each with a capacity of approximately 20 persons. It also houses a computer lab with 24 workplaces, a study room with 30 seats, and several coffee corners and meeting places. All seminar rooms and lecture halls are fully equipped with interactive beamers, computers and internet access. Two of the rooms are fully equipped with video-conference facilities, and the building has a high-speed wireless internet access system. The PhD fellows, academic staff and other researchers have their offices throughout the building which stimulates interaction between the master students and the researchers. For larger lecture halls and more facilities, the MPP programme has access to the nearby buildings of other units of Maastricht University, especially the Faculty of Law and the Maastricht University School of Business and Economics. The School makes use of a university-wide electronic learning environment called ELEUM. It supports educational activities by offering a collection of information and communication tools for students as well as staff. It is an interactive platform that informs about modules, timetables, and tests and is used to distribute course material. Additionally, it provides the opportunity to exchange assignments and discuss topics. ELEUM can be accessed from any internet PC, making it an important service feature especially for the students who are currently completing their study abroad. The University Library is accessible seven days a week, with opening hours until midnight during exam weeks. The library building is wirelessly connected and can seat up to 840, of which 370 places are equipped with computers. It contains a large and constantly updated book collection of over half a million items. The library also provides remote access to more than 12,000 online and full-text available academic journals, numerous software packages, and a myriad of databases. Assessment The committee received a guided tour to gain an impression of the available facilities. It concluded that the buildings and facilities of the School are eminently suitable to achieve the learning outcomes and that the School pays close attention to ensuring that the facilities are QANU / Public Policy and Human Development, Maastricht University 39

40 made available to its students. The committee concluded that there are currently enough classrooms, computer rooms and study places available. The committee also confirmed that there are adequate ICT facilities, including an electronic work environment. Students can use the computers and wireless internet during opening hours. During the site visit, it became clear that the students are more than satisfied with the facilities and the number of computers, except for the fact that the computers sometimes are a little slow. Regarding this, the School has already taken action by planning on replacing the computers with new ones. The committee greatly appreciates the fact that all of the facilities are located in one building. The committee is deeply impressed with the condition of the building and classrooms. This is the first time it saw a place with facilities based on the vision and learning outcomes of a programme in such a successful way. The committee verified that the building is indeed specially equipped to house this graduate school, and also perfectly fits the didactical approach of problem-based learning. The building offers lecture rooms in all sizes, and students can easily work together on assignments in one of the many rooms. The committee was also impressed by the room at the top of the building, where students can work in silence. Students frequently use and appreciate all the different rooms. According to the committee, the School succeeded in creating a community of committed staff and students, partly thanks to the facilities. The facilities are set up to create an interaction between staff and students and stimulate contact between all those involved. For example, the coffee corners are strategically placed at different spots in the building, where corridors in which different groups are housed converge. In this way, the PhD students, master students, support staff and lecturers meet regularly on an informal basis. In addition, the School succeeded in creating a place in which students feel at home. The committee established that all of the students are very enthusiastic about the building and only complain about the times that the building closes at 18:00 hours. As stated before, the committee would like to point out that if the number of students increases, the facilities should increase accordingly. The committee established that the School is aware of this, and is well informed about, for example, the possibilities to offer courses in other buildings. The committee considers it a challenge to maintain the extremely high standard of the facilities at this moment. Master s programme in Public Policy and Human Development: the committee assesses this standard as excellent. S16: Tutoring Tutoring and information provision for students are adequate in view of study progress. Tutoring and information provision for students correspond with the students needs. Description At the university level, career service officers give individual advice on continuing to study or give training in labor market issues. The Student Service Centre provides students with general information, for instance with regard to housing, financial issues or career advice. Special guidance, for example in case of illness is given by student deans with access to a broad range of information resources. Moreover, student psychologists may be consulted for personal problems. The Student Affairs Office provides the students with general information, brochures, transcripts, and fulfils a broad range of cross-functional tasks. All information that students need for their studies is available on the electronic platform ELEUM (see standard 15). The monthly E-Gazette keeps the students informed about 40 QANU / Public Policy and Human Development, Maastricht University

41 activities. The weekly UM Magazine Observant gives notice about educational activities, such as block openings, examinations, internships, and information sessions. The small-scale teaching within the MPP programme enables teaching staff, supporting staff and students to interact closely. The primary contact persons for study support are tutors, teachers, specialization and thesis coordinators, and the MPP Programme Director. Students are also closely monitored regarding their course and exam performance. Students who regularly fail courses are invited to discuss their lack of progress and possible solutions. At the moment, the School is of opinion it will have to organize the student counselling more professionally, as it is expanding rapidly and the demand for these services is growing. The School is considering to organize a career office, assisting MPP students to find their way into the labour market after graduation. Assessment The committee studied the ways in which the MPP programme arranges tutoring and information provision for students. It concluded that adequate tutoring is provided in view of study progress, and correspond with the students needs. The committee appreciates the School s effort to provide information to students by means of information meetings, individual interviews and weekly and monthly magazines. This ensures that students are well informed. The committee views favourably the efforts of lecturers and support staff to interact closely with students and discuss their study progress. The committee noted that students feel free to contact the staff at any time, and the staff is well aware of the individual students, their problems and needs. In addition, it established that when students fall behind, this is noticed immediately and at that point, lecturers or coordinators inform the support staff. This keeps the support staff aware of all problems that occur, so they can act when necessary. The committee applauds this way of tutoring. If the number of students grows, the committee advises expanding the core staff and formulating procedures to facilitate the contact between students and staff concerning study progress. The specializations, each having its own coordinator, decrease this problem because the groups are smaller. Master s programme in Public Policy and Human Development: the committee assesses this standard as good. Assessment of the theme Services The committee comes to an overall assessment of the theme Services on the basis of its assessments of the separate standards. In the case of the master s programme, it assesses this theme as satisfactory Internal quality assurance system S17: Periodical evaluations The curriculum is periodically evaluated in the light of verifiable objectives and other measures. Description The self-evaluation reports noted that the School strives for high standards of education, and places great emphasis on how the teaching is perceived by its students. The formal feedback is organized within the Educational Committee, in which evaluation and intended actions for improvements are discussed. The Educational Committee is formally represented by four students, the Programme Director, the Dean of the school, and the programme and logistics coordinators. The formal and informal discussions lead to clearly set objectives for QANU / Public Policy and Human Development, Maastricht University 41

42 improvement and to ongoing discussions about the success or failure of the innovations in the MPP course programme. The written and oral feedback from the students on the curriculum and each of the courses is of great value for the quality assurance of the programme. In addition, as a part of the management information system of the School, the IWIO system focuses exclusively on the regular evaluations done by students. At the end of each course, evaluation sheets with 20 to 60 questions are given to students, asking for their feedback. These questionnaires are carefully designed according to essential parameters of the objectives. The results are reported by the Student Affairs Office and forwarded to the lecturers, the Programme Director and also the Educational Committee for discussion and recommendations for action. Apart from regular written evaluations of the courses, the Programme Director holds formal and informal discussions with students in order to get an impression of how the students experience and perceive the separate courses and the curriculum as a whole. These discussions are particularly useful for decisions on actions to be taken. In addition, the cohorts of students are relatively small. That means that the informal feedback from the students to the staff is very important to the MPP programme. Assessment The committee studied the periodic evaluations conducted at the MPP programme and the system behind it. It concluded that the curriculum is evaluated periodically according to verifiable objectives, also reflecting past measures. The MPP programme makes sure that both courses and the curriculum are evaluated on a regular basis according to set guidelines. The committee noted that the School does a great job of evaluating courses in which different groups of staff and students are involved. Because the School is a small-scale organization, immediate feedback is provided as a result of course evaluations that will lead to improvement. The evaluation at the curriculum level, during which an entire programme or specialization is evaluated, could be improved since it is not done regularly. The committee advises evaluating the curriculum as a whole more often. The committee noted that an Educational Committee is present which is responsible for evaluating the courses and the programme. This committee advises the Programme Director about educational issues of the MPP programme and about the Teaching and Examination Regulations (OER). The Educational Committee is actively involved and influential in the quality assurance system. However, the committee deems it desirable for the Educational Committee to consider its composition. By making the programme management part of the Programme Committee, it is not able to perform its verifying and advisory function as independently as it should. Both students and staff indicated that the students are heard, but the Programme Committee in its current composition will hardly give unsolicited advice to the programme management. The School endorses this friction of roles and plans to change the composition of the Educational Committee as soon as the core staff is extended. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S18: Measures for improvement The outcomes of the evaluation form the basis of verifiable measures for improvement that contribute to the achievement of the objectives. 42 QANU / Public Policy and Human Development, Maastricht University

43 Description The MPP programme has only existed for four academic years. The initial curriculum was designed using the input of US APPAM schools and of staff members from the other departments of Maastricht University. Every year, small changes have been made following on the students comments and the lecturers reports. These adjustments were discussed with the Programme Director before implementation and were evaluated afterwards. The specializations have been revised on some minor points by the coordinators. In the next academic year (2010/2011) a major change will be made in the curriculum of the first semester (see table 2b). This change was inspired by feedback from both students and teachers and is considered a major improvement on the course outline. As stated before, new specializations and cooperation with the Maastricht School of Management are being considered (see standard 4). Assessment The committee studied the way in which measures for improvement are dealt with by the School. It concluded that the evaluation procedures are the basis for verifiable measures for improvement, contributing to the achievement of the learning outcomes. The committee also ascertained that the results of evaluations give rise to improvement measures when necessary. The outcomes of the evaluations are taken seriously and are discussed with lecturers and students. When a lecturer underperforms, the School reacts adequately. The committee is convinced that improvements are implemented as a result of the evaluation process. During the site visit, the committee established that students notice the improvements as a consequence of their feedback. Examples that students came up with were related to the spread of the workload (see standard 7) and the number of hours the curriculum offers to prepare for courses. Most suggestions are dealt with quite quickly. The committee finds it remarkable that changes are implemented within four months, and appreciates the pace with which the staff reacts to the evaluations. The committee has its doubts about the achievements of the Educational Committee, and feels that the School could improve the role of the Educational Committee by changing its composition (see standard 17) and strengthen its influence by formalizing procedures. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S19: Involvement of staff, students, alumni and the professional field Staff, students, alumni and the relevant professional field will be actively involved in the internal quality assurance system. Description According to the self-evaluation report, students and the staff are actively involved in the functioning of the internal quality assurance system (see standard 17). The relevant professional field is automatically involved because several lecturers are both active as professionals and as academics. Apart from that, there are regular meetings held with professionals in the specializations of the programme. This involves contact with international organizations, national and supranational government bodies, research institutes, NGOs and consultancy companies. In addition, the staff and researchers are actively involved in commissioned research, students are stimulated and assisted to cooperate with professional organisations during internships and the student association Demos and the alumni association are active in organizing trips and events enabling the students to meet professionals. QANU / Public Policy and Human Development, Maastricht University 43

44 The School has active contact with its alumni, every two years an alumni day is organized. Remarkably enough, many of the alumni come back to the School for that day, even if it means they have to fly in from other continents. These alumni days are valuable occasions to discuss the new developments within the School and the MPP programme as well as to listen carefully to the experiences of the alumni and their recommendation about the programme. Alumni are also invited to play a role in informing the MPP students still at the School. Assessment The committee studied how the staff, students, alumni and professional field are involved within the School and its quality assurance system. It concluded that the staff and students are actively involved in the internal quality assurance system, partly because of the great atmosphere in which informal contacts take place. The committee states that the level of involvement and interaction of all members is approaching the point of excellence. The committee noted that students have different ways to influence the quality of their education, for example through the course evaluation, the Educational Committee, seminars, and directly by informal contacts with staff. The committee finds that the contacts between the Educational Committees and their student constituency run primarily through their informal contact network. The committee states that the involvement of alumni and the professional field does not focus strongly on quality assurance. During the site visit, alumni indicated that they are being approached on a regular basis. Nevertheless, the committee observed that alumni are not involved with the quality assurance in a standardised way. The committee appreciates that the School tries to stay in touch with its alumni and succeeds in doing so, but it feels that this effort could be better structured. Contacts with the professional field do exist, mostly on an informal basis and involving staff members who also work in the professional field. The committee is in favour of formalising contacts with the professional field. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. Assessment of the theme Internal quality assurance system The committee comes to an overall assessment of the theme Internal quality assurance system on the basis of its assessments of the separate standards. In the case of the master s programme, it assesses this theme as satisfactory Results S20: Achieved learning outcomes The achieved learning outcomes correspond with the aims and objectives regarding level, orientation and subject- /discipline-specific requirements. Description The self-evaluation report states that the quality and level of the final thesis can be used as a proxy for indicating the quality of the end-products delivered by the students. The thesis writing process is very strictly structured, especially in the beginning. Already in November/December students are given information sessions and a thesis manual specifying deadlines that have to be met. Step-by-step the students select a thesis topic related to their specialization, formulate a problem statement, search through the main literature, develop an analytical approach, find a supervisor (senior staff), and finally construct a thesis outline. This training process guarantees that students can actually start with their research at the beginning of the second semester. 44 QANU / Public Policy and Human Development, Maastricht University

45 The thesis is graded by the supervisor and a second examiner. The second examiner is assigned by the department in consultation with the thesis supervisor. Subject matter, reasoning and method together constitute the partial grade for content, which forms 80% of the final grade. Appearance and structure together constitute the partial grade for form, which constitutes 20% of the final grade. Students with a grade of 5.5 and higher for both partial grades have to defend the final thesis before the supervisor and the second examiner. The examiners supply a so-called exam scrip, with the final grade and a short report. Assessment The committee assessed the achieved learning outcomes by inspecting a selection of the master theses from the programme. It made a random selection of 11 from the list of the 25 most recently completed theses and also received the associated assessment forms. Consideration in selecting the theses was given to the grading (low, average and high grade). The committee members read the theses, and assessed their presentation of the problem and review of the literature, methods and justification, conclusion and discussion, structure, legibility and verification. It found that almost all theses met the aims and objectives regarding level, orientation and subject/discipline-specific requirements. In addition, the committee appreciated the fact that most students find a permanent job immediately after graduation. In general, the committee agreed with the grades awarded by the supervisors. According to the committee, the grading of the theses is fair and reflects the differences in the dissertations. However, theses with low grades tend to be graded too generously according to the committee. In contrast, some theses with high grades were of such quality that they could be published. According to the committee, the contrast between theses with low and high grades is quite large, especially regarding the use of methodology and empirical research. The theory and the empirical foundation were sometimes absent or not explained sufficiently, or the selected methodology was not justified. The committee stresses that including empirical research in the thesis is preferable, so that students can display their research skills and methods. The committee was pleased to observe that most theses are well written and structured, and complements the School on the writing style and the presentation format. Although the overall quality of the theses is satisfactory, the committee had expected better results, taking into consideration the quality of the School and the students. The committee suspects that the quality of the theses might increase from now on, because of the improvements the School has implemented in the supervision procedure. Alumni noticed that the way supervision is arranged now has visibly improved and will support students in writing theses of high quality. In the past, students could choose their own thesis subject and had to find a supervisor with knowledge of this subject. Although the committee is positive concerning this freedom of choice, it established that this did not always work out, which is not unusual given the number of lecturers and disciplines. The committee observed that the School currently tries to steer students towards subjects that lecturers cover. Students have to write a proposal and hand this in to the coordinator of the specialization in question. The coordinator will then decide which of the staff members should be the supervisor. The committee applauds this progression. Nevertheless, the committee wants to make the School aware that they should stay conscious of the effects of the freedom of choice and the multiple disciplines on the quality of supervising. The committee wants to advise the School to draw a line regarding the knowledge of supervisors on different subjects and disciplines and has to create a clear view of which subjects can and cannot form the basis of a thesis. QANU / Public Policy and Human Development, Maastricht University 45

46 Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. S21: Study progress Target figures that are comparable to other relevant programmes are formulated to express the expected success rate. The programme s success rate complies with these target figures. Description The School believes that the vast majority of the students admitted to the MPP programme should be able to obtain the degree. Table 4 provides an overview of the study progress of the students by cohort. Cohort Percentage of students graduating 12 months 15 months 18 months 24 months Not yet Feb Sept Feb Sept Feb Sept Feb Sept Feb Table 4. Percentage of students by number of months of study at graduation The table illustrates that between 60% and 70% of the students of the September cohorts graduate after 12 months. Around 75% of the students from the MPP programme have graduated after 18 months. Drop outs are unavoidable, but the MPP programme is dedicated to keeping that number as low as possible. Studying the reasons for these delays reveals that it involves postponing writing and finishing the master s thesis. Therefore, the MPP programme experimented successfully in with a more structured and better monitored thesis process (see standard 20). The action has been successful, as approximately 60% of the next cohort, starting in September 2008, graduated after 12 months and 75% after 15 months. Only 1 student dropped out from that cohort. Assessment The committee examined study progress at the MPP programme. It concluded that the School set a target and succeeds in achieving this. Although the committee considers the programme too new to make a well-informed and valid judgement based on the available numbers, the committee finds that the success rates are acceptable. The committee established that alumni and students confirm that it is feasible to finish the thesis within the set time. Almost all students with a delay chose it deliberately. Reasons behind the delay are the wish to deliver a thesis of outstanding quality or the decision to combine the programme with an internship or a job. Master s programme in Public Policy and Human Development: the committee assesses this standard as satisfactory. Assessment of the theme Results The committee comes to an overall assessment of the theme Results on the basis of its assessments of the separate standards. In the case of the master s programme, it assesses this theme as satisfactory. 46 QANU / Public Policy and Human Development, Maastricht University

47 Overview of the committee s assessment Master s programme in Public Policy and Human Development: Theme Assessment Standard Assessment 1. Aims and Satisfactory 1. Subject-/discipline-specific Good objectives requirements 2. Bachelor and master level Satisfactory 3. Academic orientation Satisfactory 2. Curriculum Satisfactory 4. Requirements for academic orientation Satisfactory 5. Correspondence between the aims and Good objectives and the curriculum 6. Consistency of the curriculum Satisfactory 7. Workload Satisfactory 8. Admission requirements Good 9. Credits Complies 10. Coherence of structure and contents Good 11. Learning assessment Satisfactory 3. Staff Satisfactory 12. Requirements for academic orientation Satisfactory 13. Quantity of staff Satisfactory 14. Quality of staff Satisfactory 4. Services Satisfactory 15. Facilities Excellent 16. Tutoring Good Satisfactory 17. Periodical evaluations Satisfactory 18. Measures for improvement Satisfactory 5. Internal quality assurance system 6. Results Satisfactory 19. Involvement of staff, students, alumni and the professional field Satisfactory 20. Achieved learning outcomes Satisfactory 21. Study progress Satisfactory The committee s overall assessment of the master s programme in Public Policy and Human Development The committee concluded, on the basis of its assessments of the themes and standards from the assessment framework, that the master s programme in Public Policy and Human Development fulfils the formal requirements which are a prerequisite for accreditation. QANU / Public Policy and Human Development, Maastricht University 47

48 48 QANU / Public Policy and Human Development, Maastricht University

49 APPENDICES QANU / Public Policy and Human Development, Maastricht University 49

50 50 QANU / Public Policy and Human Development, Maastricht University

51 Appendix A. Elaborated learning outcomes Knowledge K1. An advanced knowledge of the functioning of the policy-making in the context of market economies, emerging markets and developing countries; K2. An advanced knowledge of theories relevant for policy analysis and for the chosen specialization K3. A strong grasp of methods aimed at evaluating both ex ante and ex post policy interventions; K4. An understanding of connections between the various disciplines relevant for policy analyses; K5. A strong grasp of approaches to recognize, analyze and solve problems related to public and valuation and decision-making; K6. An genuine understanding of differences in the context (national, political, economic) of policy analytical problems; K7. Understanding the link between theories and methods and the moral philosophical ideas. Skills S1. Capable of reading and understanding original research articles in the area of policy analysis and in the specialization; S2. Capable of analyzing policies problems in their actual context and within a specialization field; S3. Capable of recognizing links between different theoretical frameworks and stemming from various disciplines; S4. Capable of analyzing policy options using the existing approaches; S5. Capable of adapting the existing approaches for the analysis of specific cases with varying contexts; S6. Capable of mapping and analyzing alternatives to existing policy interventions, understanding that the options have to adapted to the local situation or the historical situation at hand; S7. Capable of assessing, quantitatively as well as qualitatively, the consequences of different policy options; S8. Capable of reporting in English both orally and in written; S9. Capable of presenting their results to both academic audience and to wider audiences; S10. Capable of understanding and respecting cultural and moral differences in audiences; S11. The ability to search for relevant academic and professional contributions and to relate specific cases to the existing literature on the subject. The ability to search for and analyze relevant data; S12. Capable of learning new theories and methods and applying them outside their usual field; S13. Capable of formulating a (policy) research problem in a way that it can be analysed. QANU / Public Policy and Human Development, Maastricht University 51

52 52 QANU / Public Policy and Human Development, Maastricht University

53 Appendix B: Curricula vitae of the committee members Professor P. (Paul) Verweel studied anthropology at Utrecht University and graduated in 1983, in 1987 he completed his PhD study in social sciences (Title of thesis: University, change and planning).verweel became professor in 2000 and full professor in Since October 2007 he chairs the Richard Kraijeck endowed chair for Governance. He is head of department and director of education for the Utrecht School of Governance, of the Faculty of law, economics and governance of Utrecht University. Paul Verweel sits on the boards of many organizations in the public domain and has published widely on the subjects of Governance, the dynamics-, theories and cultures of organizations and on the public values of sports (and minorities). Professor A. (Armando) Barrientos is professor and Research Director at the Brooks Wolrd Poverty Institute at the Universityof Manchester. He gained his BA and PhD from the University of Kent at Canterbury in the UK. He was formerly a Research Fellow at IDS, University of Sussex, and Senior Lecturer at IDPM in Manchester. He is also a Senior Researcher with the Chronic Poverty Research Centre, leading research on Insecurity, Risk and Vulnerability. His research interests focus on the linkages existing between social protection and labour markets in developing countries, and policies addressing poverty, vulnerability, and population ageing. His articles have appeared in journals such as World Development, Applied Economics, The Manchester School and Geneva Papers in Risk and Insurance, and is the author of Pension Reform in Latin America. He has acted as an adviser to the ILO, the World Bank, DFID, UNRISD, UN-DESA, UNICEF, IADB, WIEGO, and the Caribbean Development Bank Mr. B. (Bob) Paulussen BSc is the student member of the committee. At this moment, he is student of the Utrecht School of Governance (bachelor s programme, Utrecht University. He is starting his master in Public Administration at the Utrecht School of Governance in september. His interest is mainly in the dynamics within the public sector. During his study he worked for a consultancy firm in The Hague and organized an international course for students all over Europe. QANU / Public Policy and Human Development, Maastricht University 53

54 54 QANU / Public Policy and Human Development, Maastricht University

55 Appendix C: Programme of the site visit at Maastricht University Day 1 09:00 13:30 Preparatory meeting / inaugural meeting 13:30 14:00 Lunch break 14:00 15:00 Meeting with management and staff responsible for the degree programmes 15:00 15:45 Students (6 to 8) 15:45 16:00 Break 16:00 16:45 Lecturers (10 at most) 16:45 17:30 Alumni (6 tot 8) 19:00 21:30 Dinner with committee Day 2 09:00 09:30 Student members Educational Committee 09:30 10:00 Staff members Educational Committee 10:00 10:30 Examination Board 10:30 11:00 Round tour facilities 11:00 11:30 Office hour 11:30 12:00 Internal meeting committee 12:00 13:00 Lunch break 13:00 14:00 Second meeting with and feedback to management of the programme 14:00 16:00 Internal committee meeting preparation of conclusions and presentation 16:00 16:30 Presentation of initial results 16:30 17:00 Reception QANU / Public Policy and Human Development, Maastricht University 55