Assessment of the degree programmes in Public Administration at Erasmus University Rotterdam

Assessment of the degree programmes in Public Administration at Erasmus University Rotterdam QANU & EAPAA / Public Administration, Erasmus University Rotterdam / March 2011 / Q252

Quality Assurance Netherlands Universities (QANU) Catharijnesingel 56 P.O Box 8035 3503 RA Utrecht The Netherlands Phone: +31 (0)30 230 3100 Fax: +31 (0)30 230 3129 E-mail: info@qanu.nl Internet: www.qanu.nl European Association for Public Administration Accreditation (EAPAA) P.O. Box 217 7500 AE Enschede The Netherlands Phone: +31 (0)53 483 6346 Fax: +31 (0)53 483 6347 E-mail: secretariat@eapaa.org Internet: www.eapaa.org 2011 QANU/EAPAA 2 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

TABLE OF CONTENTS Part I: General Part 5 1. About this report 7 2. Task and composition of the assessment committee 7 3. Working method of the assessment committee 10 4. Domain-specific requirements Public Administration, Public Governance, and Governance and Organization (PAGO) Programmes, 2010 13 Part II: Programme Report 19 1. Report on the bachelor s programme Public Administration and the master s programme Public Administration at Erasmus University Rotterdam 21 Appendices 67 Appendix A: Curricula vitae of the members of the assessment committee 69 Appendix B: Composition of the assessment committtee per site visit 73 Appendix C: Programme of the site visit to Erasmus University Rotterdam 75 Appendix D: Joint QANU-EAPAA assessment framework 77 Appendix E: List of materials studied by the committee during the site visit 83 QANU & EAPAA / Public Administration, Erasmus University Rotterdam 3

4 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

PART I: GENERAL PART QANU & EAPAA / Public Administration, Erasmus University Rotterdam 5

6 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

1. About this report In this document, the Public Administration 2010 assessment committee reports its findings. The report consists of two parts: a general part and a programme report. The assessment committee assessed fifteen degree programmes offered by seven universities in the Netherlands. Its findings are laid down in separate reports for the various universities. The assessment was jointly organized and conducted by QANU (Quality Assurance Netherlands Universities) and EAPAA (the European Association for Public Administration Accreditation). The purpose of the co-operation between QANU and EAPAA was to enable programmes to apply for accreditation both at national level and at European level. The general part, which is the same in every report produced by the assessment committee, summarizes the tasks, the composition, the documentation and the working methods used by the committee. This part of the report also contains the domain-specific requirements for Public Administration, Public Governance and Governance and Organization Programmes that were used by the assessment committee. The programme report describes the evaluation and assessment of the programmes offered by a specific university. The programme part is structured in accordance with the accreditation criteria of the NVAO (the Accreditation Organization of the Netherlands and Flanders) and EAPAA. 2. Task and composition of the assessment committee 2.1. The task of the assessment committee The task of the Public Administration 2010 assessment committee was to evaluate and assess fifteen degree programmes at seven universities in the Netherlands on the basis of an assessment framework which incorporates the accreditation criteria defined by the NVAO (the Accreditation Organization of the Netherlands and Flanders) and EAPAA (the European Association for Public Administration Accreditation). The assessment committee was expected to assess relevant aspects of quality of the programmes involved on the basis of information provided by the faculties and programmes in self-evaluation reports and of the outcomes of discussions held with representatives of various target groups during the site visits. The assessment committee has been requested to assess the following programmes (including the Dutch name when applicable, the modes of study and the CROHO registration number): Utrecht University Bachelor s programme Public Administration and Organization Science (Bestuurs- en organisatiewetenschap, full time, 50007) Master s programme Public Administration and Organization Science (Bestuurs- en organisatiewetenschap, full time and dual, 60446) Tilburg University Bachelor s programme Public Administration (Bestuurskunde, full time, 56627) Master s programme Public Administration (Bestuurskunde, full time, 66627) QANU & EAPAA / Public Administration, Erasmus University Rotterdam 7

Leiden University Bachelor s programme Public Administration (Bestuurskunde, full time, 56627) Master s programme Public Administration (full time, 60020) Radboud University Nijmegen Bachelor s programme Public Administration (Bestuurskunde, full time, 56627) Master s programme Public Administration (Bestuurskunde, full time, 66627) Erasmus University Rotterdam Bachelor s programme Public Administration (Bestuurskunde, full time, 56627) Master s programme Public Administration (full time, 60020) VU University Amsterdam Bachelor s programme Public Administration and Organization Science (Bestuurs- en organisatiewetenschap, full time, 50007) Master s programme Public Administration (Bestuurskunde, full time and part time, 66627) University of Twente Bachelor s programme Public Administration (Bestuurskunde, full time, 56627) Master s programme Public Administration (full time, 60020) Master s programme European Studies (full time, 69303) 2.2. The composition of the assessment committee The assessment committee consisted of four chairs and twelve members. The site visits were conducted by a chair and four additional committee members. The list below contains the names of all members of the assessment committee and specifies in which assessments each of them participated. Appendix A lists short curricula vitae of the committee members. Chairs: prof. dr. C. (Christoph) Reichard, emeritus professor of Public Management, Universität Potsdam (DE), for the assessment of the programmes of Leiden University, Radboud University Nijmegen, VU University Amsterdam and the University of Twente; prof. dr. J.J. (Jaap) Boonstra, professor of Organizational Change and Learning at the University of Amsterdam, professor of Organizational Dynamics at Esade Business School in Barcelona, for the assessment of the programmes of Utrecht University; prof. dr. J.A. (Hans) de Bruijn, professor of Public Administration at Delft University of Technology, for the assessment of the programmes of Tilburg University; prof. dr. H. (Harald) Sætren, professor of Administration and Organization Theory, Universitetet i Bergen (NO), for the assessment of the programmes of Erasmus University Rotterdam. Members: prof. dr. J. (Juraj) Nemeç, professor of Public Finance, Univerzita Mateja Bela (SK), for the assessment of the programmes of Utrecht University, Tilburg University, Leiden University, Radboud University Nijmegen and Erasmus University Rotterdam; 8 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

prof. dr. J. (John) Loughlin, professor of European Politics at Cardiff University (UK), as of 1 October 2010 Fellow and Affiliate Lecturer in Politics at St Edmund's College, Cambridge University (UK), for the assessment of the programmes of Utrecht University and Tilburg University; prof. dr. T. (Tony) Bovaird, professor of Public Management and Policy, University of Birmingham (UK), for the assessment of the programmes of Leiden University and Radboud University Nijmegen; prof. dr. H. (Harald) Sætren, professor of Administration and Organization Theory, Universitetet i Bergen (NO), for the assessment of the programmes of VU University Amsterdam; prof. dr. M. (Michael) Hill, emeritus professor of Social Policy, University of Newcastle upon Tyne (UK) and visiting professor at Queen Mary College, University of London (UK), for the assessment of the programmes of Erasmus University Rotterdam, VU University Amsterdam and the University of Twente; prof. dr. M. (Mirko) Vintar, professor of Informatics in public administration and e- government, Univerza v Ljubljani (SI), for the assessment of the programmes of the University of Twente; mr. drs. A.J. (Arthur) Modderkolk, former director of the Province Noord-Brabant, former secretary of De Open Ankh foundation, for the assessment of the programmes of Utrecht University, Radboud University Nijmegen and the University of Twente; dr. A.A.M. (Louis) Meuleman, secretary/director of The Advisory Council for Research on Ppatial Planning, Nature and the Environment (RMNO), for the assessment of the programmes of Tilburg University and Leiden University; dr. C.J.M. (Kees) Breed, Secretary of the Council for Public Administration and the Council for Financial Relations, for the assessment of the programmes of Erasmus University Rotterdam; drs. H. (Henk) Nijhof, party chairman Groen Links, for the assessment of the programmes of VU University Amsterdam; T. (Tom) Degen, student Public Administration at Leiden University, for the assessment of the programmes of Utrecht University, Tilburg University, Erasmus University Rotterdam and VU University Amsterdam; J. (Janneke) van der Heijden, student Public Administration at Tilburg University, for the assessment of the programmes of Leiden University, Radboud University Nijmegen and the University of Twente. Given the large number of degree programmes to be assessed and the short period during which the site visits were planned, QANU and EAPAA decided at an early stage to work with a pool of committee members. For each site visit, QANU and EAPAA selected a committee from this pool. Several factors were taken into consideration for the composition of the committees, including the independence of the committee members, any potential conflicts of interest, the availability of the committee members, and their content-related expertise. Appendix B contains a list of the composition of the committee at the various site visits. Drs. S. (Sietze) Looijenga, QANU staff member, acted as overall project coordinator of the assessment of the Public Administration programmes. He also served as secretary for the site visit at Tilburg University. Secretary for the site visit at Utrecht University, Radboud University Nijmegen and the University of Twente was drs. L.C. (Linda) te Marvelde, QANU staff member. Secretary for the site visit at Leiden University was drs. R.L. (Reinout) van Brakel, advisor at PWC, on secondment at QANU for this purpose. Secretary for the site visit at Erasmus University Rotterdam was drs. J. (José) van Zwieten, advisor at PWC and on QANU & EAPAA / Public Administration, Erasmus University Rotterdam 9

secondment at QANU. Secretary for the site visit at VU University Amsterdam was drs. R.L. (Renate) Prenen, self-employed advisor. As required by the protocols used by QANU and EAPAA, all members of the assessment committee signed a declaration of independence to establish that they were in a position to judge independently, without any bias, personal preference or personal interest, and that their judgement would be made without undue influence from the institutes or the programmes to be assessed or other stakeholders. The secretaries of the assessment committee signed a declaration of independence as well. 3. Working method of the assessment committee 3.1. General preparations for the assessment In the autumn of 2009, the representatives of the programmes in Public Administration and Organization Science decided that they would use the upcoming assessment of their programmes for a dual purpose: to obtain accreditation both from the NVAO, the Accreditation Organization of the Netherlands and Flanders, and from EAPAA, the European Association for Public Administration Accreditation. They invited Theo van der Krogt, secretary-general of EAPAA, and Sietze Looijenga, deputy director of QANU, to develop a proposal for the assessment to achieve this purpose. Theo van der Krogt and Sietze Looijenga developed an assessment framework which combined the requirements of the NVAO and EAPAA. They used the NVAO s framework for existing degree programmes as a starting point and added standards and criteria for aspects of the programmes which are covered by EAPAA s assessment framework, but which do not play a role in the NVAO s framework. The joint assessment framework consists of six themes and 26 standards, five of which refer to requirements imposed by EAPAA exclusively. The joint assessment framework has been formally approved both by the NVAO and EAPAA. It served as the starting point for the process of self-evaluation conducted by the programmes that were assessed. The self-evaluation reports all follow the structure laid down in the assessment framework. The representatives of the programmes in Public Administration and Organization Science produced a domain-specific framework of reference which was formally approved by EAPAA. The assessment committee used this domain-specific framework as the contentrelated starting point of its assessment. QANU and EAPAA agreed on dividing the task of composing the assessment committee. EAPAA approached experts in the field of public administration and organization sciences from its own network on the basis of a proposal from the participating programmes, while QANU contacted potential student members and representatives of the professional field. Because the site visits were planned in a rather limited period of time, it was not possible to establish a committee which had the same composition at every site visit. Similarly, it was not possible to appoint one secretary who supported the committee during each site visit. Sietze Looijenga, overall project coordinator, was present at all site visits to prepare committee members for their task and to enhance the continuity and consistency of the committee s assessment. 10 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

3.2. Preparations for the site visits Linda te Marvelde, staff member of QANU, checked the self-evaluation reports of the programmes to ensure that they could serve as a starting point for the assessment. She established that all reports fulfilled the relevant criteria of relevance and completeness. QANU s secretariat distributed the self-evaluation reports and the additional information among the appropriate committee members. The committee members were asked to phrase their remarks, comments and questions regarding the self-evaluation report and the additional documents prior to the site visit. In addition to the self-evaluation reports, the members of the committee who participated in a specific site visit read two theses for each programme. In addition, committee members studied additional theses during the site visit. The secretary of the committee selected the theses from the list in the self-evaluation report, making sure that the theses to be distributed represented the full range of marks assigned. The members of the committee received QANU s checklist for the assessment of theses to ensure that their assessments were comparable and took the relevant aspects into account. The committee members paid particular attention to the scientific level and quality of the theses, to the accuracy and transparency of the assessment and to the assessment procedure used. Before each site visit, Sietze Looijenga (and, in most cases, Linda te Marvelde) had a meeting with representatives of the programmes to be assessed to discuss and agree on the programme for the site visit and various practical arrangements. The programme was the same for all site visits, with one exception. In the case of the University of Twente, which offers the master s programme European Studies next to the regular master s programme in Public Administration, the committee had an additional interview with students of this particular master s programme. In all cases, the programme included an office hour. Both staff members and students were informed about this opportunity to speak to the committee confidentially at least one week before the site visit. 3.3. The site visits Every site visit started with a preparatory committee meeting, in which the committee members discussed the self-evaluation report, the additional documentation and the bachelor s and master s theses they had received prior to the site visit. The committee also discussed and agreed on the questions and issues to be raised in the interviews with representatives of the programmes and various groups of stakeholders. In every case, the committee conducted interviews with the board of the faculty which is responsible for the programmes, the management of the programmes (programme director or coordinator, chair of department or institute et cetera), students, lecturers, graduates, representatives of the professional field, members of the Education Committee and the Board of Examiners, and the study advisor(s). In addition, the members of the committee studied additional materials made available by the programmes, including learning materials, written exams, assignments and other assessments, minutes of meetings of the Education Committee and the Board of Examiners, and it explored the electronic learning environment provided by the programmes. After the last interview with the board of the faculty offering the programmes, the committee held another internal meeting, in which it discussed its findings, phrased its conclusions and gave its assessment of the themes and standards making up the assessment framework. Finally, the chairman of the committee presented the committee s preliminary findings. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 11

3.4. After the site visits After each site visit, the secretary of the committee produced a draft version of the report about the programmes which had been assessed and presented it to the members of the committee who had participated in the site visit. The secretary processed all the corrections, remarks and suggestions for improvement provided by the committee members and thus produced the first final draft report. QANU s secretariat sent this draft report to the institution and the faculty offering the programmes, inviting them to check the report for factual errors, inaccuracies and inconsistencies. The secretary forwarded the comments and suggestions provided by the institution and/or faculty to the chairman of the committee, and, if necessary, to the other committee members as well. The (chairman of the) committee decided whether the comments and suggestions were incorporated in the report or ignored. On the basis of the committee s decisions, the secretary compiled the final version of the programme report. 3.5. Assessment of the themes and standards As agreed beforehand by all parties involved, including EAPAA, the assessments for the themes and standards of the joint QANU-EAPAA framework followed the criteria and guidelines laid down in the NVAO s accreditation framework for existing degree programmes. This means that the committee assessed the standards on a four-point scale (unsatisfactory, satisfactory, good, excellent), while it used a two-point scale (unsatisfactory, satisfactory) for the themes. In accordance with the formal framework, the committee interpreted the scale for the standards in the following way: unsatisfactory: the programme does not meet the requirements for basic or generic quality; satisfactory: the programme meets the requirements for basic or generic quality; good: the programme exceeds the requirements for basic or generic quality; excellent: the programme exceeds the requirements for basic or generic quality by far and is a clear example of (international) best practice. The committee used satisfactory as its default score for the standards. 12 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

4. Domain-specific requirements Public Administration, Public Governance, and Governance and Organization (PAGO) Programmes, 2010 Introduction The study of public administration has developed and expanded into a broad interdisciplinary body of knowledge, which tackles a variety of themes and practices on public administration, governance and organization (PAGO). The academic community in the Netherlands acknowledges that throughout the years this field has widened and now includes not only public administration but also governance and organization. This entails a diversity of approaches on the one hand, but on the other, the conviction that these approaches are connected and interrelated and worthwhile to keep together. Programmes may share basic components, but also may differ to express their specialisation in this broadened field. This parallels developments in the profession. Alumni are increasingly challenged in a wide variety of fields that put varying demands regarding professional knowledge, skills and attitudes. In this frame of reference we will address this field as the PAGO-field: including public administration, public governance, and governance and organization. In this domain-specific frame of reference we start with a brief summary regarding the development of the PAGO-field and argue that the broadening of the field is due to various exogenous and endogenous changes. Accordingly we will outline the programme principles of PAGO-studies as well as related learning outcomes. Developments The societal impact of processes like globalization, individualization and ICT has altered the nature of public problems. Issues like risk and security, environment and ecology, economics and welfare, and nationality and culture are high on the societal and political agenda. The impact of such problems has consequences for the abilities of (national) governments. It challenges them to reach beyond traditional approaches. This has led to manifold changes in political and administrative landscapes. New expectations and demands are expressed towards politics and administration, including moral standards. New criteria for performance have emerged that aim at value for money, new businesslike concepts of management, and reformed public service delivery. There have been new interpretations of democracy and accountability, and of relations between state, civil society and the market. Government and public administration not only changed its own practices, it also changed its relationship with society. Public administration thus moved towards governance, i.e. dealing with public problems through dispersed networks of organizations and actors, including social institutions, non-governmental organizations (NGO s), and private companies. Government and public policy are still relevant, but new outlooks and mechanisms are designed and used to make things work. These developments have also changed the field of study of PA. Scholars started to use new concepts to understand developments, broadening categories such as governmentgovernance, and crossing boundaries between the public and private world. These concepts include focused attention to issues like interdependence, ambiguity, networks, contextuality, governance, and the role of institutions, trust and integrity. These developments invited researchers to cross disciplinary borders and take aboard theories, concepts, methods and ideas, from organization studies (structure, culture, management, strategy, networks, et cetera) as well as other bodies of knowledge (new fields within economics, political science and QANU & EAPAA / Public Administration, Erasmus University Rotterdam 13

sociology, communication theory, ethics and philosophy, geography, international relations and law, et cetera). Another issue that needs to be highlighted is that the study of Public Administration in the Netherlands includes several fields that elsewhere are situated in political science. The PAGO-studies not only focus on classical PA issues, but also on public organization and management issues, as well as on subfields like public policy, policy making, public governance, public culture and ethics. Scholars of these issues are part of the broad PA community, in research as well as in educational programmes. Resulting Fields of Study This PAGO-community consists of three fields of study. The first embodies the classical features of the discipline, concentrating on politics, administration and the public sector. Public administration often started within the context of (departments of) politics and/or law, with an emphasis on the study of government and bureaucracy as well as public policymaking and implementation. The second emerged through the fact that public interests and public problems are increasingly tackled by a multitude of public and private actors. It broadened the scope of study to include nongovernmental actors, as part of the often complex public-private, multiactor networks that deal with collective and public interests. The third field focuses on questions of governance and organization that surpass the traditional public-private boundaries. It includes the study of private actors in social contexts. This orientation links the worlds of business administration and public administration and pays attention to what we know about management, strategy and behaviour in corporations. This approach can be labelled as governance and organization. PAGO today is a broad multi- and interdisciplinary field of science. The classical core disciplines of political science, law, sociology and economics are important, and there is an increasing involvement of disciplines that focus on organization, culture, and communication. Also, challenging new interchanges with bodies of knowledge in (for example) social and organizational psychology, planning studies and geography, philosophy and ethics and history have demonstrated added value. The PAGO-community acknowledges that there are different views regarding object and focus of the field of study. For instance: is PAGO about knowledge by description, explanation and prediction, or is evaluation and improvement the prime goal? Or, how do we relate to and communicate with practitioners in public (and private) administration, governance and organization? Rather than excluding certain views, the PAGO-community welcomes a variety in approaches, ideas and outlook. This variety is also visible in the PAGOprogrammes. Defining programme principles PAGO-programmes are academic programmes aiming at the development of academic knowledge, skills and attitude in students that are relevant for understanding public administration, governance and organization. They pay particular attention to social and political contexts and developments, relevant (interdisciplinary) bodies of knowledge, aim at developing research capacities, and contribute to working professionally in public and private domains. In this frame of reference we have listed elements that are to be seen as building blocks for academic programmes. As far as knowledge is concerned, contemporary 14 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

programmes encompass various disciplinary views supporting the PAGO-domain, and various sorts of domain-specific knowledge. As far as skills are concerned, they encompass skills for applying and reflecting on scientific methods and approaches, integrating knowledge and skills for working in public domains/organizations. As far as attitude is concerned, it encompasses critical stances and moral stature. Each of these subfields is briefly elaborated in order to circumscribe specific learning outcomes at Bachelor and Master levels (see next paragraph). Knowledge Knowledge of society and changing contexts Activities in public domains influence, are influenced by, and interact with social systems and developments. On the one hand, they constrain public sectors, as they reproduce values, traditions and culture(s). On the other hand, they call for public action; (new) facts, events and problems, fuelled by new technologies, pose new challenges. PAGO-programmes enhance understandings of social structures and behaviours, societal trends and changes. This calls for an awareness of political, sociological, cultural, historical, philosophical, ethical, economic and judicial contexts. Knowledge of political and administrative systems The organization, processes and activities in public domains are shaped by and within political systems. PAGO-programmes should devote attention to the institutions, structure, organization and activities of such political systems, at different levels (local, regional, national, transnational). PAGO-programmes encompass political and social theories, including those regarding legitimacy and the democratic design and functioning of organizations in public domains. They also pay attention to the application of these theories in everyday practice. Knowledge of (public) policy, decision making and implementation Governance for societal problems includes many insights derived from various bodies of knowledge, ranging from high-level decision-making to everyday service delivery. PAGOprogrammes address both classic and contemporary theories, methods and techniques of policy-making, management, decision-making, and their implementation in everyday practice. Knowledge of organizations and organizing principles Public domains entail a variety of organizations, some organized as classical government bodies, some as between the public and private sectors, while others have been influenced by and/or have taken on the characteristics of private organizations. There is a growing awareness that policies and service delivery must be organized and require well-trained and motivated professionals. This leads to a more explicit emphasis on organizational studies. PAGO programmes entail knowledge of organizational concepts/perspectives on organizing, domains of managerial activities, insights in organizational change and management tools. Knowledge of governance and networks The powers and authorities to intervene have become less governmental and more distributed. Due to organizational fragmentation, the rise of network relations, and the spread of (normative) governance models e.g., joined up government, public-private partnerships, and corporate social responsibility (CSR) multiple parties have become active in dealing with public problems and representing public interests. PAGO programmes pay attention to new relations and new governance regimes, having both theoretical and empirical consequences. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 15

Skills Research skills The role of knowledge in (public) policies and organizations is crucial for its effectiveness, especially for understanding the complexity of contexts, structures, outcomes and behaviours. PAGO-programmes include methods of quantitative and qualitative social-scientific research to analyse and also emphasise a clear understanding of contextual aspects. Integrative skills Public domains can be analysed from different angles; theories are grounded in various disciplines. The quality of research and capacities of civil servants and other functionaries in public domains depend on integrative skills, i.e. abilities to combine, integrate and apply different bodies of knowledge. PAGO-programmes devote attention to and provide opportunities to practice integrative skills. Cooperation and communication skills The functioning of the public domain largely depends on the skills of actors to exchange ideas, to negotiate when necessary, and to cooperate in constructive ways. Civil servants and other functionaries use a repertoire of skills and attitudes to communicate ideas to audiences of experts as well as laymen. Cooperation is at the heart of PAGO and includes a sense of responsibility and leadership. PAGO-programmes devote attention to and provide opportunities to practice cooperative and communicative skills. Attitude Critical stances PAGO programmes are academic programmes that not only facilitate cognitive learning and skill development, they also develop critical powers. Students are taught how to critically analyze arguments used by others, how to relate fashionable statements, e.g. by politicians, to more traditional as well as to scientific insights, and how to reflect upon political and normative implications of policy choices and organizational design. PAGO-programmes devote attention to the development of a constructive, critical attitude. Moral stature and professionalism The eloquence and credibility of PAGO has two features. First is its ability to approach societal problems in effective ways, but second is the degree to which government and governance principles serves as a moral compass. PAGO-programmes train students in this respect for occupying positions in governance regimes (public and private), they also train students in developing appropriate or professional conduct. This is a matter of guarding values, such as accountability and integrity, and of practicing values, such as entrepreneurship and innovation. Academic learning outcomes for PAGO studies The broad fields identified and circumscribed in the above are to be seen as programme criteria and, thus, as the building blocks of a programme. Each programme will emphasize a specific selection of these building blocks to impose specific learning outcomes on students. In the table below we list such learning outcomes. This is a generic list, both applicable for bachelor and master programmes. The difference between both studies is in the degree of complexity; in the level of analysis; and in the independence of the student. Here we follow the distinctions made in the so-called Dublin descriptors. In this system a distinction is made between first cycle learning for bachelors and second cycle learning for masters. First cycle learning involves an introduction 16 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

to the field of study. It aims at the acquisition and understanding of knowledge, ideas, methods and theories, elementary research activities, and basic skills regarding communication and learning competences. At second cycle learning we find a deeper understanding of knowledge; problem solving skills are developed for new and unexpected environments and broader contexts. Here students can apply knowledge in various environments. At the master level we also expect a well-developed level of autonomy regarding the direction and choices in a study. In generic bachelor PAGO-programmes most of the learning outcomes will apply that are listed below. Master programmes, however, usually have a much stronger thematic focus and may especially focus on a particular set of these learning outcomes that are best suited for that specialisation, but not covering all the learning outcomes listed below. We propose that the learning outcomes for the bachelor level, apply for the master level in the sense that students demonstrate that they are capable of: dealing with increased situational, theoretical and methodological complexity; demonstrating increased levels of autonomy and self-management; applying ideas, methods, theories in research and problem solving; mastering the complexity that is inherent to the field of specialisation. In the table below we have organized the learning outcomes according to the Dublin descriptors. We present the main components of the Dublin descriptors in italics, and accordingly the proposed learning outcomes. Knowledge and understanding 1 (Bachelor) [Is] supported by advanced text books [with] some aspects informed by knowledge at the forefront of their field of study 2 (Master) provides a basis or opportunity for originality in developing or applying ideas often in a research context (Basic) knowledge of (changing) societal contexts (Basic) knowledge and understanding of the distinctive nature of organization, policy making, management, service delivery and governance in PAGO domains (Basic) awareness of political traditions and politics (Basic) knowledge and understanding of the discipline, PAGO-paradigms, intellectual tradition, theories and approaches (Basic) knowledge and understanding of multi-actor and multi-level concepts A general (basic) understanding regarding the dynamics and processes of actors in public domains, how these processes influence society and vice versa Applying knowledge and understanding 1 (Bachelor) [through] devising and sustaining arguments 2 (Master) [through] problem solving abilities [applied] in new or unfamiliar environments within broader (or multidisciplinary) contexts (Basic) capacity to work at different levels of abstraction (Basic) skills in problem definition and problem solving in the PAGO domain (Basic) ability to distinguish normative preferences and empirical evidence (Basic) skills in combining, integrating and applying knowledge QANU & EAPAA / Public Administration, Erasmus University Rotterdam 17

(Basic) insight into the scientific practice (Basic) capacity to select a suitable theoretical framework for a given empirical problem (Basic) skills in combining normative and empirical aspects (Basic) capacity to build arguments and reflect upon the arguments of others (Basic) awareness of relevant social, ethical, academic and practical issues Making judgments 1 (Bachelor) [involves] gathering and interpreting relevant data 2 (Master) [demonstrates] the ability to integrate knowledge and handle complexity, and formulate judgements with incomplete data (Basic) ability to formulate research questions on problems in the PAGO-domain (Basic) knowledge regarding research on social-scientific positions and thinking (Basic) training in and application of quantitative, qualitative and mixed methods social science research (Basic) abilities to collect data and to derive judgments thereof Communication 1 (Bachelor) [of] information, ideas, problems and solutions 2 (Master) [of] their conclusions and the underpinning knowledge and rationale (restricted scope) to specialist and non specialist audiences (monologue) (Basic) capacity to use argumentative skills effectively (Basic) capacity to function in multi- and interdisciplinary teams in several roles (Basic) capacity to function effectively in governance, organization, management, policy and advocacy settings (Basic) capacity to use communicative skills effectively in oral and written presentation Learning skills 1 (Bachelor) have developed those skills needed to study further with a high level of autonomy 2 (Master) study in a manner that may be largely self-directed or autonomous Learning attitude (Basic) capacity to reflect upon one s own conceptual and professional capacities and conduct 18 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

PART II: PROGRAMME REPORT QANU & EAPAA / Public Administration, Erasmus University Rotterdam 19

20 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

1. Report on the bachelor s programme Public Administration and the master s programme Public Administration at Erasmus University Rotterdam Administrative data Bachelor s programme Public Administration: Name of the programme: Public Administration CROHO number: 56627 Level: bachelor Orientation: academic Number of credits: 180 EC Degree: Bachelor of Science Mode(s) of study: full time Location(s): Rotterdam Expiration of accreditation: 2 April 2012 Master s programme Public Administration: Name of the programme: Public Administration CROHO number: 66627 Level: master Orientation: academic Number of credits: 60 EC Degree: Master of Science Mode(s) of study: full time Location(s): Rotterdam Expiration of accreditation: 2 April 2012 The site visit of the assessment committee Public Administration to the Faculty of Social Sciences of Erasmus University Rotterdam took place on 4 and 5 October 2010. 1.0. Structure and organization of the faculty The Department of Public Administration is part of the Faculty of Social Sciences (FSW) of Erasmus University Rotterdam. The other departments within this faculty are Sociology and Psychology. The Dean of the faculty answers to the executive board (College van Bestuur) of Erasmus University Rotterdam. A small management team (FMO or Facultair Management Overleg), consisting of the Dean and the heads of the three departments, oversees the faculty. A decentralised management team (bestuursraad), consisting of a chairman, a director of research, a programme director, a financial director and a student representative supervises the Department of Public Administration. The programme director is responsible for the quality of the bachelor s and master s programmes Public Administration. The Public Administration department is relatively autonomous with respect to day-to-day matters. Within the department, both the bachelor s programme and the master s programme have their own coordinating person or team. For the bachelor s programme, this is the bachelor s committee. The various differentiations of the master s programme have a coordinating staff QANU & EAPAA / Public Administration, Erasmus University Rotterdam 21

member. These coordinators monitor the implementation of the programme and advise the programme director on all relevant issues. The Board of Examiners establishes the teaching and examination rules and acts as a committee in the event of conflicts between students and lecturers. The members of this independent committee are appointed by the Dean. The Programme Committee consists of six members of staff and twelve student members representing all programmes and differentiations. Six of the student members have the right to vote. The other student members are advisory members. The Department thus ensures that every year and every specialization is represented within the Committee. The Committee deals with strategic issues concerning education, changes to programmes and complaints concerning the implementation of programmes. It can give advice, both on request and on its own initiative. In the last six years, it was never necessary to cast a vote in the Programme Committee. The Educational Office serves the departments of both public administration and sociology. It provides all kinds of assistance with respect to educational matters to both staff members and students (such as providing information, organising course evaluations and allocating rooms). The Department of Public Administration has a Board of Advisors, consisting of about 15 members, who are all alumni of the department, now working at strategic levels inside ministries, in municipalities and other public organisations and in consultancy organisations. This board meets twice a year and advises the department on strategic issues regarding the education and research programmes. 1.1. The assessment framework 1.1.1. Aims and objectives Standard 1: Mission-based accreditation [EAPAA] The programme should state clearly its educational philosophy and mission and have an orderly process for developing appropriate strategies and objectives consistent with its mission, resources, and constituencies. From the mission a set of credible educational objectives should be formulated. Interpretations of the EAPAA standards of this accreditation must be justified in light of the programme s mission and objectives and success in fulfilling its mission Description According to the self-assessment report, the department of Public Administration formulated a mission statement in March 2010. This mission statement followed upon a process of reformulating the aims and objectives of the programmes, which was recommended by the committee responsible for the mid-term review in 2007. The starting point of this process was the department s strategy days in March 2008. The mission statement, the aims and the objectives are similar for both the master s and bachelor s programmes. However, the intended learning outcomes differentiate between the bachelor s and master s programme. The mission statement of the Public Administration programmes of Erasmus University Rotterdam is formulated as follows: to educate students in public administration, so that they can identify and analyse relevant societal questions, are able to advise on policy options to solve these, and organize the necessary processes to achieve these solutions. The aims and objectives are formulated as follows: 22 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

First, the department has the aspiration that all Rotterdam Public Administration students develop into people being able to build bridges. The department expects them to be capable of operating in complex networks with many actors. In these networks, graduates should be capable of identifying and connecting parties with different interests or perspectives in order to realise societal goals. They will do so by employing knowledge and insights derived from a range of social sciences in such a manner that they (1) are able to connect different scientific disciplines and (2) are able to practically apply the insights from these various disciplines. Second, graduates will have the scientific knowledge, skills (including research skills) and the necessary attitudes to analyse complex societal issues and they will be capable to formulate proposals to resolve these issues. They will also be capable of operating within a wide array of different areas. Rotterdam Public Administration students will therefore be able to contribute to various issues in which at least one public actor is involved. This could be the national government, but also the provincial, local level or international governmental organisations. Graduates will also be able to make valuable contributions to issues involving other public organisations, such as those in education, health or well-being. Finally, a Rotterdam Public Administration student will appreciate that the development, execution and effectiveness of policy are also affected by the way relevant organisations are structured and managed. That is why the programmes focus both on policy processes and on issues concerning the organisation and the management of public organisations. In order to achieve its mission, the department uses an educational philosophy based on the so-called academy-atelier formula. The essence of this philosophy is that public administration is not only a theoretical, but also a practical discipline. Theoretical aspects (knowledge) are taught in the academy, the practical part (skills) in the atelier. Assessment The committee studied the mission statement of the Public Administration programmes as well as their aims and objectives. It established that the mission, although formulated at the end of the strategic process, is the product of a process in which a range of stakeholders was involved. The mission itself is found to be quite general, where the aims and objectives give more insight into the core task of the department. Although not clear from the mission, the Rotterdam programme seems to have some unique characteristics. One of them is the historical foundation in sociology. This foundation means that the programmes strongly focus on interaction with society, which the committee perceived as a clear theme throughout the site visit. The committee assessed that the aim of developing students to be people able to build bridges forms a connecting thread through the programmes. This became evident in the conversations with staff, students and people from the working field. It is also reflected in the educational concept of the programme: the academy-atelier formula. Within the ateliers students combine the development of skills with the practice of different theoretical aspects. The committee assesses the mission-base of both programmes as satisfactory. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 23

Standard 2: Subject-/discipline-specific requirements [NVAO & EAPAA] The intended learning outcomes of the programme correspond with the requirements set by professional colleagues, both nationally and internationally and the relevant domain concerned (subject/discipline and/or professional practice). Description As described in the self-assessment report, the intended learning outcomes of the Erasmus University Rotterdam programmes were inspired by the Public Administration learning outcomes formulated by the Interuniversitary Committee of Public Administration (Interuniversitaire Commissie Bestuurswetenschappen, ICB) of the Association of Universities in the Netherlands (Vereniging van Samenwerkende Nederlandse Universiteiten, VSNU) for the previous assessment of the programmes in public administration. The programme director and the bachelor s coordinating committee have discussed these outcomes extensively and formulated their own intended learning outcomes accordingly. The results of this process were discussed with the board of studies and the Board of Advisors. These boards largely agreed with the proposed intended learning outcomes. Following their advice, several small changes with respect to the formulation were made. The intended learning outcomes for the bachelor s programme of Public Administration at Erasmus University Rotterdam are: A graduate of the bachelor s programme in Public Administration 1. has demonstrable knowledge and understanding of Public Administration concepts and theories; 2. has demonstrable knowledge and understanding of adjacent disciplines such as sociology, political science, economics and law; 3. has demonstrable knowledge and understanding of the nature, causes and consequences of societal phenomena; 4. is able to develop, conduct and evaluate Public Administration research; 5. is able to apply different Public Administration concepts and theories in order to comprehend societal phenomena; 6. is able to adequately apply acquired knowledge and understanding to public administration practices; 7. is able to gather relevant data and, based on these data, make a judgment and give recommendations that integrate relevant professional, ethical and academic interests; 8. is able to recognise and reflect on normative dilemmas; 9. is able to distinguish between empirical analyses and normative statements; 10. is able to function as a broker between different values and interests; 11. is able to report on public administration issues and research results to different target groups, according to scientific standards; 12. is able to reflect on his/her own learning strategies and acquired skills; 13. has developed learning skills which enable him/her to make a reasoned decision about future professionalism, for example in choosing further study at the master s level. The intended learning outcomes for the master s programme of Public Administration at Erasmus University Rotterdam are: A graduate of the master s programme in Public Administration Master 1. has advanced knowledge and understanding of the dynamic processes within society and public administration, and the ways in which these processes influence one another; 2. has advanced knowledge and understanding in an area or aspect of Public Administration and corresponding paradigms and theories; 24 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

3. has the capacity to identify, select and analyse different theoretical frameworks to address complex public administration issues; 4. is able to autonomously employ research methods and techniques, combined with a theoretical framework, to analyse, evaluate and report about societal and public administration related problems in order to contribute to possible solutions; 5. has an understanding of normative aspects of policy and management issues in the public domain, particularly the ethics of public administration; 6. has the ability to autonomously give advice on organisational, management and policy issues; 7. is able to function as a broker between values and interests, as manifested in heterogeneous environments, to solve complex problems ; 8. is able to apply acquired theories and concepts in practice in relevant single or multiple domains of public administration, such as organisation, management and/or policy; 9. possesses argumentative and communicative skills to independently and critically report on complex issues in the public domain; 10. has developed learning skills that enable him/her to undertake a new study in a selfdirected and autonomous way or to function on an academic level in and around the public sector. The new framework for Dutch Public Administration programmes developed by the Discipline Orgaan Bestuurskunde (DOB) in late 2009, after the phrasing of the intended learning outcomes of the Public Administration Programme of Erasmus University Rotterdam. In its self-evaluation report, the department relates the intended learning outcomes to these intended learning outcomes of the DOB. Assessment The committee studied the intended learning outcomes of the bachelor s programme and the master s programme and the relation of the learning outcomes with the framework of the Discipline Orgaan Bestuurskunde. It established that there is a clear correspondence between these two sets of intended learning outcomes. For the bachelor s programme for example, the third intended learning outcome of the DOB framework ( basic awareness of political traditions and politics ) corresponds with the second intended learning outcome ( has demonstrable knowledge and understanding of adjacent disciplines such as sociology, political science, economics and law ). The twelfth DOB intended learning outcome ( basic capacity to select a suiting theoretical framework for a given empirical framework ) corresponds with the fifth intended learning outcome of the bachelor s programme ( is able to apply different Public Administration concepts and theories in order to comprehend societal phenomena ). The committee established that all the DOB intended learning outcomes can be related to one of the intended learning outcomes of the bachelor s programme. The committee also studied the intended learning outcomes of the master s programme and the comparison with the DOB framework. The masters learning outcomes do correspond as well. For example the first learning outcome of the DOB framework ( knowledge of changing societal contexts ) corresponds with the first intended learning outcome of the programme ( has advanced understanding of the dynamic processes within society and public administration, and the ways in which these processes influence one another ). The last learning outcome of the DOB framework corresponds with learning outcome number 10 of the programme ( has developed learning skills that enable him/her to undertake a new study QANU & EAPAA / Public Administration, Erasmus University Rotterdam 25

in a self-directed and autonomous way or to function on an academic level in and around the public sector ). The committee also established that the competences trained in both programmes refer to skills which graduates need in the professional practice. The committee established that the explicit focus of the programme on functioning in public sector practice is a well developed characteristic of the programme and its intended learning outcomes. The committee concludes that its requirements for bachelor s and master s programmes are sufficiently covered by the professional and academic competences that the programmes have in view and that the final qualifications of both programmes correspond to the demands of the professional practice. It therefore assesses both programmes as satisfactory. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 3: Bachelor and master level [NVAO & EAPAA] The intended learning outcomes of the programme correspond with the general, internationally accepted descriptions of a Bachelor s qualification or a Master s qualification. Description According to the self-evaluation report, the learning outcomes of the bachelor s programme correspond to internationally accepted qualifications for an academic bachelor s programme, as becomes apparent when the intended learning outcomes are linked to the Dublin descriptors. The self-evaluation report contains a table which shows how the intended learning outcomes relate to the Dublin descriptors. This table can be summarized as follows: Knowledge and understanding: learning outcomes 1, 2, 3 Applying knowledge and understanding: learning outcomes 4, 5, 6 Making judgments: learning outcomes 7, 8, 9 Communication: learning outcomes 10, 11 Learning skills: learning outcome 12, 13 The learning outcomes of the master s programme also correspond to internationally accepted qualifications for an academic master s programme. These outcomes build on the outcomes of the bachelor s programme. As stated in the self-evaluation report, the master s learning outcomes aim at a deeper understanding of knowledge and advanced research abilities and communication skills. Again, the self-evaluation report contains a table which provides an overview of the relation between the intended learning outcomes and the Dublin descriptors. The table can be summarized as follows: Knowledge and understanding: learning outcomes 1, 2 Applying knowledge and understanding: learning outcomes 3, 4 Making judgments: learning outcomes 5, 6 Communication: learning outcomes 7, 8, 9 Learning skills: learning outcome 10 Assessment The committee studied the intended learning outcomes of the bachelor s programme and the master s programme from the perspective of their level. It established that the learning 26 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

outcomes correspond sufficiently to the Dublin descriptors and that they specify the level of the programmes adequately in both cases. It agrees with the way the relation between the intended learning outcomes and the Dublin descriptors has been elaborated in the selfevaluation reports. In its view, the intended learning outcomes reveal that students of the bachelor s programme acquire knowledge, understanding, skills and attitudes at a basic level that is typical for a bachelor s programme. For example, students of the bachelor s programme obtain knowledge and understanding of Public Administration concepts and theories. They acquire the ability to develop, conduct and evaluate Public Administration research. Similarly, the intended learning outcomes show that students of the master s programme obtain advanced knowledge and understanding of the dynamic processes within society and public administration and the ways in which these processes influence each other. They are able to autonomously employ research methods and techniques at an advanced level that is characteristic for master s programmes. The committee noted that the differences in level between the programmes are clearly reflected in the intended learning outcomes. The committee therefore concludes that both programmes fulfil the criterion which relates to the level of the learning outcomes. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 4: Academic orientation [NVAO & EAPAA] The intended learning outcomes of the programme correspond with the following descriptions of a Bachelor s and a Master s qualification: The intended learning outcomes are derived from requirements set by the scientific discipline, the international scientific practice and, for programmes to which this applies, the practice in the relevant professional field. An academic bachelor (WO-bachelor) has the qualifications that allow access to at least one further programme at academic master's level (WO-master) and the option to enter the labour market. An academic master (WO-master) has the qualifications to conduct independent research or to solve multidisciplinary and interdisciplinary questions in a professional field for which academic higher education is required or useful. Description According to the self-evaluation report, the learning outcomes of the bachelor s programme cover the general characteristics of academic training. For instance, general scientific skills are covered by several of the intended learning outcomes of the bachelor s programme (including 4, 5, 6, 7, 9 and 11), while several learning outcomes refer explicitly to the requirements of the professional practice (6, 7, 8 and 10). Students who have successfully completed the bachelor s programme have direct access to the master s programme Public Administration at Erasmus University Rotterdam or to any other Public Administration master s programme in the Netherlands. Bachelor s graduates can also directly start working in the professional field of Public Administration, but the self-evaluation report states that this path is seldom chosen. The master s programme emphasizes the scientific training of students. The programme aims at preparing students for further study or to function on an academic level in and around the public sector. According to the self-evaluation report, learning outcomes 2, 3, 4, 9 and 10 emphasize the scientific orientation of the master s programme. Assessment The committee has studied the intended learning outcomes to be acquired by students of both programmes from the perspective of their orientation. It already established that these learning outcomes correspond sufficiently to the demands of the scientific discipline (or the professional colleagues and the relevant domain) and the professional practice (cf. Standard QANU & EAPAA / Public Administration, Erasmus University Rotterdam 27

2). It noted that graduates of the bachelor s programme have unconditional access to the master s programme Public Administration at Erasmus University Rotterdam or any other university in the Netherlands. The committee is convinced that graduates of the master s programme have acquired the qualifications to carry out academic research in their area of specialisation and therefore qualify for relevant PhD programmes. The committee finds that the learning outcomes of both programmes correspond sufficiently to the requirements of the international scientific practice. For example, the intended learning outcomes of the bachelor s programme include skills concerning the development and evaluation of research and reflection on empirical and ethical statements. The intended learning outcomes of the master s programme comprise the ingredients by which students are enabled to conduct research autonomously and responsibly. The committee therefore concludes that both the bachelor s and the master s programme fulfil the criterion that relates to the orientation of the learning outcomes to be acquired. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Assessment of the theme Aims and objectives The committee comes to an overall assessment of the theme Aims and objectives on the basis of its assessments of the separate standards. In the case of the bachelor s programme Public Administration, it assesses this theme as satisfactory. In the case of the master s programme Public Administration, it assesses this theme as satisfactory. 1.1.2. Curriculum Description of the curriculum of the programmes In the bachelor s programme a full year is made up of eight blocks. Each block consists of five weeks, including exams. In the first 1,5 years of the programme each block consists of an academy part of lectures (5 EC) and consecutive blocks host overarching atelier parts of working groups (7,5 EC for ateliers I, II and IV, 5 EC for atelier III). The second 1,5 year of the programme consists of blocks of integrated modules of 7,5 EC each. The curriculum provides knowledge and skills in all aspects of public administration, such as management in the public sector, policy and politics, international governance, political science and side disciplines, e.g., sociology, economics and law. The curriculum (and its current structure) was introduced in the academic year 2007-2008. The structure of the programme is based on ideas and routines from educational psychology. The programme is sequential (one module at a time) so students can focus fully on each module. Teaching is made up of traditional lectures (four hours per week) plus tutorials in small groups of about 15 students (again four contact hours per week). An examination follows immediately after each module and is, if necessary, followed by a re-examination after the next module (first 1,5 year) or next two modules (second 1,5 years). An important element of both the bachelor s and the master s programme is the academy-atelier formula. In this formula, the traditional lecture format is used for the academy sessions. In the atelier sessions, students get theoretical and practical assignments. Ateliers are organised in tutorial groups in which they develop the necessary professional skills. In the first part of the bachelor s programme, 28 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

ateliers overarch two or three blocks, in these ateliers knowledge of the two or three academy parts is combined in the assignments. The minor in the third year of the curriculum is an important elective for students. They can choose from one of 56 minors offered by departments of Erasmus University Rotterdam. The minors are organised on the central level of the university. The so called educational minor of 30 EC will start for the first time in the academic year 2010-2011. Public Administration students who want to follow this minor can use the 15 EC of the normal minor, exchange one of the four 7,5 EC modules in the third bachelor s year, and have to invest 7,5 EC additional time. If they successfully complete this minor, students fulfil the required competences as stipulated in Article 36 of the Secondary education Act, and are eligible to teach in the subject of maatschappijleer in junior general secondary education and the first three years of higher general secondary education and pre-university education. This is made visible by an addendum on the bachelor s certificate. Instead of following a minor, students can opt to go abroad. The international office is responsible for the exchange programme that includes several permanent contacts with foreign universities. As part of these programmes, foreign students are welcome at the Rotterdam department to follow courses. The master s programme offers six specialisations, four of them being part of the regular differentiation: Public Policy, Public Management, Labour, Organisation and Management and Governance and Management of Complex systems. The other differentiations are International Public Management and Policy and the Evening Master. During the site visit staff and students highlighted that the wide coverage of the specialisations is a strength of the Rotterdam programme. The programme management team considers this as one of the advantages of the size of the department. The regular differentiations start with an introductory module and advanced modules within the domain of the differentiation, followed by two bounded electives within the department of Public Administration. The second part of the master s curriculum consists of an internship and the master s thesis, which are usually a combination. During this period, students meet in tutor groups for guidance and support to realize the academic thesis. The International Public Management master s and the Evening Master s do not have an internship in the curriculum. The first one consists of courses in three domains: (international) political and administrative systems, knowledge of (international) public policy, decision-making and implementation, and knowledge of (international) organisations and organising principles. The last part is a thesis and an atelier on professional development. The evening differentiation is designed for practitioners with professional experience. It is preceded by a pre-master s programme. The pre-master s and master s make up a generic programme on Public Administration with modules such as Policy and Society and Public management and organizational change. Both the international master s and the evening programme finish with the master s thesis. Alongside the regular curriculum, the department aims to stimulate talented students. Firstly, talented students are offered the opportunity to participate in the Erasmus Honours Programme. This is a generic university programme available to only 25 students in each study year, selected from all Erasmus University Rotterdam programmes. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 29

Since 2007, there has also been a Faculty honours programme. Students who achieve an average mark of at least 7.5 are invited to participate in this programme. Successful completion leads to an addendum on the bachelor s certificate. Standard 5: Public administration character of the programme [EAPAA] The core curriculum provides a thorough teaching of the basic concepts, theories, methods and history (classics) of Public Administration on the level of the programme (bachelor or master). The curriculum components are designed to produce professionals capable of intelligent, creative analysis and communication, and action in the public sector. Courses taken to fulfil the core curriculum components provide research methods, concepts and theories from the disciplines of economics, law, political science, sociology, public finances, informatization, and public management as well as the relationship between these fields. Students are obliged to give adequate proof of their ability to work independently (under the supervision of a supervisor) on real problems or research questions in the public sector, for example through essays and final papers (e.g. a thesis), adapted to the level (bachelor or master) of the programme. The core curriculum components enhance the student's values, knowledge, and skills to act ethically and effectively. Description The self-assessment report presents a table in which the curricula of the programmes are linked to the DOB-framework (see Standard 2). The bachelor s curriculum covers all eight themes of the framework, with a strong emphasis on Cooperative & communication skills. The department makes this emphasis following the mission, aims and intended learning outcomes which state an important focus on professional practice. The modules of the bachelor s programme cover a wide range of concepts and theories of Public Administration, e.g. in the courses Core issues in public administration and Introduction to public administration research. There are courses in related disciplines: Economics (course: Economics: instruments and policy), Sociology (course: Introduction to sociology), Law (course: Introduction to state and administrative law), Political Science (courses: Introduction to political science, Policy and politics and Political philosophy and democracy), Informatization (course: Information society and e-governance) and public management (courses: Core concepts of management and organization, International Governance, Public organizations and change, Management in the public sector). The synthesis of the different disciplines is made in the atelier courses and in the bachelor s internship and thesis. For example, in one atelier session students did a simulation of the formulation of the constitution of Kosovo. During the simulation they had to form four parties with the joint commission to make a constitution. Students pointed out that through this simulation they learned that governments cannot just impose new policy. In another atelier students were given an exercise to make photographs of a disadvantaged neighbourhood. These exercises and the bachelor s internship provide the opportunities to work on real problems. In the third year there are two courses that focus on the professional practice, which give reflection on the specific roles a Public Administration professional can have and dilemmas concerning integrity that can arise. The master s programme also covers the range of the DOB framework. The self-evaluation report shows the differentiations correspond with the DOB framework. This shows the particular accents the differentiations have. For example, the differentiation Public Management has a special emphasis on both Knowledge of political and administrative systems and on Knowledge of organizations and organizing principles. The International Master s and the Evening Programme have a broad perspective and cover the different themes of the framework with equal emphasis. The themes that are not highlighted for a specialization receive limited attention in the courses. Meanwhile, most of the students choose the electives that are underemphasized, so that their final curriculum of the master s programme has a complete coverage of the different themes. In the master s programme, the 30 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

thesis (and for the regular differentiations the internship) forms a core part in which integration and specialization takes place. In tutor groups, students rehearse their research skills. In atelier sessions, with a similar approach as those in the bachelor s programme, working on real problems is an important element of the programmes. Cases from the professional field and excursions are included in the curriculum and regularly there are guest lecturers from the professional field. During the site visit the committee pointed out that there are four major demands for public bodies: compliance, performance, resilience and coping with change. The study of public administration needs to prepare students to be aware of the gap between expectations from politicians on the one hand and citizens on the other hand. During the site visit the staff explained to the committee that the differentiations in the master s programme differ in the emphasis that is put on these demands. For example, the specialisation of Public Management is more focused on performance, the specialisation of Governance and management of complex systems focuses more on resilience. Assessment The committee studied the curriculum of the bachelor s and the master s programme and acknowledges the programmes to be very complete. The Public Administration character is very clear for both of the programmes, combining courses on Public Administration, Public Organisation, Public Policy and the related disciplines Economics, Law, Sociology and Political Science. The bachelor s programme stands out with the wide coverage of theories combined with exercises and assignments that have a good base in the public administration professional field. The committee especially appreciates the course on e-governance, which demonstrates that the programme has integrated current developments in the Public Administration field. Throughout the bachelor s and the master s programme there are several ways in which real social problems are integrated in the curriculum. The committee appreciates the fact that both the bachelor s and the regular specialisations of the master s programme have an internship. In the master s programme this internship is linked to the thesis, a valuable characteristic of the programme. The committee stresses that integration of the four demands of a public body in the curriculum is important. During the site visit a director at a Dutch Ministry pointed out that Public Administration students are much more perceptive of political systems than students who studied a single discipline such as Economics. The committee assesses this standard as good for the bachelor s programme and as satisfactory for the master s programme. Bachelor s programme Public Administration: the committee assesses this standard as good. Master s programme Public Administration: the committee assesses this standard as satisfactory. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 31

Standard 6: Requirements for academic orientation [NVAO & EAPAA] The proposed curriculum meets the following criteria for an academic orientation: The students develop their knowledge through the interaction between education and research within the relevant disciplines The curriculum corresponds with current developments in the relevant discipline(s) by verifiable links with current scientific theories The programme ensures the development of competences in the field of research Where appropriate, the curriculum has verifiable links with the current relevant professional practice. Description According to the self-evaluation report, in many modules of the bachelor s and master s curriculum, older and recent theories and research insights within the Public Administration field are introduced and discussed. In the first part of the curriculum of the bachelor s programme this is done on an introductory level, but in the second part of the bachelor s programme and in the master s programme this is done on a increasingly advanced level. A general criterion for academic education is that it provides an interaction between education and research. Permanent staff are responsible for the content of the modules and their presentation of the content (during lectures) to students is an important factor in achieving this. The courses are taught by researchers who contribute to developments in their own field of specialization. They help students acquire a critical, investigative and analytical attitude. As a result, most courses do not only contribute to the acquisition of domainspecific knowledge and skills, but also to the development of general academic skills and an academic attitude, such as writing, presenting, working together, conflict handling, negotiating, interviewing techniques etc. Research skills form an important element of the curriculum. The acquisition of research skills is not limited to dedicated Research Method & Techniques modules, but also integrated in substantive parts of the curriculum. During atelier sessions the substantive and the research modules are integrated. The basic research skills are instilled in the bachelor s programme (or the pre-master s programme), and these are refreshed and extended during the master s programme, especially within the atelier components. In the evening differentiation, attention to research skills is provided both as an integral part of a number of substantive modules and in separate modules. For instance, research skills (including the use of theory within research) are presented in the Methodology II module, and network analysis in the Steering in the Public Sector module. Over the years, the amount of time and effort devoted to teaching and learning these skills has been extended. Finally, the bachelor s and master s theses also require the integration of content and research. The short internship in the second year of the bachelor s programme and the internship in the master s programme comprise an important part of the curriculum and provide explicit links to professional practice. In the bachelor s internship students not only participate in a real organisational setting, but they also conduct a small research project which has to be relevant from both academic and professional practice viewpoints. In addition to the internship, students also gain familiarity with professional practice through regular excursions. For instance, in the academic year 2009-2010, bachelor s students visited the Raad van State and the Ministry of Finance, Rotterdam Stadionpark and the Feyenoord Stadium. Recent visits include the Ministry of Home Affairs for the Labour, Organisation and Management specialisation, several divisions within the council that runs the City of Rotterdam by Public Policy students, AT Osborne (a consultancy firm) by Governance and Management of Complex Systems students and to international organisations in Geneva by the International Public Management and Public policy students. 32 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Assessment The committee studied the curricula of the programmes from the perspective of the requirements that hold for an academic programme. It established that the bachelor s programme presents an adequate overview of the discipline as a whole. The committee established that the learning materials used are adequate. As stated before the curriculum contains courses from other relevant disciplines and practical courses which provide students with the necessary practical skills and techniques. Students are introduced to topical issues and practical problems within the professional field through an internship, the bachelor s thesis and discussion of current public sector issues and challenges in the programme s courses. In addition, faculty members are widely involved in policy-oriented research and in service training. They further use case material and insights from professional contacts within the field in their teaching. The committee assumes that this research base is reflected in the teaching at the department. Given that master s programmes in Public Administration in the Netherlands have a duration of one year only, the committee establishes that the Public Administration Department of Erasmus University Rotterdam puts reasonable effort to train master s students to perform academic research. The introduction of tutor groups during the thesis phase is a good way to help students to accomplish a good standard in their theses. The committee highly values an international component in academic public administration programmes and notes that this is somewhat underemphasized in the programme s. Even at master s level only the international differentiation is taught in English. The committee recommends that the department also establishes other ways to realize the international component, such as English taught courses during the bachelor s phase and more opportunities to study abroad. The committee assesses this standard as satisfactory for both programmes. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 7: Correspondence between the aims and objectives and the curriculum [NVAO & EAPAA] The curriculum is an adequate realisation of the intended learning outcomes of the programme and this regards the level, the orientation and the subject-/discipline-specific requirements. The intended learning outcomes are adequately transferred into the educational goals of the curriculum or parts thereof. The contents of the curriculum ensure the students achievement of the intended learning outcomes. Description According to the self-evaluation report, each module of the bachelor s and master s programmes has its own learning objectives, which are formulated in correspondence with the intended learning outcomes on programme level. Every learning objective contributes to at least one of the intended learning outcomes of the programme as a whole. Students are informed about the learning objectives of the modules in the study guide and through the e- learning platform. The self-evaluation report contains tables which present overviews of the relation between the courses of the bachelor s and master s programmes and the learning outcomes to which they contribute. The tables show that every learning outcome is dealt with in various courses and that most of the learning outcomes are evenly distributed over the curriculum as a whole. Each learning outcome is present in at least two modules of the curriculum. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 33

The appendices of the self-assessment report contain a table in which the curriculum is presented in relation to Bloom s taxonomy, which discerns six levels of cognitive knowledge: knowledge, insight, application, analysis, synthesis and evaluation. The table shows that student attain these six levels in the course of the bachelor s programme, with an increasing emphasis on the higher levels. Assessment The committee studied the table in the self-evaluation report which relates the courses in the curriculum of the bachelor s programme to the various learning outcomes. It concluded that the information provided in the table is adequate and correct. It established that every learning outcome receives sufficient attention in the curriculum and that the distribution of learning outcomes over the curriculum is appropriate. The committee considers the curriculum as a broad programme which provides students with a broad spectrum of knowledge which helps them to build the bridges, as stated in the programme s intended learning outcomes. In the committee s opinion, the curriculum is designed and structured in such a way that it can be taken for granted that students who complete the curriculum have successfully acquired every learning outcome at the appropriate level. The committee also studied the table which relates the courses in the curriculum of the different master s specializations to the intended learning outcomes of the master s programme. It established that all the intended learning outcomes are met within each of the six specializations. The committee therefore concludes that the curriculum is an adequate realization of the learning outcomes of the master s programme and that it sufficiently guarantees that students will actually achieve these learning outcomes. The committee concludes that both the bachelor s programme and the master s programme fulfil the criteria which refer to the relation between the learning outcomes and the curriculum. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 8: Consistency of the curriculum [NVAO & EAPAA] The contents of the curriculum are internally consistent. Description The self-evaluation report refers to a model of De Bie and De Kleijn (2001) which structures the bachelor s curriculum in four didactic lines: the conceptual line, the skills line, the integrative line and the reflection line. According to the self-evaluation report, the relation between the conceptual line and the skills line is crucial for the curriculum. The conceptual line deals with the transfer of knowledge. In the bachelor s programme, this knowledge is spread over three substantive elements: public policy, management and organization and research methods. The appendices of the self-evaluation report contain a table in which these elements are related to the curriculum. For example, the module Policy and Politics is related to the Public Policy element, just like the International Governance module. The management and organization element is, among others, present in the module Public organizations and change, as well as in the module Management in the Public sector. According to the department, Public Administration is not only a theoretical but also a practical discipline. The theoretical or knowledge part is taught in the academy sessions, skills 34 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

are taught and practiced in the atelier sessions. The skills line in the curriculum is divided in cognitive and intellectual skills (e.g. selecting relevant literature, reporting according to academic standards), interactive skills (e.g. giving a presentation, project management) and reactive skills (e.g. providing feedback, reflection on analyses of and solutions for issues). The integration line is achieved by the academy-atelier educational concept. In the first part of the bachelor s programme different academy modules are covered in one atelier module to ensure integration. In the second part of the programme, integration of knowledge and skills mainly takes place within the modules. The internship and the bachelor s thesis are two important items in the curriculum in which students need to integrate their obtained knowledge and skills. Reflection is a major thread throughout the programme. It is implicit in different parts of the programme as well as explicit in the internship and the module Reflection on the professional practice, for which students write a reflection on their own learning and development. Some students pointed out during the site visit that they would like to be challenged more, especially on the skills part: they would like to have more practice of presentation and debating skills. According to them, the atelier sessions can be too passive. They would like to participate more actively in the composition of the atelier sessions. The regular master s programme follows the same four didactical lines as the bachelor s programme. In this programme as well, the conceptual line is covered in the substantive modules, the skills line is taught in the ateliers as well as in the internship and thesis. Integration of these lines is ensured as well in the internship and thesis, especially because students are stimulated to combine their internship with the empirical research needed for their master s thesis. Reflection is part of the thesis and its defence. Within the International Public Management and Policy differentiation, the consistency is build up in three pillars: a generic/national pillar, an international pillar and an EU pillar. Each consists of two related modules to ensure the accumulation of knowledge and internal coherence within the programme. Typically, the second module in a pillar is also linked to the initial modules of other pillars to further increase coherence and knowledge accumulation. Students follow one elective module which enables them to specialise, for instance in a specific policy field, or to gain an additional perspective. The programme concludes with a final research project which offers an option to gather data through a short internship. The evening differentiation has a didactic concept which includes four stages: unfreezing, cognition, application and reflection. The four elements are represented in all modules, during the curriculum the emphasis shifts from 1 through 2, to 3 and 4. In this last stage students are able to theorise, evaluate and develop professional strategies. Furthermore, the pre-master s and master s programme consist of three main streams: substantive policy, societal trends and problems (covered in module Policy and society), institutions, organization, management networks, processes (covered in module Public management and Organisational change) and third methodology (covered in module Methodology 2). Integration of these streams takes place in the module Public Administration Professional and in the Master s Thesis. The structure of the curricula provides for consistency in the bachelor s and master s programme. The consistency is preserved as well by the responsibilities of the staff. In the bachelor s programme, a (associate) professor is responsible for each component of the conceptual line to safeguard the vertical integration of the courses over the three years. Every QANU & EAPAA / Public Administration, Erasmus University Rotterdam 35

master s specialization has a small staff team, chaired by a full professor. The master s coordinator ensures consistency over the four regular specializations. There are regular meetings to discuss aspects of the curriculum that are relevant to all specializations. Assessment The committee studied the curriculum of the bachelor s and master s programmes and spoke to staff and students of the programmes about the consistency of the curriculum. It established that the curricula are structured in such a way that they cover all knowledge and skills needed for the Public Administration practice. There are clear lines in the programme that make up for coherence and consistency. The committee appreciates the integrative approach which enables students to apply their theoretical knowledge to the practice of public administration in the atelier sessions. Students point out that there is a progression in the programme, with more deepening towards the last year of the bachelor s programme, exemplified by the fact that in the end of the programme articles are used more intensively than books. In the master s programme there is an increasing integration of theoretical modules and practical knowledge, which is obtained in the internship, case-studies and excursions. The committee has established that the coherence of the regular master s programme is visible within the different specialisations since students take an introductory module which is followed by three specialization courses and then choose two matching electives to create a coherent programme. In the International Master of Public Management and Policy this coherence is created by the three pillars. The committee appreciates that the second module of each pillar refers to the knowledge of the introductory modules of the other two pillars. The committee concludes that both the bachelor s programme and the master s programme fulfil the criterion which relates to the coherence of the curriculum. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 9: Workload [NVAO & EAPAA] The curriculum can be successfully completed within the set time, as certain programme-related factors that may be an impediment in view of study progress are eliminated where possible. Description The bachelor s programme follows a sequential structure with one module at a time. A full year is made up of eight blocks. Each consists of five weeks, including exams, and is worth 7.5 EC (including an atelier component of 2.5 EC). A week includes two two-hour lectures and two two-hour tutorials in small groups, resulting in eight contact hours per week. The remainder of the time is spent on self-study, such as reading academic literature, writing papers and preparing presentations. The principle of allocating time to specific tasks (students have to submit written assignments on a regular basis during the atelier) is an important instrument in stimulating students to study regularly throughout a module. In this way the atelier requires students to take on a specific module right from the start in the first week. According to the self-assessment report the curriculum s sequential structure helps to ensure a workload that is more or less constant throughout the year. With this structure, students can focus on one module without competition from other modules. 36 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

The so-called binding study advice was introduced in the academic year 2005-2006. Students of the first year need to achieve at least 40 EC to continue their studies in year two. Students need to achieve the full 60 EC of the first bachelor s year within the first two years of their bachelor s programme. This binding study advice contributes to a more regular workload across the first year especially as students get several advices throughout the year. That is, they get an early warning if they are not devoting enough time to their studies. They are also advised to turn to the study counsellors if study results suggest this is desirable Study materials (academic books, academic articles and assignments) are matched to the formal study load. To be able to measure the formal workload, the Education and Exam regulations specify the calculation of the normative workload for reading, assignments, etc. The department has taken extensive measures to ensure that students of the master s programme are able to finish their studies within the set time of one year. For instance, the department has concentrated all modules of the regular programme in the first two blocks, so that students can fully devote their time to the internship and the thesis in the third and fourth block. Furthermore, the differentiations already pay attention to potential research questions for the theses in the first two blocks. Still, it is difficult for many students to complete their programme within a year. One of the reasons for this is that the internship can lead to delays. According to the self-evaluation report, the self-reported study load for students has increased, from on average about 20 hours in 2006 to about 24 hours per week in 2009. This is lower than the 40 hours that is, in principle, available, but the actual figure is consistent with that in other departments of the faculty and elsewhere in the Netherlands. Student evaluations have not so far indicated that one or more modules are systematically too difficult or involve a too heavy workload. However, if such problems, noticed by evaluations, focus groups or the Programme Committee, arise, these will be discussed within the Programme Committee. Assessment The committee established that the curriculum of the bachelor s programme can in principle be completed in the nominal period of three years. The actual study load of the courses shows some variation, but the overall study load is in general evenly distributed over the curriculum and not too high. The first part of the curriculum is organized in such a way that students are more or less forced to spend many hours on their studies. The sequential system demands of the students that they are fully dedicated to one course at a time. The committee endorses the advantage of the focus this brings into the programme. However, the committee points out that for some students this brings a pressure: it is hard to catch up if they miss or fail a course. The committee has not received any information which indicates that the curriculum contains serious obstacles which lead to delays in students progress. The committee has not received any information which suggests that the curriculum of the master s programme can not be completed in the set time of one year. However, a one year master s programme is a challenge in which theory, practice (during the internship) and research make a very demanding combination. There is a heavy workload converged in one period. The committee established that students spend quite a lot of time on their internship and their thesis project because they are motivated to perform as well as possible and not because the research they conduct is too difficult or too demanding. They sometimes do tend to produce theses that are too long, in both the bachelor s and the master s courses. The committee recommends the staff to encourage students more strongly to limit the length of QANU & EAPAA / Public Administration, Erasmus University Rotterdam 37

their thesis. The committee has not encountered any structural problems which cause the delays most master s students have. The committee assesses the feasibility of both the bachelor s programme and the master s programme as satisfactory. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 10: Admission requirements [NVAO & EAPAA] The structure and contents of the intended curriculum are in line with the qualifications of the incoming students: Academic bachelor s programme (WO-bachelor): VWO (pre-university education), propaedeutic certificate from a hogeschool (HBO)or similar qualifications, as demonstrated in the admission process Master s programme (WO-master): a bachelor's degree and possibly a selection (with a view on the contents of the discipline). Description According to the self-assessment report, all students with a degree from pre-university secondary education (vwo) have access to the bachelor s programme. Students who have successfully completed the first year of a professional bachelor s programme (hbo) may also enter the programme. As a third option, students who are at least 21 years old can be admitted to enrol in the programme if they can prove they have the required knowledge by successfully completing a colloquium doctum. Most students have a vwo background, the colloquium doctum procedure is seldom used. Students need to prove that they have sufficient mathematical skills, equivalent to the vwo-6 level. If not, they are required to reach that level and the department offers a remedial mathematics course to help them achieve this. Although it is not a formal admission requirement, since 2008-2009 students have to undertake a Dutch language skills test as part of the atelier component. If they fail this test, students are encouraged to improve their skills through remedial teaching. All students who meet the admission requirements can enter the bachelor s programme. No further selection takes place. The admission requirements for the bachelor s and master s programmes are presented on the departments website. A large number of master s students enter the Rotterdam programme directly from the bachelor s programme or enter the programme through the pre-master s. A limited number of students enter the programme from another Public Administration bachelor s programme. Students with a Public Administration bachelor s degree of Erasmus University Rotterdam have access to all the master s differentiations. In order to avoid what would virtually amount to a year s gap in their study, students from the bachelor s programme can conditionally enter the master s programme if they are short of no more than 10 EC in their bachelor s programme (provided they have finished their bachelor s thesis). Students who have completed a Public Administration bachelor s degree from another Dutch university are also admitted without restrictions. All other students have to follow a premaster s programme in order to qualify for admission. There are two pre-master s programmes: one for the daytime differentiations and one for the evening one. As with bachelor s students, pre-master s students are given conditional entrance to the master s programme if they lack no more than 10 EC from their pre-master s year. 38 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

This relaxation on completing prior courses has been introduced to limit study obstacles to progressing from the bachelor s (or pre-master s) to the master s programme. The percentage of master s students with obligations left over from their bachelor s or pre-master s courses has decreased in recent years. To press students to complete these outstanding commitments more quickly, the Board of Eaminers implemented stricter regulations in 2009: since then students only start their master s internship once they have finished their bachelor s or premaster s programme. International students are admitted on the basis of their qualifications (a bachelor s degree in a relevant subject) and an adequate knowledge of English. They should fill in an application form and present a specification of their higher education, including the exams passed and the grades obtained. Students need to write a short paper on a Public Administration topic and they should submit the result of a certified English language test. If the student is admitted, he or she will immediately enter the master s programme. Assessment The committee established that the requirements for access to the bachelor s programme have been specified explicitly and are in accordance with the relevant formal demands. It has noticed that students experience the transition from secondary education to the bachelor s programme as challenging but that they settle down usually after the first months. In general students do not have problems in adjusting to their new learning environment. The committee established that the pre-master is an adequate programme to prepare for the master s. Some students who have finished another master s programme before entering the premaster s programme encounter some overlap with their earlier education. The committee therefore recommends the department to consider a more flexible premaster s programme for this group and to refer to their own responsibility to prepare adequately for the master s programme. This does not mean that it is up to the students to compose a premaster s programme, but the department could make this process more flexible. The committee has some concerns regarding the bachelor s students who have one or more unfinished courses when they enter the master s programme. The committee established that the department is conscious of this problem and has taken some measures to prevent this. It is in favour of a policy to prevent this situation by enforcing students to completely finish their bachelor s programme before entering the master s programme. The committee assesses the standard relating to the admission requirements as satisfactory for both the bachelor s programme and the master s programme. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 11: Credits [NVAO & EAPAA] The programme meets the legal requirements regarding the range of credits: Academic bachelor s programme (WO-bachelor): 180 credits Academic master s programme (WO-master): a minimum of 60 credits. Description The curriculum of the bachelor s programme Public Administration comprises 180 EC. The curriculum of the master s programme Public Administration comprises 60 EC. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 39

Assessment The bachelor s programme Public Administration complies with the formal requirements with respect to the range of credits. The master s programme Public Administration complies with the formal requirements with respect to the range of credits. Standard 12: Coherence of structure and contents [NVAO & EAPAA] The educational concept is in line with the aims and objectives. The study methods correspond with this educational concept. Description The academy-atelier formula is an important instrument to ensure that graduates will be able to build bridges. This formula is an important element of the aims and objectives of the Public Administration programmes of Erasmus University Rotterdam. The restructuring of the curriculum in 2007 required a more radical integration of the academic and atelier parts within each module. During academy sessions, the traditional lecture format is used, but during the atelier sessions small tutorial groups of about 15-20 students are formed. These groups are supervised by a tutor, and the assignments set in the atelier sessions help to ensure that students start their self-study on time. They also help to develop the necessary professional skills. Many assignments can only be completed to a sufficient standard if students have first prepared themselves using the literature recommended for the module. The first 1.5 year of the bachelor s programme has an introductory character (with respect to both public administration and the related disciplines). At the end of this period, the fiveweek short internship where students are confronted with professional practice forms a natural bridge to the second part of the programme. In this second 1.5 year part, modules have a more integrated character, with more time for reflection and making one s own judgments. The bachelor s programme concludes with a bachelor s thesis for which students carry out a research project on a topic within public administration. They do this individually within a short period and under relatively close supervision. To ensure adjustments to modules are complementary and do not lead to omissions or overlaps, each year the staff teaching on either the first half or the second half of the bachelor s programme meet to discuss the fine-tuning of modules in these phases. Vertical integration is further safeguarded by meetings within the three substantive lines of public policy, management and organisation and research skills. Some students have pointed out during the site visit that they would like to be challenged even more. The department distinguishes four didactical lines to illustrate the consistency of the curriculum: the conceptual line, the skills line, the integrative line and the reflection line. The self-assessment report shows tables in which theses lines are related to the programmes and the teaching methods. For instance, teaching of the conceptual line takes place in the academy sessions using lectures. The master s programme continues to prepare students to build bridges and operate in complex networks with many actors. In the self-evaluation report, the programmes uses the same didactical lines as the bachelor s programme to illustrate the consistency of the curriculum. In the IMP master s programme, for example, the skills line is represented in the atelier sessions where teaching methods vary: assignments, presentations, discussions and (research) papers. Excursions and workshops attribute to this line as well. The integrative line takes place in the second part of the bachelor s programme where academy and atelier aspects 40 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

are integrated and during internship and thesis. The reflection line is represented in different modules and reflection questionnaires and papers that students need to write during their programme. Assessment From the self-evaluation report and the additional explanation during the site visit, the committee has established a clear image of the educational concept of the Public Administration programmes of Erasmus University Rotterdam, which is the academy-atelier formula. The ateliers are a substantial element of both programmes. The committee appreciates this element and is aware that it is labour-intensive with the large population of students Rotterdam attracts. The committee has the impression that the department is aware of that and that it seems to cope with this challenge by involving tutors. The committee appreciates the different teaching methods that are used in these ateliers such as case studies, group assignments, debating, simulations and excursions. The committee recommends the department to challenge students more strongly by actively involving them in the construction of the ateliers. Obviously, the focus in the master s programme is on the specialization courses, internship and thesis projects. In the committee s view, the teaching methods used in the master s programme succeed in stimulating independence and in developing academic and research skills. In this respect, they correspond to the intended learning outcomes of the programme, which presuppose that students are able to work independently. The committee assesses the bachelor s programme as good and the master s programme as satisfactory, because it feels that the academy-atelier formula has been elaborated more systematically and thoroughly in the bachelor s programme, which is of course largely due to the fact that the master s programme is a one-year programme with a more substantial individual component. Bachelor s programme Public Administration: the committee assesses this standard as good. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 13: Learning assessment [NVAO & EAPAA] By means of evaluations, tests and examinations, the students are assessed in an adequate and for them insightful way to determine whether they have achieved the intended learning outcomes of the programme or parts thereof. Description The form of the assessments of the Rotterdam Public Administration programmes is related to the academy-atelier formula. As knowledge forms an essential element of the academy part of the modules, all individual modules are assessed by a written exam (multiple choice or open answer exam). The assessment form for the atelier parts of the modules depends on the skills that are being tested. The following testing and assessment methods are used: multiple choice exams and open answer exams; small individual assignments and presentations, contributions to group discussions and peer feedback, especially for the 7.5 and 5 EC atelier modules in the first and second year and for atelier components of 7.5 EC modules in the third year; larger group assignments as final projects in atelier components; QANU & EAPAA / Public Administration, Erasmus University Rotterdam 41

obligatory activities such as a language skills test and Master Class presentation; the writing of a thesis; take home exams (in the master s programme). The Board of Examiners has an important formal role in safeguarding the quality of exams and assignments. In practice, the Board has delegated this task to the bachelor s coordinating committee. The lecturer responsible for a specific module is primarily responsible for designing the examinations and assignments used in that module. With respect to the safeguarding of the quality of the assignments four criteria are important: validity, reliability, efficiency and transparency. The validity is safeguarded by asking teaching staff to cover the learning objectives of the modules in their examinations. To safeguard this procedure, the bachelor s coordinating committee reviews the coverage using a test matrix. In this review, they also ascertain whether or not the intended exam questions are sufficiently clear. Following an exam, students are asked in an evaluation survey whether or not the exam, in their opinion, reflected what had been taught in the module. If the results of this survey indicate problems (which has not yet been the case), the bachelor s coordinating committee will discuss this with the respective staff member and measures for improvement will be taken. Further, the programme director takes the results of this evaluation into account when making his preliminary judgment on staff members as part of the annual appraisal cycle. Finally, more substantive ateliers are peer-reviewed by two staff members independently. If their grades differ by more than a point, the grade is reviewed in a discussion between the two staff members and the coordinator of the module. To find out whether students have really mastered the intended learning outcomes, the last two modules of the bachelor s programme are especially important. Within their bachelor s thesis, students have to show that they are able to carry out a piece of research that is relevant to public administration. Since the thesis does not explicitly address the learning outcomes related to the Dublin descriptor Learning skills, the module Reflection on professional practice module is important as well. In this module, students are asked to reflect on the three years that they have been studying Public Administration in relation to their intellectual growth and on future work and study prospects. They are presented with a number of examples of people who have studied Public Administration in the past and are asked to picture themselves following those examples in the future. This module was extended with the Rose of Leary method in the academic year 2009-2010. With respect to safeguarding the reliability of the assessments, given the high number of students, the reviewing of exams and assignments in the first study years is usually delegated to junior lecturers. In these cases, the reviewers carry out their tasks based on reviewing instructions supplied to them by the responsible lecturer. Throughout the reviewing process, these instructions can be refined, based on the junior lecturers experiences. The responsible staff member of the module is expected to check the results of the reviewing by taking samples. With respect to safeguarding the reliability of the assessment of the bachelor s thesis, assessment of the thesis supervisor is checked by a second member of staff. With respect to the efficiency of the assessments, assessments are spread out over the study year. In principle, an examination of a module takes place every 5 weeks. Assessment of small assignments as part of an atelier takes place during the module. For each module (with the exception of the internship and the bachelor s thesis) there are two (and no possibility of more) opportunities for students to take an exam. The examination regulation requires students to finish a module within one academic year. If a student fails to complete one or 42 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

more of the requirements (that is, if one of the components is graded with a mark below a 4, or if the weighted average of all component marks is less than 5.5), a student is required to retake all the required tests and examinations for that specific module in the following year. First year students can compensate for low scores in two academy modules, provided that the mark for each is at least a 5. Each 5 must be balanced by two modules with at least a 7. A similar compensation rule applies in the second and third year, where a maximum of two modules can be compensated. The rationale behind these compensation rules is not only that they stimulate the study pace, but that they acknowledge that students can have weak spots (for instance in economics or quantitative research methods). The department accepts these weak areas to some degree through these compensatory regulations, provided that the students can also show that they have both strengths and weaknesses. Transparency is safeguarded by the fact that students know in advance of the learning objectives of the module. The regulations also stipulate that they should be informed of the type of questions they can expect on the examination. They are ensured of feedback on their exams as members of staff are obliged by the regulation to create a possibility for students to look at their marked exams. Feedback on small assignments in the atelier parts is mostly provided immediately (within the atelier session). Final assessments in the atelier modules are graded and commented upon by the teaching staff. This feedback is sent to the students who are given the opportunity to respond. A special assessment form is used for the bachelor s thesis. During the master s programme, the internship (for the regular differentiation) and the master s thesis are crucial in achieving the intended learning outcomes. To test what students have learnt during their internship, organisations offering internships are asked about students attitudes and performance during this internship. This is communicated to the student and, provided it is acceptable, signed by the supervising staff member. Only with this signature are students credited with the EC points. With respect to the master s thesis, a similar form exists with references to the learning objectives. This form has to be completed by the first and second readers jointly, who also suggest an appropriate final grade. This form is part of the evidence on the day of the final exam. This involvement of three members of staff safeguards the reliability of the grading. Within the evening differentiation, two students will often write a joint thesis. This generally occurs when they are working on an external assignment, or on a consultation trajectory linked to real-life problems. Nevertheless, the evaluation criteria remain the same and, when a thesis is a joint production, students have to stipulate who has done what so that they can be individually evaluated. Lecturers themselves are responsible for a specific module and also for the design of the tests and assignments used in a specific module. It is common practice that concept exams are first reviewed by other staff members. The quality of the master s thesis is safeguarded by the fact that two readers have to agree to its standard before students can formally defend it. Students defends their thesis before for a committee of three (a chairman and the two readers). The final degree is also determined by the quality of the defence. As with all other written assignments, theses are electronically checked for possible fraud. With respect to the master s thesis a special document specifies to the students what is expected in a thesis. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 43

Unlike in the bachelor s phase, in the master s phase it is not possible to compensate for weak scores. Over the study year, there are six moments (about every two months) for students to defend their thesis and obtain their degree. Assessment During its site visit, the committee established that both the bachelor s programme and the master s programme use a variety of assessment methods which correspond to the aims and the didactic methods of the programme components. Testing of knowledge by written exams demands memorizing knowledge. The committee feels that processing knowledge is perhaps even more important and that other assessment methods are needed for that. The methods used properly reflect the level and orientation of the programmes. Written exams, for instance, are used in the assessment of courses which focus on the acquisition of knowledge and insight, while practical tests are used to assess whether students have acquired professional skills. The committee studied written exams, assignments, essays and papers produced by students of both the bachelor s programme and the master s programme and concludes that the quality and the level of the assessment and evaluation is adequate and appropriate. However, the committee finds that the assessment methods are somewhat traditional, that the variety is rather limited and that the programmes largely focus on written exams. Furthermore, most of the modules in the bachelor s phase are assessed by different assignments that make up for one final grade. Given the substantial amount of group assignments, these composed grades may carry a risk of masking unsatisfactory skills or knowledge of individual students. During the conversations with students, the committee established that significant differences can exist in the grading between different staff members. The department is aware of this observation, but claims that these differences are restricted to grading by tutors. It has taken several measures to improve this situation. For instance, the tutors discuss marking of assignments in their weekly meetings. The committee is aware of the fact that grading will never be fully objective, but it still feels that more inter-staff checks can prevent or at least decrease these differences. Overall, the committee assesses the standard that relates to assessments and examinations as satisfactory for both programmes. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Assessment of the theme Curriculum The committee comes to an overall assessment of the theme Curriculum on the basis of its assessments of the separate standards. In the case of the bachelor s programme Public Administration, it assesses this theme as satisfactory. In the case of the master s programme Public Administration, it assesses this theme as satisfactory. 44 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

1.1.3. Staff Standard 14: Requirements for academic orientation [NVAO & EAPAA] The programme meets the following criteria for the deployment of staff for a programme with an academic orientation: Teaching is principally provided by researchers who contribute to the development of the subject/discipline. Description According to the self-assessment report, a total of 28 members of the permanent staff were at least in some way active in the bachelor s curriculum in 2009-2010. In the same year, 26 members of non-permanent staff (tutors and PhD students) were involved, almost exclusively with the atelier part of the curriculum. Until the academic year 2009-2010, atelier sessions were supervised by PhD students (who have to devote 10% of their time to educational tasks), talented recent graduates (who mostly have position for two years, devoting 70% of their time to education and 30% to research the latter focussed on obtaining a PhD position) and talented master s students (who can combine this educational task with their own studies). Evaluation talks with students and tutors showed that tutors without a master s degree more often had problems reaching the atelier objectives. The relative limited distance between students and these tutors proved to be an important reason for this. Therefore, the department has decided that, starting with the academic year 2010-2011, it will only appoint tutors for the atelier components who have at least a master s degree. All academy modules are supervised by a member of the permanent staff with a PhD degree. These staff members are also responsible for organizing the atelier part of their modules. This involves preparing the workbook for the module and weekly meetings with the tutors supervising the atelier parts of the module. All core staff responsible for the modules in the master s programme hold a PhD and almost all of them contribute to the development of the discipline of Public Administration by writing books, chapters of books and articles. Sometimes staff without a PhD (almost exclusively PhD students) do play a role by giving guest lectures or supervising master s theses which are in the field of their PhD research. Assessment The committee established that most staff members who contribute to the bachelor s and master s programme have a PhD degree and conduct research which is evaluated regularly by assessment committees made up of international experts. It therefore concludes that teaching is provided by researchers who contribute actively to the development of the discipline. The department chooses to deal with the growing student population by appointing a substantial amount of PhD students and other non-permanent staff members who work as tutors in the atelier sessions. The department has stopped deploying master s students in the bachelor s programme, a decision which the committee approves. It noted that the atelier part of the bachelor s programme still relies to a substantial extent on contributions from teachers who have not completed a PhD thesis, but that the final responsibility for the contents and the assessment of courses always rests with senior staff members who have sufficient research experience. The committee considers this to be a risk in retaining the acquired level of the programme. It has established that the department has taken measures to limit this risk and that it offers teachers the opportunity to obtain a PhD degree, but that these measures will most likely not increase the amount of teachers who have a PhD within the next few years. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 45

In the view of the committee, the staff of the department contains a number of excellent researchers who contribute significantly to the master s programme. They perform renowned research and publish frequently. Some members of the staff are considered to be part of the best in the field of Public Administration. As a result, the programme has a good and solid grounding in research. The committee concludes that the bachelor s programme fulfils the criterion which relates to the academic orientation of the staff and therefore assesses this standard as satisfactory. For the master s programme the staff clearly surpasses this criterion, therefore the committee assesses the standard as good. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as good. Standard 15: Quantity of staff [NVAO & EAPAA] Sufficient staff are deployed to realise the desired quality of the programme. Description The deployment of staff is based on the educational task distribution (onderwijstaakverdeling) that is prepared annually by the chairman of the department in close cooperation with the programme director. Staff members are also consulted before the task distribution is confirmed. Permanent staff members are mainly deployed on the academy part of the curriculum. The number of tutorial groups (for the atelier components) is not fixed as the number of students can vary considerably from year to year. This can be a challenge for the department especially if the number of students rises sharply (as for instance in 2009-2010 with a 30% increase). Based on the task distribution of the staff in 2009-2010, the selfevaluation report specifies the deployment of staff in working hours and fte in the bachelor s programme. Permanent staff put in nearly 4000 hours (or 4.4 fte) to the academy component, while non-permanent staff invested more than three times as many hours in the atelier part of the programme. Members of the permanent staff are nominally expected to devote 55% of their time to education, 40% to research and 5% to management. The calculation of the student-staff ratio shows that the department invests more educational effort in the first year of the bachelor s programme. This choice is made on the assumption that first year students need more attention. The somewhat lower ratio in the third year compared to the second year is the consequence of the existence of the minors and of the fact that supervision of the bachelor s thesis implies more effort. In total, the student-staff ratio in the bachelor s programme is 29.9:1 (permanent and non-permanent staff). The relative high student-staff ratio in the bachelor s phase as such is explained by the substantial increase in the number of students (from 68 in 2004 to 172 in 2009) and the fact that the financial compensation for this is effectuated (much) later. The daytime pre-master s programme involves 16 permanent staff members and 4 nonpermanent staff members; and the evening pre-master s programme 18 members of the permanent staff. The daytime pre-master s programme calls on 1.07 fte permanent staff and 2350 hours of non-permanent staff time (the latter linked to the use of small tutorial groups). This deployment is not directly funded by the government or the university. With respect to the evening pre-master s programme, the equivalent of 1.90 full-time permanent staff is 46 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

deployed, the costs of which are partly funded by the fact that evening students pay an entrance fee of 3,000 for this pre-master s programme. With respect to the master s programme itself, 28 members of the permanent staff are involved in the daytime differentiations and 23 in the evening differentiation. The studentstaff ratio is calculated by relating the student enrolment of 2009-2010 (182) to the deployed fte in educational effort (where 1680 hours = 1 fte). In total, the equivalent of 13.74 full-time staff members is deployed with respect to the master s programme. This substantial number reflects to a large extent the number of master s theses supervised (186 in 2009). The studentstaff ratio of the master s programme is 13.2:1. All educational tasks are included within the task distribution (including thesis supervision based on a three-year average for each staff member). The results show that, in general, sufficient staff is available to run the programme. With respect to thesis supervision the department experiences a challenge to accommodate the wishes of students as some subjects (for instance currently diversity ) can be more popular than the relevant staff can cope with. Until now, the department has been able to overcome these challenges, sometimes by employing retired staff. Assessment The committee has noted that the staff members have been under great pressure due to the increasing number of students. By employing a large amount of non-permanent staff, the department has been able to realise an acceptable student-staff ratio in the bachelor s programme. The committee remarks that this ratio is acceptable partially because of the large number of PhD students and recent graduates who teach in the bachelor s programme. According to the committee, this approach may present a risk for those non-permanent staff members who are expected to produce a PhD thesis (and whose primary task is doing research). The committee holds the opinion that the growth of student numbers should in due course be followed by a growth of permanent staff. The committee is impressed by the remarkable number of staff dedicated to the master s programme. The committee therefore assesses this standard as satisfactory for the bachelor s programme and as good for the master s programme. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as good. Standard 16: Quality of staff [NVAO & EAPAA] The staff deployed are sufficiently qualified to ensure that the aims and objectives regarding the content, didactics and organisation of the programme are achieved. Description In ensuring the quality of staff, a distinction is made between the permanent and the nonpermanent staff. All potential new permanent staff are screened during the application process for their educational qualifications. In recent years (since 2008) new personnel are obliged to obtain a basic educational qualification (Basiskwalificatie Onderwijs, BKO) to ensure their teaching capabilities. Moreover, recent permanent staff have a tenure track contract in which they are challenged to meet specific research as well as educational objectives. If they do so, tenured assistant professors will eventually get a position as an associate professor. A special faculty committee (the so-called 13-15 committee) advises the Dean as to whether staff members have met the criteria and are therefore eligible for such a promotion. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 47

Each module of the curriculum has been evaluated by students. The results are used in the annual staff appraisal cycle. Based on these results, the programme director gives a judgement of the performance of staff members, which is part of the annual performance interview between staff members and their supervisor. If this judgement is unsatisfactory, staff members may be obliged to follow training courses. This has happened several times in the past, but the most recent cycle (2010, based on 2009 performance data) found that all permanent staff members were judged at least as satisfactory in terms of their teaching performance. Each tutor is selected after an application procedure in which at least two members of the decentralized management team are involved. Demonstrable educational talent and/or experience with working in groups are the most important aspect in this procedure. Tutors are trained for their specific tasks (before the start of a study year) and closely monitored. This training and monitoring is done by RISBO an autonomous organisation in the university which deals with educational quality. Tutors are also monitored and guided by members of the permanent staff responsible for the academy part of their module. Further, the atelier modules in the first half of the bachelor s programme are also monitored by the staff members who are responsible for these atelier modules. Finally, the performance of the tutors is evaluated by the students in their tutorial group. If the evaluation turns out to be to be insufficient, the contract will not be extended. Assessment The committee assesses the content-related expertise among the staff as satisfactory. It established that the areas of specialization which the programmes offer are all represented sufficiently. The committee established that the department is conscious of the importance of a good selection procedure for tutors in order to maintain a high standard in staff quality. It has noticed that tutors receive training at the beginning of the academic year and that the department has a system of coaching and counselling for the tutors. The committee appreciates these measures, but it recommends implementing more regular observations in the tutor groups by permanent staff in order to assess the performance of the tutors. The department invests in the education of its staff during the summer period and in the course of the academic year. The university offers courses and training to staff members whose didactic skills can be improved. When the performance of a staff member is assessed as unsatisfactory during the annual staff appraisal cycle, this training can be enforced. The committee established that the university has introduced the basic didactic qualification to ensure that staff members have the required didactic skills. The committee appreciates the fact that the results of the course evaluations are discussed in the annual performance interviews. The committee has noticed that staff members are strongly committed to their educational tasks. On the basis of these considerations, the committee assesses the standard relating to the quality of the staff as satisfactory for both programmes. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. 48 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Standard 17: Diversity: gender and minorities [EAPAA] The programme strives for a percentage of women among their (professional) staff, which is in accordance with the percentage of women in the workforce. When this is an issue in the wider society, the programme strives for a percentage of persons among their staff, which is in accordance with the societal targets. Description The self-evaluation report reflects on the gender balance. The department is aware that the percentage of women within the permanent staff is still below average. Promising prospects for changing this in the future do exist. Currently, there are two female associate professors and two female assistant professors. Both associate professors have been recruited since 2007 (=40% of all recruited associate professors since then), which stresses the fact that the department is trying to increase the percentage of women. Also, in 2009, 5 out of 9 graduated non permanent staff members involved in the tutorial groups were women. With respect to the PhD students, 7 out of 14 are women. In the past three years the department followed three procedures for appointing new full professorships on three occasions. On two of them a female candidate was selected, however both of them opted for a position elsewhere in Europe. The department takes care that there are female members on the selection committees. With regard to minorities, the department also tries to increase their representation, which is especially important given the large increase in the number of students with an ethnic minority background in recent years. In May 2010, the department employed one PhD student and six tutors with such backgrounds. Assessment The committee establishes that the Public Administration department at Erasmus University Rotterdam is aware of the importance of having a diverse staff. This is especially important because their student population is very diverse. Although the diversity in staff is lower than the diversity in the student population, the committee establishes that the department makes reasonable efforts in order to establish a more diverse staff. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 18: Programme jurisdiction [EAPAA] Within the framework of institutional organisation, responsibility for the programme in Public Administration rests with an identifiable person or group of persons, chosen according to the rules of the organisation. Within the framework of organisation and process peculiar to the institution, the faculty and/or administrator exercises initiative, and substantial determining influence with respect to important aspects of the programme. Description As described earlier, a decentralised management team ( bestuursraad ), consisting of a chairman, a director of research, a programme director, a financial director and a student representative supervises the Department of Public Administration. The programme director is responsible for the quality of the bachelor s and master s programmes of Public Administration. The department is relatively autonomous with respect to day-to-day matters. Within the department, each programme has its own coordinating person or team. These coordinators monitor the implementation of the programme and advise the programme director on all relevant issues. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 49

All staff are invited to attend the annual strategy days, which focus on research and educational themes. These strategy days enable the staff to evaluate relevant aspects of the programme. Once or twice a year additional staff meetings are organized to discuss matters of importance (for example in December 2009 a meeting about the skills line). Staff members also have substantial discretion with respect to the modules they are teaching (for instance with respect to the literature they want to prescribe). Assessment According to the committee the department of Public Administration is identifiable and autonomous in its responsibility for the programmes. The department has an open culture in which all staff members are invited to influence the programme. The factual responsibility for the programmes is adequately and effectively organized and the responsible parties are able to influence decisions with respect to the important aspects of the programme. Based on above considerations, the committee assesses the standard relating to programme jurisdiction as satisfactory for both programmes. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 19: Public relations [EAPAA] The programme should provide future students and others with adequate information on the programme. Description The Faculty of Social Sciences has a special public relations officer who coordinates external public relations. In close collaboration with the staff, this officer is responsible for the faculty website, the bachelor s brochure and the special master s brochure. At least twice a year, the department organises information events where potential students can get information about the bachelor s programme. Information events are evaluated on a central level. Based on these evaluations the department assesses its public relations as adequate. Assessment The committee established that the public relations of the department are good. It is impressed by the good, informative website. Like other Public Administration programmes, the programmes in Rotterdam has a rather large amount of drop-outs in the first year. This could be related to the expectations that students get from the information. The committee stresses the importance of managing expectations of future students. Furthermore, both students and the professional field are well informed about the programmes, as the committee found out in its interviews during the site visit. Relationships in the public sector are very good. The committee recommends broadening this scope even more to the private sector, where according to the professionals the committee met during the site visit there are a lot of opportunities for students for research and internships. The committee has established that the department has strong contacts with its alumni. The alumni are very positive about their education. The committee therefore assesses this standard as good for both the bachelor s programme and the master s programme. Bachelor s programme Public Administration: the committee assesses this standard as good. Master s programme Public Administration: the committee assesses this standard as good. 50 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Assessment of the theme Staff The committee comes to an overall assessment of the theme Staff on the basis of its assessments of the separate standards. In the case of the bachelor s programme Public Administration, it assesses this theme as satisfactory. In the case of the master s programme Public Administration, it assesses this theme as satisfactory. 1.1.4. Services Standard 20: Facilities Housing and facilities are adequate to achieve the learning outcomes. Description The academy-atelier formula requires relatively large classrooms for the lectures in the academy component, and smaller rooms for the atelier tutorial groups. The department uses the collective facilities of Erasmus University Rotterdam. The classrooms are generally of a good standard and in a good condition and have ample facilities for using audio-visual equipment. The department s educational office makes an inventory of the rooms needed very early (around March in advance of the new study year) and asks the central administrative office to provide appropriate rooms. With some adjustments, the department usually gets the facilities it requires. However, finding small rooms for the atelier sessions is often a challenge as almost all programmes at the university have been introducing the idea of working in smaller groups. In addition to general provisions with regard to the accessibility of rooms on campus, special lifts and toilets have been installed for the disabled. In addition, Erasmus University Rotterdam has also implemented special provisions such as wheelchairs that can be borrowed and special parking permits for this target group. The Erasmus Student Service Centre has its own Studying Without Barriers department (Studeren zonder Drempels) to support students with performance disabilities. A step-by-step plan has been developed for each form of performance disability so that students know exactly what needs to be arranged at each point during their studies. Support may concern practical matters, for example through additional funding, and matters such as extra exam time or a separate room to sit exams. In addition, Studying without Barriers also offers various courses. If necessary, students with disabilities are also allowed to take a longer time for their examination. The study counsellors advise the Board of Examiners whether or not students are eligible for any exemptions. Most students who get an exemption suffer from dyslexia. The Erasmus University Rotterdam has a central library situated on the Woudestein Campus. Around 30% of the overall central library collection of approximately one million books can be considered relevant to the Faculty of Social Sciences (Public Administration, Sociology, Political Science and, since 2001, Psychology). Of this 30%, around one-third belongs to the Public Administration collection. Given its interdisciplinary nature, much of the literature from other disciplines, such as Economics and Law, is also relevant to Public Administration. The university library has a staff member who is responsible for the social sciences information infrastructure. Close co-operation exists between this specialist and the teaching and research groups of the faculty. A faculty library committee advises on the collection policy and other relevant matters relating to the information infrastructure. A member of the Public Administration Department is a member of this committee. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 51

Every enrolled student has free access to all library facilities. The library budget for the Faculty of Social Sciences amounts to 270,000. An overall budget of 1,770,000 is available for electronic databases. These databases can be accessed in the library on one of the 188 PCs and at home through VPN. The full texts of nearly 21,000 journals are available, in licensed publisher packages or through Open Access channels, to students and staff of the university. The Resources for Public Administration portal brings together all Public Administration information resources and Dutch and European government information that are available through the website of the University Library. A modular library instruction programme has been developed especially for PA students. This consists of combinations of general and discipline-specific training and workshops that build on each other and that can be followed throughout the curriculum. All students are provided with e-mail addresses and have free access to the Internet. Two computer rooms with 90 computers are available to students at the faculty. These rooms are so intensively used by students that the faculty has added two extra computer rooms with 40 computers to facilitate more small-group work. Some students point out that in the library available computers can be rare, but they may find a computer elsewhere on the campus. The PCs are connected to a network, allowing access to a broad selection of applications. Programmes relevant to teaching purposes that are available include MS Office and SPSS. The computers in the computer rooms naturally provide free access to the Internet. The Blackboard e-learning platform was introduced at the faculty in 2000. Staff used Blackboard as an e-learning platform to communicate with students about their courses. It was also used as a medium to test whether or not student assignments were all their own work or taken from the web (plagiarism). A second ICT platform was used until September 2009 by the educational office to communicate with students. Surveys showed that most of the students appreciated the platform and that they would like to see all the courses available on Blackboard. Having two ICT platforms, however, was not an ideal situation and, at the start of the 2009-2010 academic year, Blackboard was therefore replaced by a new opensource learning environment called BSK-web. This new environment was developed by the faculty s own staff using insights learned during the development of an e-learning platform for the Psychology department. This new e-learning platform followed up on the most important suggestions for improvements by the students, in particular their need for a single, fully-integrated e-learning platform. In addition to teaching staff, now the educational office, the study counsellors, the Board of Examiners and the University Library all are present within this new e-learning platform. By including the University Library, students are now able to find and pay for all the learning materials they need during their courses. By offering students digital readers, the faculty has been able to substantially reduce the costs of studying for the students. Due to the open character of the new learning platform, it has been possible to allow students access to all their digital content, free of charge. Assessment During its site visit, the committee studied the material facilities available to students of the bachelor s and the master s programme on the Woudestein Campus. On the basis of its own observations during a guided tour, the information provided in the self-evaluation report and the interviews with students, the committee concludes that the material facilities are good and that they enable the programmes to realise their final qualifications. It appreciates the new and modern L-building, which provides good classrooms. The quality of the lecture halls and classrooms is good. 52 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

The library is very complete. The committee established that it offers students all the literature they may need for their education and research. The computer facilities and number of workplaces for self-study are adequate. The committee has not received any information which points at problems or obstacles with respect to the facilities. The committee assesses this standard as good for both the bachelor s and the master s programme. Bachelor s programme Public Administration: the committee assesses this standard as good. Master s programme Public Administration: the committee assesses this standard as good. Standard 21: Tutoring Tutoring and information provision for students are adequate in view of study progress. Tutoring and information provision for students correspond with the students needs. Description The introduction of small tutorial groups as part of the restructuring of the bachelor s curriculum in 2007 has had an important advantage with respect to the tutoring of students. The tutors supervising these groups will note when students in their group are experiencing problems (for instance when study progress is lacking or when they perceive personal problems). If necessary, they will refer these students to the study counsellors. Two study counsellors are available for all students who experience any kind of study related problem and need advice. The study counsellors are part of the educational office. This office also assembles the study guide. To further strengthen the information supply to students, the programme director writes a general memo to all students about three times a year with relevant information. This memo also contains information about how students can find information they are looking for and how they can complain about something if they want to. Twice a year the bachelor s coordinating committee holds meetings for each year s students where they are informed about important aspects of their curriculum. Information about the minors that bachelor s students can follow is offered at the central university level. Since 2005-2006, the department has provided a binding study advice (BSA) in the first year. This consists of three pre-advices (in February, April and June), and one definitive advice in the last week of July. Unless there have been extenuating personal circumstances (agreed by the study counsellor), students who have failed to achieve 40 EC in the first year are obliged to leave the programme. Students with between 40 and 60 EC receive a temporary positive advice, meaning they can progress to the second year, but have to finish their first year courses within the following year. During the year, any students who receive negative preadvice reports are invited to a meeting with the study counsellor. There, they can discuss their problems and, if possible, the study counsellor will help them, for example, by referring them to student psychologists or to workshops about effective studying or making a realistic study plan. The department has the strong impression that the BSA approach has helped not only the average student to perform better, but also helps the weaker students to reflect on their study strategy and on whether alternative studies might be more appropriate. The department has introduced additional measures to provide information and service to students of the master s programme. Although obtaining an internship is primarily the responsibility of the students themselves, the department supports them in several ways in doing this. The individual specialisations use their networks to identify potential positions, and there is also an internship coordinator who collects information on possible internships QANU & EAPAA / Public Administration, Erasmus University Rotterdam 53

and communicates this to students. This coordinator also communicates with the students about the rules and regulations with respect to an internship. To start with the internship, students must have finished their bachelor s or pre-master s and obtained at least 10 EC from the master s programme. During the site visit, staff and students explained that there are no problems finding an internship for students. In addition to the measures mentioned in the self-evaluation report, the network of students in the evening programme is a possible source of internships for students who have difficulties finding an internship. Finding a member of staff to supervise a master s thesis is supported in several ways. In the first place, there is a separate brochure available for students which outlines the specialisations of the staff. In recent years, supervisors have been assigned in the daytime master s differentiations to thesis groups made up of about four students. Students who fall behind or for some other reason do not participate in thesis groups can still use the brochure to help identify a suitable supervisor. In the evening programme, two members of staff are available for students to consult with, and they can offer advice in the initial phase of the thesis project and also suggest supervisors. Students often take longer than programmed to complete their master s thesis. Especially if they have postponed starting the work on their thesis, it is possible that they lose contact with the department as writing their thesis is the only thing left to do. To counter this, a special unit ( tempo makers ) keeps track of these students, contacts them if necessary and connects them to a possible thesis supervisor. In order for this task to run effectively, an officer in the educational office keeps track of the progress with master s theses. Since September 2009, the department has taken steps to digitalise this process through an electronic system (DigiDesk). Assessment The committee has established that the tutoring and the provision of information for both the bachelor s programme and the master s programme are adequately organised and put into practice. It appreciates the good connection between the bachelor s tutors and the study counsellors. Tutors have regular contact with students and therefore recognise students who have difficulties with their study progress in an early stage. Their close contacts with the study counsellor make this a good early-warning system. Tutors also address the study counsellors themselves to get advice on how to cope with students who have difficulties. The committee established that the current capacity for the study counsellors is sufficient and that their work load is acceptable. It noticed that the programmes are organised in a rather informal way. Staff members are easily accessible for students and they contribute to the support and guidance of students. The committee noted that the system of tutoring is more or less the same for both programmes. However, the committee feels that the bachelor s programme has adopted a more active and systematic approach and that it follows students and their progress more closely. The committee therefore assesses this standard as good for the bachelor s programme and as satisfactory for the master s programme. Bachelor s programme Public Administration: the committee assesses this standard as good. Master s programme Public Administration: the committee assesses this standard as satisfactory. 54 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Assessment of the theme Services The committee comes to an overall assessment of the theme Services on the basis of its assessments of the separate standards. In the case of the bachelor s programme Public Administration, it assesses this theme as satisfactory. In the case of the master s programme Public Administration, it assesses this theme as satisfactory. 1.1.5. Internal quality assurance system Standard 22: Periodical evaluations The curriculum is periodically evaluated in the light of verifiable objectives and other measures. Description According to the self-evaluation report, the programmes have explicit targets that are evaluated periodically and to which the infrastructure is geared, both in the sense of teaching and from personnel and organizational perspectives. This enables a structured quality cycle according to a plan-do-check-act system. The targets focus on: positive evaluations of modules, each module should be evaluated internally and achieve at least a satisfactory score (i.e. a 3 on a five-point scale or a 6 on a ten-point scale). Further, the aim is for modules to score at least a 3.5 (or 7 on a ten-point scale); achieving a top three national position compared to the other Dutch Public Administration programmes. The nationwide independent Elsevier survey is used to measure this; on the programme level, results should fit the EUR 2013 strategy; this means that 60% of the students starting in the bachelor s programme should achieve a positive binding study advice within the two-year term and subsequently that 90% of those should finish the programme. For the master s level there is an additional target to achieve a higher pass rate and a better study pace than is currently the case. The aim is that 90% of the day students and 80% of the evening students graduate. An aim for the future is also that 60% of those who start the master s programme, having completed their bachelor s or pre-master s programme should finish their degree within one year. To measure the results with respect to the first objective, the department uses an internal measurement system. In terms of the second, the Elsevier survey, which is published every year, evaluates facilities, programme organisation, the staff, assignments, and organisation and communication. Since the introduction of the new bachelor s curriculum, the programme s position in this survey has improved substantially (from a seventh and last position in 2007 to the third position in 2009). For the master s programme, there are only data available for 2009, when the programme shared a second position. With respect to the third target, an important instrument for the evaluation of the entire programme is provided through the management information which is compiled about every six months by the educational office and which holds relevant performance information. The quality of the degree programme is monitored on several levels and in multiple, integrated ways with the so called SSK model. Firstly, each module is systematically evaluated by students on the basis of open-ended and closed (Likert-type) questions. The open-ended questions enable students to respond more specifically on certain components of a course that they particularly liked or disliked. The outcomes of these evaluations are immediately sent to the respective staff members. They can use this information (for instance about student s perception of the structure of the module, the literature or guest lecturers) to decide whether changes in the module are required for the following course year. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 55

At the level of the programme year, all course evaluations are used by the bachelor s coordinating committee, and by the programme director, to assess how modules are perceived by students and whether or not changes (for instance in module content or literature used) are called for. The programme director also uses this information in the appraisal cycle. Complementary to this, students are onvited two or three times a year to take part in focus groups in which the curriculum is discussed, for instance with respect to the synchronising of courses and the workload. These focus groups make it possible to delve deeper into relevant issues (and if necessary possible solutions). The board of studies is also sometimes explicitly asked to evaluate parts of the programme, but it can also discuss all educational matters it sees as relevant. If the evaluations suggest potential problems with a module, these are discussed by the programme coordinators with the responsible staff member, so that improvements can be made (for example, in 2009-2010 the ateliers of public management and labour, organisation and management have been slightly changed based on evaluation results). Recently, the department realised that thesis supervisors were not systematically evaluated. Steps to improve this situation have therefore been taken. Alumni surveys are held from time to time (irregularly, but the aim is to hold one every few years), the last one being in 2009 and involving 515 former students. At the curriculum level, the department organises strategy days each year (cf. standard 18), which enable the staff to evaluate relevant aspects of the programme in a more qualitative way and to reflect on longer term changes. Finally, the decentralised management team meets about twice a year with the Board of Advisors. This quality assurance system is embedded in the quality assurance policy that is being developed for Erasmus University Rotterdam by a university-wide working group in order to obtain university-wide accreditation. A periodic system of interim degree programme reviews (so-called mid-term reviews) was introduced by the Board of the university several years ago. Here, a committee assesses, on the basis of a self-evaluation and a one-day visit, whether the degree programme has adequately implemented improvements arising from the previous accreditation report. This mid-term review takes place three years after the official assessment leading to a renewal of the accreditation, most recently in 2007. Assessment The committee studied the internal quality assurance system of the department of Public Administration of Erasmus University Rotterdam. The committee established that the courses of the curricula of both the bachelor s programme and the master s programme are evaluated periodically and systematically and that the department uses target figures to assess the outcomes of the evaluations. The committee studied the questionnaires which are used for the course evaluations. It noted that these questionnaires provide useful and systematic information about students assessment of the courses. The committee is positive about the additional efforts undertaken by the department to collect information about the courses and the curricula, such as the organisation of focus groups with students. As a result, the department and the management of the programmes have a lot of information at their disposal which they can use to assess and improve the quality of the programmes. The committee established that the responsible staff of the programmes play a central and leading role in the evaluations, in the analysis of their results and in the formulation of measures to improve the quality when necessary. 56 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

The committee established that the programmes as a whole are evaluated in a number of ways: by means of evaluations among graduates of the bachelor s and master s programme and in the meetings of the department. Furthermore the committee appreciates that the quality system of the department is connected with the faculty and university quality assurance. The system makes a professional and useful impression. On the basis of these considerations, the committee concludes that both the bachelor s programme and the master s programme fulfil the criterion with respect to the evaluation of results. It therefore assesses this standard as satisfactory for both programmes. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 23: Measures for improvement The outcomes of the evaluation form the basis of verifiable measures for improvement that contribute to the achievement of the objectives. Description The self-evaluation report describes several improvement measures that have been implemented for both the bachelor s and the master s programme. the recommendations of the 2003 assessment committee were used as a starting point in the re-orientation of the bachelor s programme; all the recommendations of this committee (relating, for example, to the position of quantitative research, the integration of law modules and management information) have been implemented. It should be noted that the mid-term assessment committee positively evaluated the progress made following the assessment committee s report of 2005; in line with suggestions for further improvements made in the mid-term review, the following measures have been taken since early 2008: a. mission, aims and objectives plus intended learning outcomes have been discussed with the staff and rewritten; b. although the mid-term visitation committee questioned the option of compensating for insufficient exam marks, this possibility following positive evaluations with both the Public Administration curriculum as well as elsewhere on the Erasmus University Rotterdam campus has been retained; c. the recommendation to develop an integrated vision with respect to the skills taught in the atelier components, and to integrate the ateliers across the separate modules, has been implemented; d. the didactical concept of the curriculum has been made more explicit. The same is true for the didactical lines within the programme; e. the two ICT platforms have now been merged into one e-learning platform; f. students appear to spend more time studying than was typical with the previous bachelor s programme; g. additional measures for the mentoring and training of tutors have been implemented; h. the governance code is now more explicit. based on results of the Elsevier survey, several weak points have been successfully addressed, for instance communication to students a very weak point in the 2006 survey. Recent survey results show an improvement (from 5.9 in 2006 to 7.5 in 2009); as will be seen in the results section, in some respects the success rate of the bachelor s students has been somewhat disappointing in the past (both in overall success rate and in study progress). As noted earlier, several measures to improve this situation have been QANU & EAPAA / Public Administration, Erasmus University Rotterdam 57

taken in recent years. First, the restructuring of the bachelor s curriculum reflects the department s assessment of this situation. The same holds for the introduction of the binding study advice (BSA) in 2005 and for several measures taken by the Board of Examiners. In recent years, the Board of Examiners has introduced more stringent requirements for bachelor s students who have not yet finished their programme but want to enter the master s programme. The number of uncompleted EC allowed in order to enter the master s programme has been decreased and, probably most importantly, it is obligatory for students to finish their bachelor s thesis before entering the master s programme. the atelier sessions that have been introduced in all modules contribute both to group cohesion and a spreading of the workload. Also, they are very important with respect to skills development. Nevertheless, creating atelier sessions which are varied and challenging and have added value for students is a challenge in itself. Although formal evaluations of the atelier modules are positive, discussions within the focus groups and with the board of studies frequently show that room for further improvement certainly exists. In response, the department decided in April 2010 to include elements of the Harvard Case Method within the atelier sessions. This is something that cannot be done instantly, and the department intends a gradual implementation over the next two years. On a programme level, recent master s programme evaluations have been positive. This applies to the Elsevier survey of October 2009 and the report of the mid-term assessment committee in 2007. Further, the alumni survey shows, in terms of most of the reviewed aspects (staff, modules, lectures, curriculum design, scientific quality and societal relevance), that alumni graduating after 2000 are more positive than those who graduated in the 1980s and 1990s. This suggests that the measures the department has taken to improve the quality have had an effect. Evening alumni are more positive about their curriculum than their daytime equivalents. Looking back to the recommendations put forward by the previous assessment committee, the department has responded to an important and critical point. The previous committee was critical of the Public Administration character of two specialisations in particular: Labour, Management and Organisation (AOM); and Governance and Management of Complex Systems (GMCS) (which was then called Policy and Management of Complex Spatial Developments). Changes in response to the critical remarks have been made. For example, with respect to the AOM specialisation, the then existing cooperation with the Sociology department was terminated in 2006 and (as requested by the committee) students are obliged to follow their internship within the public domain. The department constantly aims to further improve the quality of the programmes. 1) Students often take longer than a year to finish their master s studies. After positive trials in 2008-2009, thesis groups have been introduced within all the daytime specialisations for the 2009-2010 study year in order to stimulate a faster throughput time. Similar measures have been taken in the evening differentiation. 2) Based on an assessment by the evening coordinating committee, a second methodology module was developed for the evening differentiation and a more structured time schedule for the thesis component was introduced, starting right from the beginning of the master s year. This has already led to a substantial shortening of the average study duration (especially in terms of the thesis period). 3) Following evaluations by students (supported by the Programme Committee), the choice of electives was slightly expanded in 2009-2010. Until then, only introductory modules 58 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

could be chosen as electives, whereas the choice is now larger with respect to the second elective. 4) The former Policy and Management of Complex Spatial Developments specialisation was evaluated very positively by students, and a high percentage of participants graduated on schedule after one year. However, it did not attract sufficient students to be viable and as a consequence, the specialisation has been revamped with an entirely new programme and was renamed Governance and Management of Complex Systems. The enrolment increased from 17 in 2008-2009 to 25 in 2009-2010. 5) In response to the slow study progress, measures have been taken by the Board of Examiners to stimulate faster completion of the bachelor s phase so that students can focus more fully on the master s curricula and complete this more rapidly. The number of missing EC allowed has been reduced (from 15 to 10 EC) and, more importantly, since 2008-2009 bachelor s students must have finished their bachelor s thesis before they can enter the master s programme. The management information shows that this has had a large effect as the percentage of students finishing their bachelor s thesis on time has increased significantly. 6) Following the introduction of a new bachelor s curriculum, the pre-master s curriculum has also been revised. This curriculum now also involves working in small tutorial groups. 7) The evening differentiation is continuously being improved on the basis of experience and evaluations. The curriculum level is also regularly scrutinised, both by the programme management and through consultations with students (focus groups) and staff. Curriculum level improvements in the master s programme focus on: a. facilitating the learning process and time management of students (for example, by changing the order and the time schedule of modules and by redistributing the available time through the reallocation of EC). b. substantive improvements in response to changing practices. Following the visible trend that both governmental bodies and increasingly societal groups and institutions contribute to steering in the public domain, some of the emphasis in the curriculum is changing. 8) The IMP differentiation has adopted several important measures in recent years: a. Following requests from students, a module on global governance has been included that adopts an international relations perspective. This has contributed to a more robust International Pillar. b. In order to improve the timeliness of thesis completion, the part of the curriculum on Research Design, as part of Professional Development, was intensified in 2008-2009 through a series of assignments and additional working group sessions. The aim was to help students produce a research proposal more quickly. Also a more proactive approach to monitoring student progress has been recently adopted. This has contributed to a substantial increase in the proportion of students who complete their thesis within one year. c. The IMP staff tries to concentrate thesis supervision within the core staff as this helps the coordinators of the programme monitor student progress and maintain schedules. Assessment The committee studied information about various examples of measures aimed at improving the quality of the curricula of the programmes. It is particularly positive about the process which has led to the restructuring of the curriculum of the bachelor s programme. In the committee s opinion, the process convincingly shows that the programme is able to formulate and implement measures which have the desired effect on the quality of the curricula. The committee very much appreciates the outcomes of the process and is positive about the revised curriculum of the bachelor s programme. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 59

In addition, the committee noted various examples of measures taken to improve the quality of individual courses or parts of the bachelor s and master s curriculum. For example the atelier sessions are continuously improved, even if formal evaluations have a positive trend. In the master s programme the committee appreciates the introduction of tutor groups during the thesis phase. These examples provide further confirmation that the programmes are able to formulate and implement measures which have the desired effect. This shows that the system of quality assurance functions adequately. The committee assesses the standard which relates to the development and implementation of measures for improvement as good for both programmes. Bachelor s programme Public Administration: the committee assesses this standard as good. Master s programme Public Administration: the committee assesses this standard as good. Standard 24: Involvement of staff, students, alumni and the professional field Staff, students, alumni and the relevant professional field will be actively involved in the internal quality assurance system. Description Earlier in this report the involvement of students (evaluations, focus groups, Programme Committee) and staff (Programme Committee, strategy days and other periodical meetings about educational matters) has already been addressed. The department has a Board of Advisors (consisting of alumni) which is consulted twice a year, for instance with respect to matters concerning the mid-term visitation. There is also an active alumni association (ABEUR) and alumni are approached for help in locating short internships for bachelor s students. Alumni are also regularly asked to give guest lectures (for instance in the module Reflection on professional practice). As part of the department s 25th anniversary (in 2009) a survey was held which provides information about how our alumni evaluated the Public Administration programme of Erasmus University Rotterdam programme as a whole. Assessment The committee has established that staff members and students of both programmes contribute significantly to the internal quality assurance, via the board of studies, the evaluations of the courses and the programme, the strategy days and the regular staff meetings. It has noted that the staff plays a leading role in the development of the curricula. The committee is very positive about the commitment and the dedication of staff members and students, which lead to homogeneous programmes and a good and open atmosphere. The programmes have adopted a bottom-up approach which is very effective. The committee has noted that the involvement of graduates of the programmes and of the professional field is very good as well. The Board of Advisors is actively involved in the internal quality assurance of both programmes. The committee advises the department that future political developments may make the master s programme more expensive for students. In that situation it is even more important to maintain structural contacts with the professional field, both to acquire more students and to attract more private funding of research and education. The evening programme is already 60 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

a very useful instrument to maintain these contacts. The committee furthermore appreciates the recent alumni evaluation that the department has set out. The committee assesses this standard as good for both the bachelor s programme and the master s programme because it wants to express its enthusiasm for the strong contributions from students, staff members and the professional field to the internal quality assurance. Bachelor s programme Public Administration: the committee assesses this standard as good. Master s programme Public Administration: the committee assesses this standard as good. Assessment of the theme Internal quality assurance system The committee comes to an overall assessment of the theme Internal quality assurance system on the basis of its assessments of the separate standards. In the case of the bachelor s programme Public Administration, it assesses this theme is satisfactory. In the case of the master s programme Public Administration, it assesses this theme as satisfactory. 1.1.6. Results Standard 25: Achieved learning outcomes The achieved learning outcomes correspond with the aims and objectives regarding level, orientation and subject- /discipline-specific requirements. Description Formally, each module is supposed to contribute to the intended learning outcomes of the programme. The fact that all learning outcomes are made operational as learning objectives within modules in theory safeguards this. When discussing learning assignments, the last two modules in the bachelor s programme (the bachelor s thesis and the module reflection on professional practice) are especially important in determining whether students really have acquired the intended knowledge and skills. According to the self-evaluation report these show that they indeed have acquired the intended learning outcomes. The department has an explicit and systematic organisation of the bachelor s thesis, students receive instructions and the criteria used to evaluate their end products are explicit as well. For the master s programme, the self-assessment report refers primarily to the master s thesis. The department is convinced that the theses demonstrate that the programme indeed realises the learning outcomes set out to achieve. Another, more indirect, way to assess whether or not the department achieves the intended learning outcomes is to look at what happens with the alumni. The alumni survey shows that, in general, alumni indicate that their education has helped them to gain an academic orientation, has helped them to achieve a good knowledge of Public Administration theories and of the public sector. Further, 90% evaluated the relation between their education and the labour market as satisfactory or better. This is despite the fact that alumni go on to work for a broad range of employers. The most recent alumni survey shows that 52% have a job within the public sector in a narrow sense (national and local government), 14% in the non-profit sector (health, education) and 4% at agencies. Moreover, 28% work in the private sector (including consultancy organisations). Looking at those graduating from the daytime courses since 2000, 67% obtained a Public Administration-relevant job immediately or within four weeks of graduating, 22% took longer but did find such a job, and only 12% did not find employment where there master s was of direct relevance. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 61

Assessment The committee has established that the level and the quality of the concluding theses of the bachelor s programme and the theses produced by students of the master s programme are at least satisfactory. The documents studied by the committee reveal that students have acquired knowledge and understanding at a level which suits a bachelor s or a master s programme and that they are able to conduct research at that same level as well. The committee therefore concludes that the achieved learning outcomes, as exemplified in the theses, sufficiently correspond with the intended learning outcomes of the programmes. In general the committee shares the assessment of theses. The committee took into account the fact that the master s programme is a one-year programme. This holds a risk that students aim to achieve more than reasonable in one year. It recommends the department to enforce the guidelines for the length of theses more strictly in order to protect students from doing so. The committee noted that graduates of the bachelor s programme are able to continue their studies in the master s programme without any problem. Graduates of the master s programme manage to find a position on the labour market with relative ease. They consider themselves sufficiently prepared for their first job. In the committee s opinion, this means that the actual competences which students have achieved in the course of the programmes correspond sufficiently to the demands of the discipline and the professional practice. On the basis of these considerations, the committee concludes that both the bachelor s programme and the master s programme fulfil the criterion which relates to the achieved learning outcomes. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Standard 26: Study progress Target figures that are comparable to other relevant programmes are formulated to express the expected success rate. The programme s success rate complies with these target figures. Description The department of Public Administration has extensive data on the study progress of both bachelor s and master s students. This is important because several targets to be achieved by 2013 are defined in the Erasmus 2013 strategy. Most notably, that 60% of the students starting the bachelor s programme should have a positive BSA within 2 years and that the success rate for those achieving a positive BSA should be 90%. The programmes have not yet met these criteria. Current figures suggest that they probably will soon. The self-evaluation report shows a table about the achievements of bachelor s students since 2003-2004. Two things are apparent: there has been a large number of drop-outs and many students take much longer than the nominal period of three years to graduate. Both were important reasons for restructuring the bachelor s programme and also for establishing the convenant with respect to Erasmus 2013. An important problem with assessing the relevance of the preceding data is that many things have changed since the 2003-2004 intake. Not only has the curriculum changed (with objectives included for a better success rate and a faster study progress), but since 2005-2006 the concept of binding study advice has been introduced and the Board of Examiners has imposed stricter rules with respect to entering the master s programme. It is therefore 62 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

important to have a closer look at the more recent cohorts, especially since the introduction of the BSA. The figures show that the number of students leaving before February 1st fluctuates (and was especially high for the 2006-2007 cohort). The preliminary conclusion is however that the drop-out rate has decreased following the introduction of the restructured bachelor s curriculum in 2007-2008. With the exception of 2006-2007, the percentage of students with a negative BSA is relatively consistent at slightly below 25%. The percentage of students receiving a positive advice fluctuates somewhat more, with 2006-2007 once again an outlier. The percentage was highest in 2005-2006, although it must be noted that the percentage of students that had dropped out before February 1st was also relatively high in that year. The figures of the most recent cohort (2008-2009) are slightly better than those of 2005-2006. One should not only look at students with a positive BSA at the end of the first year but also at those with a conditional positive advice (who have achieved between 40 and 60 EC in the first year). In practice, most of these students go on to receive a positive advice in their second year. Excluding those students who decide to leave the department relatively soon after joining (i.e. before February 1st), a large majority of students successfully finished the first year s components of the degree course, with the study year of 2006-2007 being clearly a negative dissonant. In that year, the percentage is below the Erasmus 2013 strategy target of 60%. In the other years it is above that target. The self-evaluation report shows a large difference in success rates between those receiving a positive BSA in their first and second year. Clearly, students who fall behind in their first year struggle to catch up. Another observation is that most students who perform well in their first year keep up this performance until the end. From this perspective, the BSA status at the end of the first year seems to be a very good predictor of how students will do later on. It may be observed that many students take a rather long time over a study with a nominal duration of three years. This can partly be explained by administrative reasons, which have recently been addressed (some students had actually finished their bachelor s training but were not aware that they could immediately apply for their degree, and sometimes waited many months with their application request). The restructured bachelor s curriculum should enable students to finish their degrees faster, but it will take a few years before the data will show that this is indeed happening. The Erasmus 2013 convenant does not provide exact figures for the expected study success rates in the master s programmes, but it stipulates that the programme should be among the best three master s programmes in the Netherlands. The department aspires to achieve an eventual pass rate of 90% for the daytime options, and 80% for the evening students. Another aim is that 60% of those who start a master s programme, having completed their bachelor s or pre-master s programme, should finish it within one year. The department has not met these targets yet, but is making progress in the right direction. The recent introduction of thesis groups is an important measure in eventually meeting these goals. The main conclusions that can be derived from management information are that: Only a small minority of students fail to graduate. In the daytime differentiations, around 10-15% fails to eventually gain a master s degree and in the evening stream the percentage QANU & EAPAA / Public Administration, Erasmus University Rotterdam 63

is slightly higher. After three years, between 85% and 90% of daytime students, and around 75% of evening ones have obtained their degree. This is close to the target. The majority of students take more than the nominal year to achieve this goal. Many take more than two years. Specialisations that have been using thesis groups for some time (GMCS for many years, AOM since 2008) have higher success rates. Measures taken in the evening differentiation to boost the study pace have been successful. The percentage finishing within eighteen months has increased from below 20% to almost 50%. Students who start the master s programme having completed their bachelor s degree or a pre-master s programme complete their master s study in a shorter period than those with missing obligations at the start of their programme. Students who have followed the daytime pre-master s programme achieve similar results (both in terms of success rate and final degree score) as former bachelor s students. In terms of final degree, evening students on average end up with slightly lower scores than those on the daytime programme. Assessment The committee established that both programmes have defined target figures for their success rates and that these target figures are realistic and sufficiently ambitious. For both programmes, the target figures have not been met yet. The committee appreciates the efforts taken by the department to improve the success rates. It is aware of the fact that the programme has limited means and that it is to a certain extent dependent on measures or decisions taken at a higher level. For instance, the programme is unable to change the rule that students who have not completed their bachelor s programme are allowed to register for courses of the master s programme. The committee welcomes the decision to apply the existing rules more strictly. It finds that both programmes should continue to investigate opportunities to further improve the success rates. Given the circumstances, and given the fact that low success rates are a more general problem in the Netherlands, the committee finds the actual success rates achieved satisfactory, even though they can certainly be improved. In this respect, there is no distinction between the programmes offered by Erasmus University Rotterdam and other programmes. The committee therefore assesses the standard related to the success rates as satisfactory for both the bachelor s and the master s programme. Bachelor s programme Public Administration: the committee assesses this standard as satisfactory. Master s programme Public Administration: the committee assesses this standard as satisfactory. Assessment of the theme Results The committee comes to an overall assessment of the theme Results on the basis of its assessments of the separate standards. In the case of the bachelor s programme Public Administration, it assesses this theme as satisfactory. In the case of the master s programme Public Administration, it assesses this theme as satisfactory. 64 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Overview of the committee s assessment Bachelor s programme Public Administration: Theme Assessment Standard Assessment 1. Aims and Satisfactory 1. Mission-based accreditation Satisfactory objectives 2. Subject-/discipline-specific Satisfactory requirements 3. Bachelor and master level Satisfactory 4. Academic orientation Satisfactory 2. Curriculum Satisfactory 5. Public administration character of the Good programme 6. Requirements for academic orientation Satisfactory 7. Correspondence between the aims and Satisfactory objectives and the curriculum 8. Consistency of the curriculum Satisfactory 9. Workload Satisfactory 10. Admission requirements Satisfactory 11. Credits Complies 12. Coherence of structure and contents Good 13. Learning assessment Satisfactory 3. Staff Satisfactory 14. Requirements for academic orientation Satisfactory 15. Quantity of staff Satisfactory 16. Quality of staff Satisfactory 17. Diversity: gender and minorities Satisfactory 18. Programme jurisdiction Satisfactory 19. Public relations Good 4. Services Satisfactory 20. Facilities Good 5. Internal quality assurance system Satisfactory 6. Results Satisfactory 21. Tutoring Good 22. Periodical evaluations Satisfactory 23. Measures for improvement Good 24. Involvement of staff, students, alumni Good and the professional field 25. Achieved learning outcomes Satisfactory 26. Study progress Satisfactory The committee concludes, on the basis of its assessments of the themes and standards from the assessment framework, that the bachelor s programme Public Administration fulfils the formal requirements which are a prerequisite for accreditation. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 65

Master s programme Public Administration: Theme Assessment Standard Assessment 1. Aims and Satisfactory 1. Mission-based accreditation Satisfactory objectives 2. Subject-/discipline-specific Satisfactory requirements 3. Bachelor and master level Satisfactory 4. Academic orientation Satisfactory 2. Curriculum Satisfactory 5. Public administration character of the Satisfactory programme 6. Requirements for academic orientation Satisfactory 7. Correspondence between the aims and Satisfactory objectives and the curriculum 8. Consistency of the curriculum Satisfactory 9. Workload Satisfactory 10. Admission requirements Satisfactory 11. Credits Complies 12. Coherence of structure and contents Satisfactory 13. Learning assessment Satisfactory 3. Staff Satisfactory 14. Requirements for academic orientation Good 15. Quantity of staff Good 16. Quality of staff Satisfactory 17. Diversity: gender and minorities Satisfactory 18. Programme jurisdiction Satisfactory 19. Public relations Good 4. Services Satisfactory 20. Facilities Good 21. Tutoring Satisfactory 5. Internal quality Satisfactory 22. Periodical evaluations Satisfactory assurance system 23. Measures for improvement Good 24. Involvement of staff, students, alumni and the professional field Good 6. Results Satisfactory 25. Achieved learning outcomes Satisfactory 26. Study progress Satisfactory The committee concludes, on the basis of its assessments of the themes and standards from the assessment framework, that the master s programme Public Administration fulfils the formal requirements which are a prerequisite for accreditation. 66 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

APPENDICES QANU & EAPAA / Public Administration, Erasmus University Rotterdam 67

68 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Appendix A: Curricula vitae of the members of the assessment committee Professor C. (Christoph) Reichard is emeritus professor of Public Management at the University of Potsdam. He is a member of the Potsdam Centre of Policy and Management and of the Institute of Local Government Studies. He was and is guest professor at several universities, including universities in Milan (Bocconi), Rotterdam, Siena and Vienna. He is chair of the European Association of Public Administration Accreditation (EAPAA). His main fields of research include (new) public management, governance issues, public financial management and public personnel. His recent research projects deal with the evaluation of national and international trends of new public management, corporate governance problems of public enterprises, the use of financial data for managerial decisions, and education and training in the German public sector. He published about 240 books and articles. Professor J.J. (Jaap) Boonstra is professor Organizational Change and Learning at the University of Amsterdam and at Esade Business School in Barcelona (ES). He is the Dean and chairman of the scientific board of Sioo, the National Inter-university Center for Organizational Change and Learning in the Netherlands. He is a member of the board of governors for a wholesale organization and a youth care institute, and a member of the advisory board of the Dutch Immigration Services and the Dutch Academy of Governmental Communication. As a consultant, he is involved in change processes in organizational networks in the Netherlands, Spain and South Africa. His research focuses on transformational leadership, barriers to organizational change and innovation, power dynamics in organizational change, and sustainable development of organizations. He published more than two hundred articles on technological and organizational innovation, management of organizational change, organizational learning, politics in organizations, strategic decision making and transformational change in the service sector and public administration. Professor J.A. (Hans) de Bruijn is professor of Public Administration at Delft University of Technology. He is one of the leaders of the Policy, Organization, Law & Gaming Group, which comprises some 30 researchers, many of whom are social scientists or legal experts. The group is part of the Faculty of Technology, Policy and Management. The Faculty encompasses experts in the fields of Policy Analysis, Systems Engineering, Economics and Business Administration and a large number of experts in technical areas of application such as energy and industry, and transport and logistics. Hans de Bruijn is member of the Management Team and research director of the faculty. He is attached as an associate to Berenschot Process Management, The Hague. As a consultant, he has been commissioned by a large number of parties. These recently included the Netherlands Ministry of Education, Culture and Science, the Province of South Holland, the Ministry of Housing, Spatial Planning and the Environment, the Confederation of Netherlands Industry and Employers, the Institute for Inland Water Management and Waste Water Treatment, Berenschot, the Ministry of Transport, Public Works and Water Management. Professor H. (Harald) Sætren is head of department of the Department of Administration and Organization Theory of the University of Bergen. In 1983, he obtained his PhD degree in at the University of Bergen. In 1987, he became full professor in the Department of Administration and Organization Theory. He was head of department in 1988-1989 and from 1999 until 2001 as well. He currently co-chairs the EGPA permanent study group on Public Policy. He was a member of international expert teams appointed by the respective ministries of education to evaluate Public Administration programmes in Estonia in 2009, 2006, 2001 and in Lithuania in 2006 and 2004. He is a member of the editorial board of the Journal of QANU & EAPAA / Public Administration, Erasmus University Rotterdam 69

Comparative Policy Analysis and a member of the Advisory Committee to the Scandinavian Political Studies journal. His research focuses on public policy and policy changes. He published numerous books and articles, including a recent book on change and continuity in public organizations. Professor J. (Juraj) Nemec is professor of Public Finance and Public Management at the Faculty of Finance of the University of Banska Bystrica. He holds an MBA in Business Administration and a Ph.D. in Public Sector Economics and he has more than 28 years of experience in teaching in public sector management and procurement. He published over 300 books and scientific articles in his field and fulfilled several academic posts, including the position of Dean of the Faculty of Finance. In 2010, he received an important reward for his performance, the NISPAcee Brunowska Award. He is Vice-President of the International Research Society for Public Management, member of the Management Board of the International Association of Schools and Institutes of Administration (IASIA), project director of the IASIA permanent working group Public Sector Financial, Performance and Information Management. He is also a member of the Accreditation Committee of the European Association of Public Administration Accreditation. He was involved in the evaluation process of the first year of the European Public Service Award. He has coordinated many research and advisory projects. Recently, he was appointed as member of the review committee of the Slovak research grant agency APVV. Professor J. (John) Loughlin was professor of European Politics at Cardiff University since 1995. In October 2010, he took up the position of Fellow of St Edmund s College Cambridge and of Affiliated Lecturer in Politics of Cambridge University (UK). Previously he was associate professor in Public Administration at the Erasmus University Rotterdam (1991-1994) and Senior Lecturer in Public Administration at the University of Ulster (UK, 1985-1991). He holds a Doctorate in Political and Social Sciences from the European University Institute in Florence (IT). He holds or has held visiting professorshops and Fellowships in Oxford, Cambridge, Princeton, Florence, Paris, Bordeaux, Aix-en-Provence and Brussels, among many others. His main fields of research have been in Comparative European Politics and Administration. He has published numerous books and articles. In 2009, the University of Umea (SE) awarded him an honorary doctorate in recognition of his contribution to research in European politics and territorial governance. In 2010, the French government named him an Officier dans l Ordre des Palmes Académiques in recognition of his contribution of the study of European politics and to the spread of French language and culture in the United Kingdom. He is a member of the Royal Society of Arts, the Royal Historical Society and the Academy of Social Sciences of the United Kingdom. Professor T. (Tony) Bovaird is professor of Public Management and Policy at the INLOGOV and Third Sector Research Centre of the University of Birmingham. He worked in the UK Civil Service and at several universities. In 2006, he accepted a position at INLOGOV. He was a member of the OECD e-governance Task Force. He chaired the Evaluation Partnership, set up by the UK government to coordinate the evaluation of the Local Government Modernisation Agenda, from 2002 2007 and is a member of the CLG Expert Panel on Local Governance. He undertook evaluation case studies of the Civil Service Reform Programme, commissioned by the Cabinet Office, and recently led the UK contribution to an EU project on user and community co-production of public services in five European countries. He helps to organize the European Public Sector Award and is on the Strategy Board of the Local Authorities Research Council Initiative. He published numerous books and articles. 70 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Professor M. (Michael) Hill was professor of Social Policy at the University of Newcastle. He was head of department from 1986 until 1994. He took early retirement in 1997 and was appointed emeritus professor of Social Policy. Currently he is visiting professor in the Personal Social Services Research Unit at the London School of Economics, the Department of Politics, Queen Mary College, University of London and the School of Applied Social Science, University of Brighton. From 1998 until 2003, he was visiting professor at the Department of Politics, Goldsmiths College, University of London and joint editor of the Journal of Social Policy. His previous appointments include positions as Senior Lecturer at the School for Advanced Urban Studies in the University of Bristol, Research Officer and Deputy Director of a DHSS funded social work research project at the University of Oxford, Lecturer in Sociology at the University of Reading, and Executive Officer in the National Assistance Board. Professor M. (Mirko) Vintar is full professor of Informatics in Public Administration at the Faculty of Administration of the University of Ljubljana. He was dean of the Faculty of Administration (1993-1995) and he has been vice-dean for scientific research since 1999. He is also head of the Institute for Informatization of Administration. He gained his doctorate in administration and information science studies at the Faculty of Economics of the University of Ljubljana. For over 20 years, his work has dealt with the informatisation of public administration, with a major focus on the development of e-government in recent years. Professor Vintar chairs and has chaired numerous national and foreign research and applied development projects. He is actively involved in various international scientific and professional bodies in the field of administration research. He is a member of the social sciences council at the Slovenian Research Agency, editor and co-author of several scientific monographs. He chaired the programme and organizing committees of international conferences held in Slovenia (EGPA 2004, NISPAcee 2006, SSPA 2007). Mr. drs. A.J. (Arthur) Modderkolk studied economics and law at the Erasmus University Rotterdam. In 1975, he joined the directorate-general for the national budget of the Ministry of Finance. He held various positions at the ministry, including the position of director responsible for the finances of provinces and municipalities. In 1985, he was appointed as general director and secretary general of the province of Noord-Brabant. In 1997, he moved to the health sector, where he became manager and secretary general of the foundation De Open Ankh. In this postion, he was manager of the largest conglomerate of AWBZ-funded health services and a member of more than ten supervisory boards of health institutions. Arthur Modderkolk has had and continues to have many additional positions in various public sectors, including education, church, youth care, and housing corporations. Dr. A.A.M. (Louis) Meuleman has almost 30 years of experience in the public sector experience, serving as a policy-maker, project manager, head of unit, process manager and project director, on national, regional and international issues, mainly in the fields of environment, sustainable development and spatial (land use) planning. Until recently, he was secretary general of the Dutch Advisory Council for Research on Spatial Planning, Nature and Environment (RMNO) in The Hague. Since January 2010, he works at the Dutch environment ministry as senior advisor international strategy and governance. He is director of the project Science for sustainable societal transformations: Towards effective governance (TransGov) of the International Institute for Advanced Sustainability Studies (IASS, Potsdam, Germany), chair of the Netherlands Association for Public Management (VOM), senior lecturer at Nyenrode Business University (Breukelen, the Netherlands), and research fellow at VU University Amsterdam. He gives workshops on (meta)governance, QANU & EAPAA / Public Administration, Erasmus University Rotterdam 71

process management and stakeholder participation. His PhD thesis Public Management and the Metagovernance of Hierarchies, Networks and Markets was published in 2008. Dr. C.J.M. (Kees) Breed graduated in political sciences at the University of Amsterdam (1973). In 2007, he successfully completed his PhD thesis Governance culture and strategy a research study into the cognitive map of top civil servants. Kees Breed has been working for several public and semi-public organizations, mainly in the fields of scientific research (Netherlands foundation for international scientific cooperation, Nuffic) and public administration (Ministry of Home Affairs, Permanent Representation of the Netherlands at the European Union). From 1998 until 2006 he acted as a private strategy consultant for organizations at different levels of government (national, provincial and local government). Since 2006, Kees Breed holds the position of Secretary of two national advisory councils: the Council for public administration (Rob) and the Council for financial relations (between government layers, Rfv). Drs. H. (Henk) Nijhof has been chairman of the Dutch political party GroenLinks since 2006. From 1994 until 2006, he was alderman in the city of Hengelo, representing GroenLinks. He also served as regional manager of the education trade union ABOP and as director of a so-called Jenaplan school in Hengelo. Henk Nijhof holds and has held various additional positions. Currently, he is chairman of the council of clients of nursing homes in Twente, member of the board of the Olympic Network Twente and a member of the advisory council for the public domain. Until 2009, he chaired the supervisory board of a cluster of four schools for severely maladjusted children in Twente. Until 2006, he was chair of the School Advisory Service (Schoolbegeleidingsdienst) Twente. T. (Tom) Degen is a master s student of Public Administration at Leiden University. He enrolled in the bachelor s programme in 2006. During his first two years of study, he was actively involved in various committees of his student fraternity and his study association. In 2008-2009, he was chairman of the Bestuurskundige Interfacultaire vereniging Leiden, the study association of Public Administration. In 2010, he was one of the authors of a book which marked the 25th anniversary of the department of Public Administration and the study association. He currently is a research assistant at the Hague Campus of Leiden University and he follows the specialization in Crisis and Security Management of the master s programme in Public Administration. J. (Janneke) van der Heijden is a student of the bachelor s programme in Public Administration at Tilburg University. As of September 2010, she is the student member of the board of the Tilburg Law School. In 2009-2010, she was the student member of the faculty s Education Board, which supports and advices the Vice-Dean for Education of the faculty. From 2007 until 2009, she was a member of the Faculty Council of the Tilburg Law School. 72 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Appendix B: Composition of the assessment committtee per site visit Composition of the assessment committee at the site visit to Utrecht University: Prof. dr. J.J. (Jaap) Boonstra, chair Prof. dr. J. (Juraj) Nemeç Prof. dr. J. (John) Loughlin Mr. drs. A.J. (Arthur) Modderkolk T. (Tom) Degen Drs. L.C. (Linda) te Marvelde, secretary Drs. S. (Sietze) Looijenga, project coordinator Composition of the assessment committee at the site visit to Tilburg University: Prof. dr. J.A. (Hans) de Bruijn, chair Prof. dr. J. (Juraj) Nemeç Prof. dr. J. (John) Loughlin Dr. A.A.M. (Louis) Meuleman T. (Tom) Degen Drs. S. (Sietze) Looijenga, secretary and project coordinator Composition of the assessment committee at the site visit to Leiden University: Prof. dr. C. (Christoph) Reichard, chair Prof. dr. J. (Juraj) Nemeç Prof. dr. T. (Tony) Bovaird Dr. A.A.M. (Louis) Meuleman J. (Janneke) van der Heijden Drs. R.L. (Reinout) van Brakel, secretary Drs. S. (Sietze) Looijenga, project coordinator Composition of the assessment committee at the site visit to Radboud University Nijmegen: Prof. dr. C. (Christoph) Reichard, chair Prof. dr. J. (Juraj) Nemeç Prof. dr. T. (Tony) Bovaird Mr. drs. A.J. (Arthur) Modderkolk J. (Janneke) van der Heijden Drs. L.C. (Linda) te Marvelde, secretary Drs. S. (Sietze) Looijenga, project coordinator Composition of the assessment committee at the site visit to Erasmus University Rotterdam: Prof. dr. H. (Harald) Sætren, chair Prof. dr. J. (Juraj) Nemeç QANU & EAPAA / Public Administration, Erasmus University Rotterdam 73

Prof. dr. M. (Michael) Hill Dr. K. (Kees) Breed T. (Tom) Degen Drs. J. (José) van Zwieten, secretary Drs. S. (Sietze) Looijenga, project coordinator Composition of the assessment committee at the site visit to VU University Amsterdam: Prof. dr. C. (Christoph) Reichard, chair Prof. dr. H. (Harald) Sætren Prof. dr. M. (Michael) Hill Drs. H. (Henk) Nijhof T. (Tom) Degen Drs. R.L. (Renate) Prenen, secretary Drs. S. (Sietze) Looijenga, project coordinator Composition of the assessment committee at the site visit to the University of Twente: Prof. dr. C. (Christoph) Reichard, chair Prof. dr. M. (Michael) Hill Prof. dr. M. (Mirko) Vintar Mr. drs. A.J. (Arthur) Modderkolk J. (Janneke) van der Heijden Drs. L.C. (Linda) te Marvelde, secretary Drs. S. (Sietze) Looijenga, project coordinator 74 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Appendix C: Programme of the site visit to Erasmus University Rotterdam Day 1 9:00 13:00: Preparatory meeting of the assessment committee: general discussion, discussion of the self-evaluation reports and the theses which have been distributed in advance, review of documentation made available 13:00 14:00: Interview with representatives who are responsible for the the contents of the programmes (chairman of the department, programme director(s), chair holder(s), authors of the self-evaluation report et cetera) 14:00 14:45: Interview with students of the bachelor s programme (approximately 3 per year, 10 at most) 14:45 15:30: Interview with students of the master s programme (6 to 8) 15:30 15:45: Break 15:45 16:30: Interview with staff members who contribute to the programmes (10 at most) 16:30 17:15: Interview with graduates of the master s programme (6 to 8) 17:15 18:00: Interview with representatives of the professional practice 19:30 21:30: Dinner with representatives of the faculty offering the programmes Day 2 09:00 09:30: Interview with student members of the Programme Committee 09:30 10:00: Interview with staff members of the Programme Committee 10:00 10:30: Interview with members of the Board of Examiners and with the study advisor(s) 10:30 11:00: Guided tour and office hour 11:00 11:30: Internal committee meeting in preparation of the interview with the board of the faculty 11:30 12:30: Interview with the board of the faculty 12:30 13:00: Lunch 13:00 16:00: Internal committee meeting: discussion leading to general conclusions and the assessment of the programmes 16:00 16:30: Oral report of the committee s conclusions by the chairman of the committee 16:30 17:00: Reception QANU & EAPAA / Public Administration, Erasmus University Rotterdam 75

76 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Appendix D: Joint QANU-EAPAA assessment framework Theme 1: Aims and objectives Standard 1: Mission-based accreditation [EAPAA] Criterion: The programme should state clearly its educational philosophy and mission and have an orderly process for developing appropriate strategies and objectives consistent with its mission, resources, and constituencies. From the mission a set of credible educational objectives should be formulated. Interpretations of the EAPAA standards of this accreditation must be justified in light of the programme's mission and objectives and success in fulfilling its mission Standard 2: Subject-/discipline-specific requirements [NVAO & EAPAA] Criterion: The intended learning outcomes of the programme correspond with the requirements set by professional colleagues, both nationally and internationally and the relevant domain concerned (subject/discipline and/or professional practice). Standard 3: Bachelor and master level [NVAO & EAPAA] Criterion: The intended learning outcomes of the programme correspond with the general, internationally accepted descriptions of a Bachelor s qualification or a Master s qualification. Standard 4: Academic orientation [NVAO & EAPAA] Criteria: The intended learning outcomes of the programme correspond with the following descriptions of a Bachelor s and a Master s qualification: The intended learning outcomes are derived from requirements set by the scientific discipline, the international scientific practice and, for programmes to which this applies, the practice in the relevant professional field. An academic bachelor (WO-bachelor) has the qualifications that allow access to at least one further programme at academic master's level (WO-master) and the option to enter the labour market. An academic master (WO-master) has the qualifications to conduct independent research or to solve multidisciplinary and interdisciplinary questions in a professional field for which academic higher education is required or useful. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 77

Theme 2: Curriculum Standard 5: Public administration character of the programme [EAPAA] Criteria: The core curriculum provides a thorough teaching of the basic concepts, theories, methods and history (classics) of Public Administration on the level of the programme (bachelor or master). The curriculum components are designed to produce professionals capable of intelligent, creative analysis and communication, and action in the public sector. Courses taken to fulfil the core curriculum components provide research methods, concepts and theories from the disciplines of economics, law, political science, sociology, public finances, informatization, and public management as well as the relationship between these fields. Students are obliged to give adequate proof of their ability to work independently (under the supervision of a supervisor) on real problems or research questions in the public sector, for example through essays and final papers (e.g. a thesis), adapted to the level (bachelor or master) of the programme. The core curriculum components enhance the student's values, knowledge, and skills to act ethically and effectively. Standard 6: Requirements for academic orientation [NVAO & EAPAA] Criteria: The proposed curriculum meets the following criteria for an academic orientation: The students develop their knowledge through the interaction between education and research within the relevant disciplines The curriculum corresponds with current developments in the relevant discipline(s) by verifiable links with current scientific theories The programme ensures the development of competences in the field of research Where appropriate, the curriculum has verifiable links with the current relevant professional practice. Standard 7: Correspondence between the aims and objectives and the curriculum [NVAO & EAPAA] Criteria: The curriculum is an adequate realisation of the intended learning outcomes of the programme and this regards the level, the orientation and the subject-/discipline-specific requirements. The intended learning outcomes are adequately transferred into the educational goals of the curriculum or parts thereof. The contents of the curriculum ensure the students achievement of the intended learning outcomes. 78 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Standard 8: Consistency of the curriculum [NVAO & EAPAA] Criterion: The contents of the curriculum are internally consistent. Standard 9: Workload [NVAO & EAPAA] Criterion: The curriculum can be successfully completed within the set time, as certain programmerelated factors that may be an impediment in view of study progress are eliminated where possible. Standard 10: Admission requirements [NVAO & EAPAA] Criteria: The structure and contents of the intended curriculum are in line with the qualifications of the incoming students: Academic bachelor s programme (WO-bachelor): VWO (pre-university education), propaedeutic certificate from a secondary school (HBO) or similar qualifications, as demonstrated in the admission process Master s programme (WO-master): a bachelor's degree and possibly a selection (with a view on the contents of the discipline). Standard 11: Credits [NVAO & EAPAA] Criteria: The programme meets the legal requirements regarding the range of credits: Academic bachelor's programme (WO-bachelor): 180 credits Academic master's programme (WO-master): a minimum of 60 credits. Standard 12: Coherence of structure and contents [NVAO & EAPAA] Criteria: The educational concept is in line with the aims and objectives. The study methods correspond with this educational concept. Standard 13: Learning assessment [NVAO & EAPAA] Criterion: By means of evaluations, tests and examinations, the students are assessed in an adequate and for them insightful way to determine whether they have achieved the intended learning outcomes of the programme or parts thereof. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 79

Theme 3: Staff and organization Standard 14: Requirements for academic orientation [NVAO & EAPAA] Criterion: The programme meets the following criterion for the deployment of staff for a programme with an academic orientation: Teaching is principally provided by researchers who contribute to the development of the subject/discipline. Standard 15: Quantity of staff [NVAO & EAPAA] Criterion: Sufficient staff are deployed to realise the desired quality of the programme. Standard 16: Quality of staff [NVAO & EAPAA] Criterion: The staff deployed are sufficiently qualified to ensure that the aims and objectives regarding the content, didactics and organization of the programme are achieved. Standard 17: Diversity: gender and minorities [EAPAA] Criterion: The programme strives for a percentage of women among their (professional) staff, which is in accordance with the percentage of women in the workforce. When this is an issue in the wider society, the programme strives for a percentage of persons among their staff, which is in accordance with the societal targets. Standard 18: Programme jurisdiction [EAPAA] Criterion: Within the framework of institutional organization, responsibility for the programme in Public Administration rests with an identifiable person or group of persons, chosen according to the rules of the organization. Within the framework of organization and process peculiar to the institution, the faculty and/or administrator exercises initiative, and substantial determining influence with respect to important aspects of the programme. Standard 19: Public relations [EAPAA] Criterion: The programme should provide future students and others with adequate information on the programme. 80 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Theme 4: Services Standard 20: Facilities [NVAO & EAPAA] Criterion: Housing and facilities are adequate to achieve the learning outcomes. Standard 21: Tutoring [NVAO & EAPAA] Criteria: Tutoring and information provision for students are adequate in view of study progress. Tutoring and information provision for students correspond with the students needs. Theme 5: Internal quality assurance system Standard 22: Periodical evaluations [NVAO & EAPAA] Criterion: The curriculum is periodically evaluated in the light of verifiable objectives and other measures. Standard 23: Measures for improvement [NVAO & EAPAA] Criterion: The outcomes of the evaluation form the basis of verifiable measures for improvement that contribute to the achievement of the objectives. Standard 24: Involvement of staff, students, alumni and the professional field [NVAO & EAPAA] Criterion: Staff, students, alumni and the relevant professional field will be actively involved in the internal quality assurance system. Theme 6: Results Standard 25: Achieved learning outcomes [NVAO & EAPAA] Criterion: The achieved learning outcomes correspond with the aims and objectives regarding level, orientation and subject-/discipline-specific requirements. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 81

Standard 26: Study progress [NVAO & EAPAA] Criteria: Target figures that are comparable to other relevant programmes are formulated to express the expected success rate. The programme s success rate complies with these target figures. 82 QANU & EAPAA / Public Administration, Erasmus University Rotterdam

Appendix E: List of materials studied by the committee during the site visit recent theses listed in the self-assessment reports which had not been distributed among committee members prior to the site visit; forms used for assessing the theses; materials and publications used for information and marketing purposes; learning materials: handbooks, readers, collections of articles, etc.; examples of papers, reports and internship reports produced by students; rules and regulations for writing theses, reports, research papers; rules and regulations applying to internships; exam regulations; written exams and assessment materials; recent minutes and reports of meetings of the Board of Studies, the Board of Examiners, annual reports on education; reports of evaluations of courses and curricula; results of surveys among graduates; policy reports and documents relating to the degree programmes. QANU & EAPAA / Public Administration, Erasmus University Rotterdam 83