A Framework for Quality Assurance at the Business Unit of Pforzheim University. A Case Study

Transcription

1 A Framework for Quality Assurance at the Business Unit of Pforzheim University A Case Study Prof. Dr. Karl-Heinz Rau and Nadja Legrum-Khaled (B.Sc.) Business School Pforzheim University Tiefenbronner Str Pforzheim karl-heinz.rau@hs-pforzheim.de Phone: Abstract Quality assurance has to be an integral part of the activities at an academic institution. Systematic approaches and processes ensure continuous improvement. This means that faculty involvement and commitment is crucial for sustainable success. At our institution quality management provides guidance for all the strategic and operational decisions. We align our quality assurance and quality development with world leading standards (AACSB, 2010). Quality management is an effective measure to build confidence for education as a kind of credence good (Kortendieck, 2008, 19; Dill/Soo 2004). We started with this approach about four years ago. We developed a well structured framework and some components are already implemented. So we have already some experiences but we are still in a complex learning process. Sometimes we are faced with the question if a formal quality management is compatible with the self conception of an academic institution (Nickel 2007, 19). Basically we are convinced that quality benefits! Content Introduction and Outcomes... 2 Overview of the QA/QD-Framework... 2 Structure of the QA/QD Framework... 2 Processes of the Business Unit... 3 Collecting and Utilizing Data... 4 Analysis and Interpretation of the Data... 6 Developing and Implementing Improvement Measures... 6 Organizational Aspects of QA/QD... 7 Conclusions... 8 Rau/Legrum-Khaled 5th European Quality Assurance Forum 1

2 Introduction and Outcomes From a timeline point of view the topic of quality assurance (QA) and quality development (QD) at academic institutions in Germany is strongly related to the Bologna accord signed in The obligation to get the study programs or the whole institution accredited by certified accreditation bodies and the increasing competition in the future enforces more and more universities to establish systematic quality assurance approaches. In 2002 the supervisory board of Pforzheim University gave the recommendation to the Business Unit (all business oriented programs) to strive for an international accreditation (Trapnell, 2007, 68). Finally this process started in 2006 and initiated activities to establish a culture of quality awareness which is more than a formal and systematic QA/QD system. This case study should provide the following outcomes for the reader. The reader will: have a basic understanding of our framework for QA/QD, know some selected elements in detail, benefit from our lessons learned and be aware of challenges and pitfalls. Overview of the QA/QD-Framework Structure of the QA/QD Framework Education and applied research are results of the processes accomplished at a university. Therefore the processes are essential for the success. To assure a continuous improvement we use a closed-loop system for quality assurance and development (QA/QD) (see fig. 1). Analysis and interpretation of data Development and implementation of improvement measures Quality Assurance and Development Collecting and utilizing data Processes of the Business Unit Figure 1: Closed-loop system of QA/QD Rau/Legrum-Khaled 5th European Quality Assurance Forum 2

3 For the purpose of QA/QD, data of different kinds and origin have to be collected and analyzed. The analysis and interpretation of data provides insights about the performance and may reveal need for improvements. If this is the case improvement measures have to be developed and implemented in the processes. According to the lead-times in the next cycle of data collection the success of the implemented actions may be assessed. Processes of the Business Unit A prerequisite for this approach is to have end-to-end processes defined. In figure 2 main processes of the business unit are represented. The definition of processes followed the mission of the business unit (Pforzheim University 2010): To provide a high level of academic education in applied business to prepare our students for sustainable and successful professional and managerial careers in a dynamic global environment. To capitalize on the potential of the many attractive companies in our area to mutually benefit both our students and the companies themselves To make intellectual contributions based on research that can be applied to contemporary business issues, to classroom instruction or to enhance the knowledge base of the faculty s disciplines. Students information and recruiting selection and admission advising and mentoring development and improvement of programs Learning and Teaching planning of the semester teaching courses planning and conduct exams privately/publicly funded research projects Research research activities in the context of theses independent research activities internship/excursions and/semester/excursions Company Relations projects/theses teaching assignment/guest lectures recruiting faculty members Faculty Members mentoring and professional development of faculty members/management of teaching load management of adjunct and visiting faculty Figure 2: Main processes of the Business Unit According to this mission statement the core process Learning and Teaching in combination with the resource related process Faculty Members and Students are focused on the first item of the mission. Company Relations is the process which relates primarily to the second statement of the mission and Research is the proc- Rau/Legrum-Khaled 5th European Quality Assurance Forum 3

4 ess which supports the mission statement which is focused on intellectual contributions. This indicates the consistence between the mission and the process structure. Beside of these main processes supporting processes like Recruiting and Development of Staff, Providing and Maintaining the Infrastructure, Build and Maintain Relationships to Partner Universities and Build and Maintain Alumni Relationship. Each of the processes may be divided in a hierarchy of sub-processes which is already indicated in figure 2. For each process a sketchy description exists so far. The intention is not to invest too much time and resources to describe and elaborate each process in all details. At first the intention is to have an architecture of processes to identify fields with high priority for improvements. Collecting and Utilizing Data To identify the areas for improvements it s necessary to collect and/or utilize data (see the blue box in fig. 1). There are a variety of data resources available (see fig. 3). Freshman survey Alumni survey Diary study Graduate survey Collecting and utilizing data Course evaluation Outcome assessment Existing statistics Student feedback Number of applicants Admission numbers Enrollments Share of types of higher education entrance qualifications Grades of higher education entrance qualification Retention rates Figure 3: Collecting and utilizing data The alumni survey addresses graduates who are already integrated in professional life for more than four years. In this survey the graduates reflect their studies from their professional perspective. They assess the value contribution of their learning to their professional success. Additionally rich data is provided about the career entry which helps to evaluate the career service activities. The graduate survey provides an evaluation of the studies at the time of graduation and is focused on the various elements of the curriculum In winter semester 2008/09 the freshman survey was conducted for the first time. It is carried out with first semester students in the first weeks after semester beginning. The results provide important data about relevant communication channels and competitors. The diary survey was conducted as pilot study with selected students. It delivers data about the students studying behavior. The students are forced to note all activities for 24 hours. Information about the real time input for individual courses in relation to ECTS credits are of special interest. This information is extremely helpful for optimizing the curriculum. Rau/Legrum-Khaled 5th European Quality Assurance Forum 4

5 The course evaluations by students have been conducted by the faculty member on an individual basis for many years now. Since 2006 the standardized evaluating regulations are valid. The implementation is supported by a software system. The course evaluation provides important information about the teaching and learning process in the individual courses from the students` point of view. The information is primarily important for the teaching staff itself. On the basis of these results, wellfounded decisions about the further development of the courses and the curriculum can be taken. Existing statistics produced by the IT-systems of the students admission office and the exams office are a valuable source for information about various facts. Scholarly research shows that the entrance qualification of students is crucial for the graduation success. Therefore data about the entrance qualification in connection with retention rates are valuable for monitoring the admission process. Although the informal student feedback is not always systematic this information contributes to better understanding of the various processes. On the dean s level the student committee is a rich source for the administrators because it follows the principle of an open-door -policy. Furthermore the feedback of student representatives in different committee provides input for improvement activities. The outcome assessment has the goal to provide information whether students have learned what they are expected to learn. Faculty members assess students output. Basis of this assessment could be course embedded activities like specific questions in an examination, student s presentations, project work, the thesis, term papers, etc. The assessment is focused on measuring the achievement of program goals (competencies) which are detailed on an operational level as learning outcomes (learning objectives). The learning outcomes are used as indicators to assess the achieved competence. Typically the assessment is done by using a rubric which provides a list of criteria and well described expected levels of performance. To collect and analyze the outcome assessment data the following steps are necessary (Martell/Calderon 2005, 9): Defining of program goals Defining of learning outcomes/objectives Mapping of learning outcomes and courses of the curriculum Developing of rubrics Conducting of assessment For our institution the outcome assessment is a new instrument. We invested a lot in the learning process to understand the methodology. The faculty members are still in a learning process but realized already some successes. In some cases we collected the assessment data, the faculty did the analysis, and came up with improvement activities. An example is the learning outcome oral communication which was assessed in a seminar course. The assessment was based on presentations of a term paper subject. The data revealed that the students have shortcomings in some aspects. The consequence was that the communication training was changed in some aspects. The next assessment is scheduled according the lead-time and we will see if the changes will make an impact. The superior benefit of the approach compared to the former approach is that the discussions are based on statistical data instead of Rau/Legrum-Khaled 5th European Quality Assurance Forum 5

6 anecdotic experiences. The discussions and decisions are much more straightforward. Nevertheless we also have experiences that the intended assessment approach didn t work or the collected data was biased, etc. Analysis and Interpretation of the Data Crucial for a continuous improvement process is the analysis and interpretation of the data (see yellow box in fig. 1). Figure 4 shows that the data has to be evaluated by comparing it with goals, objectives, requirements, etc. Strategic goals Requirements of stakeholders European and national qualification framework/accreditation agencies Analysis and interpretation of the data Quality objectives Competence goals on study program level Organisational, physical and financial restrictions Competitors Figure 4: Analysis and interpretation of data The question whether the requirements are adequate can only be answered if there is an alignment with given qualification frameworks or requirements of stakeholders. The analysis has to consider as well the circumstances of the data collection. For example, the evaluation of team working skills can show quite satisfying results because the teams are made up of student friends, who get along well and do not have problems to find time together, etc. As the different data sources often complement each other, the analysis and interpretation should always consider interdependencies of data. From this, impulses for changes in the data collection approach can arise. The analysis of the outcomes assessment can indicate that certain course evaluations may provide helpful and complementing information. Another important task of the analysis and interpretation is the elaboration and determination of priorities. This is especially necessary if there is a variety of actiontaking necessities and there is neither time nor resources in unlimited extent. Developing and Implementing Improvement Measures The effectiveness of all the improvement activities could be measured by the implemented changes. Because quality is sometime difficult to measure a basic principle of QA/QD could be: What we did yesterday was good, but what we will do tomorrow is better! Following this principle there is nearly always room for improvement. The figure 5 provides only an initial idea of possible improvement measures. A recom- Rau/Legrum-Khaled 5th European Quality Assurance Forum 6

7 mendation to start with could be Take simple measures with low risk and high effectiveness first. Professional/didactic training of faculty Further development of course methodology/ didactics Changes in course assignments Changing exam modalities Developing and implementing improvement measures Further development of offered degree programs Changes in admission/enrollment of students Improving the infrastructure... Improving student counselling Figure 5: Developing & Implementing Improvement Measures The examples noted in the ovals of figure 5 are more or less self-explanatory. Organizational Aspects of QA/QD To make the QA/QD effective organizational responsibilities have to be defined. The business unit is structured into disciplines (departments) like Business Information Systems, Marketing, or Economics. Each faculty member is assigned to one discipline. The programs are assigned to disciplines but there are also disciplines without a program assigned; an example is Economics. Each discipline is supervised/coordinated by a department. In a lot of cases the department head is also in the position of a program director. From a product perspective the faculty team who is involved in a program is responsible for QA/QD activities. Finally the program director represents the responsibility. For all cross-program activities and processes the dean and the associate deans as member of the faculty board are responsible for QA/QD. For coordination purposes a Continuous Improvement Committee was established. Beside of these coordination activities this committee has the mission to cultivate the QA/QD spirit of the Business Unit. To monitor the QA/QD activities across the Business Unit each discipline has to create an annual report. The same is true for the Business Unit in total. The main focus of these reports is that it has to be demonstrated that the loop which is defined by the closed-loop system of figure 1 is closed. Rau/Legrum-Khaled 5th European Quality Assurance Forum 7

8 Conclusions The Business Unit of Pforzheim University has developed a framework for its QA/QD initiative. It s characterized as a work in process. Perhaps this should be always true if the intention is to strive for continuous improvement. Recent discussions at a quality management workshop organized by the German Rectors Conference encouraged the authors to discuss the framework in a broader audience. Perhaps other institutions can benefit from the experiences and see value in adapting the framework for their institution. All in all the authors did not intend to draw a rosy picture. Following topics are food for thought: QA/QD more than bureaucracy? New terminology but no new insight! How to achieve faculty involvement? Availability of resources? Backing of the top administrators? Does quality benefits? The authors are looking forward to an inspiring discussion. Bibliography AACSB (2010): Eligibility Procedures and Accreditation Standards for Business Accreditation. Revised January 31, 2010, Tampa ( 01/09/2010) Dill, D. und Soo, M. (2004): Transparency and Quality in Higher Education Markets. In: Teixeira, P., Jongbloed, B., Dill, D. & Amaral, A. (Eds.), Markets in Higher Education: Rhetoric or Reality? Dordrecht, Kluwer Academic Publishers 2004, pp Kortendieck, G. (2008): Bildung ein Vertrauensgut. Über das Verhältnis von Betriebswirtschaft und Erwachsenenbildung. In: Kortendieck, G./Summen, F. (Hrsg.), Betriebswirtschaftliche Kompetenz in der Erwachsenenbildung. Bielefeld: Bertelsmann, pp Martell, K. and Calderon T.G. (2005): Assessment in Business Schools: What it is, where we are, and where we need to go now. In: Assessment of Student Learning in Business Schools: Best practices each step of the way. Edited by K. Martell and T. Calderon. Association for Institutional Research. Florida State University, Tallahassee 2005, pp Nickel, S.(2007): Institutionelle QM-Systeme in Universitäten und Fachhochschulen Konzepte Instrumente Umsetzung. CHE Arbeitspapier Nr. 94, Gütersloh: CHE Pforzheim University (2010): Mission Statement. Pforzheim ( 01/09/2010). Rau/Legrum-Khaled 5th European Quality Assurance Forum 8

9 Trapnell, J.E. (2007): AACSB International accreditation. The value proposition and a look to the future. In: Journal of Management Development. Vol. 26 No. 1, 2007, pp Rau/Legrum-Khaled 5th European Quality Assurance Forum 9

10 Author s Bio: Dr. Karl-Heinz Rau is a full Professor for MIS at the Business School of the Pforzheim University, Germany, since Before his time in academia he worked for IBM Germany in business planning and strategy consulting. He earned his doctoral degree from the Tuebingen University. Dr. Rau published three books and wrote more than 30 articles in journals and books in the area of information systems, management accounting, and quality management. Currently he serves as an associate dean at the Business School of Pforzheim University. Nadja Legrum-Khaled is the Quality Management coordinator at the Business School of the Pforzheim University, Germany, since She received her Bachelor of Science degree in Information Engineering and Management at Karlsruhe University/Karlsruhe Institute of Technology with focus on Organization Management and Theories as well as contract law on the internet. Rau/Legrum-Khaled 5th European Quality Assurance Forum 10