AUSTRALIAN UNIVERSITIES QUALITY AGENCY. Report of an Audit of University of South Australia

Transcription

1 AUSTRALIAN UNIVERSITIES QUALITY AGENCY Report of an Audit of University of South Australia October 2009

2 AUQA Audit Report Number 86 ISBN Australian Universities Quality Agency 2009 Level 10, 123 Lonsdale Street Melbourne, VIC 3000 Ph Fax admin@auqa.edu.au The Australian Universities Quality Agency receives funding from the Australian Government and state and territory governments of Australia. The views expressed in this report do not necessarily reflect the views of these governments.

3 CONTENTS 1 EXECUTIVE SUMMARY Audit Findings Main Points Matters from Cycle 1 Audit Theme 1: Building Workforce Capacity to Improve Teaching and Learning and Enhance Research Performance Theme 2: Internationalisation National Protocols Other External Reference Points Institutional Context Strategic Context Commendations, Affirmations and Recommendations...7 Commendations... 7 Affirmations... 8 Recommendations CONTEXT AND DIRECTIONS AND MATTERS FROM CYCLE 1 AUDIT Aspirations and Directions Improvements to Quality Processes Academic Standards and Benchmarking Standards and Benchmarking The Academic Board Follow up from 2004 AUQA Audit Stakeholder and Community Engagement Student Representation Schools and Institutes THEME: BUILDING WORKFORCE CAPACITY TO IMPROVE TEACHING AND LEARNING AND ENHANCE RESEARCH PERFORMANCE Background and Panel Overview Research Capacity Building Research Capacity Building Research Capacity Building Post Research Supervision Teaching and Learning Teaching and Learning Framework Teaching Research Nexus and Workloads Using Student Feedback Examples of Outcomes Enabling Systems and Support Support Programs...24

4 3.4.2 Sessional Staff Workforce Strategy and Professional Staff Performance Management THEME: INTERNATIONALISATION Planning Internationalisation Key Result Areas International recognition Internationalisation of the Curriculum Internationalisation of Research International Students Onshore Transnational Education Context and Teach out Points Common to all TNE Programs Taylor s University College, KL APMI Kaplan, Singapore Hong Kong Baptist University Jiangxi University of Finance and Economics, Nanchang Academic Integrity English Language Proficiency DATA...41 APPENDICES...51 APPENDIX A: THE AUDIT PROCESS...51 APPENDIX B: AUQA S MISSION, OBJECTIVES, VISION AND VALUES...53 APPENDIX C: THE AUDIT PANEL...55 APPENDIX D: ABBREVIATIONS AND DEFINITIONS...56

5 University of South Australia Audit Report 1 EXECUTIVE SUMMARY 1.1 Audit Findings The scope for the 2009 audit of the University of South Australia (UniSA) is the two themes of Building workforce capacity to improve teaching and learning and enhance research performance and Internationalisation, together with the follow up of selected recommendations from the 2004 AUQA Audit Report. In addition this Report includes comments on the University s compliance with the MCEETYA National Protocols for Higher Education Approval Processes, other external reference points and on academic standards and benchmarking. AUQA s findings are contained in sections 2 to 4. A selection of data that support the findings is provided in section 5. Information on the conduct of the audit is contained in Appendix A Main Points The Portfolio states that UniSA has achieved much since 1991 and AUQA agrees. The University sees the implementation of strategies directed at recruiting, retaining and developing productive, well rounded research active academic staff as its principal means for securing improved outcomes in teaching and learning and in research and research education. Given the significance of staffing policies and practices to the achievement of the University s aspirations, a theme that encompasses workforce planning is therefore a natural choice. Likewise the theme of internationalisation is particularly pertinent for a university with a substantial presence offshore and expanding levels of international activity onshore. The strategy paper New Horizons provides a clear direction for the UniSA community to move forward and outlines concisely how a significant repositioning of the University might be secured. The paper focuses on building a strong research presence and embracing studentengaged learning whilst still maintaining UniSA s strong commitment to values of social justice, equity and engagement. These key priorities are reflected in the Corporate Plans and New Horizons enjoys widespread support and is favourably referred to by internal stakeholders, who praised it for its inclusivity and for the clarity of direction it offers. The Vice Chancellor, as the principal author of the paper, and other members of the Senior Management Group are actively engaged in interaction with academic and professional staff to advance the agenda that New Horizons presents. From AUQA s perspective, New Horizons articulates in an appropriate manner a transformational agenda for UniSA, setting in relative terms a high standard for achievement over a comparatively short period of five years. The challenges that this represents are not to be understated. To date, UniSA has made considerable progress in securing major improvements in its national standing both in relation to learning and teaching outcomes, and research and research training. For the future, further relative improvements in national standing may prove more demanding in an increasingly competitive higher education sector in which all institutions are seeking to enhance their own positions. Australian Universities Quality Agency

6 Audit Report University of South Australia An aspiration centred on building a research led academic community requires a major shift in a university that traditionally has focused primarily on teaching, education for various professions and the provision of transnational education. The Excellence in Research for Australia initiative (and its antecedent Research Quality Framework) now provides the stimulus, and a greater sense of urgency, for UniSA to move to effect the required change in university culture. Many of the initiatives already taken or planned by the University are positive and laudable. From AUQA s perspective, however, it is too early to tell if these initiatives will be sufficient to effect the significant repositioning of a large and comparatively young university. There remain considerable challenges in securing sustainable enhancements in its core academic endeavours. Furthermore, it could take years to ascertain the degree to which UniSA s reform initiatives have succeeded.unisa has a strong sense of direction, is well organised, effectively governed and managed and is tackling core issues through its focus on workforce reform. It seems well placed, therefore, to deliver on its ambitious agenda Matters from Cycle 1 Audit In reflecting the University s Portfolio, section 2 of this Report contains matters from the Cycle 1 audit as well as some matters of quality improvement. In summary, the University has developed excellent capabilities in collecting, presenting and analysing a wide variety of data. This includes the creation of a data warehouse to store data and information originating from various university systems such as the Finance, Human Resources, and Student systems. AUQA commends the University on its approach to datadriven, evidence based change management. As noted in section 1.1.1, the University has set itself demanding objectives on a national and international scale. Those aspirations have to be tested against demanding benchmarks at the national and international level. In AUQA s view the University has to take a more strategic approach to benchmarking and make a more explicit attempt in setting and assuring academic standards. This will assist in the continuous improvement of programs and outcomes at the school and institute level and within individual disciplines. On matters of quality processes and quality assurance, the Academic Board has a significant role in determining and monitoring academic standards and seeking ways of maintaining them, including in relation to English Language Proficiency and much more. AUQA recommends that UniSA consider strengthening the role and current standing of the Academic Board in line with the Board s charter. In following up affirmations and recommendations from the University s 2004 AUQA Audit Report, AUQA concluded that satisfactory progress has been made in responding to the issues raised. One of the 2004 recommendations related to community engagement, including a strategic planning framework for its objectives. AUQA commends UniSA for the substantial improvements that have been achieved on this and in particular the systematic and valued engagement with local communities on the Whyalla Campus and with the City of Salisbury in developing the Mawson Lakes Campus Theme 1: Building Workforce Capacity to Improve Teaching and Learning and Enhance Research Performance The topic of this theme is pertinent to the current stage of development of UniSA. It relates to the core academic activities of a university, research, and teaching and learning. AUQA 2 Australian Universities Quality Agency 2009

7 University of South Australia Audit Report acknowledges the considerable effort and resources that have gone into supporting and developing these activities in recent years. These initiatives have as their foundation a thorough analysis of the relative weaknesses and strengths of the institution, a review that was followed in 2006 by a series of measures to facilitate improvements in performance and outcomes. AUQA commends the University on its strategic and comprehensive approach to establishing a research culture across discipline areas generally. Various initiatives designed to build research capacity were adopted such as the creation of highly visible and well supported chairs, research fellowships, support mechanisms for emerging researchers, and the allocation of resources to attract higher degree by research students. Equally, the University s new Teaching and Learning Framework and particularly its current vehicle of implementation, STEP2010, are commended by AUQA as an effective process for reinventing teaching and learning, curriculum development and pedagogy. UniSA s strategic approach to implementing support programs designed to enhance the human resources, and learning and teaching capacities of its workforce is also commended, as is the strategic way the University approaches workforce planning and the monitoring of outcomes. AUQA expresses some concern about the level of understanding at UniSA of the meaning and implications of the teaching research nexus, a relatively new issue for the University emerging from its strategic goals. Related to this is a need to build significantly on recent developments in identifying a new strategy for research concentrations and to settle relationship issues regarding schools and institutes, a topic that was previously raised in Another area where AUQA affirms the University s actions and encourages further improvements is the management and development of sessional staff Theme 2: Internationalisation The University is a significant player on the international higher education landscape. It has been in the lead group of Australian universities active in transnational education. With the advent of New Horizons, UniSA is in the process of a major repositioning of its international priorities. A fresh approach to internationalisation places more emphasis on activities that are based in or originate from Adelaide and less on transnational education. There is currently a substantial teach out program of many transnational programs, but some of these will not be concluded until Given the changes in strategic direction as articulated in New Horizons, including in relation to internationalisation, AUQA recommends that UniSA provide a clearer definition of its revised internationalisation agenda and improve internal communication and understanding of its objectives, priorities and core activities. UniSA is commended for the quality of planning, implementation, overall leadership and management of the decision to substantially withdraw from transnational education and for the manner in which it is handling the ensuing teach out process. This latter point reinforces the sensitive manner in which UniSA seeks to engage with students. AUQA commends UniSA on the systems and processes in place that facilitate this supportive climate. Notwithstanding these comments, from both the overseas and the Audit Visit in Australia the Panel noted the absence of a consistent approach to informing students of actions taken as a result of student survey feedback. AUQA recommends that this be rectified. Australian Universities Quality Agency

8 Audit Report University of South Australia The audit comprised visits to four transnational program locations. AUQA acknowledges that excellent working relationships are in place at each location. There is some concern expressed in the Report regarding mechanisms in place for assessing actual English language proficiency of students in an academic setting. AUQA affirms UniSA s intended actions to strengthen processes and support in this area. Similarly, AUQA affirms UniSA s extensive measures on academic integrity and encourages continuing vigilance National Protocols The National Protocols for Higher Education Approval Processes require all universities to meet a range of criteria, in particular nationally prescribed criteria A1 to A10 for higher education providers and D1 to D5 specifically for universities. At AUQA s request, the University provided a self assessment against the Protocols, as part of a requirement in the arrangements for the 2009 audits. On the evidence considered by the Panel, the University of South Australia satisfies the criteria in the National Protocols Other External Reference Points The University refers to the following documents in its Portfolio documentation: Australian Qualifications Framework Higher Education Support Act 2003 (Cwlth) (HESA) and guidelines DEST 2007, National Code of Practice for Registration Authorities and Providers of Education and Training to Overseas Students (the National Code) under the Education Services for Overseas Students Act 2000 (Cwlth) UNESCO/OECD 2005, Guidelines for Quality Provision in Cross border Higher Education AUQA 2009, Good Practice Principles for English Language Competence. The audit did not identify any matters of concern regarding the University s compliance with these external reference points. 1.2 Institutional Context The University of South Australia was established in It was formed as a result of an amalgamation of the South Australian College of Advanced Education and the South Australian Institute of Technology. Thus the University has its origins in education for industry and the professions. Its Act of establishment combines a strong social mission with a commitment to excellence and community engagement. UniSA is a multi campus university and has campuses at City West and City East in Adelaide, Mawson Lakes and Magill within metropolitan Adelaide, plus a campus at Whyalla and a regional centre at Mt Gambier. The University has four academic divisions: Business; Education, Arts and Social Sciences; Health Sciences; and Information Technology, Engineering and the Environment. Each of the academic divisions is headed by a pro vice chancellor (PVC). The divisions comprise a number of schools. The Whyalla Campus and the regional centre at Mt Gambier constitute the Centre for Regional Engagement. There are 18 schools in total. The five University wide coordinating portfolios are headed by the DVC (Deputy Vice Chancellor): Academic, DVC: Research and Innovation, PVC: Strategy and Planning, PVC: International and Development and an Executive Director: Finance 4 Australian Universities Quality Agency 2009

9 University of South Australia Audit Report and Resources. The Vice Chancellor chairs the Senior Management Group which includes the heads of the academic divisions and of the coordinating portfolios. The University is part of the Australian Technology Network of universities comprising: RMIT University, University of Technology, Sydney, Queensland University of Technology and Curtin University of Technology. Selected UniSA statistics, Item Number of Total 32,951 32,266 33,680 34,603 36,176 students Commencing 12,025 12,232 13,519 13,417 14,085 Student Total 22,126 21,378 22,535 23,302 24,443 load Undergraduate 17,229 16,958 17,984 18,621 19,580 (EFTSL) Postgraduate 4,897 4,420 4,551 4,681 4,863 Commonwealth operating grant 14,294 n.a. n.a. n.a. n.a. Student load by funding source (EFTSL) Research income Staff (FTE) Commonwealth grant scheme n.a. 14,024 14,756 14,512 14,798 Commonwealth Research Training Scheme Domestic fee paying International onshore 2,533 3,037 3,317 3,505 3,844 Transnational 4,265 3,185 3,364 4,135 4,795 Other DEEWR categories 1 4 ($m) p Total 2,086 2,103 2,162 2,266 2,396 Academic ,010 1,043 Professional* 1,195 1,183 1,205 1,256 1,353 Teaching and research Research only Other function 1,187 1,179 1,196 1,235 1,309 Source: PF p13, extract from Table 1.1. *Professional staff at UniSA are called general staff or non academic staff at some other universities in Australia. Key performance indicator results for UniSA for the period 2004 to 2008 are in data item Strategic Context The AUQA audit of an institution is guided by the objectives that an institution sets for itself. The purpose of an institution is defined by its Act and governing body. In May 2008 the University of South Australia Council adopted the new statement of Vision, Mission and Values which, it says, retains the enduring values of the institution but signals a clear, contemporary and futureoriented vision for the University s role in the life of regional, national and global communities. Australian Universities Quality Agency

10 Audit Report University of South Australia Vision Mission UniSA will be a leading contributor to Australia having the best higher education system in the world, supporting the world s best educated and most innovative, cohesive and sustainable society. UniSA educates professionals and citizens to the highest standards; creates and disseminates knowledge; and engages with our communities to address the major issues of our time. Values Scholarship Engagement Social justice Sustainability Innovation Openness UniSA promotes and sustains open intellectual enquiry characterised by high standards of ethics and integrity. Academic rigour, excellence and relevance underpin our actions in research and its application, in teaching and learning and in engagement with our communities. UniSA s teaching and research connect strongly with the issues of our local, national and international stakeholders students, alumni, staff, partners, professions, government, industry, academic peers and community groups. UniSA gives effect to reconciliation with Indigenous Australians and builds social cohesion by achieving equitable educational access and outcomes across our diverse student community, through research that aims to improve the quality of life and by acting responsibly as a corporate citizen. UniSA contributes to environmentally, economically, socially and culturally sustainable development, and we aim to reduce our own environmental impact. UniSA anticipates change and acts quickly to seize opportunities and solve problems. UniSA is outward looking, welcoming diversity and the wide range of perspectives it brings, international in outlook, collaborative, creative, agile and enterprising. These aspirations and values are congruent with the seven Qualities of a Graduate of the University of South Australia which the University considers to have been a fundamental plank of its Teaching and Learning Framework since 1996: A Graduate of the University of South Australia: 1. operates effectively with and upon a body of knowledge of sufficient depth to begin professional practice 2. is prepared for lifelong learning in pursuit of personal development and excellence in professional practice 3. is an effective problem solver, capable of applying logical, critical and creative thinking to a range of problems 4. can work both autonomously and collaboratively as a professional 5. is committed to ethical action and social responsibility as a professional and as a citizen 6. communicates effectively in professional practice and as a member of the community 7. demonstrates international perspectives as a professional and as a citizen. The Vice Chancellor s position paper New Horizons states that UniSA will be guided by the needs of its students as key stakeholders and that the performance of its people will be critical to its success. The 2009 New Horizons: A Progress Report outlines for Council the major priorities and change projects which are critical to UniSA s success in teaching and learning, research, internationalisation and business processes. All these priorities are underpinned by enhancing the capability of staff, and are therefore the focus of theme 1. 6 Australian Universities Quality Agency 2009

11 University of South Australia Audit Report As for other Australian universities, the environment in which institutional strategies and priorities are currently being determined is largely shaped by the series of policy announcements relating to the higher education sector made by the Commonwealth Government. In 2008 there was the Cutler Review, Venturous Australia Building Strength in Innovation and early in 2009, the Bradley Review of Australian Higher Education. These announcements foreshadowed institutional targets for certain categories of student enrolments, a new research agenda shaped by the Excellence in Research for Australia (ERA), imminent discussions about the introduction of compacts between each university and the government and a sharper focus on academic standards and internationalisation. The three Australian universities located in South Australia, UniSA, Flinders University and the University of Adelaide, face similar challenges. While the population base of Adelaide and of South Australia is relatively small and stable, all three universities are said to have plans to grow, to increase their research activity and secure success in the new ERA system, to engage with their communities and to enjoy significant international standing. There is also a small but growing presence of overseas universities in Adelaide. There is therefore a heightened level of sensitivity about the impact of the AUQA audit and its associated processes. As a consequence, some of the data and strategic information presented to the Audit Panel by UniSA has been treated as being of a commercial in confidence nature. This Report seeks to respect the request by the University not to publish material that might reasonably be regarded as commercially sensitive. 1.3 Commendations, Affirmations and Recommendations This Report contains commendations, affirmations and recommendations. A commendation refers to the achievement of a stated goal, or to some plan or activity that has led to, or appears likely to lead to, the achievement of a stated goal, and which in AUQA s view is particularly significant. A recommendation refers to an area in need of attention, whether in respect of approach, deployment or results, which in AUQA s view is particularly significant. Where such matters have already been identified by the University, with evidence, they are termed affirmations. There were no high priority affirmations or recommendations marked as urgent in this Report. It is acknowledged that recommendations in this Audit Report may have resource implications. The themes for Cycle 2 audits are chosen for their potential risks to institutions and to the higher education sector and are likely also to reflect the institution s own assessment of its developmental and strategic needs. The nature of the findings is influenced by the broad nature of the themes. As for Cycle 1 audits, AUQA aims to assist the University to enhance the quality and standards of its operations. Commendations 1. AUQA commends UniSA on the clarity of direction and the manner in which New Horizons has been communicated to engage and inspire staff at all levels in the University AUQA commends UniSA on the widespread and sophisticated use of management information tools such as data cubes, the development of its business intelligence capabilities and their deployment in initiating and monitoring evidence based change and improvement Australian Universities Quality Agency

12 Audit Report University of South Australia 3. AUQA commends UniSA for the substantial improvements on issues related to community engagement raised in the AUQA Cycle 1 audit AUQA commends UniSA for the strategic and comprehensive approach it has taken to instil a greater level of commitment to a research culture across discipline areas AUQA commends UniSA on its approach to addressing teaching and learning issues since 2005, with the STEP2010 project implementing a process of reinvention of teaching and learning, curriculum development and pedagogy AUQA commends UniSA for its strategic approach to implementing human resources and learning & teaching support programs and its commitment to providing access to them AUQA commends UniSA for its strategic use of resources in support of workforce planning initiatives including changed responsibilities of professional staff in schools AUQA commends UniSA for the quality of the planning and implementation of its decision to withdraw from the substantial provision of transnational education programs and for the overall leadership and management of the associated teach out processes AUQA commends UniSA on the systems and processes in place, both onshore and offshore, for dealing with student issues Affirmations 1. AUQA affirms UniSA in its attempts to address the implications of the teaching research nexus, including workload measures and the workload implications of the implementation of the Teaching and Learning Framework (STEP2010) and the new research concentrations strategy (ReNEW) AUQA affirms UniSA s recognition of the need for more consistent management of the experience of newly appointed academic staff, in particular sessional staff AUQA affirms UniSA in its programs and activities to enhance the international dimension of the student experience AUQA affirms UniSA in its policy approach to academic integrity and encourages continued vigilance across all academic programs irrespective of location AUQA affirms UniSA s recognition that English language proficiency for students is a significant and immediate issue that needs to be addressed, and supports timely conclusion of the current discussion about the implementation of the English language proficiency project, including the testing of student proficiency and where required the provision of additional support and guidance for students...40 Recommendations 1. AUQA recommends that UniSA take a more strategic approach to benchmarking and the validation of its academic standards to facilitate continuous improvement at school and institute levels and within disciplines Australian Universities Quality Agency 2009

13 University of South Australia Audit Report 2. AUQA recommends that UniSA consider strengthening the role and standing of the Academic Board primarily in relation to academic standards and quality assurance in line with its charter AUQA recommends that UniSA provide a clearer definition of its revised internationalisation agenda and improve internal communication and understanding of its objectives, priorities and core activities AUQA recommends that UniSA design and implement a consistent mechanism, including in relation to transnational education programs, that enables students to be informed more productively of actions taken as a result of student feedback and student surveys and the outcomes of those actions...35 Australian Universities Quality Agency

14 Audit Report University of South Australia 2 CONTEXT AND DIRECTIONS AND MATTERS FROM CYCLE 1 AUDIT Section 1 of the University s Portfolio contains contextual information, details of plans for the future and some data and results relating to the 2004 AUQA Audit Report, also referred to as Cycle 1 audit. In keeping with the style of AUQA audit reports, some of these topics are included in section 1 of this Report and some form part of this section. 2.1 Aspirations and Directions In 2004, despite having a well established Teaching and Learning Framework and a good understanding of quality assurance processes, the University came to the realisation that these processes would not necessarily guarantee quality outcomes. This was evidenced when the first round of funding outcomes for the Learning and Teaching Performance Fund (LTPF) were published in The University did not receive funding and was seen publicly as having performed poorly in the round. This was confronting for UniSA and affected staff morale. On the research front, the perceived outcomes of the then Research Quality Framework (RQF) were identified in 2006 as a major risk to the achievement of the University s research aspirations. Reasons for the perceived negative impact of the RQF on UniSA were related to the quality and consistency of research performance and the absence of a pervasive research culture across the institution. The University identified that it needed to exceed the sector s average rate of improvement in both teaching and research in order to increase its relative performance. Initial steps were taken to address this challenge. After the UniSA Senior Management Group (SMG) retreat in 2005, the University introduced strategic initiatives to improve teaching and learning, build research capacity, strengthen its data collection and analytic capabilities, and its performance development and management of staff. The current Vice Chancellor was appointed in Following his first few months of internal discussions, the initiatives identified in 2005 were considerably extended and a substantial repositioning of the University was proposed. In early 2008 the Vice Chancellor issued the position paper New Horizons that both encapsulated the achievements of UniSA to date and set out a concise statement of the vision, goals and strategies designed to guide the development of UniSA over the next three to five years. New Horizons is an exposition of a visionary and inclusive strategy aimed at securing UniSA s long term future. The paper includes five interlocking aspects that have subsequently been adopted in UniSA s Corporate Plan , namely: 1. teaching and learning quality, including experiential learning in UniSA programs through a new teaching and learning framework, implemented through STEP enhancing research performance by building research capacity and growing research income 3. a high achiever strategy regarding students 4. a new workforce strategy 5. budget and administration. The Corporate Plan identifies items 1 to 3 above as major priorities. Items 4 and 5 are also incorporated into the University s planning processes. Developing and implementing a 10 Australian Universities Quality Agency 2009

15 University of South Australia Audit Report new workforce strategy is a core project being led personally by the Vice Chancellor, see section New Horizons is an aspirational future directions paper and the initial horizon is implicitly set to be a five year period. However only two years into the planning period, it is premature to assess how realistic the aspirations of New Horizons are. In the Panel s view the aspirations are ambitious but, if current levels of improvement are maintained, they remain eminently achievable. The Audit Panel formed the impression that staff at UniSA appreciated and endorsed the Vice Chancellor s approach as outlined in New Horizons, and even more so welcomed its aspirational tone and sense of vision and direction. There was not one expression of disagreement or doubt regarding New Horizons amongst those interviewed by the Panel about the wisdom of the New Horizons strategy or the manner of its implementation. The paper reinforces the key message that people are the key to UniSA s future and if UniSA is to take its place among the best universities it will need to attract, develop and retain outstanding people, both academic and professional. Apart from the clarity of the vision and strategy of New Horizons, staff throughout UniSA appeared to value in particular the manner in which the Vice Chancellor and his senior team had sought to engage with staff in its communication and subsequent implementation. Commendation 1 AUQA commends UniSA on the clarity of direction and the manner in which New Horizons has been communicated to engage and inspire staff at all levels in the University. 2.2 Improvements to Quality Processes The University has developed excellent capabilities in collecting, presenting and analysing a wide variety of data that relate to the core activities of UniSA. These data are being used for a multitude of purposes. Some of the high level information is for Council and for the SMG, more detailed data are provided to assist heads of school in their roles as academic leaders, and some even more focused information is of benefit in the performance management process of individual academic staff members. Examples of the steps being taken by UniSA to improve its capabilities in the compilation and analysis of data are as follows: With additional funding the University s Planning and Assurance Services (PAS) unit has been able to build a data warehouse in which information merged from various systems (eg Finance, Human Resources and Student systems) could be used for planning and performance monitoring Data cubes were developed and are being regularly updated to enable users to explore combinations of data by choosing dimensions (ie broad groups of descriptive data about major aspects such as academic divisions, students or funding) and performing detailed calculations to enhance strategic and management information a Strategic Information Management and Reporting Plan was approved by SMG in late A Business Intelligence System Steering Group provides oversight of the implementation of the plan and monitors progress. It also oversees quality assurance processes and enforces data quality standards. Recurrent resources have been allocated to the Business Intelligence function and within that function priorities in 2008 have been improved levels of reporting and analysis on student demand, student satisfaction and workforce data which informed the major quality improvement priorities and processes of the Teaching and Learning Framework Australian Universities Quality Agency

16 Audit Report University of South Australia An integrated data suite was developed as part of a Workforce Strategy and Planning project, see section Research performance dashboards have been in use since 2008 to inform discussions between a head of school, research centre or institute, the DVC: Research and Innovation, Dean of Graduate Studies and the relevant PVC and Dean of Research. The Panel was given a demonstration by staff of the PAS unit of the types of data that were collected and analysed and of the various ways they were used to make them easily understandable. Staff at different levels of the institution appreciated the improvements in and the accessibility to management information. The use of the data is well integrated and supports other processes such as performance management and workforce planning. Commendation 2 AUQA commends UniSA on the widespread and sophisticated use of management information tools such as data cubes, the development of its business intelligence capabilities and their deployment in initiating and monitoring evidence based change and improvement. 2.3 Academic Standards and Benchmarking Standards and Benchmarking The Panel approached its consideration of the University s academic standards through: a close investigation of the sample programs that were identified for the offshore visits (section 4.4), a range of documentation provided by UniSA, and through interviews seeking information about the processes in place. In this context the Panel investigated the setting and monitoring of standards and how UniSA ensures comparability of academic standards nationally and internationally. On the one hand, the Panel identified some good examples of policies, procedures and practical approaches that contribute to the determination and maintenance of standards. On the other hand, the Panel also noted some gaps. As a positive example, there is the Improving Assessment Practices initiative that commenced in 2006 after extensive discussion during The University s relatively poor performance in the LTPF gave the impetus for the University to focus on data for the first time beyond the program level. Responses in the Course Evaluation Instrument (CEI) and in the Student Experience Questionnaire were systematically analysed. The provision of constructive feedback on assessment and workload issues arising from the timing of assessment were identified as two critical issues. As a result of actions taken in relation to Improving Assessment Practices the University s Portfolio points to improvements of performance on the Good Teaching Scale by 5 per cent from 2004 to 2008 and mentions improvements in CEI item 7 (feedback) from 63 per cent to 71 per cent over the same period. In 2008 the University implemented further amendments to improve the Assessment Policy and Procedures Manual. Despite these efforts it seemed to the Panel that many academic staff remained unaware of the linkage of the various aspects of assessment to academic standards. In interviews academic standards were frequently either equated with processes of moderation or with outcomes of external reviews where these are required for professional accreditation purposes. AUQA regards these accreditation processes as general and threshold in nature. At UniSA, external accreditation reports seem, at least in some disciplines, to be used as a proxy for defining and 12 Australian Universities Quality Agency 2009

17 University of South Australia Audit Report monitoring academic standards. Their use and value were not well understood or properly integrated into the quality assurance culture of the institution. One example is a specialisation of the Bachelor of Engineering program. This program is offered by UniSA in Singapore and in Adelaide. The Panel investigated the comparability of the intensive mode of face to face delivery in Singapore with the semester length method of delivery in Australia, including in particular student access to laboratory based work. The consistent explanation given to the Panel was that, because the professional association Engineers Australia had accredited the dual forms of delivery, academic standards were assured. Despite the evident progress since the first audit, there appeared to the Panel still to be a lack of clear institutional definition of and approach to the setting and maintaining of academic standards and only some attempts at external validation of them. For example, the fact that academics have studied or worked at other institutions and therefore have an individual understanding of what is happening elsewhere is not an acceptable substitute or proxy for a common understanding of benchmarked standards. Informal judgements reached on the basis of personal experience needs to be replaced by systematic, rigorous formalised benchmarking for and commitment to quality assurance and improvement purposes. As far as external monitoring is concerned, the Portfolio contains several references to benchmarking, using the Australian Technology Network (ATN) cohort or national data sets, including Course Experience Questionnaire (CEQ) data. Despite these references in the Portfolio and an evident understanding at senior executive level of the benefits of benchmarking, apart from the use of standard national data sets, there does not seem to be any systematic use of it across programs and schools for the purposes of institutional and academic quality assurance and continuous improvement. The Portfolio refers to the University s objective to instil in all its graduates the qualities and confidence to continue to develop and achieve at the highest levels both in Australia and internationally. It also mentions the development of an Academic Standards Framework, which has been discussed at, and approved by, Academic Board in In relation to both these areas, the international dimension of benchmarking appears to be at present largely missing at both institutional and academic unit levels. AUQA notes that Recommendation 1 of the Cycle 1 report states: That UniSA provide a clear definition of benchmarking, and provide assistance to divisions and units in identifying, collecting and interpreting benchmarking data. Notwithstanding some examples of the use of benchmarking at the discipline level, such as nursing, or at the institutional level, in the case of research data, the above observations lead AUQA again to include a recommendation that encourages increased attention to benchmarking and to linking it to academic standards. Recommendation 1 AUQA recommends that UniSA take a more strategic approach to benchmarking and the validation of its academic standards to facilitate continuous improvement at school and institute levels and within disciplines The Academic Board The Academic Board is one of the major committees of Council. Its charter states The principal academic advisory body to Council is Academic Board. This charter establishes the Australian Universities Quality Agency

18 Audit Report University of South Australia responsibilities of Academic Board within the governance system of the University of South Australia. It then lists the primary role and functions as follows. 1. To act as a forum for the debate of University wide academic issues. 2. To foster excellence in teaching, learning and research in the University. 3. To oversee quality assurance and the administration of academic matters. 4. To approve proposals for the development of all undergraduate and postgraduate programs and courses, and the phasing out of, or amendments to, existing programs and courses. 5. To approve policies and procedures related to teaching, learning, research, equity and student administration, and any variation to standard academic procedures. 6. To provide advice on the development of compacts with the commonwealth and associated matters. 7. To provide formal input into the corporate planning process through the review of priorities and parameters for the annual Corporate Plan. The Panel explored the current role of the Academic Board through its consideration of documentation and questioning of members of the Board, academic staff at various levels within the University and members of the SMG. It reached the conclusion that there is a lack of general awareness or understanding across the University of the role of Academic Board in determining and monitoring academic standards. The Panel notes that a further confounding factor in wider understanding of the Board s role may be the high profile of some of the Board s key sub committees such as the Teaching and Learning Committee, the Academic Programs and Policy Review Committee, and the Research Degrees Committee as well as division Teaching and Learning Committees. The Panel also had difficulty in substantiating that the role described for the Academic Board in the Portfolio accorded with its actual standing within the UniSA academic community. On the basis of the available information, the Panel was not convinced that the Academic Board contributes as effectively as it might to the development and progression of academic matters as provided for in its terms of reference. It also was rare in interviews for individuals to refer to the Academic Board as an influential or overarching body. This was in sharp contrast to the frequent reference made to the importance of academic divisions and the SMG. The prominence of the latter group may, over an extended period, have contributed toward the current perception of the role of Academic Board within the academic community. As already mentioned in the context of New Horizons, the Panel was assured of the culture of openness and inclusivity prevailing at the University. AUQA is of the view that this should extend to and include the Academic Board and its remit in relation to the development and review of academic standards. Recommendation 2 AUQA recommends that UniSA consider strengthening the role and standing of the Academic Board primarily in relation to academic standards and quality assurance in line with its charter. Across institutions, academic promotion is a key factor and incentive in influencing academics performance and behaviour. The Portfolio states that since 2004 administrative processes in relation to academic promotion have been reviewed and revised on a number of occasions. Despite this it seems to be taking a long time to adjust promotion criteria to reflect changes in University objectives. This process commenced in 2006 and a study of academic promotions 14 Australian Universities Quality Agency 2009

19 University of South Australia Audit Report processes in 2008 was said would form the basis of a policy review. This is a matter to which UniSA might warrant giving greater priority. 2.4 Follow up from 2004 AUQA Audit The 2004 AUQA Audit Report contained 16 commendations, five affirmations and 10 recommendations. As part of this audit, the Panel evaluated a first progress report dated 2005, an updated report from 2007 and a description of the current status of affirmations and recommendations as part of the Portfolio. On the basis of this desk audit the Panel decided that several of the affirmations and recommendations would be addressed through in depth consideration of the two themes which form the basis of this audit in In addition, the Panel signalled that it would follow up Affirmation 3 and recommendations 7 and 10. Overall AUQA concludes that UniSA s responses to the items from the 2004 AUQA audit demonstrated satisfactory progress in responding to the issues raised at that time. In reaching this conclusion it should be noted that the subject of the previous Recommendation 1 reappears in a broader context in section 2.3 and that Recommendation 6 and Affirmation 4 seem still to be work in progress Stakeholder and Community Engagement Affirmation 3 of the 2004 AUQA Audit Report states: That UniSA make more systematic use of stakeholder input in program evaluation. Enhancing the effectiveness of Divisional advisory boards may assist with both this and other aspects of external advice and input. The Panel was satisfied that UniSA s documented response to this affirmation addresses the issues. Recommendation 7 states: That UniSA develop a conceptual and strategic planning framework for its objective of community engagement, developing indicators and measurements, and pay explicit attention to evaluating the desired objectives. The 2006 Strategic Plan of UniSA adopted the Regional, Industry and Community Engagement strategy and thus formalised the University s commitment to community engagement. This includes a statement of what the University seeks to achieve through such engagement, key result areas for achievement and related performance dimensions. Research income through engagement has been included in the relevant set of key performance indicators since The adoption of a conceptual framework for engagement at UniSA is firmly embedded as it arose from a corporate project in 2006 and 2007 to establish a comprehensive framework for measuring and monitoring engagement activities. In recognition of its achievements in engagement, UniSA in 2008 was awarded the Business and Higher Education Round Table s inaugural Ashley Goldsworthy Award for sustained collaboration between business and higher education sectors. Other examples of outcomes are the systematic and valued engagement initiatives being undertaken with local communities under the leadership of the Centre for Regional Engagement based at the Whyalla Campus, and through a University level partnership with the City of Salisbury, the development of a major innovation hub at the Mawson Lakes Campus and its surrounding environs. UniSA is demonstrating a genuine commitment to engage with its regional communities and has secured a number of significant achievements as a result of this engagement. Australian Universities Quality Agency

20 Audit Report University of South Australia Commendation 3 AUQA commends UniSA for the substantial improvements on issues related to community engagement raised in the AUQA Cycle 1 audit Student Representation Recommendation 10 from the 2004 AUQA Audit Report states: That, consistent with its aspiration to be student centred, UniSA investigate jointly with the students association ways to improve the levels of student representation on committees across schools and programs. Students are now represented on key University committees and appeared to be generally positive about their opportunities to contribute to decision making. It should be noted, however, that it was not possible for the Panel to speak with the President of UniLife, the student association at UniSA, to seek his confirmation of the level of genuine engagement with the student leadership. Despite these improvements in representation, student programs and amenities seem to have been affected markedly as a result of cost containment action taken by UniSA following the introduction of legislation prohibiting the collection of compulsory fees for student services and amenities in This therefore appears to be an area that will warrant further careful consideration by UniSA Schools and Institutes Recommendation 6 from the 2004 AUQA Audit Report states: That UniSA clarify the relationship between schools and research institutes, centres and groups, and resolve a number of related organisational issues. Although this was not specifically identified for explicit follow up at the Audit Visit, the Panel observed that UniSA took some time to grapple with and resolve issues raised in this recommendation, as the following chronology shows. As part of the University s initial followup, an external review on the reporting relationships between various entities was conducted in As a result, a UniSA Research Concentrations Strategy was approved in 2008 and frameworks are being developed during This will address the reporting relationships and responsibilities for performance development and management of staff working between an institute and a school. One academic division head outlined to the Panel how this will work in his division from next year. It will include clarification of the role of schools in relation to higher degree by research students when they are enrolled in a school but working at a research institute. See also section of this Report and a related affirmation. 16 Australian Universities Quality Agency 2009