Course sustainability = relevance + quality + financial viability

Transcription

1 Course sustainability = relevance + quality + financial viability Sonia Ferns, Beverley Oliver, Sue Jones, Rosemary Kerr Curtin University of Technology Abstract National ranking systems that determine funding, a competitive global market and increased student diversity are placing pressure on universities to provide quality teaching and learning experiences for students. Curriculum 2010 (C2010) is an initiative of Curtin University of Technology to ensure all courses offered at the university are of strategic significance, comprise quality teaching and learning experiences and are financially viable. C2010 aims to ensure Curtin remains competitive within the higher education market well into the future. To achieve this, a range of indicators has been identified to determine the level at which a course is operating under a set of criteria. Collectively, the indicators will be considered to establish an overall rating to determine sustainability. This paper reports on the progress of developing a model for determining course sustainability. Four courses one from each faculty of the university were identified in consultation with representatives from the owning schools. These courses will be rated against each indicator and assigned an overall sustainability rating. The study has involved extensive collaboration with representatives from the teaching teams to ensure all issues are addressed and considered in the development of a course sustainability model. This paper outlines the key criteria and associated indicators. It has emerged that rating courses against specific measures and (ultimately) determining overall sustainability is complex and encompasses many factors. As the model development is still in progress, a description of the process is provided and issues to date are highlighted. Keywords course sustainability, course quality, strategic relevance, financial viability Introduction A competitive global market, an increasingly diverse student population and a greater demand for specific workplace proficiency is placing greater demands on universities and teaching staff to provide quality experiences for students. In an era where accountability is measured, student evaluations of teaching and learning impact on Commonwealth funding, and universities are subject to national ranking systems the quality of courses on offer is central to the generation of positive outcomes, positive community perceptions and successful marketing opportunities. According to Ward (2007), higher education systems are being asked to educate more students, provide more support for them, address workforce needs, solve social, scientific and technical problems and do it all more efficiently. Consequently, universities need to offer courses that encompass all facets of

2 excellent educational opportunities and ensure students are recipients of relevant and fulfilling teaching and learning experiences. Australia has experienced the massification of higher education, which has led to concerns about quality because in the absence of any major reductions in unit costs, the expansion of access has far exceeded its anticipated costs (Ward, 2007). The higher education sector is experiencing a continuing emphasis on quality assurance and quality improvement with the establishment of performance indicators capable of yielding nation-wide and increasingly, global scale comparisons (Meek & Wood, 1998) These trends have placed pressure on all Australian universities to find ways to adapt rapidly to external change in a knowledge-based economy (DEST, 2005). The increasing demands for accountability and program consolidation means that institutions need to demonstrate to stakeholders that concerted efforts are being made to enhance institutional efficiency and effectiveness (Michael, 1998). Pivotal to a university s success is the quality of its courses. Courses must be quality teaching and learning experiences, and be strategically significant and financially viable to ensure long-term sustainability. Institutions need to re-examine their methods of operation and innovation, while capitalising on their strengths, in order to remain viable now and in the future (Sorensen et al., 2005, as cited in Furst, Bowe & Bauer, 2007; Michael, 1998). The processes used should aim to produce sustained improvement in areas such as student learning and performance, stakeholder satisfaction, staff wellbeing and overall organisational effectiveness (Furst, Bowe & Bauer, 2007). Program reviews (also known as course reviews ) are one aspect of the renewal process. Michael (1998) outlines various models for program review: the cost model, quality model, market model, employment model, political model and academic model. Essentially, since the delivery of an academic program is extremely complex in relation to all its stakeholders, most program reviews use a combination of the models, known as the eclectic model, described by Michael. Program reviews usually involve a combination of performance indicators, selfstudy and external review to evaluate an academic program (Dill, 2000). Dill states that comprehensive reviews enable an institution to make strategic decisions about the future sustainability of a course. Program review is a vital component of a sustainability study. According to Meek and Wood (1998) it is at the program level where much of the competition in higher education occurs. It is through programs that institutions attract students, and it is through students that institutions attract money. It is essential that all staff and stakeholders (including students) have confidence in the process of determining course sustainability. An inclusive collaborative process has been undertaken so that all staff have ownership of the final model. Once the model is in place, it is intended that staff will be empowered to make

3 decisions based on sound evidence, thus creating a culture at the university that embraces the concept of long-term sustainability. In order to remain a competitive entity in the higher education market, Curtin is implementing a comprehensive course review process that comprises three main criteria: course quality, course relevance and course viability. Comprehensive course review aims to ensure the university operates in a financially sustainable manner and establishes work patterns for staff that are manageable, fair and efficient. Embedded in each criterion is a range of specific indicators and measures that generate a course sustainability rating. This paper reports on the progress of the development of a model for determining course sustainability at Curtin. Criteria and indicators The Comprehensive Course Review Process incorporates and acknowledges the importance and relevance of student, external and other stakeholder feedback including professional accreditation reviews. The review process focuses on three main aspects: course quality, course relevance and course viability. Data are collected for a range of indicators embedded in each of these aspects to identify emerging trends of a course. The model proposed here suggests that these measures must be considered as a whole, and not in isolation. Even though it may be tempting to judge a financially unviable course as unsustainable, the model suggests that all three factors must be considered together to make a reasonable judgement. While it is difficult for universities to continue to offer unviable courses, other factors, such as strategic significance within a community, may warrant the continued existence of a course. Conversely, a profitable course that is low in quality and relevance may not be deemed sustainable. even though the course generates healthy income, poor quality and relevance will mean that ongoing success is unlikely. Therefore, such a course is unsustainable. Provisos such as these dictate caution when a course performs poorly on one measure but highly in others. If the reasons for poor performance are valid, careful thought and.on balance. judgment will be required to avoid compromising the overall ranking of the course. Course quality Course quality may be judged according to many criteria, but in this model the following indicators are being considered to measure course quality and collectively provide a teaching and learning profile that better facilitates an overview of course quality and student satisfaction levels. In this model, these criteria are a mix of internal and external student feedback measures, graduate destination, entry criteria, pass and retention rates, as shown in Table 1. Table 1

4 Indicator Description Ranking evaluate unit survey: Quantitative item 11 for each unit in the course Student feedback on teaching and learning using Curtin s internal system Over 80% Overall satisfaction. GREEN 60 to 80% Overall satisfaction. AMBER Overall satisfaction less Course Experience Questionnaire (CEQ) Graduate Destination Survey (GDS) Tertiary Entrance Rank (TER) Pass rates Retention rates Good teaching scale Generic skills scale Overall satisfaction Graduates in full time or part time employment Graduates in further study? Tertiary entrance ranking of enrolled students Percentage of students passing units in the course? Percentage of all students retained than 60%. RED Above the national average. GREEN Equal to the national average. AMBER Below the national average. RED 80% or greater employment. GREEN 60% to 80% employment. AMBER Less than 60% employment. RED Average TER is consistently high?. GREEN Average TER declining 5% over 5 years. AMBER Average TER decline greater than 5% over 5 years. RED Pass rate greater than 95%. GREEN 70% to 95% pass rate. AMBER Pass rate less than 70%. RED Retention rate greater than 85%. GREEN Retention rate 80% to 85%. AMBER Retention rate less than 80%. RED In this model, a course is assigned an overall status of RED, AMBER or GREEN on course quality using the indicators in Table 1. If two or more of the indicators achieve a rating of RED, the overall rating for course quality will be

5 RED. Two or more ratings of AMBER will result in an overall rating of AMBER. Course relevance In this model, relevance is defined as a composite of the course s potential to achieve strategic advantage for the university. A range of indicators are considered when reviewing the strategic direction of a course, using the indicators in Table 2. Feedback from the sources listed in Table 2 identifies existing competition and potential risks for the ongoing sustainability of the course in its present form. The information may indicate that minimal changes to the course can maximise opportunities to partner with other organisations. For example, scrutiny of courses offered through TAFE may highlight opportunities for articulation pathways to be established. Such partnerships boost the potential market, increase marketing opportunities at low cost, tap into existing resources and comply with industry requirements. Developing productive partnerships is also aligned with Curtin s strategic focus on nurturing relationships for mutual benefit both internally and externally. Table 2 Indicator Is this course a named priority area for the university? Is there competition from other educational providers, local and remote? Are the future directions of industry buoyant? Are there established articulation opportunities? Are there collaborative partnerships and research activities? How did the course rate in recent reviews by the professional accrediting body? Sources University Strategic Plan Research undertaken by Curtin.s Office of Strategy and Planning Web searches CEQ data Interviews with personnel from teaching areas Research undertaken by Curtin.s Office of Strategy and Planning Peak industry bodies Interviews with personnel from teaching areas and advisory boards Interviews with personnel from teaching areas and from external providers and delivery agencies Articulation and RPL committee Interviews with teaching staff, Course Coordinators and Executive Deans Office of Research and Development Accreditation documentation Criteria in Table 1 are largely quantitative, and this clearly makes the assessment of sustainability much easier. Indicators in Table 2 are clearly far from

6 quantitative. This is an area where.on balance. Judgments must be made on qualitative data. In this model, and using this method, a course is assigned RED status if any three of the following occur and AMBER if any two of the following occur: Course is not a priority area for the university Significant competition exists from other providers Industry is experiencing a down turn No articulation opportunities exist if appropriate No collaborative activities are underway Feedback from accrediting body is unsatisfactory. Course viability Course viability is most closely related to indicators that determine costs. The measures for this criterion, as listed in Table 3, examine the market trends, course demand and financial indicators. Table 3 Indicator Description Ranking TISC preferences Trends of TISC preferences over a 5- year period Increased preferences. GREEN Stable. AMBER International demand Direct applications Financial profile Demand of fee paying students Number of students applying directly to the university Income versus cost of course delivery Decreasing preferences. RED Increasing demand. GREEN Stable demand. AMBER Decreasing demand. RED Increasing. GREEN Stable. AMBER Decreasing. RED Cost of delivery less than income. GREEN Decreasing trends in income. AMBER Cost of delivery exceeds income. RED Enrolments Number of new enrolments Rising trend over 5 years. GREEN Stable trends (+/. 5%). AMBER Declining trend over 5 years. RED Financial viability is of major concern in the consideration of course sustainability. Teaching teams need a consistent set of rules to determine course financial viability. In the past, the relevant information about costs and income has not

7 been readily available to Course Coordinators, which has hindered decision making regarding course viability. One of the key tasks generated from this study has been the development of a course costing model. This model is being trialled with the courses involved in testing the sustainability model. The Course Coordinator will manually enter data that may vary from unit to unit to determine the significant cost drivers within a course. The software will then automatically generate the income and outgoing costs for a course. Using these indicators, a course is assigned an overall status on of RED, AMBER or GREEN on course viability. If two or more of the indicators achieve a rating of RED, the overall rating for course quality will be RED. Two or more ratings of AMBER will result in an overall rating of AMBER. Overall assessment of sustainability Using the indicators above, a rating of GREEN, AMBER or RED is assigned for each of the three key criteria. The next step is to make a judgement about the overall sustainability of a course. The data and information collated for each course can be used to highlight specific threats to a course and alert the course team to the need to develop strategies to address the issues. By using the data, course teams can work to improve the course rating over a given timeframe. There are likely to be several scenarios, as indicated in the examples that follow. Course A Course relevance Course quality Course viability Overall rating GREEN GREEN GREEN GREEN Course A is clearly sustainable, according to this model, since all three criteria are green. Course B: Course relevance Course quality Course viability Overall rating GREEN GREEN RED AMBER Course B has strong relevance and quality, but falters on viability. In this case, the overall rating is AMBER. Course C: Course relevance Course quality Course viability Overall rating GREEN AMBER RED AMBER Course C has strong relevance but falters on quality and viability. In this case, the overall rating is AMBER. Course D:

8 Course relevance Course quality Course viability Overall rating AMBER AMBER RED RED Course D falters on relevance and quality, and particularly viability. In this case, the overall rating is RED. Procedure A range of approaches have been implemented to facilitate collection of data that incorporate all aspects of course delivery in a university context and reflect the multifaceted nature of course sustainability. The courses selected for testing the sustainability model have been deidentified to ensure anonymity and reinforce the intention to test the criteria rather than the actual courses. Widespread consultation with staff has been (and will continue to be) a feature of this model development. An initial paper was written outlining proposed criteria and measures for sustainability and placed on the website where staff could provide online feedback. Staff also requested a meeting with the authors of the document, which facilitated open discussion. This strategy has been valuable for two reasons. Firstly, the university community has had an opportunity to consider and reflect on the concept of course sustainability resulting in frequent discussions on the topic. Secondly, it has provided excellent feedback for the team working on course sustainability and enabled a continual cycle of reviewing and refining the model. Data, both qualitative and quantitative, examines a course from a range of perspectives including student, staff, employer and industry stakeholders. Data pertaining to each course was collected from a range of sources, including national and university websites and specific documentation. To support comprehensive course review, the university is developing a data warehouse where relevant data will be stored for easy access. The indicators under consideration for the course sustainability model have informed the development of the data warehouse. Staff consultation is considered a priority as a means of acknowledging the insights and expertise of teaching staff. A meeting was held involving the Course Coordinator, Head of School, Dean (Teaching And Learning) and appropriate C2010 staff. Course data was tabled at this meeting, prompting significant discussion about strengths and areas for improvement pertaining to the course. A needs analysis document was produced from this discussion and shared with relevant staff. Meetings were held with the Executive Dean of each Faculty. To ensure the conversation was focused and obtain relevant information, three focus questions relating to the relevance of this course were forwarded to the Executive Dean prior to this meeting. A meeting with a similar format was held with the Course Coordinator, providing an opportunity for him/her to express views honestly and openly. All staff involved in teaching the courses were invited to

9 team meetings (which have been well attended, indicating an eagerness from staff to provide input). At the time of writing this paper, one team meeting is yet to be held. Researchers sought staff permission to tape the meetings to ensure an accurate record of comments. Prior to the meeting, staff were sent 12 questions. The intention of meeting was to be.conversational. in format, the questions providing a focus and enabling staff to consider comments prior to the discussion. All meetings were of immense value as they a provided feedback on sustainability of the course from an executive management and course management perspective. All of this qualitative data is triangulated against the qualitative data from the course relevance indicators. The Executive Dean and Course Coordinator were also invited to predict the sustainability rating of the course. If the final outcome contradicts significantly with these predictions, further scrutiny of the course will be undertaken. Discussion While this model development is incomplete, much has already been learned. Challenges in measuring course sustainability are significant. Firstly, many of the indicators may sit in more than one key criterion. Retention, for example, could sit with course quality or course relevance. Students may be driven to complete because they are engaged in the quality of the learning experience or the lure of highly paid employment opportunities in a buoyant industry climate. The model development undertaken to date has also highlighted the need for intensive collaboration in defining and refining measures. The complexity of measuring and determining sustainability requires input from a range of stakeholders to ascertain that all perspectives are considered in the final analysis. The concept of course sustainability is a relatively recent phenomenon for university staff and may be perceived as a threat to the traditional academic culture incorporating academic freedom, intellectual capital and a knowledge based infrastructure. It is important that the entire university community embraces the concept of sustainability and has confidence in the measures used to indicate the level of sustainability. Given the sensitivities surrounding the concept of sustainability, the approach needs to be considered and carefully engineered. This model development and testing will help to answer some integral questions pertaining to course sustainability. Under which criteria do the indicators belong? Are the indicators a meaningful measure of sustainability and do they collectively provide a true overall indication of the future sustainability of a course? Do staff have confidence in the indicators and perceive the process to be comprehensive considering all the facets of teaching and learning in a university context? Some answers will only come with the passage of time and may require revisiting the model to reflect the changing circumstances.

10 References Department of Education, Science and Training (DEST). (2005). Building university diversity: Future approval and accreditation processes for Australian higher education. Canberra: Author. Dill, D. (2000). The academic audit project: Implementing academic audits: Lessons learned in Europe and Asia. Retrieved May 11, 2007, from FurstBowe, J., & Bauer, R. (2007, Spring). Application of the Baldrige model for innovation in higher education. New Directions in Higher Education, 137. Retrieved May 11, 2007, from Wiley Scientific Online. Meek, V., & Wood, F. (1998). Managing higher education diversity in a climate of public sector reform. Canberra: Department of Employment, Education, Training and Youth Affairs (DETYA). Michael, S. (1998). Restructuring U.S. higher education: Analysing models for academic program review and discontinuation. The Review of Higher Education, 219(4), Ward, D. (2007). Academic values, institutional management, and public policies. Higher Education Management and Policy, 19(2). Retrieved May 10, 2007, from Organisation for Economic Cooperation and Development (OECD).