Accelerating the strategic use of student data: a collaborative approach

Transcription

1 Accelerating the strategic use of student data: a collaborative approach A/PROF ANNE YOUNG Director Strategy, Planning and Performance Vice-Chancellor s Division 21 November 2014

2 The University of Newcastle at a glance The University of Newcastle

3 Strategy, Planning and Performance unit Main responsibilities Strategic and corporate planning Institution-wide KPIs and trend indicators Business intelligence (Data warehouse, dashboards, reports) Enrolment projections and management Academic QA Surveys/focus groups Predictive analytics The team Statisticians Quality assurance experts Qualitative researchers Business analysts Data warehouse / ETL staff Teaching and research experience 3

4 The Importance of information The most meaningful way to differentiate your company from your competition, the best way to put distance between yourself and the crowd, is to do an outstanding job with information. 4 How you gather, manage, and use information determines whether you win or lose."

5 Types of student data? During the application process During the enrolment process Program and course level information Retention Success (percent of load passed) Completion Engagement (attendance, library, LMS, PASS, workshops,..) Survey data (internal, national, international) Focus group data Social media Graduate outcomes 5

6 Management Information System 6 Student records HR LMS Course & Program Surveys Research MIS (DW) Finance Applications Engagement Outcomes Data Silos Integration - Aggregation Holistic Perspective

7 Common issues in large organisations Multiple disparate data collections and systems 7 Inconsistent terminology and definitions Reactive approach to data quality Difficulties accessing data Insufficient training and education about data Silo mentality and turf wars The Data Governance Institute

8 Link to Strategic Plan? 8 Strategy 1.3 Achieve student success and retention above national and international benchmark levels through the provision of diagnostic information to inform targeted action plans by Faculties and Divisions. How did we get to this point?

9 New approach in 2009 Focus on improving the student experience Student surveys moved from paper to online All units of study, every time offered M/C with open comments encouraged All comments read & coded (55,000 in 2009 alone) 9 Integrated quantitative and qualitative reports prepared for each unit of study (>1500 reports in 2009) Summary reports for Executive Committee and each School and Faculty with comparative data Reports on the web for staff and students Ongoing meetings with Heads of School and academic staff lots of conversations!

10 What did the students say about teaching? Students think of their teachers holistically 10 Teaching quality is at least as important as subject knowledge to students A good teacher is usually able to overcome any perceived weaknesses in a course Students link their own motivation to the quality of teaching they experience

11 What did the students say about relevance? Learning is interesting when it connects study to personal aspirations and to the real world. 11 Teaching that shows clear relevance between what is being taught and why it is being taught and how that will help with study, assessment, outcomes, career paths is highly valued.

12 What did the students say about structure? Learning works well when content is organised, themes are linked and all course elements are well integrated 12 Lectures, tutorials, laboratories, practicum and other learning should all support each other Learning and teaching methods, assessment approaches and course content should be coherent Courses/programs should be designed to provide supported learning by building on content and assessment throughout the course/program

13 What did the students say about flexibility? Having the opportunity to learn in ways most suitable to one s style, environment and pace 13 Flexibility of delivery may be required to accommodate personal circumstances Learning happens in different ways, and at different speeds

14 What did the students say about assessment? There is, and should be, connection between assessment and learning 14 Information about requirements of an assessment task is appreciated Getting feedback soon after tasks and other learning demonstrations aids learning Relevance of assessment to content, course or aspiration is often unclear to students

15 Common drivers behind passive response or resistance to change 15 Response rates too low to be valid Only whingeing students respond Data not relevant / out of date Student deficit Not enough time or resources Change not a priority Having results that demonstrate improved outcomes is critical to making sustainable changes from resistance to acceptance to enthusiasm.

16 Actions taken by early adopters of change Student forums, student centred approach Respond positively to student feedback Student support, social events Quality assurance mechanisms strengthened Teaching improvement initiatives Peer review of teaching Actively promote excellence in teaching Focus on academic development Support and training for program and course coordinators Better induction for casual staff More accountability Embedded career advice and work integrated learning 16

17 Institution-wide strategies Phase 1 Improvements to course and program management Improvements to course and program design Accommodating diversity of student cohorts Provision of future generation learning spaces Academic development Tailored support Language support Diagnostic tools developed Phone calls to students who withdrew to better understand attrition Longitudinal student focus groups to track progress 17

18 Institution-wide strategies Phase 2 Predictive modelling to enable early contact and targeted support Using historical data to develop predictive models of attrition, particularly attrition in the first few weeks of semester Program/discipline specific where possible Test and refine models ahead of launch In parallel, prepare targeted strategies in collaboration with academic staff and student support staff based on key predictors Run prediction models in Week 0 and implement targeted support Continue to engage with students during term Evaluate outcomes 18

19 Institution-wide strategies Phase 3 Provide a range of analytical tools and act to improve retention and success Expand the range of data products available to staff Integrate new sources of data into the Management Information System Provide more self-service access to Business Intelligence for staff Launch a visual analytical platform to encourage engagement with data 19 Discuss appropriate provision of data at a local level and then how to best integrate systems for more strategic use Monitor results and provide feedback to students and staff Encourage more collaboration academic and professional staff Celebrate success!

20 Assessing Business Intelligence Maturity central coordinated team breadth of data (education, research, HR, finance) awareness/role of business units type of products available (reports, dashboards, analytics) who is using the outputs contribution of users to development of outputs data management & data quality perceived business value positioning in the institutional strategy 20

21 Big data: The next frontier for competition McKinsey Global Institute, May Analysing large data sets will be a key basis of competition as long as the right policies and enablers are in place. Companies and policy makers must tackle significant hurdles to fully capture big data's potential including a shortage of skilled analysts and managers. How is your institution managing these challenges and accelerating the strategic use of student data?

22 Thank you DISCUSSION ALASI 2014 Australian Learning Analytics Summer Institute Sydney, November 2014 CRICOS Provider 00109J