Introduction to UTS/OLT Educational Data Mining Projects. Prof. Longbing Cao Director, Advanced Analytics Institute

Transcription

1 Introduction to UTS/OLT Educational Data Mining Projects Prof. Longbing Cao Director, Advanced Analytics Institute

2 Outline Introduction to Educational Data Mining (EDM) IEEE Task Force OLT & UTS LT Projects Briefing The Underlying EDM Techniques Future Work

3 Outline Introduction to Educational Data Mining (EDM) IEEE Task Force OLT & UTS LT Projects Briefing The Underlying EDM Techniques Future Work

4 Educational Data Mining (EDM) In a nutshell, Educational Data Mining (EDM) is a newly emerging interdisciplinary research field which focuses on Knowledge Discovery and Data Mining techniques to analyze data from educational settings, including interactive learning systems, intelligent tutoring systems and institutional administration data. The primary goals of EDM is to uncover scientific evidence or patterns that are useful to gain insights and explain educational phenomena.

5 EDM Stakeholders

6 EDM Goals For students Improve learning performance Evaluate learning effectiveness ( learning is defined as reduction in error made over times of practice) Understand social, cognitive and behavioral aspects For teachers Improve teaching performance understand student preference, learning experience Respond to student needs Adapt to each individual student For institutions Distant learning (online) Adaptive tutorials (focus on individual student's needs and weak points) Evaluate teaching effectiveness (how well teachers respond to student needs) Predict student performance (risks of dropping out or failing a subject)

7 EDM Goals Educational data processing and representation Educational data acquisition Educational domain representation Educational data preparation Educational data quality issues TL behavior construction EDM benchmark data TL behavior analysis TL behavior modeling TL behavior pattern analysis TL demographic analysis Replication analysis Plagiarism analysis TL group analysis TL sequence analysis TL evolution analysis TL history analysis Mobility analysis

8 EDM Goals EDM social analysis Educational social factor analysis Educational psychological factor analysis Educational pedagogical analysis TL hidden network and its behavior Performance, effect and impact analysis TL performance profiling TL cause effect analysis TL intervention evaluation Evaluation and validation EDM evaluation methods TL validation methods EDM software and applications EDM software and tools Mobile computing EDM tools Educational teacher support Web-based EDM tools Applications Lessons

9 Outline Introduction to Educational Data Mining (EDM) IEEE Task Force OLT & UTS LT Projects Briefing The Underlying EDM Techniques Future Work

10 IEEE EDM Task Force

11 Outline Introduction to Educational Data Mining (EDM) IEEE Task Force OLT & UTS LT Projects Briefing The Underlying EDM Techniques Future Work

12 UTS Project Pipelines A collection of projects conducted at UTS: Improving The Support for Student Learning Needs Through Pattern Analysis of The Student Experience; Curriculum Renewal and Better Support of Student Learning Needs Through Pattern Analysis of Student's Learning Experiences; Inform Course Learning and Renewal by Deeply Understanding Cause-Effect of Student Learning Behaviors and Performance; Data Mining Modelling and System Prototype for the Detection and Prediction of Students at Risk in the "Killer Subjects ; At-risk student's prediction for early intervention in key engineering course; University Admission Centre Data Analysis.

13 OLT Project Aims Title: Data mining of learning behaviors and interactions for improved sentiment and performance Aims: This project aims to develop innovative methodologies for an in-depth discovery of: the learning patterns that positively and negatively contribute to the students academic performance. how the poorly-performing students can be instantly identified and transferred to a superior level.

14 Challenges in Killer Subjects

15 Some questions to be addressed by EDM Which students are at risk? How to improve the performance of students at risk? Where should UTS go to source good students? Is student happy or sad (emotional well being)? Which students for further education? Education pathway What is next best step? What should you do? Do different campuses exhibit vastly different performance? Where and when should we hold the class for best performance? What is optimum mix of blended learning for student? How to evaluate the knowledge gap among subjects?

16 At-risk students prediction for early intervention in key engineering course C10061 The service is provided to all students in course C10061 i-educator Subjects involved: All core subjects; Cross major subjects; Killer subjects in EE; Subjects with high failure rate and large student number FEIT students Recommendations: Free English courses Financial Assistant Engagement Improving Review math knowledge 1 Data collected by student systems 4 Recommendations proposed to students according to their risk reasons 2 Student Systems (CASS, UTS:Online, etc) Hi We re calling to see if you re going OK. Data integrated for analysis 3 SSU Outreach program Risk scores and reasons generated for intervention Stu No: Risk level: 0.7 Risk reasons: English level: poor Socio-Eco Status: low Math backgrd: low Engagement: Medium

17 What does/will i-educator do for student risk management? Data: any relevant private and public data Risk rating: rate vs. rank at different levels Risk factors: why risky Risk patterns: how students behave Intervention: risk causes, areas to be improved, what to be taken or where to change student behaviors Validation: response from the intervention back to the system input and modeling

18 Data Data CASS data Allocate Plus data Assessment Data Sanction Data UTS Online data UTS Library data SFS data Housing data ABS Collected or not

19 i-educator (Lecturer dashboard for identifying academically risky students)

20 i-educator (Lecturer dashboard for identifying academically risky students)

21 i-educator (Lecturer dashboard for identifying academically risky students) Key factors in common Historical education level Stu_suburb Birth_cntry_fail_rate GPA Engagement Exclusion_sanction Activity_num Sanc_p Fundemental subjects in common (math mod1) (Phys mod) Distinct key factors for different subjects Previously taken subjects scores and failure times (reflects students knowledge fundation for this subject) Prerequisite Non prerequisite

22 Performance of i-educator

23 Performance of i-educator (Evaluation conducted on Autumn 2012 first-year student data) Target student group: All FEIT first year students in Autumn 2012 (1109 in total) - 96 with academic cautions failed at least one subject Evaluation results Students with academic cautions - Top 100 (around 10%) covered 71 (recall 74%) - Top 200 (around 20%) covered 94 (recall 98%) Students with at least one subject failure: - Top 10 with 10 failed at least once (precision 100%); - Top 20 with 20 failed at least once (precision 100%); - Top 30 with 28 failed at least once (precision 93%); - Top 40 with 38 failed at least once (precision 95%); - Top 100 (10%) with 92 failed at least once (precision 92%); - Top 200 (20%) covered 153 failed at least once (precision 76.5%);

24 UTS:Me (Student dashboard for understanding student sentiment)

25 Factor Linkage Driving Student Intake and Preferences

26 Preference Streaming of Student Intake Dynamics

27 Factor Correlation for Student Intake Data

28 Outline Introduction to Educational Data Mining (EDM) IEEE Task Force OLT & UTS LT Projects Briefing The Underlying EDM Techniques Future Work

29 The Underlying EDM Techniques

30 Future Work Courseware improvement Learning process improvement Career planning Life-long learning recommendation i-educator enhancement

31 Thank You! Any comments?