Adaptive Learning Systems

Adaptive Learning Systems The State of the Field Michael Madaio

What is Adaptive Learning? Individualized learning: Content difficulty Content form (text, video) Sequence of content Pace Adapted based on: Student knowledge, performance Student learning style, preferences, goals

1920 s - Personalized learning at scale (Dewey, Montessori) 1960 s - Mastery-based learning (Bloom) 1970 s / 1980 s - Computer-assisted Instruction, Cognitive Tutors Koedinger et al., 1997; Mödritscher et al., 2004 Brief History

Why now? Proliferation of learning technologies Students, teachers, institution Learning mediated through technology Flipped classes, MOOC s Learning apps, student collaboration on forums

Why now? Volume of learning data generated Variety, velocity, granularity Improvements in big data analytics Machine learning Hadoop, MapReduce Student clustering Learning recommendations

Improved student engagement Improved student performance on learning outcomes Improved student retention in courses Improved research-based interventions Magnisalis, 2011; Natriello, 2011; Poelhuber, 2008 Potential

Challenges Material: Bandwidth Laptops, tablets Cost to license software Infrastructural: System setup and support Integration with existing LMS s Training and adoption curve

Challenges Pedagogical: Integration into existing teaching processes Course authoring Accuracy of recommendations Social dynamics of self-paced learning

Current Adaptive Landscape We will look at 16 major adaptive learning providers These 16 are currently used in several public and private universities, by a variety of public K-12 school districts, and by corporations and individuals

Adaptive Providers Major companies Knewton Cerego CogBooks DreamBox Others SmartSparrow Open Learning Initiative LoudCloud ALEKS

Adaptive Providers

Sites of Adaptive Learning Emporium math courses have seen an 18 percent increase in pass rates and 47 percent drop in student withdrawals, from 2011-2013. ASU leadership estimates that the institution has retained $12m in what would have been lost tuition revenue, for 2012-2013. University of New South Wales (SmartSparrow) Developed with 6 other Australian universities to teach threshold concepts in mechanics courses Saw a decline in average course drop-out rate from 31 percent to 14 percent, even as course enrollments increased by nearly 30 percent, in 2012-2013. Education Growth Advisors, 2013 Arizona State University (Knewton)

Other Institutions Carnegie Mellon University New York University University of New Hampshire Southern New Hampshire University American Public University Western Governors University

Adaptive System Domain Model Learner Model Adaptation Model

A conceptual map of the course or domain Course elements are assigned to nodes Could be videos, articles, assignments, quizzes Edges are hierarchical relationships Linear or nonlinear With prerequisites defined Nesbit, 2006; Chen, 2008; Graf & Ives, 2010 Domain Model -

Created and arranged either by the provider or teachers Some adaptive systems use pre-authored content Some allow for teacher authoring and arrangement Multiple conventions for domain representation and navigation Zooming, panning, filtering Karampiperis & Sampson, 2005; Magnisalis, 2011; Chung & Kim, 2012 Domain Model - Method by

Trade off of teacher autonomy and effectiveness of system Issues of consistency and sufficiency of metadata Ability for students to view their own position in the domain Risks of cognitive overload and frustration Bargel et al., 2012; DiBitonto et al., 2013 Domain Model - Risks

Reimann, 2013

Falmagne, 2011

Domain Model - Systems

Model of each learner s current knowledge state Either an overlay model or a stereotype model Overlay - compared to the overall domain model Stereotype - clustered with similar learner models Stereotype model is based on assumptions about student similarity Nitchot et al., 2010; Knauf et al., 2010; KlašnjaMilićević et al., 2011 Learner Model -

Data is collected either: Statically or dynamically Explicitly or implicitly Static data Cognitive characteristics, background knowledge Non-cognitive - topic preference, learning goals Explicit data collection methods Collected via pre-test, survey, feedback prompts Karampiperis & Sampson, 2005; Chen, 2008; Klašnja-Milićević et al., 2011 Learner Model - Method

Dynamic data Knowledge state Learning style (may be static or inferred dynamically) Time spent on course element or LMS Clickstream data Assessment scores Student feedback (prompted explicitly) Implicit data collection methods Automatically collected through interactions with the system. Moreno-Ger et al., 2007; Akbulut & Cardak, 2012 Learner Model - Method

Risks for explicit data collection Non-completion or inaccuracy Interrupts learning process Risks for implicit data collection Data provenance Accuracy of inferences made with data Data privacy issues Klašnja-Milićević et al., 2011; Magnisalis et al., 2011; Lo et al., 2012 Learner Model - Risks

Klašnja-Milićević, 2011

Chen, 2005

Learner Model - Systems

Unit of Adaptivity - Course element being adapted: Difficulty Content media Sequence (micro or macro) Pace Method of Adaptation Variety of algorithm types used for different purposes Direct or indirect presentation of adaptation Mitrovic & Martin, 2004; Kumar, 2006; Ullrich et al., 2009; Sosnovsky, 2010 Adaptation Model -

Bayesian network tracing - clustering students into groups Hidden Markov models - predicting likelihood of learner success Genetic algorithms Refine models and construct optimal learning paths from pre-tests Neural networks Pattern recognition updated with input (eg: inferring learning styles from interactions) Brusilovsky, 2001; Chen, 2008; Magnisalis et al., 2011 Adaptation Model - Method

Direct Adaptation Students are given a visible recommendation for the next course element Micro-level - problem feedback, explanations, links Macro-level - learning path presented as optional Indirect Adaptation Link hiding Learning path presented as only option Hauger & Köck, 2007; Magnisalis et al., 2011; Akbulut & Cardak, 2012 Adaptation Model - Method

Risks to student agency if the system controls too much Feelings of paternalism Systems with learner choice Lack of student ability to choose optimal path Design of adaptation engine influences the learning process Bargel et al., 2012; Akbulut & Cardak, 2012; Kirschner and Mierrenbeer, 2013 Adaptation Model - Risks

www.smartsparrow.com

www.knewton.com

Adaptation Model - Systems

- Technical Must be embedded in a platform Either adaptive learning platform Integrated with LMS Used in a digital textbook Need adequate bandwidth, available hardware, technical support for set-up and maintenance

- Pedagogical Choose an adaptive provider that fits your goals Consider pedagogical goals and mode of instruction How much teacher agency involved in authoring the course? Need faculty and student buy-in Overcoming cultural resistance to new models of learning Need departmental or institutional buy-in Freedom to experiment Competency-based credit

- Pedagogical Mode of instruction Online course Blended, face-to-face Teacher goals Address differences in background knowledge Appeal to different learning styles, goals

- Pedagogical How does whole-class instruction happen when students are not working on the same content at the same time? Challenges for peer learning, mentoring, collaborative groups, project-based learning Could be an opportunity for changing how we view those experiences

- Research Opportunities Improving student learning outcomes, retention, engagement Informing curricular development Connections between cognitive factors and performance (ie: self-regulation, motivation)

- Research Challenges Data acquisition - greater volume, variety, and velocity of data than many teachers or researchers may be equipped to deal with Data privacy - involvement of 3rd party adaptive providers raises questions about privacy and security of data

Thank you! Questions? Michael Madaio mmadaio@gatech.edu @mmadaio