Learning Analytics and Learning Tribes

Transcription

1 Learning Analytics and Learning Tribes Kari Lehtonen 1, Ilkka Autio 2 1 Helsinki Metropolia University of Applied Sciences, Finland 2 TribaLearning, Helsinki, Finland Abstract Traditional mass education is characterized by relatively static learning materials and environments, a slow feedback cycle, and indifference to the variety of learning styles. Improving the learning process is slow and corrective interventions often come too late. Also, we do not take full advantage of the rich potential of individual learners. The quality of learning outcomes is less than ideal and there are too many failures, especially in difficult subjects such as mathematics. We offer students a new interactive learning environment, provided by TribaLearning, which the students can personalize themselves according to their interests and tastes. The TribaLearning application provides teachers with a tool for predictive analysis of factors influencing learning results as well as ways to improve them at personal and group levels. The TribaLearning tool gathers data that is used to build mathematical models to analyze and predict factors with positive and negative impacts in learning. The results indicate problem areas in learning processes and quantifies their importance from learning results perspective. Learners are divided into 'learning tribes', that is, groups defined by a shared learning behaviour. Presenting results in an understandable graphical way helps teachers to support learning and intervene in the case of predicted learning problems early enough. Personalized Learning strive for enhancing education in the future. Personalised Learning (also called Adaptive Learning) is one of the most interesting visions for the future in the field of education. The topic has been taken up on multiple discussion platforms around the world and it is something that teachers in different parts of the world are interested in. Personalised Learning will be a norm in the future and currently different stakeholders are researching different ways to produce something that would fulfil the promise of Personalised Learning. New technologies are emerging and the current trend of digitalization of education can be seen as a base for the future developments that will ultimately bring forth a disruptive change in the field of education. Learning Analytics is one of the areas investigated by researchers and innovative teachers in their attempts to bring this change to education to the practitioner level. According to Horizon Report (Johnson et al., 2014), the following is the best way to describe what Learning Analytics can be used for: Learning analytics research uses data analysis to inform decisions made on every tier of the education system, leveraging student data to deliver personalized learning, enable adaptive pedagogies and practices, and identify learning issues in time for them to be solved. Adaptive learning data is already providing insights about student interactions with online texts and courseware. Teachers willing to test new technologies on their courses and in classrooms are of key value for the development of the technologies tied to Personalised Learning.

2 2 Cooperation models between education technology companies, teachers and educational institutions are the base for piloting and bringing in new innovations available for users, namely the students. What does the future look like and what are the challenges? The United States Department of Education (2012) describes Personalised Learning as follows: Education is getting very close to a time when personalization will become commonplace in learning. The instructor is responsible for supporting student learning, but her role has changed to one of designing, orchestrating, and supporting learning experiences rather than telling. Rather than requiring all students to listen to the same lectures and complete the same homework in the same sequence and at the same pace, the instructor points students toward a rich set of resources, some of which are online, and some of which are provided within classrooms and laboratories. Thus, students learn the required material by building and following their own learning maps. There are still challenges that need to be addressed however. Challenges can be found in a lack of teacher to student engagement, a lack of hardware (computers) in schools, a lack of financial resources and so on. According to the research paper Teachers Know Best (2014), the technologies brought to market do not cater to the needs of the teachers or the students particularly well. Either they are too simple, performing simple tasks, or they are too complicated, becoming too difficult to use. The needs of the teachers are quite simple when it comes to new educational technologies. Teachers identified six instructional purposes for which digital tools are useful: 1. Delivering instructions directly to students, 2. Diagnosing student learning needs, 3. Varying the delivery method of instruction, 4. Tailoring the learning experience to meet individual student needs, 5. Supporting student collaboration and providing interactive experiences, 6. Fostering independent practice of specific skills. (Teachers Know Best, 2014). Many companies are trying, as noted above, to bring something new to the educational technology market, but the products that are launched are somehow lacking important elements. This means that, while the products are good as technology, they are not good enough to serve the purpose of education, especially for the teachers and students. TribaLearning and learning theories All Learning Analytics providers have data gathering and data presentation as the base of their system. They state that they are providing data that will help the teachers and students to understand their knowledge level which, in turn, will give them systemaided feedback. These learning analytics systems give the students either recommendations on how to proceed with their studies or they intervene when the system detects that the student is lacking in knowledge. In the latter case, in order to proceed to step M, the system tells the student that he or she must return to step N. 2

3 3 TribaLearning, established in 2013, also tries to bring 21 st Century Skills into the field of Learning Analytics, namely cooperation, communication, creativity, innovation and problem solving. Therefore the company develops new algorithms from pedagogical theories that are based on social activity and these algorithms are incorporated into the personal learning environment. The analytics will be developed to also include motivation, emotions, collaborative knowledge building, stress and anxiety during the learning process. (Litmanen, 2012). The Triba Learning Environment consists of the student user interface (UI) and the analytics interface for the teacher. The student interface is arranged topically as boards that include articles, namely documents and web resources. The student can include any board or article in their collection of favourites. The student can highlight and make comments on the articles, down to the level of a phrase in the article. Analytics - Algorithms and Models In its present state the Triba analytics tool gathers data on how often and when the students read the articles, how much they comment or highlight the articles and how much they make private notes. Results can be analyzed and visualized in several ways. Two analytics views: Timeline and Tribes Personal learning analytics: 1. Time series: Hours studied (per student, per day or hour), article views, commenting activity. See the section An example: Metropolia, Learning orientations and moods. Students learning orientations are estimated using a 5-factor model developed at the University of Helsinki. Individual learning orientations affect moods and feelings of satisfaction, which in turn are known to affect learning outcomes. For example, social learners may thrive in groupwork settings, elevating their mood and learning outcomes. Target-oriented learners or individually oriented learners are often independent and they organize their study track themselves. Cookbook learners need clear 3

4 4 instructions and are primarily interested in just passing the course. (Litmanen, 2012). Teacher analytics supporting interventions and course design: 1. Time series: Hours studied (per student, per day or hour), article views, commenting activity. See the section An example: Metropolia, Cluster analysis, grouping students into tribes sharing similar characteristics, for example similar time series, tastes (favoured articles), and interactions (commenting and replying binds the participants together; outliers may be isolated socially). 3. Learning orientations and moods. See above (personal learning analytics). Knowing the students psychological orientations may help in designing the appropriate studying environment, for example social learners may be satisfied in groupwork settings while students with high individual orientation may be less satisfied by this. Peer-to-peer tutoring: Student comments can be tagged as open questions. Teachers and other students can use topic based filtering to find open questions related to topics they are comfortable with. The student who posted the original question gets to decide whether to close the question (an answer was satisfactory) or keep it open. Interactions that result in questions being closed generate topic-specific expertise points for those that provided the satisfactory answers. In the future, estimated topic-specific expertise will be used in clustering the students, so that, amongst other steps, the potential tutors (assistants) can be found in larger groupwork settings. Personalised recommendations Topic (via tags) and article level difficulty (challenge) ratings from students enable personalised recommendations (within topic). For example, if a student finds an article about linear algebra very difficult compared to his peers, the system can recommend related books that his peers rated as easier. Depending on the student s motivation, this may help him in catching up with his peers. An example: Metropolia, 10 th March to 27 th April The Triba learning environment was used during a Metropolia Industrial Management calculus course between March 10 th and May 8 th in The students (n = 40) could attend lectures and laboratory or practice sessions, or they had the option to study at home. All study material and instructions (roughly 50 documents) can be accessed in the Triba learning environment. On the average, 25 students attended the lectures and laboratory or practice sessions regularly. They relied mainly on classroom notes and 4

5 5 only partially on Triba documents. The other students used the documents in Triba more (excluding drop-outs). Time spent reading (in hours). Daily sums were taken over all 40 students and all articles during the period from to 27.4.: Mon Tue Wed Thu Fri Sat Sun * Some observations: Based on measured reading times, Triba s usage appears to have declined over time. Possible reasons could include usability problems, reading and exercising offline and the Easter holiday around the week 14 th to 20 th April. Most activity tends to occur on Mondays and Thursdays when students have dedicated laboratory practice hours in their weekly schedule. Sunday 23 rd March (*) was the deadline for the first course project. The second project workload was distributed more evenly across the week 31 st March to 6 th April. As the final test and the third project deadline is 8 th May, daily sums will probably become higher during the two weeks that are not included in the table. * A more detailed view: an hourly breakdown of Sun 23.3.: * * The most popular hour, 19-20, had about 4 active readers contributing to the sum of 4.68 hours. Suggestions for further analysis: In the future, identify the students who tend to stay up late the night before a lecture / an exam / an exercise session. Try to encourage healthier studying habits. Generate a separate table or figure for each course chapter, that is, they do not take sums over all articles to see whether students progress as planned. For example, if the intended schedule is two chapters per week, the data could show students abandoning earlier chapters and moving on. The analysis above examined reading times only. The same analysis can be done with article views and commenting activity. Using the official UI, individual students are always compared to peer averages. 5

6 6 Applications Course design: Perform a similar analysis for several courses in parallel. Try to design courses so that students workload is distributed evenly over weekdays. Course design 2: Compare several courses to rank them based on how time consuming they are compared to the credits earned by that course. Interventions: Once usability / utility problems are solved, declining student work hours may be a sign of student disengagement interviewing a subset of students based on these statistics may be more economical than interviewing them all. Interventions 2: Examining individuals odd working hours (for example working late at night on weekdays and an emphasis on weekends) might indicate planning problems (such as having a job instead of being a full time student). Administrators are able to look at detailed data across different classes to examine progress for all students attending a given institute to easily distinguish factors that promote success. Using the data, administrators can set, implement, and adapt their policies and programs to improve learning results. References Litmanen, T. et al. (2012) Capturing teacher students emotional experiences in context: does inquiry-based learning make a difference? Instructional Science, 40: Johnson, L., Adams Becker, S., Estrada, V., Freeman, A. (2014). NMC Horizon Report: 2014 Higher Education Edition. Austin, Texas: The New Media Consortium. Teachers Know Best: What Educators Want from Digital Instructional Tools. (2014). Bill & Melinda Gates foundation. U.S. Department of Education, Office of Educational Technology. (2012). Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief. Washington, D.C. 6