The skinny on big data in education: Learning analytics simplified

Transcription

1 The skinny on big data in education: Learning analytics simplified By Jacqueleen A. Reyes, Nova Southeastern University Abstract This paper examines the current state of learning analytics (LA), its stakeholders and the benefits and challenges these stakeholders face. LA is a field of research that involves the gathering, analyzing and reporting of data related to learners and their environments with the purpose of optimizing the learning experience. Stakeholders in LA are students, educators, researchers, institutions, and government agencies. The way in which analytics information flows from students to other stakeholders involves a hierarchy, where all stakeholders are able to provide input and offer recommendations to enrich the learning process for the student. Challenges faced by stakeholders include the movement of traditional analytics to learner-centered analytics, working with datasets across various settings, addressing issues with technology and resolving ethical concerns. Despite these challenges, research points to solutions that will allow LA to transform teaching and learning. Keywords: big data, learning analytics, trends in education Introduction B ig data large and complex datasets collected from digital and conventional sources that are not easily managed by traditional applications or processes have transformed business industries affecting millions of organizations and individuals (Manyika et al., 2011). This data comes from sources such as the New York Stock Exchange, Facebook and Ancestry.com and can be used to predict consumer behavior, improve products and services and make better-informed business decisions (White, 2012; Manyika et al., 2011). From this data, businesses are able to generate advertising, recommend media such as music and movies and connect people within social networks. With the current shift in educational settings to blended and online learning and the introduction of learning management systems such as Moodle and Blackboard (Table 1), it is no surprise big data has found its place in education and is predicted to be extensively implemented in institutions of higher education in two to three years (Johnson et al., 2013). Derived from business intelligence and data mining, the Volume 59, Number 2 TechTrends March/April

2 Table 1. Examples of Types of Learning Analytics Resources Moodle Blackboard Analytics Overview.aspx GISMO SNAPP Meerkat-ED SunGard Assessment and Curriculum Management Desire2Learn Pittsburgh Science of Learning Center DataShop Mulce Mulce/ LinkedEducation.org Open-source learning platform Packaged self-service analytics applications Interactive tracking system built for Moodle that displays data through a graphical interface Web tool that provides social network analysis Web tool that analyzes participants and their interactions in discussion forums Learning performance solution for assessment management and analysis and curriculum management Integrated learning platform that addresses challenges with engagement, retention, and learning outcomes Data repository and analysis service that provides access to intelligent tutoring systems datasets Research project that shares datasets of learners online interactions Open platform that promotes sharing of educational data and resources process of gathering, analyzing and reporting educational big data is referred to as learning analytics (LA); it is an emerging field of research that can provide teachers, students and other stakeholders insight into the learning process (Buckingham Shum & Ferguson, 2012; Clarke & Nelson, 2013). The most widely acknowledged definition of LA is the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environment in which it occurs (SoLAR, n.d., para. 3). When learners interact in online academic environments such as libraries and labs and use web tools within these settings, like SNAPP or Meerkat-ED (Table 1) tools that analyze discussion in online forums they leave a trail of breadcrumbs and contribute to datasets (Friesen, 2013). While there are numerous datasets of learner information available for the field of education, there is still a need for improvement in the process of measuring, collecting, analyzing, reporting, and sharing data across institutions (Verbert, Manouselis, Hendrik, & Duval, 2012). Technological advancements and improved access to datasets have transformed the way educators use national datasets. Innovations for accessing, storing and analyzing datasets include data warehouses, the cloud, and sophisticated computer systems that calculate data and detect motivational patterns (Wagner & Ice, 2012). The acceleration of LA began as applications emerged in the form of learning performance solutions, like SunGard and Desire2Learn (Table 1), and learning management systems where data specific to the school or university could be collected (Brown, 2011). Prior to this, systems could not be purchased and instead had to be built from scratch, which often took several years to design. New technology also allowed practitioners and institutions to access, collaborate and contribute their own data in an effort to build single platforms with the capacity for sharing multiple datasets. Several of these projects that have been used as community resources to aid in understanding learner interactions and performance include: the Pittsburgh Science of Learning Center DataShop (Table 1), which provides access to intelligent 76 TechTrends March/April 2015 Volume 59, Number 2

3 tutoring systems datasets; Mulce (Table 1), a platform for sharing datasets of learners online interactions; and LinkedEducation.org (Table 1), which provides access to datasets that can be discussed by researchers directly on the site (Verbert et al., 2012). Both learning management systems and platforms with multiple datasets available for research share a key element of LA called visualization, a way in which analysis results are displayed so they are easily understood by decision makers (Brown, 2012). Visualization is made possible through a user interface or dashboard that displays charts or graphs and presents users with customized views of the data. One example of visualization can be found in GISMO (Table 1), a tracking system that works within Moodle, which takes data and displays it as a graphical interface that instructors can interact with (Romero & Ventura, 2013). With LA, stakeholders will have easy, visualized access to massive amounts of digital data left behind from learners about experiences in various online systems in the same way that the business intelligence market analyzes consumer data today. When data are appropriately leveraged through a validated framework, educators will be able to draw from multiple datasets simultaneously to detect patterns that lead to informed decision-making and strategic actions in a way that was not possible before (Johnson et al., 2013). Ultimately, LA has the capability of driving improvement in a student s learning process and procuring academic success. Stakeholders and Benefits With exceptional amounts of digital data about the interest and activities of learners becoming more accessible, there is an opportunity to discover the best ways to achieve significant learning results (Buckingham Shum & Ferguson, 2012). This process of discovery involves several stakeholders; to understand how these stakeholders interact with the flow of information, one might consider them in a hierarchical model that resembles the structure of traditional education (Greller & Drachsler, 2012). At the top, there is the government, which includes policy makers and agencies involved in educational affairs. These decision makers can use big data collected across institutions to evaluate education on national and regional scales. While the data analysis process may not change significantly for these stakeholders, they will have access to new types of data, such as characteristics of learner motivation that may aid in making decisions for school systems. Next in the hierarchy are institutions and then teachers; at the foundation, below teachers, are students. When data are analyzed at the student level, it can inform all stakeholders from one level to the next. For instance, with LA, a learner s progress can be measured at any stage and during any activity in a course, an advantage over traditional methods of identifying students who are at risk of failure (Brown, 2011). Where a traditional course evaluation may offer some insight into a learner s motivations and opinions once a course is over, LA allows for the understanding of how students are using content, interacting, and participating in a course as it is occurring so that early intervention can be done to aid in course completion. Teachers might collect this type of analytics information through a learning management system. This data could enable them to identify knowledge gaps, which might lead to positive intervention in student learning, changes to the design of curriculum or modification of teaching strategies. Institutions, in turn, could analyze the data from both students and teachers to aid in the development of new policies, reorganize resources or design professional development programs for educators. Included in the flow of information, but not directly linked in the hierarchy, are researchers. Researchers are charged with the duty of validating and reporting research results so that best practices are shared with stakeholders at every level. For example, researchers might compare data mining methods to determine and recommend what technique is most appropriate for uncovering or addressing a specific issue at a particular type of institution (Romero & Ventura, 2013). They should also gather data from stakeholders in order to understand how different roles work together in the decision making process. In this way, researchers will be better suited to evaluate or initiate processes and services that aid in making appropriate decisions for the target audience and improve the learning experience for students (Greller & Drachsler, 2012). Overall, LA can be used by all involved in educational decision-making in a variety of ways to meet the objective of improving learning outcomes. Not only can LA be used to predict a learner s performance through predictive analytics techniques, it can recognize patterns through sophisticated calculations of data that result in a suggestion of learning resources relevant to the learner s needs and a personalized learning experience that can be adapted in real time (Siemens, 2012; Verbert et al., 2012). LA can also increase learner Volume 59, Number 2 TechTrends March/April

4 awareness, as educators share and discuss data with the learner and offer the ability for reflection on the learning process. Furthermore, LA can enhance social learning, identify undesirable behaviors in learners, and allow for the detection of affects such as frustration, confusion or boredom. Teachers benefit from LA because they are able to monitor their courses to identify knowledge gaps as they occur with students and then address those gaps immediately. For institutional entities, LA provides a way to detect and address issues with retention of students, monitor graduation rates, and evaluate and improve courses (Greller & Drachsler, 2012). As stakeholders in the field of education continue to test LA in various environments, their choices and decisions will inform research, producing a stronger literature base that will eventually lead to a more solid framework for understanding and using LA. Challenges Despite the potential of LA, there has been hesitation and skepticism due to the challenges and unanswered questions that must be addressed in order for its use and implementation to be effective in achieving desired learning outcomes. Issues faced by stakeholders include the movement of traditional analytics to learner-centered analytics, working with datasets across various settings, addressing issues with technology and resolving ethical concerns. To attain full adoption of LA, there must be a distinct separation from statistical or technical activities associated with traditional analytics and data mining (Siemens, 2012). Educators should understand that LA goes beyond these activities to include the aspect of human judgment: making sense of information, coming to decisions based on data, and taking action. In order for LA to adequately address the needs of learners, the focus of data must be on learner perspectives (Ferguson, 2012). When views such as a student s enjoyment, confidence, motivation, satisfaction and achievement of career-related goals come into play, it widens the range of criteria to determine success. Educators should also be aware that this shift may mean they may need to adjust their teaching methods to accommodate data analysis results that paint a different picture than their past interpretations of traditional data. Much of the process of analytics should also be understandable by the learners so they are empowered and can provide input into refining their learning (Clarke & Nelson, 2013; Ferguson, 2012). In this sense, educators and institutions can better inform instructional design and pedagogy while enabling students to become more aware of their own learning behaviors (Ferguson, 2012). Since data can be collected from a variety of sources such as advising, assessment scores, social media profiles, and library use researchers will be faced with the challenge of developing methods for working with data sets across environments so they can understand and solve the problems learners encounter in settings outside learning management and student information systems (Siemens, 2012). A framework that combines accessibility to multiple datasets with analytics practices and that visually presents information to the end user is ideal so that institutions can share data for maximum impact on the learner s experience (Clarke & Nelson, 2013; Greller & Drachsler, 2012; Siemens, 2012). Verbert et al. (2012) suggested such a framework should be able to achieve the following objectives: predict learner performance, suggest learning resources relevant to the learner, increase reflection and awareness, enhance social learning, detect undesirable behaviors, and detect learner affects. Several frameworks have been developed by researchers, but many have not gone beyond the prototype phase, particularly because new technology emerges that makes prototypes obsoleter (Verbert et al., 2012; Wagner & Ice, 2012). Thus, there is not a single framework that is widely accepted and used for LA. Siemens (2012) also commented that there is a gap between research and practice in the sharing of LA information, tools, and datasets. With increased communication between software vendors, researchers, practitioners, and end users, awareness of the needs of stakeholders will be heightened, thereby increasing the possibility of achieving an ideal framework. There are also technical concerns with using data from multiple sources. If datasets are not in the same format, it may be difficult for one system to read from another (Greller & Drachsler, 2012). Datasets may also contain erroneous information or be incomplete. For example, when an instructor wishes to view content from a learner perspective in a learning management system, he or she may create a fictitious student profile that will add to the data collected from learners if the profile is not removed from the system. Developers and users of datasets must therefore be aware that data should be reviewed and updated on a periodic basis to be used effectively. 78 TechTrends March/April 2015 Volume 59, Number 2

5 LA also comes with new ethical issues that must be addressed. Evolving technology, such as location tracking and biometrics, allows for a variety of data to be collected that goes beyond just academic performance data. If learners feel their privacy is being invaded, they may be reluctant to allow their data to be used for research and analysis. In some instances, it is not clear who the data belong to; data might belong to an individual, an institution, or an outside vendor who owns the tool for data collection (Greller & Drachsler, 2012). Unlike traditional methods of accessing data in a research setting, there is currently no definitive framework for researchers to apply when obtaining consent to use data or for learners to have a record of analytics removed, nor are there any established guidelines for the anonymity of data (Ferguson, 2012; Greller & Drachsler, 2012). For these reasons, ethical guidelines are required to ensure stewardship and ownership of data are clearly defined and issues of privacy are taken into consideration so that data are protected from abuse. Conclusion Despite the challenges the field of education faces in implementing LA, research points to solutions that will enable LA to transform teaching and learning in the near future. The expectation is there will be an increase in the development of personalized learning experiences, the visualization of learner interactions, the push for social learning, and improved early-warning systems. Overall, stakeholders will soon be able to take ownership in the progression of educational processes by utilizing information about student success factors, the allocation of resources and effectiveness of teaching and institutional programs. These improvements, in turn, will allow for real accountability and efficiency, more accurate measurement of the quality of learning and the raising of completion and retention rates. Jacqueleen A. Reyes is a doctoral student at Nova Southeastern University, North Miami Beach, FL. Direct correspondence regarding this article to her via JL1420@nova.edu. References Brown, M. (2011). Learning analytics: The coming third wave. EDUCAUSE Learning Initiative Brief, 1-4. Brown, M. (2012). Learning analytics: Moving from concept to practice. Louisville, CO: EDUCAUSE Learning Initiative. Retrieved October 10, 2013, from Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics [Electronic version]. Educational Technology & Society, 15(3), Clarke, J., & Nelson, K. (2013). Perspectives on learning analytics: Issues and challenges. Observations from Shane Dawson and Phil Long [Electronic version]. The International Journal of the First Year in Higher Education, 4(1), 1-8. Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges [Electronic version]. International Journal of Technology Enhanced Learning, 4(5/6), Friesen, N. (2013). Learning Analytics: Readiness and Rewards/L analyse de l apprentissage: état de préparation et récompenses. Canadian Journal of Learning and Technology/La revue canadienne de l apprentissage et de la technologie, 39(4), Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics [Electronic version]. Educational Technology & Society, 15(3), Johnson, L., Adams, S., Cummins, M., Estrada, V., Freeman, A., & Ludgate, H. (2013). The NMC Horizon Report: 2013 Higher Education Edition. Austin, TX: The New Media Consortium. Retrieved October 2, 2013, from Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., & Byers, A. H. (2011). Big data: The next frontier for innovation, competition, and productivity. Retrieved October 2, 2013, from com/insights/business_technology/big_data_the_next_ frontier_for_innovation Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), Siemens, G. (2012, April). Learning analytics: envisioning a research discipline and a domain of practice. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 4-8). ACM. SoLAR, (n.d.). About. Society for learning analytics research. Retrieved October 1, 2013, from solaresearch.org/mission/about/ Verbert, K., Manouselis, N., Drachsler, H., & Duval, E. (2012). Dataset-driven research to support learning and knowledge analytics [Electronic version]. Educational Technology & Society, 15(3), Wagner, E., & Ice, P. (2012). Data changes everything: Delivering on the promise of learninganalytics in higher education. Educause Review, 47(4), White, T. (2012). Hadoop: The definitive guide. Sebastopol, CA: O Reilly Media. Volume 59, Number 2 TechTrends March/April

6 80 TechTrends March/April 2015 Volume 59, Number 2

7 Copyright of TechTrends: Linking Research & Practice to Improve Learning is the property of Springer Science & Business Media B.V. and its content may not be copied or ed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or articles for individual use.