Analytics: An exploration of the nomenclature in the student experience. Rhonda Leece Student Administration and Services University of New England Abstract The student journey through higher education, from first point of enquiry to graduation, is signposted by significant events and interactions which inform the students satisfaction, engagement and perseverance with their studies. These signposts are well understood through the shared language of student support. Extraordinary technological advances have resulted in traditional physical interactions being replaced in many instances by online learning interactions with the consequential ability to efficiently monitor, analyse and deploy student related data. This capacity has its own language which includes terms such as business intelligence, predictive modelling, learning analytics and action analytics. In an attempt to move towards a shared understanding of these terms, this paper suggests that it is useful to determine what you want to analyse, why you want to analyse it, how you are going to analyse and what action will result from the analysis. These questions form the basis of the Nuts & Bolts discussion. 1
The student journey through higher education, from first point of enquiry to graduation, is signposted by significant events and interactions which inform the students satisfaction, engagement and perseverance with their studies. These signposts are well understood by practitioners through the shared language of student support. The massification and commercialisation of higher education, combined with extraordinary technological advances, have seen the traditional classroom being replaced in many instances by online learning spaces and the reliance on technology to reshape teaching and learning is widespread. Increasingly, mixed modes of delivery of education comes with the additional challenge of how best to identify and then respond to the teaching and non-teaching support needs of students with whom we may never have an in-person interaction. University staff members are dis-located from their students, students are dis-located from their peers and the learning community may be dispersed across local and international locations. The increased use of technology however, presents opportunities to create non-physical neighbourhoods or communities of practice and one byproduct of the increased use of emerging technological solutions is the ability to monitor, analyse and deploy data collected during the online interactions with students. A google scholar search for articles using key terms related to data analytics and higher education delivered over 18,000 articles published since 2000. These articles illustrate an emerging language which combines terms from both the business and education sectors and includes concepts such as business intelligence, data mining, predictive modelling, learning analytics and action analytics to mention a few (van Barneveld, Arnold and Campbell 2012, Goldstein and Katz 2005, Gabriel, Lau, Gosper, Marrone-Burgoa, Myles, Warren and Wilson, 2012). Such terms signal a fundamental attitudinal and transactional shift in thinking within higher education. Higher education providers have always used qualitative and quantitative methods to investigate trends, to quantify achievements and to improve teaching and service delivery often through time consuming and manual data collection and analyses. The increased use of student management systems, learning management systems, customer relationship management systems, on-line portals and social media tools have resulted in access to more data than ever before (Baer and Campbell 20102 p53). It is both the volume and velocity of the new forms of data analyses that are being advocated as valuable in supporting evidence-based decision making in teaching and learning areas Van Barneveld et al (2012) in their review of analytics in higher education revealed a language which uses similar terms with different conceptual or functional definitions, as well as different terms with similar conceptual or functional definitions. Watson s perspective (2011) suggests that the changed terminology around analytics in general is not because the previous terminology was wrong, more likely, vendors, consultants, writers, and others who offer decision support products and services saw the opportunity to have people and companies take a fresh look at their offerings by promoting them as new and different. This paper explores the nomenclature applied to analytics in higher education. Analytics is most often used as an umbrella term (Watson 2011) to describe analysis of data using statistical and mathematical techniques (p.5). The definition of analytics as datadriven decision making (van Barnevld et al. 2012) refines the more general definition of the discovery and communication of meaningful patterns in data to describe, predict, and improve business performance (Wikipedia last accessed 8/2/13). Both these definitions are in common use in Australia. Many private sector organizations use business data sets to measure and 2
evaluate their return on investment and to determine how and where resources should be allocated to deliver the best financial outcome. In the higher education sector, data analysis to date has focused primarily on efficiencies in the way that student enrolments (and their associated financial load) are managed and the proposition that business analytics can improve fiscal management is reasonably well accepted. Current thinking suggests that analytics contribute to a socio-technical system (van Harmelen Workman 2012) where human decisionmaking and consequent actions should be aligned to optimise the benefits. The terms It is clear that the current language of analytics in higher education is derived from the use of Business Intelligence (BI) in the corporate sector and for many institutions the first foray into analytics has been through the introduction of BI systems and data warehouse functions. Wellman and Soares (2011) define BI as the use of quantitative measures of past financial performance to inform future planning and decision making (p.1) and this relies on the analysis of large datasets to understand and manage business effectiveness. The term Academic Analytics, evolved out of the BI principles and is most commonly applied in higher education to the administration of courses and units of study through monitoring processes such as admissions and prospective student enquiries and the management of students and financial load. Goldstein and Katz (2005) use the term to describe the intersection of technology, information, management culture, and the application of information to manage the academic enterprise (p.2). There is limited difference between the terms Academic Analytics and Business Intelligence and the term may be considered an attempt to contextualise the corporate term Business Intelligence within the academic arena. Academic Analytics are designed to benefit the institution, faculty, support staff and students by targeted and informed changes in the way education is delivered and to use information technology support for financial and operational decision making. The term Learning Analytics, is often used interchangeably with Academic Analytics but it is more specifically used to describe the use of data and models to predict student learning progress and performance. Interpretation of data produced by and gathered on behalf of students is analysed but their usefulness is highly dependent on their intended use and purpose (Gabriel et al. 2012). The data can provide a hindsight understanding of student performance, it can provide insight into why for example, particular groups of students do not succeed and it may provide some insight into future results, based on the available data. Predictive analytics attracts a high level of attention specifically because it implies an ability to predict what will happen with specific types of students. It uses large datasets, to determine relationships and patterns from which to draw reliable conclusions about past and future events. There is some discussion about the possible negative implications of academic analytics being used to develop predictive modeling of students at risk, to identify areas of need amongst specific student cohorts and the possible consequences of these value judgments and labelling. Gabriel et al (2012) identify the risk of reducing the subtle complexities of the learning experience, not to mention the learner, into predetermined statistical indicators of success (or failure). While the intentions of this type of analytic are to help universities support the success of students identified as being at risk of dropping out, to improve retention rates and to contribute to the financial viability of an institution, there is also a risk that the data may be 3
used in unidentified negative ways. Additionally, the misuse of generalised group characteristics may prove contrary to the social inclusion agenda being promoted within the higher education sector in Australia. These types of discussions appropriately raise the question of ethical use of data and this is an area that will require deeper investigation as the use of analytics evolves. Action Analytics also uses large and complex datasets and is focused on academic and administrative productivity and performance. The data is considered powerful, useful and immediate and if deployed in a timely manner, may provide the answer to the question of what to do with the data. Institutions are increasingly using the information they accumulate about their students to gain insights into big issues such as academic performance, student success, persistence and retention but irrespective of the type of analytics adopted, it will be the way in which the analysis is applied that will determine its value. The use of analytics, irrespective of the form it takes, can drive particular models of student support; discard those services for which there is no supporting evidence of use or efficacy in supporting student success; and create targeted and purposeful interventions based on the available metrics. The student experience can be positively impacted if institutions leverage the available data. The discussion Watson (2011) in a discussion about analytics suggests that to avoid confusion and to add clarity to the terminology, practitioners should distinguish between the different kinds of analytics being proposed. Figure 1 Defining learning analytics (Adapted from van Barneveld et al 2012; Oblinger 2012; Gabriel et al 2012) 4
Watson (2011) suggests that it is useful to determine what you want to analyse, why you want to analyse it, how you are going to analyse and what action will result from the analysis. The terms currently used in the higher education sector loosely fall into three categories along the hindsight-insight-foresight spectrum, distinguishing specific analytics types according to their capacity to provide an understanding of: what happened (hindsight); why it happened (insight); and what will happen (foresight). Lynch (referenced by Gabriel et al. 2012 p.4) suggests that there are three critical questions that should ideally be posed and answered before undertaking any learning analytics initiative: what do you want to know; why do you want to know it; and, what are you going to do when you find it out? These questions encourage practitioners to question their objectives in embracing analytics and the intended purpose or actions resulting from the analysis. This is a useful starting point for a naming convention of analytics in higher education. It is intended that the discussion today will go some way to establishing a shared understanding of the terms related to data analytics in the higher education sector. These questions, integrated into the figure above, form the basis of the Nuts & Bolts discussion. References Baer, L., Campbell, J.P. (2012). From Metrics to Analytics, Reporting to Action: Analytics Role in Changing the Learning Environment. Education and Information Technologies, edited by Diana Oblinger. Gabriel, J., Lau, G., Gosper, M., Marrone-Burgoa, M., Myles, M., Warren, V., Wilson, M. (2012) Learning and Teaching Analytics: the Current State of Play A review of the technological, pedagogical and ethical impact of learning analytics in Higher Education Institutions from http://learning-genome.science.mq.edu.au/static/images/litreview.pdf Goldstein, P. J., & Katz, R. N. (2005). Academic analytics: The uses of management information and technology in higher education. Educause. Oblinger, Diana (2012) Game Changers: Education and Information Technologies from http://www.educause.edu/research-publications/books/game-changers-education-andinformation-technologies van Barneveld, A., Arnold, K. E., & Campbell, J. P. (2012). Analytics in higher education: Establishing a common language. Educause Learning Initiative, 1, 1-11 van Harmelen, M. and Workman, D. (2012) Analytics for Learning and Teaching CETIS Analytics Series Vol.1, No.3, from http://www.jisc.ac.uk/ Watson, H. J. (2011).Business analytics insight: Hype or here to stay? Business Intelligence Journal, 16,4-8.Retrieved from Http://search.proquest.com.ezproxy.une.edu.au/docview/858361624?accountid=17227 Wellman, J.V. and Soares, L. (2011) Bringing Business Analytics to the College Campus. Using Fiscal Metrics to Steer Innovation in Post Secondary Educations. Centre fo American Progress. Retrieved from www.deltacoastproject.org. 5