Research on the Analytics Model Design of Online Learning Behavior

Transcription

1 Research on the Analytics Model Design of Online Learning Behavior Jun Xiao, Minjuan Wang, Lamei Wang, and Bingqian Jiang Abstract Drawing on current research about online, we constructed an analytics model, which consists of four major analytical stages: data collection, data organization, data analytics, and data application. The four stages form a continuous cycle during the implementation process. Meanwhile we identified several key factors of the analytics model for online, including the design of data collection index, data collection and organization mode, as well as key techniques of data analytics. The analytics model of online can support relevant systems, which are capable of collecting, analyzing, and extracting teaching data from mainstream platforms. Learning systems built on this model can also provide intelligent services for teachers and students in distant education. Keywords Knowledge and Technology, Adaptive and Learning Systems, analytics model, analytics system I. INTRODUCTION With the improvement of science and technology, online has significantly changed people s understanding towards education. In addition, the emergence of new technologies and ideas are propelling online forward. According to a survey report by the Sloan Consortium of the United States, many colleges and universities in the U.S. have made online an important part of their long-term development plan from 2010 onwards [1]. The trend is on a rise annually, and a majority of colleges and universities surveyed has realized the importance of online. In 2012, elite schools such as Stanford, Harvard and the Massachusetts Institute of Technology took the same action of raising a trend of MOOCs, which drew an extensive attention This paper is supported by Shu Guang award MOOCs design and empirical research oriented Shanghai lifelong (13SG56) from the Shanghai Municipal Education Commission and Shanghai Education Development Foundation. It is also supported by the 2014 Shanghai Education Scientific Research project The Study of online mode for Shanghai lifelong (A1403). Besides, thanks for the support of Science and Technology Commission of Shanghai Municipality research project Shanghai Engineering Research Centre of Open Distance Education (13DZ ). Jun Xiao is with the Shanghai Engineering Research Centre of Open Distance Education, Shanghai Open University, Shanghai China. (phone: (+86) ; fax: (+86) ; xiaoj@shtvu.edu.cn). Minjuan Wang is with the San Diego State University, San Diego, USA. ( mwang@mail.sdsu.edu). Lamei Wang is with the Shanghai Engineering Research Centre of Open Distance Education, Shanghai Open University, Shanghai China. ( wanglamei@shtvu.edu.cn). Bingqian Jiang is with the Department of Education Information Technology, East China Normal University, Shanghai China. ( @student.ecnu.edu.cn). from global universities, enterprises, the whole society and even individuals. An increasing number of people started to register courses on mainstream MOOCs platforms such as Coursera, edx, Udacity, which have different focusing areas. With an expansion of the scale of online education learners, resources and interactions, more and more people choose online. At present, online has become one of the main modes [2]. Therefore, analyzing and evaluating learners online can help promote and improve outcomes. In addition, it can help to improve teaching practice and optimize the development of a lifelong platform. An essential part in developing an online platform is to meet the demands of big data processing. II. LITERATURE REVIEW In recent years, analytics has aroused a widespread interest in the education industry. As an education data analysis technique, analytics, based on the big data, has become an indispensable part of the development of online. The biggest benefits can be pursued through the discovery and understanding of the data s hidden information, as well as an efficient utilization of researches (for example, teaching intervention, prediction) [3],[4]. This literature review reveals that current studies on online analytics has mainly focused on the following aspects: A. Description and feature analysis of online Analysis of online is an important pre-requisite for carrying out the design of online system and the development of online education resources. In order to improve the efficiency of online, to motivate learners, and to improve learners interests, it is necessary to have a clear understanding of the basic concept of online, and the characteristics and styles of online among individuals and groups [5][6]. In current research findings, quite a few researchers have carried out investigation on online among specific groups, and they also focused on the patterns, characteristics and frequencies of learners using online self-directed. Moreover, these studies analyzed the status quo and level of online learners and explored causes [6]. B. Research on the data model of online Experts in different disciplines have different opinions ISBN:

2 towards the models of analyzing online. For example, Siemens (2010) believes that analytics is composed of several stages, including data collection, data analysis, data prediction and data adjustment. He constructed a linear analytics model that includes these four elements. According to Elias (2011), analytics has three stages, namely data collection, information process and knowledge application, which form a continuous cycle. Learning analytics also has six activities, which are acquirement, selection, gathering, prediction, use, and optimization. Based on these components, she proposed an application model of analytics that is relatively more complicated. Greller and his colleagues (2012) built a theoretical model of analysis in terms of data sources, analytical modes, constraint condition, competitiveness and interested parties. In recent years, researchers have started to apply the analytics techniques to the online analysis of MOOCS. Summarizing theories and models from existing studies, analytics has core elements such as targets, objects, restraints, data resources, and processing methods, which reflect the internal and external conditions for carrying out analytics. The process of analytics consists of three stages: data collection, data processing, and data application and feedback, which form a continuous cycle during the implementation process [7][8]. C. Research on data collection techniques and system implementation of online Online learners data are acquired from Web blogs, network sniffing, questionnaires, platform database, and web data mining technique, mobile Agent intelligent agent technique, standard SCORM online technique, and electronic portfolio technique. For example, Hummel adopted a way of obtaining relevant data of learners from the database access records and server access logs of online systems.. Wu and his colleagues from Xi'an Jiaotong University created an algorithm using resolution function for attribute reduction, based on the rough set theory. They discovered that learners key feature dimension is merely 1/4 of its original dimension. Besides, learners key feature can be detected automatically, and the objective relation between personality characteristics and strategy can also be revealed. This algorithm can simplify the data analysis [9]. D. Research on the analysis techniques of online Online analytics is a newly-developing educational research, which adopts techniques including business Intelligence, network analysis, education data mining, academic analysis, etc. These intelligent techniques can be used to analyze and process massive data. Among which, data mining method has been applied in the online platform, which initiates the research on learners using analytics technique. Apart from the above common analysis techniques, the analysis technique of online has been taking in and integrating other techniques and methods, including analytic methods of social network, discourse, and content. Introduction of these new analytic methods has largely enriched the data processing approaches and strategies for analyzing online [10][11]. E. Evaluation research of online Researchers have also paid close attention to the evaluation research of online. For example, Xu and his colleagues (2003) have proposed an evaluation technique for the interaction of online education and environment. He believes that interaction is the core of online, while current online lacks feedback. Websites using the analysis method that has been applied to business websites, together with an analytics model, using mining technique, serve as the final evaluation methodology. Outtaj and his colleagues (2007) divided online learners into different types, and applied different evaluation standards to analyze the different types of online learners [12]. Although this research has touched on the fact that various aspects should be considered when evaluating a student, it only studied the interactions between learners, such as interactions among learners, interactions between learners and the platform. In conclusion of above research results, there are still some shortcomings in the analysis research of [13]: Firstly, an absence of the comprehensive collection and analytics of a multi-platform, multi- terminal data, as well as data of students dominant. Currently, research on the data collection of online is mainly dependent on a fixed platform, and data obtained tend to have an apparent mode, which lacks data collected from present multi-platform and multi-terminal, as well as that of learners dominant and physical signs. Therefore, it is relatively difficult to analyze and share via different platforms. Second, the index of analytics and evaluation index needs alignment. Online analytics is a comprehensive, dynamic and systemic process. Although there is a relatively in-depth research, in theory, on the existing index system of online, these theories cannot properly serve as the evaluation objectives of specific online analysis. The key problem lies in the fact that the index cannot be properly matched to the evaluation objectives. Third, there is a lack of smart feedback from the evaluation results of online. Although a majority of current online platforms provide evaluation functions, such as online homework and testing, the evaluation results are not ideal. Most of the information provided by existing researches is periodical or conclusive, which lacks the individualized resources for learners, or recommendations for their study. Fourth, a lack of in-depth analysis for big data. Due to the fact that not a technique is capable of extracting learners from a virtual environment, most of the current research discusses online only by observing and analyzing the backend database of a few web-based teaching platforms. And current researches on online appear to be a simple statistical description of data, ISBN:

3 which lack the in-depth mining of big data. III. MODEL DESIGN OF ONLINE LEARNING BEHAVIOR ANALYSIS Acting upon the aforementioned gaps, integrating the characteristics, method, and technique of online learners' analysis, and the objective function of analytics, we propose an online analysis model as shown in the Fig.1. The process of analysis will occur in four phrases: data collection, data organization, data analysis, and data application, which form a continuous cycle during the implementation process [14]. Identification of potential dropouts Behavior patterns Personalized recommendation Course optimization Decision support Platform optimization Evaluation of outcome Multiple terminals: PC, flat-panel TV OTT, smart phones, etc. Online platform data application Collection analysis Multiple devices: eye tracker, brain wave detector, etc. Physical, Dominant Organizati on Mapreduce algorithm, rough set theory, genetic algorithm, etc. Big data platform Fig. 1 Analysis model of online (1) collection stage: In this stage, based on the online feature of teachers and students separating from each other, a wide selection of data resources are included, which are data from different terminals (PC, tablet, TV OTT, smart phones, etc.) and a third party data platform. Techniques used are not only the common ways of obtaining virtual data, namely Web blogs, database, etc., instruments like eye tracker and brain wave detector are also adopted to get data of learners physical sign, all of which contribute to a comprehensive and objective data. This has set up a basis for the following in-depth analysis. (2) organization stage: It is unavoidable to have repetitive or invalid data among those obtained from collection stage, therefore the sorting and cleaning process is needed, and the final data will lead into the developed big data platform. (3) analysis stage: In this stage, certain algorithm and analysis model will be adopted to process the data from Big. And supports will be provided for all kinds of decisions and services needed in the next stage. (4) application stage: In this stage, a wide range of services and supports are provided, by means of the operation results from data analysis stage, including identifying potential dropouts, establishing learners patterns, providing personalized recommendation service for students, offering decision support for education administrators, offering support to the optimization of both platforms and resources, carrying out effectiveness evaluation for learners etc. IV. KEY FACTORS IN THE MODEL DESIGN OF ONLINE LEARNING BEHAVIOR ANALYSIS AND SOLUTIONS Several key factors need to be considered in the model design of online analysis, including data collection index design, modes of collecting and organizing data, as well as key techniques of data analysis. The analytic system of the above online analysis model is able to collect various kinds of learners al data in the digital process, and carry out a rapid analysis to all data, such as learners browsing habit, ways to open webs, in the process [15]. The system can also have a direct access to a range of third party platforms, and analyze on the basis of historic data. A. collection index design of online In the design of analysis index, we adopted the concept of Objectives and Key Results as the guiding ideology for the index design, i.e. applying the cycle of target decomposition quantification of key results evaluation of measures in the index system design. It has overcome the limitation caused by a low compatibility between the evaluation objective and the former index design method of using Key Performance Indication (KPI) as the core. Therefore, it is a more reasonable and scientific index design. Based on the current researches on online index, together with specific evaluation objectives of online analytics, we made the online data collection model [16]. This model caters to teachers and students with seven evaluation objectives, which is shown in Fig. 2. Identification of potential dropouts Decision support for administrators Course Learners patterns Interactive optimization of platform General Personalized recommendation Types of online Evaluation of outcome Optimization of course resources Target dimension analysis Fig. 2 collection model of online Ⅱ -level evaluation indexes incudes the above seven objective dimensions: identification of potential dropouts, learners patterns, personalized recommendation, evaluation of course resources, decision support for ISBN:

4 administrators, optimization of platform, and evaluation of outcome. There may be intersections, for example indexes that reflect learners pattern, style and path can also be used as specific indexes that support personalized recommendation service. Therefore, according to the basic principles of activity theory and the principle of al science management, we have applied three dimensions, learners general al habit, interactive and course, to collect online learners data, on the basis of the above seven objective dimensions. Moreover, after the comprehensive analysis of current evaluation indexes, classical index group is formed, by means of a preliminary selection of those frequently used indexes, adopting statistical method for frequency. On the basis of that, the most representative and non-repetitive indexes have been selected by applying clustering analysis method in order to analyze the chosen evaluation indexes [17]. Furthermore, adjusting and revising index sets for several rounds, and formulating a detailed second-level index, as shown in Table 1. Table 1 collection index of learners I-level Index General Behavioral habit Interactive Learning Behavior Course Learning Behavior Ⅱ-level Index(encoding) Learning Path(1) Learning Properties(2) Ways of Collecting Information(3) Ways of Processing Information(4) Ways of Distributing Information(5) Concentration Level in Learning(6) Involvement in online FAQ (7) Participation in BBS discussion (8) Usual interactive ways(9) Usage rate of different interactive tools(10) Learning progress of the course(11) Hit rate of different types of courses (12) Hit rate of different theme courses (13) Meaning Learning order Length, time and equipment of Quantity and quality of making notes Number of posts and logs, etc. Degree of concentration Effective times for questioning or answering Effective times for questioning or answering Synchronous or asynchronous Learning progress of individuals and groups Hit rate of textual course, audio course, video course, image course, etc. Teaching contents, course tests, course assignments, etc. Learning outcome(14) Study on the academic achievements, etc. Finally, encoding these Ⅱ -level Indexes, and making an independent assortment of them to have correspondence with the above seven evaluation objectives (mode sets), as shown in Table 2. Analysis Objectives Index Identification of potential dropouts Learners patterns Personalized recommendation Optimization of course resources Decision support for administrators Optimization of platform Evaluation of outcome B. collection method of online Analytics data of learners al process include learners personal information, time records, distribution of areas, academic achievements recording, and records about other objective. These data can be collected, quantified and stored in the al database. It can be checked up in many ways, such as from the database backup of production system, or from the real-time/ semi-real-time recording logs, or on a regular or periodical basis, or from an event-driven way, or from an initiative order at the frontend, or pushing data automatically to the analysis page. As for the data obtained, system can establish a correlation between them, so as to carry out an in-depth correlation analysis, and provide a visual multidimensional display at the frontend by means of the report chart. In the meantime, learners various using TV OTT, PC, tablets and smart phones are connected, which will help collect data of individual user s in different periods, as well as that of modes. In light of learners diversified, various methods for leading third party system data should be provided as well, for instance database, log files, excel, web service interface, etc., in order to support a real-time process of big data. Therefore, we need to set up an a statistical analysis module for inquiring and invoking, based on the web service interface modes, by means of database backup, data importing function of the administrative page. The system imports the third party Excel data via the frontend administrative page, and shows the results after analysis. Users can have a direct access to the third party SQL database, or isomorphic Mongo database, they can also have a direct access to the third party data source through Web Service interface. In data collection, considering the traditional educational measurement and ways of data collection from online platform, on the one hand, process data from platform is collected, while on the other, physical signs data of the students dominant should also be collected by intellectual devices, then using different algorithms to manage and analyze different types of data, which will reflect the students factual thinking ability and. The procedure of al data collection and analysis in the process is shown in Fig. 3. Table 2 Analysis index catering to evaluation objectives ISBN:

5 Third party database Production database Learning management module Collection 1 Collection 2 Collection 3 Read the backup Read the backup Learning analysis module Analysis sub-module 1 Event tracking interface HTTP Analysis sub-module 2 Analysis sub-module 3 Historical database Fig. 3 The procedure of al data collection and analysis in the process C. Key techniques and algorithms for analyzing online data The first step of online analysis is to collect and sort out data, and at the same time carry out an in-depth data mining and al analysis using all kinds of algorithms, which mainly include feature sets, Mapreduce algorithm and theme mining algorithm[18][19], etc. 1) Feature data sets Feature data sets contain the static data and application data of the resources, state data and so forth, which respectively correspond to data sources like historical data, real time data, attribute data, etc. Feature data sets need to go through several processes, namely pre-processing, feature extract and feature selection, as well as the final machine, in order to get the analytics model. The core of feature set algorithm lies in the setting of the feature sets, as Fig.4 shows. Based on the cover degree of the feature sets (X axis) and description ability, analytics system trains data from each quadrant adopting different algorithms so as to obtain a model with high degree of fitting[20]. Description ability ID ID Combination Perfect features Low Cover degree Float type feature (various kinds of Weak features similarities) Some low dimension features Weak First quadrant: less general amount; a lot of Bias features Second and third quadrant: a relatively large amount; complement each other Fourth quadrant: weak influence Fig. 4 Setting of the feature sets 2) Mapreduce Algoritm Mapreduce technique is a typical example of non-relational data management and analysis techniques, which include three levels of contents: distributed file system (DFS), parallel programming model, and parallel execution engine. Mapreduce technique is a concise parallel computational model. It can solve problems like expansibility and fault tolerance in the system level, and can also carry out a parallel execution automatically in the elastic large-scale cluster by accepting the Map function and Reduce function written by users. Therefore, it can process and analyze the mass education data. 3) Theme mining algorithm Theme mining algorithm is a technique widely used in analytics, which is applied in both context targeting and targeting. In the analytic system of, we choose the supervised theme mining algorithm to map the page content onto the label system that has been defined in advance, rather than in an unsupervised way. The frame is to get to know learners interests according to the learners historical visiting, and set up an analytics model. Behavior analysis is important because it provides a general way in making the most out of the online users logs to have an in-depth analysis of teaching. Therefore, the frame, algorithm and evaluation index of targeting establish the substantive characteristics of online data driven analysis. If you regard context targeting as the one that aligns with the user s single access, targeting can be regarded as the integration of a series of context targeting. Therefore, context targeting lays the foundation for targeting analysis, and each type of context targeting can have its correspondent targeting mode. V. CONCLUSION The online system we described in this paper can collect data from different approaches, including process data collection from online platform, and sort out typical physical signs data from the actual scenario to conduct a comprehensive calculation and analysis. The system can also analyze data from multiple terminals and across different screens, which fills up the gap of cross-screen analytics in the education industry. It aims at providing perceptible resources and recommendation services for learners, and also to assist with personalized. In the near future, we will analyze some learners data in the Lifelong Learning Platform for Shanghai Citizens to verify the feasibility of this study, and find out problems and rules existed in the current online situation. Besides, we will also make adjustments and improvements to boost the outcomes, and at the same time improve system function and analytic model according to the application situation. ACKNOWLEDGMENT This paper is supported by Shu Guang award MOOCs design and empirical research oriented Shanghai lifelong (13SG56) and the Oriental Scholar program (TPKY052WMJ) from the Shanghai Municipal Education Commission and Shanghai Education Development Foundation. It is also supported by the 2014 Shanghai ISBN:

6 education scientific research key project The Study of online mode for Shanghai lifelong (A1403). Besides, thanks for the support of Science and Technology Commission of Shanghai Municipality research project Shanghai Engineering Research Centre of Open Distance Education (13DZ ). REFERENCES [1] Hong Yan, Tang Hui, Liang Linmei. The New Trend of American Higher Online Education-2010 & 2011 Sloan Consortium Survey Overview, Distance Education in China, pp , Jan [2] China Internet Network Information Center. The 33 rd China Internet network development state statistic report [EB/OL].2014: [3] Gu Xiaoqing, Zhang Jinliang, Cai Huiying. Learning Analytics: The emerging data technology, Journal of Distance Education, no. 208, pp.18-24, [4] Zhu Zhiting, Shen Demei. Learning Analytics: The Energy of Smart Education, E-Education Research, no.5, pp , [5] Li Fengqing, Yang Shulin. Higher Education in the Information Age: Future Trends and Challenges-The horizon report of NMC, Modern Distance Education, no. 5, pp , [6] Wei Shunping. An Analysis of Online Learning Behavior and Its Influencing Factors, Open Education Research, vol. 18, no.4, pp , [7] Peng Wenhui, Yang Zongkai, Tu Qingshan. The Survey and Analysis of Online Learner s Learning Behavior, China Educational Technology, no.12, pp , [8] Pen Wenhui, Yang Zongkai, Huang Kebin. Analysis and Model Research of Online Learning Behavior, China Educational Technology, no.10, pp , [9] Yang Jinlai, Zhang Yixiang, Ding Rongtao. Online Learning Behavior Monitoring Based on Online Learning Platforms, Computer Education, no.11, pp , [10] Yang Jinlai, Hong Weilin, Zhang Yixiang. Research and Practice of Real-time Monitoring of Network Learning Behavior, Open Education Research, vol. 14,no.4, pp ,2008. [11] Hu Yunan, Lv Zhihui. Design and Implementation of Learning Behavior Collection Based on SCORM, Computer Engineering and Applications, no.22, pp , [12] Gao Yi, Shen Ruimin. Learning Behavior Analyzing Center Based on Open E-Learning Platform, Computer Engineering, vol.30, no. 15, pp , [13] Jia-JiunnLo,pai-Chuanshu. Identification of styles online by observing leamers browsing through a neural \network[db/ol].32ndasee/eieefrontiersinedueationconferenee [14] George Siemens.What are Learning Analytics [DB/OL]. /, [15] Elias, T. Learning Analytics: Definitions, Processes and Potential[EB/OL]. onsxprocessespotential.pdf, [16] [Wolfgang Greller. Learning Analytics framework [EB/OL] [17] Yassine Tabaa,Abdellatif Medouri. Karin Anna Hummel,Helmut Hlavacs. Anytime,Anywhere Leaming Behavior Using a Web-Based Platform for a University Lecture[DB/OL]. [18] Xu Lei, Claus Pahl. An evaluation technique for content interaction in Web-based teaching and environments [DB/OL].In Proceedings of The 3rd IEEE International Conference on Advanced Learning Technologies. Psu.edu /viewdoc/summary?doi= [19] Wu Xiyuan, Zheng Qinghua. An Attribute Reduction Algorithm to Find Learner s Key Characteristics Based on the Discernible Function, Journal of Xi an Jiaotong University, vol. 42, no. 12, pp , [20] Benaceur Outtaj, Raehida Ajhoun. Towards a model for evaluating the e-learner`s [DB/OL].ICTA`07, s/docs/p27.pdf Jun Xiao is an associate professor of Shanghai Engineering Research Center of Open Distance Education, Shanghai Open University, China, the visiting scholar in department of computer science, San Diego State University, USA, and also the committee member of China E Technology Standardization Committee. His research specialties focus on analytics, lifelong, digital system and educational resource repository research. He has led many large-scale Research and Development projects, such as Shanghai Educational Resource Center, Shanghai Lifelong Learning Network, Shanghai Learning Network, and this Cloud-Based Intelligent Learning System. He has published more than 30 articles on major publications, and he is the author of 5 books. Address for correspondence: Dr Jun Xiao, Learning Square, Room 505, No.288 Guoshun Rd, Shanghai, China. Tel: (+86) ecnuxj2003@163.com Minjuan Wang is an oriental scholar at Shanghai International Studies University, China, a professor of Learning Design and Technology at San Diego State University, and a Program Manager for the Chancellor s office of California State University. Her research specialties focus on the sociocultural facets of online, and the design and development of mobile and intelligent. She has published peer-reviewed articles in Educational Technology Research and Development, Computers and Education, Educational Media International, TechTrends, and the British Journal of Educational Technology. She has also published book chapters on engaged in online problem solving, Cybergogy for interactive online, informal via the Internet, and effective in multicultural and multilingual classrooms. Address for correspondence: Dr MinjuanWang, 5500 Campanile Dr. PSFA 315, SDSU, San Diego, CA Tel: mwang@mail.sdsu.edu. Lamei Wang is a researcher in Shanghai Engineering Research Center of Open Distance Education at Shanghai Open University, China. Her research specialties focus on analysis, and the design of E- system. She has published articles in China Educational Technology, Chinese Education Informationization. Address for correspondence: LameiWang, No 288, Guo Shun Road, Shanghai, China. Tel: (+86) wanglamei@shtvu.edu.cn Bingqian Jiang received the B.S. degrees in educational technology from East China Normal University. She is currently working towards the M.S. degree in the Department of Education Information Technology, East China Normal University. Her research interests include technologies, analytics, and lifelong. Address for correspondence: Ms. Bingqian Jiang, Room 405 Computer Building, ECNU, No North East Zhongshan Rd., Shanghai, China. Tel: (+86) jbq6888@163.com ISBN: