Student Progress Monitoring Tool using Treeview Jungsoon Yoo csyoojp@mtsu.edu Sung Yoo Chris Lance Middle Tennessee State University Murfreesboro, TN 37132, USA +1 615 898-2397 cssung@mtsu.edu lancecg@hotmail.com Judy Hankins csjudy@mtsu.edu ABSTRACT In this paper, we present an extensible visualization tool that is being used in a web-based closed laboratory system. The goal of this project is to provide a tool for both students and teachers that can help trace deficiencies in a student s understanding back to individual concepts. This visualization tool has been developed by using the tree abstract data type (ADT) which is built from the concepts to be covered in a lab, lecture, or course. Once the tree ADT is built, each node can be associated with different entities such as student performance, class performance, or lab development. Using this tool, a teacher can help students by discovering concepts that need additional lecture coverage, and students may discover concepts for which they need to spend additional time working on reinforcement exercises. Categories and Subject Descriptors K.3.1 [Computer Uses in Education]: Computer-assisted instruction, Computer-managed instruction, Online learning. General Terms Management, Design, Human Factors. Keywords Courseware, Visualization 1. INTRODUCTION A teacher typically spends a significant amount of time assessing and evaluating the students level of understanding on concepts covered in class/lab. The result of the assessment can be used to promote student learning by identifying each student s weaknesses and strengths. A teacher can use this information to make and/or modify lesson plans to reinforce certain subjects. It Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGCSE 06, March 1 5, 2006, Houston, Texas, USA. Copyright 2006 ACM 1-59593-259-3/06/0003 $5.00. can also be used by the teacher to help establish a personal learning plan for each student. However, the assessment and evaluation task is often a time consuming and tedious task. This task has been automated with an online laboratory/tutoring system. An online tutoring system has been developed for Computer Science I and II closed labs [10]. Each lab covers particular concepts from the respective course. For example, in CS-I, one lab might cover decision statements, loops, or arrays. For each lab, a concept base has been created that contains organized concepts to be covered in the lab. The tutoring system provides two components: a testing component and an evaluation component. To evaluate the student s weaknesses and strengths properly, the testing component and the evaluation component are closely coupled with desired learning concepts. The concept base has been used to link the testing and evaluation components in this tutoring system. A graphical progress monitoring tool has been developed for the online tutoring system. The system analyzes student work on each lab and presents the results to the teacher so that the student s understanding can be monitored in a structured, systematic way. Teachers may use the tool to discover topics their classes are having trouble understanding and to identify the problems individual students are having. The monitoring tool is also used by the student to view his or her performance with respect to the topics being covered in a particular lab. This will help the student see exactly what topics he or she is struggling with and should spend additional time reinforcing. To effectively visualize the performance of a student (or class), a tree ADT (Abstract Data Type) was used to associate the performance of that student (class) with concepts to be learned in a tutoring session. Each concept is represented as an individual node in a tree. Student performance is not the only area in which this type of visualization has proven to be useful: the same abstract idea can be applied to many objects which can be quantitatively measured. One example of another use of the system is visualization of a lab design. The online laboratory system has a well-defined set of requirements for the design of labs. These requirements specify the number of programs, the number of questions at different levels of difficulty, and post-test questions required for each exercise. Using the concept tree, the tool has been used to map out the differences between these requirements and the actual number of components set up in the system thus providing a completeness check for lab preparation.
Figure 1. A part of the system architecture related to the monitoring tool. In Section 2, the design of the monitoring tool is described in terms of the system s goals, architecture, and graphical visualization with the Tree ADT along with some implementation issues. Section 3 describes experimentation and results. We conclude with related work and the future direction of the current project. 2. DESIGN OF MONITORING TOOL The goal of the system is to develop a visualization tool that will help a user monitor the progress of student learning. The design goals of the system are: The system should provide an assessment tool that captures student activities during a tutoring session. The system should provide a visualization tool that helps a teacher and/or student to monitor a student s progress. The system should provide a visualization tool that helps a teacher to monitor class progress. In the following subsections, the design details of the visualization component are described. 2.1 Architecture We have developed an online tutoring system for CS-I and CS-II closed labs [10]. Figure 1 shows a part of the architecture of the online tutoring system that displays the components related to the monitoring tool. Our system provides a question editor equipped with a GUI to help a teacher create questions [11]. In the lower left corner of the figure, the question editor interfaces with the concept base. The concepts are organized in a concept base so that the dependencies and relationships between concepts and sub-concepts are represented. Each question created using the question editor is associated with a concept(s) in the concept base. This association allows the system to monitor student progress on each concept. The system also contains the rule base editor 1 that provides a graphical user interface for a teacher to create/edit a hierarchical 1 Concept relationships are represented as rules. Thus, we call the editor a rule base editor instead of concept base editor. Figure 2. The one-dimensional array lab page. concept base. A teacher also needs to create lab pages for each lab session. Figure 2 shows an example of a lab page for the onedimensional array lab in CS-I. Students will access the online system though the lab page(s). The student may view an explanation of each concept complete with examples by clicking on the concept prior to the exercises. Our on-line laboratory system captures each student s performance during the session and stores it in the tutoring system database. The performance monitoring tool analyzes student data and displays the results either to a student or a teacher. Using the monitoring tool, a student can view his/her own progress, while a teacher can view the result for an individual student or for a class as a whole. 2.2 Tree ADT The first decision that was made in the creation of the monitoring tool was how to organize the concept base used to link the question and evaluation components of the system. As with topics in many other subjects, topics in computer science are generally hierarchical in nature: a concept is composed of subconcepts which are composed of sub-sub-concepts. This means they lend themselves to being laid out in a tree structure. The concepts covered in each lab can be organized into a single tree; the entire course, then, consists of a collection of trees called a forest. For example, Figure 3 shows a partial view of the concept base built for the one-dimensional array lab in CS-I. This concept base can be used to associate all questions in a test/lab with the one-dimensional array concepts. 2.3 User Interface Design The second major decision involved the user interface for displaying the results to students and teachers. In software development, design is widely misunderstood and undervalued [8]. The user interface is, perhaps, the most crucial component of
Figure 3. A partial view of the tree ADT for the one-dimensional array lab. any software system because, without a good interface, novice users will have difficulty learning to use a new application. When designing the user interface for the tool, the principles of usability proposed by both Karat [5] and Neilsen [7] were taken into account. The performance monitoring tool utilizes dynamic HTML (DHTML) in many aspects of its user interface. DHTML allows a web-page, through client-side scripting, to modify its own content based on input from the user. While DHTML provides powerful means for adaptive user interfaces, it becomes very easy to violate Berkun s conservation of user interface principle: if you add something to a user interface, you must replace something else [2]. The DHTML-driven search and options screen in the performance monitoring tool do, in fact, conform to this rule. 2.4 Visualization using Treeview Treeview is a class that is used to display data that is hierarchical in nature, such as organizational trees, the entries in an index, or the files and directories on a disk. The monitoring tool uses Treeview to create a visual, navigable representation of the concept trees in the user s web browser (see Figure 4). The tree produced by Treeview is a convenient visualization technique for novice users because it is easy to learn and for experienced users because of the efficiency in the number of clicks involved [9]. Our tree visualization uses colors and shapes to represent the status of each node: green circles to indicate a high level of performance, yellow triangles for an intermediate level, red squares for a low level, or grey diamonds to indicate incomplete or unavailable data for the particular node. Each node is also capable of displaying context-sensitive information allowing the user to see the detail values of that node. Each child node represents a sub-topic of the parent node, so the parent node is displayed in green when the average performance of each node in the sub-tree of the node is also high enough for the green color. A teacher/student may find a deficiency by locating the highest level node that shows in non-green color. Figure 4. A screen shot of the monitoring tool that displays a student s performance for the onedimensional array lab. He/she can then view the tree at any level by collapsing or expanding nodes into their respective sub-trees. When a student is logged into the system and accesses the tool, the system presents a tree that displays the student s performance of the working lab. However, the instructor must choose whether they wish to view performance for a student or for the class as a whole. If student performance is chosen, the instructor may either search for a student by user id or select from the available class list. Similarly, if class performance is chosen, a list of available sections is displayed from which the instructor may select. Instructors may also customize the range of scores included in each color of node in the tree. For example, for an advanced student, a node with less than 80% competency may be considered a problem area and the teacher may want that displayed in red for that particular student; however, for other students, 80% may be considered reasonable competency and should be displayed in green or yellow. A mechanism for persistence of these options is also provided. 2.5 Implementation The monitoring tool is written in the Java programming language including elements from both J2SE and J2EE. The building blocks of Java web applications are components called Servlets. While traditional web pages and dynamic content created using scripting languages rely on a web server application to service client requests, control script execution, and transport output back to the client, Java web applications rely on an application called a Servlet container to provide these services. This tool has been implemented on an open source Servlet container created by the Apache Software Foundation called Tomcat. All pages are written using Java Server Pages. Java Server Pages allow Java code to be embedded inside of HTML to provide dynamic content in a format that is easy to write and maintain.
An important implementation issue in a system such as this is security. Every possible precaution must be taken to ensure that students are able to access only their own performance and not that of any other students. To achieve this requirement, a special component called a filter is used. A filter is a piece of code that is mapped to a URL on the server. When any URL is accessed, the Servlet container looks through its list of filters and executes each filter that is mapped to that URL. Each access to the tool is passed through an authentication filter which determines the current user s level of access and, based on this access level, will either allow or deny their request. 3. EXPERIMENTATION Figure 2 shows the lab page of the one-dimensional array lab that was developed for the online tutoring system. First, a concept base of the array concepts was created (Figure 3). For each of the most specific concepts in the hierarchy, one or more exercises were created using the question editor [11] and stored in a database so that a question could be retrieved using a keyword. The one-dimensional array lab has been used in CS-I since fall of 2004, and it is unique in few ways: The lab was structured to reflect the relationships between concepts in the concept base so that students can see what they are learning in a big picture. Details and examples are still available by clicking on the link provided for each concept. Exercises are created using the problem generator that generates questions on demand [11]. Students can solve problems that address a certain concept as many times as they wish until they feel confident with the concept. The result of student surveys administered after the lab shows that 83% strongly agree with solving similar questions multiple times helped understand the concepts. The survey shows that this approach helps weaker students more than advanced students. The survey also shows that 88% of the students think a hierarchical structure helped understand the overall picture of the material. We have divided the students into two groups; the first group used the traditionally formatted (unstructured and no concept base) one dimensional array lab while the second group used the structured format with the concept base. Students are given a pretest before the lab and a post-test after the lab. We measured the difference in students performance between the pretest and the post-test. Results show that the second group improved more than the first group and the average difference is statistically significant with 99% confidence. In summer 2005, the performance monitoring tool was used for the first time in a CS-I class for the one-dimensional array lab. Students in the class were surveyed to determine their attitudes concerning the monitoring tool. Eighty-five percent of the students in the class agreed that the tool was helpful. Students in fall 2005 will be using the tool and will be polled to determine whether they find the tool useful. Teachers were also asked to use the tool and polled for their opinions. Figure 5 shows a screenshot of the monitoring tool that displays the performance of a class for the Selection lab in CS-I. The selected class was having a problem with logical operators; Figure 5. A screenshot of the monitoring tool that displays the performance of a class for the Selection lab. especially one question (ID 594) turned out to be very hard for many students. All the teachers found the system very useful and the following summarizes the feedback: The collapsible feature of the tree enabled teachers to focus on their desired topic and not be distracted by others. By seeing what questions were missed, it may be possible to narrow the focus of class discussion to specific topics. For example, the student may understand the if statement but does not understand the formation of logical expressions. The teacher can see immediately where to concentrate efforts in class. The detailed question information in the right window provided teachers with useful information. Statistics on students answers helped teachers to determine the understanding/misunderstanding of the concept (see the bar chart in Figure 5). The system has all the benefits of the student response gathering system (remote clicker) with additional benefits such as accommodating individuals needs in the lab environment. It was easy to identify a problematic concept or question by recognizing a node associated with questions with incorrect answers or improper levels of difficulty. By having the options of setting lower bounds for yellow or green, the tool helped teachers adjust for honors classes as opposed to regular classes. The system also provides a visualization tool for the design of labs. In lab design mode, the only options available are the customization of the node color boundaries. Figure 6 shows an example of this interface. The online laboratory system has a well-defined set of requirements for the design of labs. However, manually maintaining the completeness of the lab is a very tedious task and incompleteness can cause serious problems. The visualization tool can map out the differences between the requirements and the actual number of components set up in the system thus providing a completeness check for lab preparation. Teachers who used the lab design tool responded, Using the lab design tree quickly made it easy to see where questions needed to be added.
facilitate diagnosis of students competencies in a set of related topics. As the design phase progressed, it became apparent that the tool could be more extensible than originally planned. With a very little amount of coding, the tool can be used to visualize a number of different types of data. We also have plans to use the tool as the teacher s command console for a PDA-based in-class application that allows students to receive and answer questions on PDA s given to them for use during class-time. The tool will be used to push questions to the student s PDA s and to view the results of the questions. This extensibility is one of the best features of this tool and one of the many reasons this project has been a success. 6. ACKNOWLEDGMENTS This research was partially supported by the URSCA Scholarship Program and the FRCAC grant at MTSU and the NSF grant DUE-0311367. Figure 6. A snapshot of the tree displaying lab design status. 4. RELATED WORK This visualization tool is analogous to the dashboard that is used in project management [3]. A dashboard in a car is used to display vital information such as whether gas is needed or whether the car has become too hot. In a similar manner, a dashboard in project management displays vital information about the project such as which areas of the project are over budget or will keep the project from being completed by the target date. The dashboard is a decision support tool that is used to convert expert knowledge in such a way that non-experts (managers and group leaders) can understand, internalize, and combine explicit expert knowledge to inform their decisions [3]. Originally in 1999, GM developed the dashboard to track a project s status for workgroups and management [6]. The dashboard project tracking system used color-coded status of all IT projects: green when it's progressing as planned, yellow when at least one key target has been missed, red when the project is significantly even if just temporarily behind [6]. The monitoring tool developed in the online tutorial system uses the dashboard idea as well as GM s color scheme. The Treeview concept has been used in many applications such as a bookmark manager and a web file manager, as well as task management systems [1, 4]. The performance monitoring tool uses Treeview with color coding to help the student quickly have an understanding of his/her performance and teachers to quickly have an understanding of class performance. Treeview was used in addition to a typical grade reporting system that itemizes grades and displays an overall score. The choice of Treeview with color coding allows the user to immediately determine which areas (concepts) need to be studied further. 5. CONCLUSION AND FUTURE WORKS The goal of this project was to design and implement a visualization tool for both students and teachers that would 7. REFERENCES [1] Bellotti, V., Ducheneaut, N., Howard, M.A., & Smith, I.E. Taking email to task: the design and evaluation of a task management centered email tool. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2003); 345-352Fort Lauderdale; FL. NY; 2003. [2] Berkun, S., The Importance of Simplicity: Create Ease of Use Without Losing Power, July/August 1999, http://msdn.microsoft.com/library/. [3] Cassaigne, N: The Dashboard: A Knowledge Conversion Tool, Engineering Management Conference, Vol. 1, pp292-297, 2002. [4] Dragunov, A., Dietterich, T., Johnsrude, K., McLaughlin, M,. Li, L., & Herlocker, J. TaskTracer: a desktop environment to support multi-tasking knowledge workers. In Proceedings of the 10th international conference on Intelligent user interfaces, pp75-82, San Diego, CA, 2005. [5] Karat, J., Evolving the Scope of User-Centered Design, Communications of the ACM, 40, 7, July 1997, pp. 33-38. [6] Mayor, T., Red Light, Green Light, Case Files: Value Proposition, Oct. 1, 2001 Issue of CIO Magazine [7] Neilsen, J., Ten Good Deeds in Web Design, October 3, 1999, http://www.useit.com/alertbox/991003.html. [8] Smith, P., Debunking the myths of UI design, March 1, 2001, http://www.ibm.com/developerworks/web/library/usmyth.html. [9] "Treeview JavaScript Applet Cross-browser DHTML tree", September 7, 2005, http://www.treeview.net. [10] Yoo, J. Seo, S. & Yoo, S. Designing an Adaptive Tutor for CS-I Laboratory. Proceedings of the 5th International Conference on Internet Computing, pp 459-464, Las Vegas, NV, 2004. [11] Yoo, J., Yoo, S., & Rusek, S. A Question Generator for an Online Tutoring System, Proceedings of the ED-MEDIA conference, pp 1820-1825, Montreal, Canada, 2005.