Students Track Their Own Learning with SAS Data Notebook

Students Track Their Own Learning with SAS Data Notebook Lucy Kosturko SAS Institute, Inc. Lucy.Kosturko@sas.com Jennifer Sabourin SAS Institute, Inc. Jennifer.Sabourin@sas.com Scott McQuiggan SAS Institute, Inc. Scott.McQuiggan@sas.com Keywords: Self-regulated learning, Mobile learning, Digital notebooks, Metacognition, Motivation, Learning strategies, Communication Abstract The adoption of the Common Core State Standards for many schools implies greater academic rigor on behalf of the students and an expectation of modernized, thought-provoking instructional methods from teachers. Ultimately, students are expected to not only master academic content, but also demonstrate proficiency for skills associated with academic independence and self-regulated learning. Extensive research has shown that these behaviors are associated with greater academic motivation and success (Zimmerman, 1990). One successful method for developing self-regulatory behaviors is encouraging students to set explicit, measureable goals, use quantitative and qualitative means to track progress, and utilize data-supported evidence to reflect on academic achievements and refine previous goals (Zimmerman, 2008). Therefore, the purpose of this research is to investigate the utility of a Digital Data Notebook an aggregated digital space for setting, tracking, and reflecting on academic goals for improving student s self-regulatory skills. Similar non-digital tools have been successfully implemented in the classroom (Bryne & Schaefer, 2006) and research-based evidence supports the use of resources targeting specific, individual facets of self-regulated learning (Zimmerman, 2008). However, to date little empirical work has been devoted to analyzing the efficacy of digital tools for the regulation of learning goals. Consequently, this work will explore the use of a Digital Data Notebook for fostering student selfregulatory behaviors through direct modelling and capitalizing on the affordances of technology for data tracking and visualization. Theoretical Framework The ultimate mission of the Common Core State Standards Initiative is to ensure all students leave high school with the necessary skillset to thrive in both career and college settings (Common Core State Standards, 2010). Therefore, the Standards challenge students to not only build and master academic content but also become independent, high-level thinkers. Students should leave high school equipped with the cognitive and metacognitive abilities required of the lifelong learner. In short, today s students should embody the characteristics of a self-regulated learner. Self-regulated learning is a term used to describe the behaviors of students who actively control their own learning by [sustaining] cognitions, behaviors, and affects that are systematically directed toward the attainment of goals (Schunk, 2008, p. 245; Schunk & Zimmerman, 2003; Winne & Hadwin, 1998; Pintrich, 2000). Self-regulated learners are not only equipped with a sufficient set of learning strategies, but also have the motivational control to put forth the necessary effort to engage in these cognitive processes (Pintrich 2000). Therefore, self-regulation is often represented by three components:

metacognitive awareness, strategy use, and motivation (Zimmerman, 2000). While students often show great variance in their self-regulatory behaviors (Ellis & Zimmerman, 2001), proficiency of such skills has shown to be influenced and improved through direct instruction and modelling (Zimmerman, 2002). Furthermore, there is evidence that students who are better able to regulate their learning in an intentional and reflective way often demonstrate greater academic motivation and achievement (Zimmerman, 1990). Although encouraging self-regulation is critical for the Common Core era student, the new curriculum is intentionally ambiguous in techniques for increasing these behaviors. According to official documentation, The Standards do not mandate the full range of metacognitive strategies that students may need to monitor and direct their thinking and learning. Teachers are thus free to provide students with whatever tools and knowledge their professional judgment and experience identify as most helpful for meeting the goals set out in the Standards (Common Core State Standards, 2010). With respect to developing self-regulatory skills, research-based best-practices suggest teachers should turn to contextualized methods that require students to explicitly engage in behaviors such as setting specific, measureable goals, diligently monitoring progress toward such goals, engaging in strategies that encourage metacognitive awareness, and reflecting on previous actions (Zimmerman, 1990; Schunk & Swartz, 1993; Schunk & Zimmerman, 2003; Schunk, 1990). One method for modelling and developing students self-regulatory behaviors proposed by the Baldrige Education Criteria for Performance Excellence is through the use of a student data tracking system or data notebook (Byrne & Schaefer, 2006; Baldrige, 2013). Data notebooks provide a single aggregated space where students can set their own measureable goals, evaluate and reflect on their progress, and refine their goals accordingly. Empirical studies have shown significant differences in academic motivation between students who set and track their own goals and students who are assigned goals (Zimmerman, 2008). Furthermore, as reported by Byrne and Schaefer (2006) several qualitative reports such as it is very empowering for students, giving them ownership of their education support the use of data notebooks in the classroom. Beyond self-regulatory behaviors, the Common Core State Standards emphasize the use of data and evidence as a necessary component for effective communication at all grade levels. Specifically, anchor standard CCSS.ELA-Literacy.CCRA.SL.4 and anchor standard CCSS.ELA-Literacy.CCRA.Sl.5 respectively state that students should present information, findings, and supporting evidence such that listeners can follow the line of reasoning and the organization, development, and style are appropriate to task, purpose, and audience and make strategic use of digital media and visual displays of data to express information and enhance understanding of presentations (Common Core State Standards, 2010). With respect to reflecting upon and detailing one s academic progress, explicit goal setting and monitoring provides an excellent platform for utilizing data and evidence for communication. Therefore, the present research sought to provide preliminary findings for the following research questions: 1) How do teachers perceive Digital Data Notebooks affect students self-regulatory behaviors and academic achievement?, and 2) How are students using Digital Data Notebooks as a data point when communicating educational progress?

Rating Research Method Participants The SAS Data Notebook app was developed in partnership with teachers from a large socioeconomically diverse elementary school in the Midwest (644 students, 61% free/reduced-price lunch); therefore, participants for this study were recruited from the partner elementary school, as well as a blanket call for participation extended to other teachers using SAS Data Notebook in their classroom via social media outlets. A total of eight participants were included in the study ranging in teaching experience, educational technology experience, and level of comfort with technology in the classroom (see Figure 1). Participants also represent a range of grade levels from kindergarten to 8 th grade. All participants had experience using a non-digital form for data notebooks in their classroom prior to implementing SAS Data Notebook. Technology Experience Technology Comfort 5 4 3 2 1 0 P1 (10+) P2 (6-10) P3 (10+) P4 (10+) P5 (6-10) P6 (10+) P7 (6-10) P8 (3-5) Participant (Teaching experience in years) Figure 1. Participant teaching and technology experience and level of comfort with using technology in the classroom. Materials Teacher Demographic Inventory is a 7-item survey to gather information about participants teaching experience, technology use, and familiarity with traditional data notebooks (Appendix A). SAS Data Notebook (Figure 2) is a free tool that brings data notebooks from the traditional 3-ring binder to the ipad. Data Notebook provides tools for students to set personal goals, monitor their own learning, reflect on previous work, and communicate their progress to teachers and parents using real data. Within the notebook, students can use the built-in templates to create mission statements, set goals, generate checklists, reflect using plus/deltas, create and practice spelling lists, and plot histograms. Data Notebook also includes a blank page and scratch paper template that enables students to load pictures, drawings, and more into their notebook. Students can also add sections for additional organization. Thus, students goals, daily progress, and achievements are aggregated in one place for easy access allowing for reflection and communication with parents and teachers.

Figure 2. SAS Data Notebook. Teacher Reflection Survey is a 15-item, researcher-constructed metric designed to collect use, satisfaction, efficacy, communication and observational data regarding the use of SAS Data Notebooks over the course of the 2013-2014 school year. Questions were answered using a 5-point Likert-scale, and open-ended reflections in order to extend and understand the effect of using the Data Notebook app had on students self-regulatory behaviors (Appendix A). Procedure Over the course of the 2013-2014 school year, teachers and students utilized the SAS Data Notebook tool as part of their regular coursework. Depending on how the teacher decided to implement Data Notebook, students updated their notebooks by collecting and archiving data, revising goals, and reflecting on performance on a daily or weekly basis. Notebooks were also reportedly used during parent-teacher conferences as a data point for student reviews. Although all teachers at the target school were required by administration to implement Data Notebook in their classroom, only those who choose to participate in the research study were asked to complete the Teacher Demographic Inventory and Teacher Reflection Survey. Participants were asked to complete the surveys in the spring of 2014. Results Using a mixed-methods exploratory design, quantitative ratings of overall satisfaction, student selfregulatory behaviors, and communication with parents were triangulated with qualitative reports to further validate the data. How do teachers perceive Digital Data Notebooks affect students self-regulatory behaviors and academic achievement? To understand teacher s perceptions of self-regulatory behaviors, participants were asked to specifically reflect on the components of self-regulated learning: metacognition, strategy use, and motivation (Zimmerman, 2000). As reflected in Figure 3, on average, participants felt SAS Data Notebook did have a positive effect on their students metacognition (M = 3, SD = 1.18), strategy use (M = 3.63, SD = 1.44), and motivation (M = 3.63, SD = 1.40). In line with these findings, participants commented that their

Rating students are eager to enter their data and see their scores on the graphs and like to track their data and see progress. With regard to metacognition, one teacher reported that having successful data to support it was useful when students are trying to meet their weekly and daily goals. Finally, teachers felt that using Data Notebooks was beneficial to overall academic performance (M = 3.75, SD = 1.38). 5 Motivation Strategy Use Metacognition Academic Performance 4 3 2 1 0 P1 P2 P3 P4 P5 P6 P7 P8 Participant Figure 3. Participants perceptions of SAS Data Notebook s effect on students self-regulatory behaviors and academic performance. How are students using Digital Data Notebooks as a data point when communicating educational progress? Similar results were found for using SAS Data Notebook as a means to communicate educational progress (Figure 4). On average, participants felt the app was beneficial for aiding their communication with parents and students (M = 3.38, SD = 1.68). One noted that Keeping track of how each child does is helpful when I meet with each student. During our conference time, we use their data from SAS to discuss weekly and daily goals. Another stated, During conferences, I shared the students SAS work with their parents. Also, I use their data to share with parents during the school year. Furthermore, participants felt the app helped students communicate with their parents about their academic performance (M = 3.89, SD = 1.33). One teacher pointed to the portability of the ipad as particularly useful for communication stating that It's nice to have the data for each student right there on their ipad. It can also be shared with parents daily when the child takes his/her ipad home each night.

Rating Teacher-Student/Parent Student-Parent 5 4 3 2 1 0 P1 P2 P3 P4 P5 P6 P7 P8 Participant Figure 4. Participants perceptions of SAS Data Notebook s effect on teacher-student/parent communication and student-parent communication. Implications The educational rigor mandated by the Common Core State Standards challenges students to go beyond what has been traditionally required by state curricula. Furthermore, as students approach college and career readiness, they are expected to demonstrate academic independence through both cognitive and metacognitive actions and embody the characteristics of a self-regulated learner. Such qualities (e.g., motivational control, metacognitive awareness, and effective learning strategy implementation) are beneficial for thriving in the Common Core era. While self-regulatory behaviors are believed to be acquired over time through guidance and practice, research suggests students, even those of the same age, demonstrate a wide range of proficiency (Ellis & Zimmerman, 2003). Such findings motivate the need for more explicit, contextualized self-regulated learning instruction. Flexible tools, such as SAS Data Notebook, provide a great resources for encouraging students to take control of their own learning across the content areas as reflecting in the findings from this study. Overall, integrating the SAS Data Notebook app was perceived to have a positive effect on students self-regulatory behaviors and overall academic achievement. Furthermore, students Data Notebooks were posited as a useful tool for communicating educational progress. In fact, only one of the participants indicated she does not plan to integrate SAS Data Notebook in her classroom instruction next year. However, it should be noted, implementing programs like SAS Data Notebook must be done with care. Students self-regulatory skills are developed overtime and proficiency often requires direct instruction and modelling (Zimmerman, 2002). Therefore, instruction around the use of these tools and the motivation for setting goals, monitoring progress, and reflecting on results is critical. In fact, two of the eight participants in this study felt SAS Data Notebook did not improve their students self-regulatory skills; however, these teachers pointed to conceptual misunderstanding as a potential problem. One noted that she, will spend more time in August emphasizing how to stay organized and updated. This

will alleviate issues that I have encountered this year. The other said he will, have more upfront training for students to be more independent with their Data Notebook. Finally, the results should be generalized with care as there are several, notable limitations. First, the sample used for this investigation was very small and does not necessarily represent the larger population of classrooms utilizing the SAS Data Notebook app. Furthermore, the degree to which Data Notebook was integrated over the course of the 2013-2014 school year was not controlled nor was the manner in which students used the app; therefore, direct comparisons between the participants should be made with caution. Moreover, it is important to consider students self-regulatory skills prior to using the Data Notebook app in order to better understand the direct effect of using the tool throughout the school year. Lastly, the measures used for this study were researcher-constructed and self-report in nature leaving room for measurement error. More rigorous investigations with more vigilant experimental control, sufficient power for more sophisticated statistical analyses, and validated measures for each of the study s variables are necessary to ensure the validity of these preliminary findings. Additionally, as self-regulated learning behaviors develop overtime, it would be interesting to understand the differences in effectiveness of using Data Notebook between grade levels. Nonetheless, design considerations for SAS Data Notebook are grounded in research-proven principles and the findings from this exploratory study show promise for supporting the development of students selfregulatory skills. References Byrne, S. & Schaefer, C. (2006). The Baldrige Program: Self-Assessment for Continuous Improvement. Principal, 24-27. Ellis, D. & Zimmerman, B. J. (2001). In H. J. Hartman (Ed.), Enhancing self-monitoring during selfregulated learning of speech (pp. 205-228). Dordrecht: Kluwer. National Institute of Standards and Technology (2013). Baldrige Criteria for Performance Excellence 2013-2014. Washington, DC. National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for English language arts and literacy in history/social studies, science, and technical subjects. Washington, DC: Authors. Pintrich, P. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation. San Diego, CA: Academic Press. Schunk, D. H. (2008). Attributions as motivators of self-regulated learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 267-286). New York: Routledge. Schunk, D. H. & Schwartz, C. W. (1993). Goals and progress feedback: effects on self-efficacy and writing achievement. Contemporary Educational Psychology, 18(3), 337-354. Schunk, D. H. & Zimmerman, B. J. (2003). In W. M. Reynolds & G. E. Miller (Eds.) Self-regulation and learning (Vol. 7, pp. 59-78). New York: Wiley & Sons.

Winne, P. H. & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 279-306). Hillsdale, NJ: Erlbaum. Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: an overview. Educational Psychologist, 25, 3-17. Zimmerman, B. J. (2002). Becoming a self-regulated learner: an overview. Theory into Practice, 4(2), 64-70. Zimmerman, B. J. (2008). Goal setting: A key proactive source of academic self-regulation. In D. H. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 267-286). New York: Routledge.

1. What grade(s) do you teach? (Pick all that apply) Appendix A Teacher Demographic Inventory Kindergarten 1st grade 2nd grade 3rd grade 4th grade 5th grade 6th grade 7th grade 8th grade Specialist Administrator Tech Coordinator 2. How long have you been teaching? 1-2 years 3-5 years 6-10 years 10+ years 3. Rate your level of experience with using technology in the classroom. 1 - No Experience 2 - Novice 3 - Moderate Experience 4 - A lot of Experience 5 - Expert 4. Rate your comfort level with using technology in the classroom. Very Uncomfortable Uncomfortable Uncomfortable at times Comfortable Very Comfortable

5. What technology(ies) do you use in the your classroom? (Pick all that apply) ipad/ipod/ios devices SmartBoard Andriod mobile devices Chromebooks Desktop computers Laptop Computers No technology/i am not a classroom teacher 6. On average, I use SAS Data Notebook in my classroom: Daily 2-3 times per week Once a week Never 7. Before using SAS Data Notebook, I used: (Pick all that apply) (Required.) 3-ring binders other I did not use data notebooks in my classroom Teacher Reflection Survey 1. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' motivation. Disagree Slightly disagree Neither agree nor disagree Slightly agree Agree 2. Explain. 3. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' study habits/strategies.

Disagree Slightly disagree Neither agree not disagree Slightly agree Agree 4. Explain. 5. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' metacognition. Disagree Slightly disagree neither agree nor disagree Slightly agree Agree 6. Explain. 7. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' academic performance. Disagree Slightly disagree Neither agree nor disagree Slightly agree Agree 8. Explain. 9. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my communication with parents about students' academic performance. Disagree Slightly disagree Neither agree not disagree

Slightly agree Agree 10. Explain. 11. To what extent to you agree with the following statement? I believe SAS Data Notebook improves students' communication with their parents about their academic performance Disagree Slightly disagree Neither agree nor disagree Slightly agree Agree 12. Explain. 13. I plan to integrate SAS Data Notebook into my classroom again next school year. No Yes Undecided 14. If yes, what will you change for next year? What will you keep the same? 15. Do you have any best practices, lessons learned, or advice for integrating SAS Data Notebook into the classroom? Submit Your Answ ers