Creating the Conditions and Know How for Effective Data Driven Dialogue around Student Performance and Behavior

Transcription

1 Doing What Matters Creating the Conditions and Know How for Effective Data Driven Dialogue around Student Performance and Behavior Research on school improvement places data driven dialogue at the heart of any successful reform effort. 1 Two major challenges face educators trying to make data driven dialogue work in their schools and districts: how can accurate and timely data be made available to all who need it when they need it; and what strategies should be used for converting data about students into knowledge and actions that will benefit students. The Institute for Research and Reform in Education (IRRE) offers technical assistance and support to address both of these questions. What is DWM? Doing What Matters (DWM) is a partnership between IRRE and schools and districts to provide new data technology, technical assistance and professional development supports to strengthen data reporting systems, engage in a proven process of data driven dialogue; and develop specific improvement strategies to increase student performance and commitment. Strengthen and Enhance Data Reporting Systems Make them more accurate, useful, and accessible Make them less institutional and more about student performance Build System Capacity around Data Driven Dialogue Dialogue about student performance and commitment Dialogue among teachers and between teachers and administrators To use data to make decisions and target strategies for improvement at all levels of the system and then track the results. 1 Cawlti & Protheroe, 2001; Massell & Goertz, 2002; McLaughlin & Talbert, 2003; Skrla, Scheurich, & Johnson, 2000; Snipes, Doolittle & Herlihy, 2002; Togneri & Anderson, IRRE,

2 DWM is focused on putting in place the conditions and capacities necessary for a culture of data use to emerge and be sustained in schools. The four building blocks of a data driven culture are: 1. System Leadership At all levels of the district (teachers, school administrators, central office staff, senior district leadership, and board) leaders create the conditions, implement supports, set expectations and accountability for, and model data driven dialogue in order to create a culture of data use. 2. Capacity and Structures At all levels of the system leadership and staff are trained and have opportunities to practice using data driven dialogue processes. Teachers, counselors and administrators have the time and space in the workday to have regular and on going data driven dialogue about student performance, behavior and commitment. 3. Data Governance The district implements an intentional program of active data management and quality control processes and creates policies and procedures to ensure that district and school data holdings are accurate, upto date, and supported by a system of on going data audits. Accountability for data integrity is distributed to identified data owners and data stewards throughout the system. 4. Enhanced Technology District data systems integrate disparate databases into a single data warehouse where users can easily access up to date, accurate, and actionable reports of student performance and behavior. Teachers, students, parents and administrators can each access reports on a limited number of key research proven indicators of student progress and behavior through customized user dashboards. IRRE,

3 Data Governance Enhanced Technology System Leadership Culture of Data Use Structures and Capacity Figure 1. Building Blocks of a Data Driven Educational System What is the History of this Work? Measuring What Matters (MWM) is IRRE s data based continuous improvement system and focuses educators on data collected about the quality of teaching and learning in classrooms and the use of proven effective practices during instruction, common planning time, and academic advocacy. MWM was developed by IRRE to support the comprehensive high school reform framework First Things First. 2 MWM tools and processes have been developed and fieldtested over IRRE s ten years of experience supporting high school reform in nine states. These applications of MWM involved diverse student information systems and hundreds of district and building based educators. In doing this work, IRRE and its district and school partners learned some hard lessons all of which have shaped DWM. Examples include: Too much data can undermine data driven decision making as much as too 2 From 2005 to 2008, MWM was supported by a project grant from the Bill and Melinda Gates Foundation. DWM is currently underway in two school districts in Texas with support from the Michael & Susan Dell Foundation. IRRE,

4 little data. Data must be on time as well as on point. Data now available to school and district staff are often not actionable. For example, average daily attendance isn t actionable; knowing the names of students who missed more than a day of school last week is actionable. Teachers and administrators won t access and use data on a regular basis to drive their daily work unless they are expected and supported do so. What Are the Project Activities? Doing What Matters is designed to make student outcome data a more central and powerful driver of dialogue and actions at every level of the system. In this document we provide a brief description of how IRRE works with district staff from multiple departments as well as teachers and administrators in participating schools to craft and then use new tools and processes in order to achieve this goal. Each of the project activities described below was designed to build upon and strengthen existing district and school efforts toward this same goal. The activities fall into two major categories: strengthening, enhancing and targeting data reporting systems and building system capacity around data driven dialogue. During the first several months of the project, IRRE and our technology partner Mizuni, Inc., learns about the district s existing data management and reporting infrastructure and data use systems and then collaborates with district and school staff to strengthen these systems so that 1) data integration into a single data warehouse can occur; 2) new reports of student performance, progress and behavior are available to users; and 3) customized entry points (i.e., performance management dashboards) are available for parents, students, teachers and administrators. District technology staff commit to collaborating with Mizuni to install new servers and software and have the new data infrastructure in place within 8 weeks of the project start date. IRRE brings to this process our extensive experience working in and with schools around data quality and data driven dialogue and the results of our national study of other districts that have substantial experience and success with data warehousing. IRRE,

5 As district and school personnel work with Mizuni staff to create the data infrastructure and bring new student outcome reports online, IRRE works intensely with data owners and users to create a coherent data governance approach that will ensure accurate, timely, actionable and useful data are being captured in the new data warehouse and student outcome reports. IRRE also provides districts a planned sequence of professional development opportunities and several options for varying levels of supports and training to use the new data system to maximum effect. Training and support activities by IRRE focus on having district and school leaders and teacher leaders learn to more effectively facilitate and participate in datadriven dialogue focused on student performance and behavior. Data driven dialogue is expected to occur across multiple settings : Board and cabinet level meetings, central office departmental meetings School level teams and groups of administrators and teachers; Professional learning community meetings (e.g., content area teams, grade level teams, course level teams) Teachers with individual students and/or parents. The purposes of these discussions range from: Designing and monitoring student focused action plans and interventions; Creating and evaluating targeted professional development activities for staff; Planning and reviewing systemic district and building wide improvement efforts. IRRE offers two foundational training and support activities: 1) four half day onsite training sessions for district and school leaders to provide focused skills training on a proven process of data driven dialogue and decision making, and 2) IRRE,

6 two 2 day on site coaching sessions for district and school leaders to address specific challenges being encountered in the implementation of data driven dialogue. On site Skill Building Sessions. DWM provides to participating districts four halfday training sessions focused on learning a process for engaging in and facilitating data driven dialogue. Participants have multiple opportunities to learn and practice the process. Sessions focus on how to access and select data reports, using data reports for data driven dialogue and decision making; and gaining proficiency in using data to identify areas of student performance and behavior that need improvement. IRRE assists district and building leadership in identifying areas of concern, identifying potential improvement strategies, creating action plans and setting targets for on going monitoring and evaluation of implementation of improvement strategies. On site Coaching Sessions. DWM provides one on one coaching, support and assistance to participating district and building leaders to address barriers to implementation of data driven dialogue. IRRE leaders spend time meeting with senior leaders, teachers, and principals on creating and sustaining the conditions that allow educators time to study, engage in collaborative dialogue, and act upon data. Additional DWM Supports Available Districts can choose to expand their measurement and reporting of student outcomes and increase the capacity of district and school administrators to engage in data driven dialogue with the following three options: Implementation of student and staff surveys measuring behaviors, supports and experiences known to be important for student academic success; Capture, analysis and presentation of historical trends for key outcomes to establish a baseline before new improvement strategies are implemented; and IRRE,

7 Participation in DWM Network Meetings. Student and Staff Surveys. Districts may elect to participate in IRRE s student and staff surveys that measure important constructs shown to predict proximal and distal student outcomes. A number of validated scales offer districts additional information about student engagement, teacher expectations and support, peer support, and quality of relationships. Staff surveys allow districts to gain valuable information about teacher engagement, experience of support, commitment, and use of data to make decisions. Districts will receive access to IRRE s proprietary web based surveys and be able to access reports immediately after the survey windows have closed. Available reports provide item and construct level results disaggregated at multiple organizational levels. Capture, Analysis and Presentation of Historical Trends. District research and technology staff work closely with IRRE s Research and Measurement group to secure historical achievement, attendance, discipline, and graduation data to construct long term trends and to set a baseline before new improvement strategies are implemented. Districts can then track the impact of data driven decision making on instructional practice and ultimately on student outcomes. DWM Network Meetings. Participating districts may elect to participate in oneday network meetings held each year. Participants will have the opportunity to learn from national thought leaders and practitioners successfully implementing data driven dialogue in their districts and schools. Past DWM Network Meetings have included speakers from the Data Quality Campaign and invited guests from the Bill and Melinda Gates Foundation, the Michael & Susan Dell Foundation, and others. Attendees will participate in: Interactive simulations and demonstrations of data driven dialogue using actual student outcome reports from partner districts; and IRRE,

8 Opportunities for cross site discussions of structures and processes to be implemented to strengthen data driven dialogue around newly implemented student outcome reports. For more information please contact: William Moore, Ph. D. Director, Research and Measurement Services IRRE,