Establishing Effec/ve Data Governance

Transcription

1 Establishing Effec/ve Data Governance Data Quality Council Much of what I say is taken from 2 publica/ons put out by the na/onal Center for Educa/on Sta/s/cs and Na/onal Forum on Educa/on Sta/s/cs

2 Forum Guide to Building a Culture of Quality Data Forum Guide to Building a Culture of Quality Data A School & District Resource This gives a good overview of the components of high- quality data and the factors that go into crea/ng a culture in an LEA that can consistently produce high- quality data.

3 Forum Curriculum For Improving Educa/on Data Forum Curriculum for Improving Education Data A Resource for Local Education Agencies And the Curriculum is based on the Building a Culture book. It has lesson- plan outlines that can be used for district professional development of staff regarding data quality issues and prac/ces and forms to guide the planning and communica/on it takes to consistently produce high- quality data. Both free. Can order up to 5 copies at a /me. I will have copies, 1 per district, at our next me/ng. probably only need 1 or 3 of the green one, the curriculum, but want mul/ple copies of building a culture of quality data

4 Ordering Books Building a Culture of Quality Data hrp://nces.ed.gov/pubsearch/pubsinfo.asp?pubid= Curriculum for Improving Educa/on Data hrp://nces.ed.gov/pubsearch/pubsinfo.asp?pubid= Here are links or you can Google the /tles. On these pages you can also download PDFs of the books so you can print out some of the forms and concept sheets

5 Components of Quality Data Accuracy Timeliness Security U/lity Here is how NCES defines quality data. Some defini/ons include consistency as a component, but this is a reasonable defini/on. For the most part, people think of accuracy as the sole component: data that are accurate are high- quality. PDE and the PIMS ini/a/ve emphasize accuracy and implicitly /meliness, as there are deadlines for submi_ng data. But as you progress in improving the data your district su

6 Impact of Quality Data More effec/ve decision making Achieving AYP Program Funding Educator Effec/veness School Performance Profile Reduced /me on PIMS To consistently produce high- quality data, district need to create a culture that recognizes the importance of quality data and puts in place the resources, procedures, and infrastructure that makes producing such data CONSISTENTLY both possible and a priority. Here is what that takes

7 Data Cycle We will start with a general understanding of the data produc/on cycle. Please note that this cycle does not apply only to data submired to PIMS. It applies to any data that is used for making decisions or conveying informa/on to the public or repor/ng to state and federal authori/es. A data event is anything that generates new data or changes to data kindergarten registra/on; assessments; students who are late, tardy or disciplined; teachers who go on parental leave; high school gradua/on. A data event requires that data be entered into some sort of data system, whether that is the SIS, a special educa/on system, DIBELS, an data management tool, etc. At some point, before the data is submired to a higher power or used for decision making, someone will need to make sure the data is accurate, which means outpu_ng or repor/ng the data; inspec/ng the data for errors; correc/ng the data in the source system; and outpu_ng it again. So that Enter, Output, and Correc/on can be a loop of mul/ple itera/ons within a single data cycle. Eventually, the now highly accurate data gets submired to PIMS or given to the Curriculum Development commiree or used by a teacher to plan lessons. And the cycle begins again.

8 Data event Entering data Data output/reports Correc/ng data Data Cycle Data Production Submi_ng data to decision makers/pims Repeat So here is the same cycle presented as a bullet list. Note that the first 4 items on that list are what can be thought of as Data Produc/on. It is the deliberateness of the procedures and prac/ces of that produc/on that determines the quality of district data. Once the data is submired to PIMS or decision makers, the data enters into data consump/on or data applica/on. And note that different people are probably involved in different steps in the data produc/on part of the cycle; different people have knowledge about the details and processes of of each of these steps. Furthermore, who those people are, par/cularly with regard to data events and correc/ng data, will change depending on what data we are talking about: the special ed coordinator knows when an event affects the special ed data; the teacher knows when his or her enrollment data is inaccurate; etc.

9 Implica/ons of the Data Cycle Communica/on Planning Pa/ence Because a number of people are involved in producing all of a district s data, keeping the quality of that data consistently high requires good communica/on among all the players. In our districts, when people have mul/ple responsibili/es, establishing parerns and procedures for good communica/on requires delibera/on and planning. And most of all, consistently producing high- quality data requires pa/ence: implemen/ng any plan these days takes /me. So we are going to start today with some ac/vi/es to help support the planning and communica/on that data governance requires. The pa/ence you are going to have to come up with on your own. First, let s look at the factors in a district that affect the quality of its data.

10 Factors Affec/ng Data Quality Policies and Regula/ons Standards and Guidelines Training and Professional Development Data Entry Environment Technology Timelines and Calendars This is a lot to contemplate developing. It is not going to happen all at once, it cannot be the responsibility of only one or two people. It requires an understanding of the needs and a commitment of the necessary resources by everyone from the borom of the organiza/on chart to the top. Iden/fying who those people are in your LEA is one of our tasks today. For one thing, no one person knows enough: the issues regarding when data would most effec/vely be presented to make instruc/onal decisions is not the same for every building; issues regarding security and privacy of data are different for different popula/ons of kids; different data systems may require different data formats; etc. For another, producing consistently high- quality data means a district or school must, as an enterprise, devote /me and staffing to the ac/vi/es required. The factors listed here that affect the quality of a schools data are the factors that comprise Data Governance

11 Data Governance Policies and Regula/ons Standards and Guidelines Training and Professional Development Data Entry Environment Technology Timelines and Calendars Making the decisions, assigning the specific responsibili/es, and se_ng up the processes that lead to all these things is the job of a school s or district s data governance structure, which requires a lot of players. And because data produc/on and data governance involves a lot of players, building a district culture that understands and values the produc/on of high- quality data happens incrementally over a number of year, and involves almost everyone in a district in one way or another. Everyone is affected if the data is wrong or late or if security breaks down and personal data leaks to the public. What we are going to start with first is iden/fying the people in each district who need to be involved in the development of the policies, procedures, guidelines, standards, and /melines required for good data governance. Those people should ideally include

12 Who Has a Role in Data Governance? Superintendent Teachers Principals Coordinators/Supervisors Central and Building Office Staff Technology Staff Board Members prery much everyone in the district. But you need to get specific for your LEA, so we have provided a spreadsheet to help you begin this planning. Here is a handout

13 Iden/fying the DQ Team that looks something like this. On the lej side are the various PIMS data submissions. Along the top are various roles and responsibili/es with regard to producing, collec/ng, correc/ng, submi_ng, and understanding that data.

14 Iden/fying the DQ Team In each row, name person responsible for each role May be same person in mul/ple cells, an en/re row, an en/re column Should end up with mul/ple names in table Work individually or as district team 15 minutes For the next 15 minutes or so, for each of the data submissions on the lej, enter in each column the name of the person with each of the responsibili/es labeled across the top. You may find that for some responsibili/es you are entering the same person in each of the data submissions. Perhaps for some submissions the same person has mul/ple roles. That s fine: just be sure that each cell is filled. You may not get this all done today, which fine, but at some point you need to. And you may get done in less /me than 15 minutes. If so, have some discussion at your table or reflect for yourself what processes or procedures are in place or should be in place to be sure that everyone on this list understands their responsibili/es and fulfills the in a /mely and effec/ve way.

15 Factors Affec/ng Data Quality Policies and Regula/ons Standards and Guidelines Training and Professional Development Data Entry Environment Technology Timelines and Calendars The next ac/vity has to do with Timeliness as a component of data quality

16 Developing a Data Calendar Work backwards from decision, publica/on, submission, and other deadlines Understand the processes of data crea/on, collec/on, assessment, and correc/on May have different deadlines, therefore calendars, for different buildings In developing a data calendar or a /meline for knowing what steps need to be taken when, the process is one of working backwards from the deadline. It also requires understanding all the processes that go into entering, collec/ng, evalua/ng, and correc/ng data, so that the amount of /me allowed for each step as you develop the calendar is realis/c. And it is important to understand that although ul/mately one wants a single calendar that covers all data collec/ons, reports and submissions in the LEA, the simpler way to proceed will be to develop separate calendars for each data ini/a/ve: one for PIMS, one for instruc/onal or curricular decisions, one for the annual report, etc.

17 PIMS Data Submission Planning This form is specifically geared to developing a data calendar regarding PIMS data collec/on and submissions. It will be the basis of our ac/vity.

18 PIMS Data Submission Planning Start with the Submission Deadline column Work backwards, allowing a realis/c amount of /me for each step This is draj: can only finalize in collabora/on with people iden/fied in the DQ team ac/vity Work as district team 15 minutes So here is what you are going to do

19 Planning for Curricular Decisionmaking Work backwards from deadlines for decisions Allow /me for data inspec/on, correc/ng Allow /me for internal processes May involve different people than PIMS process Ul/mately combine with PIMS data calendar Work on this in district with en/re team a way to begin professional development I told you that different data- based efforts might profitably begin with different data calendars. Our next ac/vity will be a start on crea/ng one for curricular decision making. But please no/ce that the process for developing the calendar is a lot like the one for PIMS. The difference will be that the processes for correc/ng data may not involve the Data Correc/on engine; that there may be LEA- internal processes that have to be taken into account. And the people involved may not be completely the same. Ul/mately, you want to have a single data calendar for whoever is the data coordinator of the district to have on order to keep all processes on track. But for today

20 Planning for Curricular Decisionmaking Think of this form as homework, something to work with back at your LEA to begin the process of thinking about developing other data calendars. Once you have filled out this form, use the column headings on the PIMS Data Calendar Planning form as a basis for developing a /meline for the kinds of internal data processes your LEA needs to begin to put in place to make effec/ve the applica/on of and decision making guided by all the data you produce.