Texas Education Agency

Transcription

1 IBM Global Business Services January 27, 29 Texas Education Agency Texas Education Agency 171 N. Congress Avenue Austin, TX TEA Data Collection, Analysis and Reporting Systems Investigation (TDCARSI) Issues and Recommendations

2 TEA Executive Sponsor Deliverable Sign-off Page Adam Jones Approver Signature Date John Cox Rick Goldgar Roger Waak Page 2 of 85

3 Table of Contents 1 EXECUTIVE SUMMARY SUMMARY OVERVIEW SUMMARY BACKGROUND SUMMARY METHOD OF INVESTIGATION STAKEHOLDER ISSUES AND SUMMARY FINDINGS SUMMARY RECOMMENDATIONS TDCARSI BACKGROUND AND CONTEXT STAKEHOLDERS OF THE TEXAS EDUCATIONAL SYSTEM TEA S STRATEGIC MISSION AND GOALS AND THE ROLE OF AN INFORMATION MANAGEMENT SYSTEM TDCARSI PROJECT SCOPE PROCESS IMPROVEMENT AND IMPACT ANALYSIS OVERVIEW OPPORTUNITIES FOR IMPROVEMENTS Best Practices from a Survey of K-12 Agencies in Peer States Strategic and Tactical Information and Data Needs CURRENT STAKEHOLDER ISSUES Issue 1: Inability of Current System to Deliver Data that is Timely, Relevant, and Actionable Issue 2: Current Data Collection Model Imposes Significant Burden on Local Districts Issue 3: Lack of Statewide Standards for ISD Data Systems Issue 4: Difficult to Integrate Student Data across Data Sources Due to Limited Use of the Unique Texas Student Identifier Issue 5: Cumbersome and Inefficient Reporting and Analysis Capabilities Issue 6: Inability to Easily Access Comprehensive Longitudinal Data Issue 7: Lack of Agency-wide Standards for Data Collection and Storage Issue 8: Lack of a Single TEA Point of Contact for all Data Collection to Resolve Issues SUMMARY OF DATA MANAGEMENT ISSUES RECOMMENDATIONS AND IMPACT ANALYSIS Recommendation #1: Streamed Data Collection Model of Granular Student Data into an Operational Data Store (ODS) Recommendation #2: District and TEA Validated and Aggregated Data Loaded into a Data Warehouse to Support Program Analysis and Reporting Recommendation #3: Business Intelligence and Reporting Tools to Support End User Analysis and Reporting Recommendation #4: Unique State-wide Texas Student Identifier (TSID) embedded in the Collection and Integration of the Data Recommendation #5: Use of a Unique Teacher Identifier (UTI) and Creation of a Classroom Link Recommendation #6: Creation of a Voluntary State Sponsored Student Information System (SIS) Recommendation #7: Establishment of Enterprise-wide Data Governance Strategy and Board Recommendation #8: Establish an TEA Enterprise Data Management Office (EDMO) Recommendation #9: Establishment of Enterprise-wide Data Standards Summary of Recommendations and the Proposed Functional Solution SOLUTION REQUIREMENTS AND ARCHITECTURE OVERVIEW TASK 1 SOLUTION REQUIREMENTS FOR DATA COLLECTION, REPORTING AND ANALYSIS PROPOSED ARCHITECTURE Data Submission Inputs Student ID System / Texas Student Identifier (TSID) Teacher ID System / Unique Teacher Identifier Classroom Link Integration Hub...7 Page 3 of 85

4 4.3.5 Business Rules Engine / ETL Engine Operational Data Store (ODS) Aggregated Data Warehouse (ADW) Reports Analytics Portal Data Users Web Services Summary of Recommendation Architecture...74 Page 4 of 85

5 Figures and Tables 1-2. TDCARSI Stakeholder Issues Key Stakeholders in the Texas Educational System Depiction of Data Usage by Groups of Texas Education Stakeholders Conceptual Overview of the Current Texas Educational Data Collection and Reporting Environment Proposed Solution Architecture...68 Page 5 of 85

6 1 Executive Summary 1.1 Summary Overview Texas is a state which has been on the forefront of delivering education services to a diverse student population. The Texas Education Agency (TEA) recognized early that data was integral to understanding the types of programs that need to be in place and the cost of delivering these programs. As a result, the TEA was an early adopter in developing a student record level reporting system, called the Public Education Information Management System (PEIMS), designed to collect student and school level data from school districts to support program analysis. The student record level data collection and the accountability program it supports became a model for elements of the No Child Left Behind (NCLB) program, which has driven other states to move toward a more granular level of accountability through data. While Texas has been a leader in education information management, its current systems and processes do not efficiently or effectively support the increasing demands for timely, transparent, accessible and actionable data. The TEA s 28 Strategic Plan recognizes this need for the central role of data to support its vision of delivering educational services to the school districts. In order to support its strategic planning efforts, the TEA secured funding from the Michael and Susan Dell Foundation (MSDF) and initiated, through a contract with IBM, a study called the Texas Data Collection, Analysis and Reporting Systems Investigation (TDCARSI). The goal of TDCARSI is to create a roadmap for developing an enhanced, statewide K-12 data capability to keep Texas at the forefront of standards-based accountability. This vision was developed with extensive input from stakeholders as well as significant research into best practices, including focus groups, interviews and surveys with a wide variety of stakeholders and five peer states This resulting report details a practical and powerful statewide data solution that will increase the availability of transparent, timely, and actionable educational data while at the same time decreasing the cost and burden of data collection to districts and the state. The solution will provide appropriate access to all stakeholders while ensuring compliance with the Family Educational Rights and Privacy Act (FERPA). 1.2 Summary Background The TEA is responsible for the following efforts for K-12 public and charter schools: Administering a data collection system on public school students, staff, and finances Administering the statewide assessment program Monitoring for compliance with federal guidelines Rating school districts under the statewide accountability system Serving as a fiscal agent for the distribution of state and federal funds Managing the textbook adoption process Overseeing development of the statewide curriculum, and Operating research and information programs Virtually every effort by the TEA requires that the TEA collect, analyze and report data pertaining to the public schools. There are over 4.6 million K-12 children in Texas public schools and over 1,2 independent school districts (ISDs) and charter schools, with very diverse demographics, ranging from large urban areas such as Dallas and Houston, to very large geographic areas with very small student populations. Providing technical assistance to the ISDs are 2 Educational Service Centers (ESCs) which are located throughout the state. Central to its role in supporting this educational environment, the TEA provides and manages funding (payments, grants, entitlements) in excess of $2 billion per year to school districts. PEIMS, developed in the late 198s, is the primary mechanism the TEA uses for the collection of compliance data from school districts. PEIMS was an early adoption of student record level reporting at a Page 6 of 85

7 state level. The data is collected four times a year through large file transfers (millions of records for some of the larger districts). The data collected through PEIMS is intended to satisfy many of the state and federal accountability reporting requirements. Over the past decade, the TEA began to drive program initiatives that were more school district and student centric. More operational and insightful data is required to support these program initiatives. In response, the TEA and its stakeholders began forging a dialog around the PEIMS environment and evaluating whether it was robust enough to support a more student and school district centric program approach. Recognizing these challenges, the Texas Legislature funded in 22 an in-depth, third party analysis of the PEIMS processes. This analysis stated a number of challenges in the existing data collection: Aged PEIMS system and processes - The existing system is still primarily a batch collection system; the process for reviewing and approving data elements is slow and difficult, the efforts to support the system are labor intensive and expensive for the state. Untimely reporting of and access to data - The data is collected infrequently (many elements only once a year) and due to the time it takes to analyze and report the data, new reports often represent data that is at least nine months old. The data is not available to stakeholders in an easy to obtain, easy to manipulate fashion. Data reporting redundancies - Due to the inefficiencies of PEIMS, many other data collections have evolved at TEA and some of these overlap with PEIMS. Data quality needs improvement - The districts must perform their own aggregation, business rules and analysis to provide the data as defined in the PEIMS data collections. Due to the complexities of creating the required data locally at the districts, these efforts are prone to error. Labor intensive reporting burden to districts - The requirement for districts to create the required data from their source systems creates an expensive and time consuming effort for both the districts and for the TEA, who must monitor their submissions for quality and completeness. Barriers to data sharing - Current stakeholders find it difficult to get data from the TEA. Many data requests require significant programming efforts by the TEA staff and may take weeks to provide. This is true for both internal TEA stakeholders who want to use the data and other, external stakeholders. This 22 investigation provided a number of recommendations to address these issues. While a few process improvements and recommendations have been implemented, a lack of state funds in 22 meant that the state could not support major system or process changes that would streamline collection, reduce local burden or facilitate broader access to data. Therefore, many of the same challenges that were present in 22 persist today. This TDCARSI study allows TEA to re-evaluate the challenges and needs of various stakeholders and determine their data-related priorities and information needs. This study also allows TEA to investigate more recent practices and technologies that can help the TEA better support its strategic initiatives. 1.3 Summary Method of Investigation TDCARSI involved the following efforts: Focus groups and interviews with over 25 districts and charters, 18 Educational Service Centers (ESCs), legislators and other government officials, researchers, and internal TEA staff Surveys of five other states education systems and processes (California, Florida, Illinois, North Carolina, and Ohio) An analysis of the existing and proposed requirements and processes currently in use for state and local data collection and reporting A vendor forum for Student Information Systems with 18 attending vendors from over a list of 1 invited A voluntary sample survey of PEIMS costs to districts with over 2 district responders Multiple meetings with an external study group consisting of other key stakeholders. Page 7 of 85

8 As a result of these activities, the IBM Team worked with the TEA and MSDF to document findings and develop recommendations for a more efficient and productive data collection system, useful for all stakeholders. 1.4 Stakeholder Issues and Summary Findings Based on background research and interviews with the educational stakeholders in the state, the IBM Team identified the eight major issues highlighted in Figure 1-2 that are further detailed in Section 3. These issues arise from four practices associated with the current state education information management system: The TEA data management environment primarily enables meeting state and federal compliance reporting requirements. This approach limits the actionable data being made available to the various other education stakeholders. The TEA makes data available to stakeholders via its portal. It also provides data to researchers in a very controlled, FERPA compliant environment. However, as education programs are evolving, program evaluators and local school leaders are seeking more useful and timely data to evaluate and make decisions about instructional and program effectiveness. (Issue #3) At the local level, Texas school districts struggle to maintain a comprehensive set of data systems that can meet the needs of state reporting. Texas is a state with local school districts that cherish their autonomy. However, a large number of districts struggle to keep up with the staffing, training, infrastructure and applications needed to support school district operations and simultaneously meet state data collection and reporting needs. One reason is that the current model does not 1. Inability of current system to deliver data that is timely, relevant, and actionable 2. Current data collection model imposes significant burden on local districts 3. Lack of statewide standards for ISD data systems 4. Difficult to integrate student data across data sources due to limited use of the unique Texas Student Identifier 5. Cumbersome and inefficient reporting and analysis capabilities 6. Inability to easily access comprehensive longitudinal data 7. Lack of agency-wide standards for data collection and storage 8. Lack of a single TEA point of contact for all data collection to resolve issues Figure 1-1 TDCARSI Stakeholder Issues align with, nor is it easily supported by local data systems. The vast majority of Texas school districts serve fewer than 5, students and many of these districts struggle with budgets and staff to support even basic local information technology efforts. The complexity of the current state reporting system puts demands on local administrations that are not balanced by value back to those districts. (Issue #5) The TEA data management environment has evolved into a data collection environment driven by multiple, often isolated (with regard to data management), organizations within the TEA. While PEIMS serves as the backbone of the TEA data collection environment, a number of departments have each developed their own data collection mechanisms that have evolved in response to federal and state regulatory changes as well as program changes. The departments efforts are likely a response to the lack of timeliness in getting other data elements included in the PEIMS collection or a lack of timeliness in the periods of PEIMS collections. Nonetheless, these, multiple and separate data collections confuse the data providers and result in multiple systems and multiple TEA data owners that each school district must support. The complexity of supporting multiple collections is exacerbated by a lack of data standards at the state level. Because present data collections models rely on snapshot data, the ISDs must sometimes submit similar data sets multiple times to the Agency during the year. This model places undue burden on the school districts. Likewise, its current decentralized data collection paradigm does not allow for a central point of contact which an ISD and ESCs can call to resolve data or policy related issues. (Issues #2, #7, and #8). Page 8 of 85

9 There are currently significant challenges in creating a linked student record which can be used for timely analysis and decision making. While the TEA conducts student level data collections, the key data (e.g., statewide unique ID) needed to link a student record across demographic information and performance outcomes is not consistently used by the TEA and districts in a way that allows for an integrated student record. Specifically, TEA creates a Person Identification Database (PID) number for each student in the state. However, this number is used internally in TEA to link and track students longitudinally. The usage of the PID to link student records is not available to school districts or other research organizations. As a result in responding to requests for longitudinally linked student data, TEA staff, researchers, and districts spend an inordinate amount of time linking and resolving student information across subject areas and across time in order to create a meaningful data set. This often results in a significant time delay between the request for the analysis and the delivery of a meaningful data set upon which decisions can be made, if the correlation can be made at all. (Issues #1, #4, and #6) In addition, to the background research and facilitated sessions with stakeholders, the IBM team analyzed the responses on practices from five other state departments of education, including: California, Florida, Illinois, North Carolina, and Ohio. These states were chosen as having best practices in place or populations and challenges similar to those in Texas. IBM used this information to identify suggested directions for developing an information management system for the TEA and its stakeholders. This effort yielded the following findings: All the surveyed states have in place or are in the process of developing a statewide data governance structure. A statewide governance structure dictates the data elements and their meaning for all data collected by the state. A statewide data architecture or data dictionary is a blueprint for the data needed across the agency to support decision making needs. Currently, Texas does not have a statewide governance structure. Though some Texas education data is well defined and some standards are published, this is not consistent for all data. All the surveyed states intend to produce a more flexible and responsive data collection and reporting environment while minimizing burdens to the school districts. All the surveyed states desire to collect more granular data on a more frequent basis to meet ongoing program evaluation needs (as distinct from compliance needs), and they are all investigating how to develop a streamlined data collection method to reduce the burden to the school districts and provide data back to districts to support classroom level instruction. For example, Illinois is now at the point that their student enrollments and associated demographics and program indicators are continually updated throughout the year so that their state student system has become the authoritative source for student data. This has enabled the state to sunset departmental legacy data collections. In addition, Illinois has been able to have all its assessment vendors use the state s student IDs and demographics to populate the pre-code ID label for assessments, thereby streamlining Illinois efforts to integrate assessment results and making available a linked longitudinal student record for its school districts to view. Other surveyed states, as Ohio and California, are also moving toward a more real time data collection framework to support data analysis and reporting. All the surveyed states are moving toward embedding a unique student ID and teacher ID into the data collection and reporting process. While TEA was originally the leader in implementing student record level reporting, more recently, the states surveyed have extended their record level reporting capabilities further. They have done this by embedding the state unique student in the local data systems (typically by having it as part of their local means of student identification, either as the primary local identifier or more typically, as an additional, but mandatory, field attached to the local student record). They use this identifier to link student demographics, program participation, and student performance data, thereby making the data more cohesive, timely and available for the needs of multiple stakeholder groups. Several states are also taking actions to assign a unique state teacher ID and embed it into staff local level data systems in order to support research that is centered on teacher program investments and student outcomes. The teacher identifier also assists in the assessment and reporting for NCLB s Highly Qualified Teacher requirements. All surveyed states are in the process of developing more flexible data reporting environments for their multiple stakeholder groups. In developing their data warehouse and reporting strategies, these states are all moving toward a self service reporting environment where Page 9 of 85

10 stakeholders with privileges based on their roles see the appropriate level of data needed to support their decision making needs. This data includes local, raw data used by local entities for their own daily needs and higher level, extracted and aggregated data for use by both local and state stakeholders. None of the surveyed states currently have a full pre-k through 16 data system in place. While the surveyed states are at various stages in their efforts to support pre-k through 16 systems, they all recognize that the development of a data warehouse based on longitudinally linked student records will best allow for the connection of higher education data. One surveyed state offers a statewide student information management system that is used by school districts to manage their schools and students. States with strong local school district autonomy have not deployed a mandatory single statewide student information system to their school districts. One state, North Carolina has a stronger central role by organizing school districts at the county level and funds school districts based on head count. It has defined district level processes for a state course catalogue, state transcript, and school activity reporting, as examples. To support this type of school district management, North Carolina has deployed a centralized student information system package that the local school districts can implement to support both their operations and state reporting. Ohio was another state that had tried in the 199s to implement one statewide student information system for the school districts but found that it did not have the governance structure to meet the operational needs of school district with local autonomy and unique practices. Ohio instead has moved toward a model that emphasizes data standards and a set of best practice student information systems for local school districts. Similarly, other states have considered offering an optional student information system or best practices student information system guidelines as a service to school districts seeking additional support. 1.5 Summary Recommendations Based on the findings of this investigation, the TEA and its stakeholders should consider transforming the current information management environment from a point-in-time data collection and reporting environment, as depicted in Figure 1-2, to a more timely and dynamic environment. The proposed environment promotes a new model whereby raw granular operational data is delivered from the districts local source systems into an operational data store. On a periodic basis data snapshots are taken and the snapshot data is aggregated, transformed and loaded into an aggregated data warehouse. The operational data store is used by the districts for their own operational analysis and reporting needs. The aggregated data warehouse provides the repository for compliance reporting and research needs. The aggregated data warehouse data would be available to Educational Research Centers (ERC). The proposed environment should result in: The creation of a data collection method that is less burdensome to the school districts, ideally through automated delivery of raw data from multiple local source data systems and moving sophisticated derivations, calculations and aggregation of data, currently burdensome to the districts, to a central/tea environment that ISDs do not need to manage. This method would replace the current PEIMS and EDIT+ applications as well as the need for certain specialized applications now utilized to collect and process non-peims data. The aggregated data warehouse that supports the longitudinal tracking of student data without expensive and time consuming manual intervention to join records from disparate sources. Page 1 of 85

11 4 Cyclical Submissions Figure 1-2: Conceptual Overview of the Current Education Information Data Collection, Integration, and Reporting System in Texas Figure 1-3: Conceptual Overview of the Recommended Data Collection, Integration, and Reporting System for Texas Page 11 of 85

12 A reporting environment that enables relevant data to be used by different stakeholders for their own needs, in a manner that is compliant with the Family Education Rights and Privacy Act (FERPA). An analytics environment that enables appropriate stakeholders to gain access to the data in a FERPA compliant manner in order to research, benchmark, and take actions to improve the teaching and learning environment for their students in a timely and proactive fashion. This broadened capacity will assist districts with immediate instruction and classroom support needs, as well as state-level evaluation of the policies and programs impacting graduation rates and college readiness. Figure 1-3 displays this proposed information management environment. To achieve this proposed information management environment, IBM recommends that TEA include the following elements in its information system architecture: Streamed data collection model of granular student data into an Operational Data Store (ODS): Data generated by source systems (student data, financial data, etc.) will be streamed on a regular and recurring basis from the ISD source applications to an ODS supporting districts needs and serviced by the TEA. The ODS represents actual raw operational data used by the districts for their own reporting, analysis and local actions. Though the TEA will provide service and support for the ODS, the TEA would not use the ODS for its own analysis unless authorized by districts through mutual agreements, The ODS would exist for the operational needs of the districts and not for TEA use. This approach provides immediate value to the districts and shared cost savings through state level hosting and support. It also helps assure a reasonable level of standardization of data across local districts in the state. District and TEA validated and aggregated data loaded into a data warehouse to support program analysis and reporting: The aggregated data warehouse (ADW) would consist of data used by TEA to satisfy its reporting and analysis mandates. The TEA would populate the ADW through automated periodic extracts or snapshots of data for specific compliance and accountability reporting purposes, which would be validated by school districts and TEA through a workflow and approval process. These extracts will take the raw data in the ODS and perform extraction, transformations (including business rules, calculations and aggregations) and loads into the ADW. This process alleviates the need for districts to perform these complex and time consuming actions and allows the transparent and cost effective creation of the state reporting data required by the TEA. The rules and processes for these extractions will be published. The resulting data will be counts, derivations and aggregations of data based on the ODS data. Individual and non-personally identifiable data will also reside in the ADW as appropriate. The ODS snapshots and data warehouse reconciliation reports will be made available to the districts to compare the snapshots of their ODS data with the resulting derived ADW contents. Business intelligence and reporting tools to support end user analysis and reporting: Analysis and reporting tools should be made available for the end users of the ODS and a set of tools should be made available for the ADW. Accessing the ODS, districts will use the reporting and analysis technology, for authorized, role-based access to the data. This technology will provide user-friendly, self service, FERPA compliant, report functionality as well as robust analysis and reporting capabilities. This solution will also include readily-available, standard reports that provide insight for common analysis needs. At the ADW, TEA and authorized researchers will have access to reporting and analysis technology that is also role-based and FERPA compliant. Business Intelligence, analysis and reporting tools will need to support the statistical analysis needs of both sophisticated and robust users and those with little or no programming skills. Unique statewide Texas Student Identifier (TSID) embedded in the collection and integration of the data: The state of Texas currently has a unique student identification number called the student Person Identification Database (PID), which has been in use for more than 2 years. However, the maintenance and traceability of these numbers are kept centrally at the TEA and not used by local districts for any meaningful efforts outside of their role in supporting TEA reporting. To streamline the linkage of student data across source systems, the TSID should be managed by the state, but captured as part of the student s local record and maintained locally in the ISD source applications. This will allow for greater mobility tracking and graduation/drop-out tracking, more efficient data submission, as well as consistent local and statewide longitudinal analysis within K-12. Placing the Page 12 of 85

13 state TSID in the local student information system student record helps assure that the unique student is locally identified. This becomes especially important in a state with high student mobility. This also alleviates much of the need to use other, more sensitive student identifiers (e.g. SSN) that may be subject to security issues. Once linked, student records have been created through the TSID, student personally identifiable information can then be masked and then made available in a manner compliant with FERPA to Texas Higher Education Coordinating Board and the Texas Workforce Commission to support the analysis of college and workforce readiness. Use of a Unique Teacher Identifier (UTI) and creation of a classroom link that can better support the research and analysis of teacher and classroom program investments. Similar to the TSID, the assignment of a UTI at the state and then embedded in staff and course level data collections will support the type of analysis related to teacher program investments and effectiveness in the classroom This will provide educator-level data from existing source systems, including credentials, post-secondary education, professional development, and employment data, so that information from these systems can be longitudinally linked to classroom assignment and student performance. This recommendation has the same benefits for teacher tracking and linkage as the TSID has for student tracking. Creation of a voluntary state sponsored Student Information System (SIS) that helps school districts save costs and resources associated with student data management: More than 8 percent of the districts in Texas each have fewer than 5, students. The vast majority of these districts do not have the budget or available staff to support sophisticated information technology departments, yet most must currently purchase operational systems that support their daily student accounting, staffing and financial operations needs. They must then typically pay for expensive (and often delayed or non-compliant) modifications to those systems to support the specific state reporting needs that are not part of their daily operations. A cost effective alternative to this current situation is for the state to provide and maintain a standard system that any district can optionally use. Through solicitation of a state hosted solution, a state sponsored SIS would be made available for voluntary use. A standard student system would provide a minimum level of operational and maintenance support that is safe and secure, as well as built in extractions to support state reporting. Moreover, it will provide a more cost efficient way to support changes to data standards. There would be no penalty for districts that do not participate, and there will be shared cost savings for all districts that do participate. This recommendation does not include support for local financial or human resources systems as these tend to be more unique to the local district operations, but, the state sponsored SIS would support some level of teacher information (either through direct support or by data import) to assure linkage between the teacher and classroom information and student information. The above recommendations represent the technical building blocks of an information management strategy that delivers the type of information needed to support the various stakeholders in the Texas educational system. However, this proposed information management strategy will not be transformative if it is not supported by a data governance structure that identifies the data needed to support a streamlined data collection and reporting environment that is FERPA compliant. Failure to do so will result in an environment that will continue have the same types of challenges as the current environment. Therefore IBM recommends the creation of data governance strategy which sets policies, rules and processes that guide the use, development and protection of information. The data governance strategy should include the following: Establishment of an Enterprise-wide Data Governance Strategy and Board: The governance organization (Data Governance Board) should include representatives from all pertinent stakeholder groups (including various size districts, legislators, researchers and TEA program staff); however, the management of the governance organization should be independent of any specific data users, in order to limit program area bias and support fair evaluation of the policies, rules and processes. The Data Governance Board should address the policies, people, processes, and technologies required to develop and enforce standards regarding educational data. Establishment of a TEA Enterprise Data Management Office (EDMO): This administrative unit of the TEA would be responsible for implementing and monitoring the policies, standards and procedures developed by the Data Governance Board and related committees. The EDMO would Page 13 of 85

14 provide 1) leadership within TEA regarding the data it collects and stores; 2) integration between internal and external data users and the ITS Division and Project Management Office that develop and maintain data management applications; and 3) a centralized unit that responds to internal and external data questions and information requests. As with the Data Governance Board, the management of the EDMO should be independent of any specific data users, in order to limit program area bias and support fair evaluation of the policies, rules and processes. The senior manager of the EDMO may act as the chairman of the DGB, thereby providing the linkage between the policy making authority (the DGB, consisting of representatives from both within and outside of the TEA) and the EDMO implementation and support authority residing within TEA. Establishment of an Enterprise-wide Data Standards: Once in place the DGB and EDMO should work toward the development of a comprehensive set of data standards for all school data collected, stored, reported and shared within the agency and between the multiple stakeholders. State standards for education data will promote consistent meaning and usage across districts and the TEA. These consistent data definitions will support a common data dictionary that will be made available to all ISDs, state agencies and other authorized stakeholders. These recommendations, which are summarized in Figure 1-4, are consistent with the best practices advocated by National Data Quality Campaign. In addition, the proposed solution will result in significant cost savings at the state and local levels. Preliminary PEIMS LEA cost analysis shows that districts with fewer than 5, students spend, on average, between $48 and $189 per student to aggregate and submit their data. The one time costs of the proposed solution will be offset by the cost savings associated with the current environment. District size (number of students) Number of Districts Responding Range of students in the districts Range of costs per student in the districts Average PEIMS cost per Pupil Total Average Cost per District Total no. districts (from 27 snapshot report) Projected District Costs for the entire state (total districts average cost per district) min max min max <1, $ 11 $ 562 $ $16, $76,591, , - 4, ,336 4,56 $ 14 $ 117 $48.54 $156, $53,254,88. 5, - 49, ,773 29,696 $ 19 $ 49 $36.51 $862, $13,21, ,+ 2 62, ,534 $ 18 $ 37 $32.35 $4,233, $63,52,72.5 Projected District Costs for PEIMS Support 1,222 $323,559,68.99 It is clear that there are economies of scale. The largest districts have a much lower per student cost for supporting PEIMS than the smallest districts which comprise over 8% of the districts in the state, but, regardless of economies of scale, the current cost for supporting PEIMS reporting is enormous. The largest single district that reported in this survey (in the Houston area) spends over $7.3M on PEIMS support; for a system that the districts believe provide them little or no immediate value. Also note that these costs do not include the TEA costs for administrating and supporting PEIMS or any of the district or TEA costs for other state data collections. Executive sponsorship and involvement of high-level management is critical, and the TEA will need to work with ISDs, ESCs, and other key stakeholders to facilitate organizational and process change management that support the new model. Communication plans should be developed to inform constituents of major phases and initiatives related to the new collection and reporting system. Likewise, mechanisms for stakeholder input and feedback should be developed to ensure that the solution is meeting user expectations. And, as each project progresses, the team should seek to involve department management support to communicate project messages, prepare the workforce for change, and accomplish other activities related to the transition to the new system and related business processes. Page 14 of 85

15 In summary, these recommendations are intended to bring the data collection, analysis and reporting capabilities for Texas in line with the expectations of 21 st century business processes. These recommendations fundamentally change how education data in Texas is collected, maintained, accessed, and reported. The proposed information management strategy facilitates the use of data by local school districts and other end users for operational and performance improvement purposes, in this case, student achievement. Second, it supports the paradigm shift occurring within the TEA to provide more school district and student centric program support. Similarly, this information management strategy helps ensure a consistency in statewide data standards to support accountability programs as well as the evaluation of district and student centric program approaches. The ability to deliver this type of information management platform will mean that districts can quickly learn about and respond to trends as they occur; that the results of funding for grants and entitlements can be understood during their implementation, in time to truly affect the future of the students these monies are meant to help; that little successes, hiding in small districts or a few schools, can be discovered and embraced to help others; and that legislation and education policy will become a more proactive and dynamic cooperation with districts, yielding more immediate results. 2 TDCARSI Background and Context Like many states, Texas is under increased pressure to improve its educational outcomes for student performance, college and workforce readiness. Many factors contribute to the challenges facing the education system in Texas, including: diverse demographics; high student mobility; a large English learner population; and increasingly rigorous academic standards for graduation. The TEA has recognized these 1. Streamed data collection model of disaggregated student data into an Operational Data Store (ODS) 2. District and TEA validated data loaded into a data warehouse to support program analysis and reporting 3. Business intelligence and reporting tools to support end user analysis and reporting 4. Unique statewide Texas Student Identifier (TSID) embedded in the collection and integration of the data 5. Use of a Unique Teacher Identifier (UTI) and creation of a classroom link 6. Creation of a voluntary state sponsored Student Information System (SIS) 7. Establishment of an Enterprise-wide Data Governance Strategy and Board 8. Establishment of a TEA Enterprise Data Management Office (EDMO) 9. Development of Enterprise-wide Figure 1-4. Data Summary Standards of Recommendations challenges and developed a five year strategic plan in July 28. This plan identifies three key challenges and the supporting goals needed to move the Texas educational system in a forward direction. In its Strategic Plan, the TEA defines the need for real time and relevant data for all stakeholders involved in education within the state. To assist their strategic planning efforts, the TEA, with funds provided by the Michael and Susan Dell Foundation (MSDF), contracted with IBM to conduct the TEA Data Collection, Analysis and Reporting Investigation (TDCARSI) project. The objective of this project is to analyze the TEA's current data collection process and provide a strategic roadmap for addressing the educational, administrative and research needs of key stakeholders in the state educational system. These stakeholders include the school districts and the students and parent communities they represent, the Educational Service Centers (ESCs), Texas Education Agency, research groups, higher education, state agencies that interact with educational data, the governor s office, and legislators. To develop the TDCARSI plan, the project team conducted a four month study which included background research, interviews with these Texas stakeholder groups and an analysis of best data management practices in other comparable states. Based on the research and interviews, the project team identified the strengths and weaknesses of the current data management and reporting environment. Analyzing the weaknesses and best practices in Texas and in other states, IBM identified a set of to be data management processes and a recommendation for a more efficient and flexible Page 15 of 85

16 environment for data collection, analysis, and reporting that will effectively support the TEA vision and all of its stakeholders. This report provides a summary of the results gleaned from the project s investigative activities, and presents recommendations, impact analysis and a proposed solution environment for the TEA s consideration. 2.1 Stakeholders of the Texas Educational System While the TEA initiated this study with the support of MSDF, the TDCARSI plan is not an internally focused TEA initiative. The TEA, instead, recognizes that it has key stakeholders which it must serve and interact with in order to carry out its educational mission for the children of Texas. The key stakeholders, as it relates to TDCARSI, are as follows: Parents and their Children/General Public There are over 4.6 million students attending public schools and receiving education services through the local school districts in Texas. Over half of the student population is economically disadvantage, and over 6 percent of whom are minority. Independent School Districts (ISDs) and their Member Schools ISDs provide instructional and related services to the students in Texas. There are 1,2 ISDs that cover over 8, schools in the state. These school districts employ approximately 385, professional staff, including teachers, to provide educational services to students in the state which is based on a combination of funding from the state and local real estate taxes. Educational Service Centers (ESCs) ESCs provide implementation support to the ISDs. Implementation support can come in the form of training, professional development, instructional support, and back end business process and system support (e.g., student data management support, financial management system support). Texas Education Agency (TEA) The TEA is funded by the legislature to define the standards for a public state educational system in the State and provide the needed compliance, monitoring, and assistance support to school districts to implement educational services. Texas Legislature The Legislature consists of the House and the Senate and their staff and they are responsible for determining the authority and the scope of the TEA and the education system it defines and delivers to the students of the State. Texas Governor s Office The Governor appoints the TEA Commissioner. The Governor s Office also signs or vetoes legislation/appropriations proposed for the Agency. Institutes of Higher Education Higher education covers the 35 public universities and 5 community college districts as well as other public and private higher education institutions, which collectively serve a total post-secondary student population of more than 1.2 million Researchers A number of independent organizations exist within the state who may receive funding from various public and private funding sources to research the effectiveness of the Texas education system, Other State Agencies These may include any State Agencies who may need to leverage the results of the Texas Educational System. Examples include Texas Workforce Commission which periodically examines high school outcomes in order to evaluate the skill sets that are matriculating into higher education and ultimately the workforce for purposes of economic and job creation analysis. While these are not all the stakeholders of the Texas educational system, they constitute the major groups which have data needs and are believed to have a role in shaping the future of an information system for the State. Figure 2-1 summarizes their role in the State Educational System and their need for data. Page 16 of 85

17 2-1 Key Stakeholders in the Texas Educational System 2.2 TEA s Strategic Mission and Goals and the Role of an Information Management System The TEA Mission is to provide leadership, assistance, oversight, and resources so that every Texan has access to an education that meets his or her needs. Supporting this mission, the TEA has identified the priority goals of: Ensuring students graduate from high school and have the skills necessary to pursue any option including attending a university, a two-year institution, other post-secondary training, military, or enter the workforce. Ensuring students learn English, math, science, and social studies skills at the appropriate grade level through graduation. Demonstrating exemplary performance in foundation subjects. The TEA and other education stakeholders, including campus and district administrators, use data to determine whether the Texas educational system is meeting these priority goals. In fact, the TEA recognized early that data was integral to understanding the types of programs that need to be in place and the cost of delivering these programs. As part of its oversight function, the TEA developed a data collection and reporting system, called the Public Education Information System (PEIMS), and it requires districts to provide the agency with information, through regular data collections authorized by Education and Administrative codes. This data is used by the TEA to support a variety of key compliance and monitoring activities, such as: Calculating school and district accountability ratings Measuring student and staff performance Allocating and monitoring state and federal school funds Managing grant participation and performance Page 17 of 85

18 Monitoring federal compliance Education research Developing statewide curriculum guidelines However, the use of education data should typically go far beyond compliance and monitoring mandates. Stakeholders use the data to support its strategic planning and performance measurement efforts to support student achievement. They also use data to identify program strengths and weaknesses and to determine the effectiveness of state initiatives. As depicted in Figure 2-2, all key stakeholders in the Texas education system must be able to access data that is timely and accurate; data that must be actionable if it is to have the power and relevance to make a difference. For example, parents need to know if schools are providing appropriate and quality educational services for their child. Is the child learning at expected levels, on target for graduation and acquiring the skills necessary for advance training and education. Districts need to know, in a timely manner, when special programs such as those targeting English proficiency, are successful and when they are not, so that adjustments may be made early enough to improve outcomes. Educators at both the state and local level must be able to watch a student over time, to determine at what point academic advances or regressions take place. Finally, the state must be able to determine, through quantifiable analysis, if grants and tax-funded education initiatives are yielding the expected gains. Each of these scenarios requires a broad set of timely and accurate data. The figure below illustrates how this data is essential to each stakeholder and each stage of a student s academic career. 2-2 Depiction of Data Usage by Groups of Texas Education Stakeholders Page 18 of 85

19 While Texas has been a leader in information management, PEIMS has not evolved enough to efficiently or effectively support the increasing demand for timely, transparent, and actionable data, as depicted in Figure 2-2. Recognizing these challenges, the Texas Legislature funded in 22 an in-depth, third party analysis of the PEIMS processes. This analysis stated a number of challenges in the existing data collection: Aged PEIMS system and processes - The existing system is still primarily a batch collection system; the process for reviewing and approving data elements is slow and difficult, the efforts to support the system are labor intensive and expensive for the state. Untimely reporting of and access to data - The data is collected infrequently (many elements only once a year) and due to the time it takes to analyze and report the data, new reports often represent data that is at least nine months old. The data is not available to stakeholders in an easy to obtain, easy to manipulate fashion. Data reporting redundancies - Due to the inefficiencies of PEIMS, many other data collections have evolved at TEA and some of these overlap with PEIMS. Data quality needs improvement - The districts must perform their own aggregation, business rules and analysis to provide the data as defined in the PEIMS data collections. Due to the complexities of creating the required data locally at the districts, these efforts are prone to error. Labor intensive reporting burden to districts - The requirement for districts to create the required data from their source systems creates an expensive and time consuming effort for both the districts and for the TEA, who must monitor their submissions for quality and completeness. Barriers to data sharing - Current stakeholders find it difficult to get data from the TEA. Many data requests require significant programming efforts by the TEA staff and may take weeks to provide. This is true for both internal TEA stakeholders who want to use the data and other, external stakeholders. This 22 investigation provided a number of recommendations to address these issues. While a few process improvements and recommendations have been implemented, a lack of state funds in 22 meant that the state could not support major system or process changes that would streamline collection, reduce local burden or facilitate broader access to data. Therefore, many of the same challenges that were present in 22 persist today. This TDCARSI study allows TEA to re-evaluate the challenges and needs of various stakeholders and determine their data-related priorities and information needs. This study also allows TEA to investigate more recent practices and technologies that can help the TEA better support its strategic initiatives. 2.3 TDCARSI Project Scope The following defines the scope of the TDCARSI project: Examination of data needs and challenges of the key stakeholder groups that attended the facilitated sessions (see Appendix C for Stakeholder Matrix). Analysis of the current environment (which primarily focused on PEIMS but also touched on the other departmental student related data collections conducted by the agency). Examination of other state s best practices based on the states responding to the survey and guidance published by the Data Quality Campaign, a national group funded to identify the best practices for state education longitudinal data management systems. The following was not included in the scope of the TDCARSI project: An analysis of school district operating processes. The focus of this effort was not to understand the data needs of a school district in performing scheduling, transportation, meals delivery, purchasing, payroll, etc. which are needed to operate a school district. An analysis of other State Agency and researcher processes. Though several other Agency interviews were performed (with The Higher Education Coordinating Board and The Texas Workforce Page 19 of 85

20 Commission) the primary focus of this effort was not to detail specific process functions of other state level stakeholders except insofar as they have needs to access, use or integrate with K-12 data. Page 2 of 85