The Technical Coordinators Guide to Organizing & Importing Different Types of Data Into the Warehouse Documentation & Template Examples that will ease the process of getting your data into Data Director This guide will help to eliminate some of the confusion you may be experiencing when looking at your data and trying to determine the best way to start getting it into Data Director. It will also attempt to answer many of the questions you may have concerning building queries, exporting & importing different data sets, building templates and organizing your data. Jennifer DeGrie Technology Applications Coordinator Lenawee Intermediate School District Data Director Midwest Users Conference Lansing, Michigan 4/20/2010
Table of Contents 1. Getting Started... 3 A. Core Files... 3 i. Demographics... 3 ii. Teachers... 4 iii. Courses... 5 iv. Rosters... 6 v. Transcript Data... 7 2. Adding State Assessment Data... 8 A. MEAP Data... 9 B. ELPA/MI Access/MEAP Access/MME... 9 C. NWEA... 9 i. Exporting your NWEA data out of the NWEA Site:... 9 ii. Importing your NWEA data into Data Director... 11 D. PLAN/EXPLORE... 11 E. DIBELS... 11 i. Exporting from DIBELS data from the University of Oregon Site... 11 ii. Importing your DIBELS data into Data Director... 14 F. Things to consider when uploading State Assessment data... 17 3. Adding Local Assessment Data... 17 i. Creating Your Template... 17 ii. Creating Your Assessment... 21 iii. Importing Data in to Your Assessment... 22 iv. Using the Enter/Edit Function... 23 4. Final Note... 24 2 P age
1. Getting Started A. Core Files There are 4 core files that make up the building blocks of the Data Warehouse. It is these 4 files that make the students visible within Data Director and enable you to upload other types of data into the system and link that data to individual students. As a rule, these files are exported from your Student Management System. The 4 core files are: Demographics, Teachers, Courses, and Rosters. i. Demographics Your Demographic file consists of information regarding your individual students. The key field in this file is the students UIC (Unique Identifier Code). This could be a field of its own or it could be the student ID for your SMS. A basic Demographic file should contain the following fields: Building State Code UIC (in the absence of a UIC, you must use a Unique Identifying number for each student). Student Last Name, First Name, and Middle Name Student mailing address, city, state, zip code, and phone number Student Gender Student Graduation Year Student Ethnicity, Primary Language, ESL Placement, etc. Student Birthdate, Student District Entry Date Could also contain IEP Status, Disability code, NSLP Status, Migrant status, Homeless status, etc. An example query using Cognos to pull this data could look like: 3 P age
Some items to note regarding the demographic file: You can pull data by individual district and/or building and compile the data in an Excel spreadsheet. Save the spreadsheet as a Tab delimited file with good headers and submit 1 file for the county or district. Be sure to check for any student that may be missing a UIC. These students will not import into Data Director. If your dates are pulled from your SMS with a timestamp be sure to remove the time stamp. For example, 01/01/2010 12:00AM. Remove the 12:00AM so that only date shows. Be sure to pull a separate file per academic year. Example Demographic Template: ii. Teachers This file is also called the User Data file and is used in 2 ways, to load the teacher information so user accounts can be created and to create the teachers in the system for the Roster file. You will want to make sure that each teacher has a unique identification number assigned to them and that the unique ID pulled in this file matches the ID pulled in the roster file. A basic Teacher/User file may contain: Unique Teacher ID Building State Code Teacher Last Name, First Name Teacher Email Address (Optional) An example query using Cognos to pull this data could look like: Some items to note regarding the Teacher file: You may need to run a query for Course Teachers and Primary/Secondary Homeroom teachers 4 P age
Be sure that none of teachers use a unique ID of 1. You will need to add a record to your teacher file for each building with an id of 1 and name it Unassigned Staff. Be sure to remove any duplicate teacher ID s listed in the same building. Each teacher should be listed only one time per building. Make sure each teacher has a unique ID and that none of the ID s are blank. Be sure that the ID pulled for teachers in this file, matches the ID pulled in the roster file. You can pull data by individual district and/or building and compile the data in an Excel spreadsheet. Save the spreadsheet as a Tab delimited file with good headers and submit 1 file for the county or district. Example Teacher Template: iii. Courses The course list is a comprehensive list of all courses offered in all buildings of your district and/or your county. The course ID that is pulled must match the course ID that is pulled on student schedules and transcripts. A basic Course file may contain the following: State Building Code Unique course ID Course ID pulled here must match Course ID in Roster file. Course abbreviated name and/or course full name Credits, Department and/or Subject Area An example query using Cognos to pull this data could look like: Some items to note regarding the course file: If pulling data for a consortium, the course ID should be unique for each district. 5 P age
Course data will be pulled for every building that schedules students in classes. This may extend to Elementary. This would not include buildings that strictly use Primary and Secondary homeroom scheduling. An example course template may look like: iv. Rosters Simply put, the Roster file is the most crucial file of all. This file is what connects the dots for the previous three files and makes the students visible within Data Director by linking students to teachers and also to the courses. Be sure that when this file is created you are using the same fields in your query that were used in the previous files. A Roster file may include the following fields: State Building Code (Same as used in 3 previous files) Student UIC (From Demographic File) Primary Staff ID (From Teacher/User File) Grade for current year, Marking Periods student is scheduled, Period that course begins Course ID (From Course File) An example query using Cognos to pull this data could look like: Some items to note regarding the Roster file: Be sure that there are no blank Student UIC s. These students will not import into Data Director. 6 P age
Be sure that there are no blank Teacher ID s or Teacher ID s that are invalid (ex. 0 or n/a ). Replace the blanks and invalids with the Unassigned Staff ID of 1 as was added to the Teacher file. Students with no valid assigned teacher will not import into Data Director. You may need to run separate queries for students scheduled into courses versus students assigned to homerooms. The above example is for students scheduled into courses. The example below shows a Cognos query for students scheduled into Homerooms: An example Roster template may look like: v. Transcript Data Transcript data is optional and will enable the use of Power Data Sets and more robust reporting abilities. You will not want to import this data until you are confident in your four core files. Fields that you will want to consider adding to your Transcript Data Template are: Student UIC State Building Code as pulled in other queries Course ID (Must match what was used in Course and Roster File) School Year, Term, and Grade course was taken in Mark and Credit received for course taken 7 P age
An example query using Cognos to pull this data could look like: Some items to note regarding the Transcript File: Once you get the data uploaded you will need to run the Power Reports Eligibility to determine if your data is complete. Continue to tweak your data until all criteria is met. An example Transcript template may look like: 2. Adding State Assessment Data Once your students have been rostered into Data Director, you will be ready to move on to the next step of importing State Assessment Data. State Assessment data should be downloaded directly from the State Website and then uploaded directly into Data Director. You should never open the file after it has been downloaded. Additionally, to maintain the integrity of the data, the file should not be emailed or pass through any additional hands. You may appoint one person at an ISD/RESA level to download and import all files for all districts or you may elect that each building is responsible for downloading and importing their own data. 8 P age
A. MEAP Data To download your MEAP data, you will need to have a username and password to access the OEAA Secure Website. The link to the site is: https://oeaa.state.mi.us/meap/login.asp. Once the data has been downloaded, you may log into Data Director and browse to: Admin Tools>Import Tools>Directive Import Tool. Select MEAP as your Upload File Type and select the corresponding Academic Year then click Go. You will need to Browse to the location that your file was saved and select it then click Go to start the import process. Please note that files should be in.csv format. Additionally, if you are uploading multiple files for individual buildings you may not open and combine the files. You will need to upload each file individually. Once the import completes be sure to check the log files to see if any records in the file were skipped or duplicates. Please note that once you have completed importing all MEAP files, you will need to submit a request for Data Director to run the MEAP Strand Calculator before you will be able to run any Strand Reports. B. ELPA/MI Access/MEAP Access/MME ELPA, MI Access, MEAP Access, and MME are additional State Assessments that follow a similar process to getting the MEAP data into Data Director. You will need to download your data directly from the State (http://www.michigan.gov/mde/0,1607,7 140 22709,00.html) and again, you will want to be sure not to open the files once they are downloaded and ensure data integrity by importing them directly into Data Director. If you run into an instance where Data Director has not yet created a directive for these types of assessments, you will need to upload your data file to the status listing and request that they upload the data for you and that a directive be created. You may do this by browsing to Admin Tools>Status Listing>Upload Data. C. NWEA i. Exporting your NWEA data out of the NWEA Site: To obtain your NWEA data, you will need to browse to the NWEA Report site: https://reports.nwea.org/login.aspx and login using your secure username and password. Once logged on, you will see Order Reports in the left hand menu. Click on that to fill out the order form. Here is an example of the items that you will need to fill in: 9 P age
Check this box so that small groups of students will be retrieved. 10 P age
Once you are done filling out the form, click Submit Order. The system will take you to a confirmation page. Now click on Retrieve Reports in the left hand menu. You will see that your order has been placed. Continue to click on Retrieve Reports in the left hand menu to refresh the screen until you see that your report is Ready. Click on Ready and save the zipped folder to your computer. When the folder is opened, the last file listed will be a.txt file. This is the file that you will want to upload into Data Director. ii. Importing your NWEA data into Data Director Log onto Data Director and click on Admin Tools>Import Tools>Directive Import Tools. Select NWEA as the File Type and the corresponding Academic Year and select Go. Next select the appropriate Term and file type and select Go. Upload your file and you are done. Be sure to check the log file to see if any students were skipped or invalid. D. PLAN/EXPLORE To obtain your PLAN and/or your EXPLORE data in electronic format; you must request the Enhanced Reporting Package when ordering your testing supplies. If you do not (or have not in the past) you will only receive your test results in paper format. I do not know at this time if you can request/purchase CD s for prior year information. The links to view purchase options are: http://act.org/plan/packages.html http://act.org/explore/packages.html Once the tests are scored a CD will be sent to the testing building. You can either elect to have each building import their data from the CD or the CD s can be sent to an ISD/RESA contact for import. There will be a.txt file on the CD. This is the file that you will want to import into Data Director using the Directive Import Tool; to get there, browse to Admin Tools>Import Tools>Directive Import Tools. Once there you will select ACT as your Upload File Type. Select the corresponding year and select Go. Next, you will need to select the correct file format. E. DIBELS i. Exporting from DIBELS data from the University of Oregon Site To obtain your DIBELS data you will need to browse to the University of Oregon DIBELS Data System site: https://dibels.uoregon.edu. Once you have logged into the site, click on the REPORTS tab and click on DATA SETS then DATA FARMING. 11 P age
Next, select the SCOPE: and then select SUBMIT. Next, select the SCHOOL: and click SUBMIT. 12 P age
Next, select the Term and Year that you would like to download the data for along with the applicable grade. Under STUDENT INFORMATION, select STUDENT NAME and STUDENT ID. Under ANALYSIS, select OUTCOME MEASURES. No other fields are required so you can now select SUBMIT. 13 P age
On the next screen, you will select the operator to use. Select Operator: ALL and click SUBMIT. Next, you will be able to click the link that says, Download Data Here. Your tab delimited text file will now be displayed in your internet browser. To save, select FILE>SAVE AS and save your file to a desired location. Once your file is saved, you will need to open it Excel so that it can be formatted to match the template so the data can be imported into the corresponding Assessment in Data Director. Please note: At this time, each school year, term and grade level must be imported into Data Director separately. As a result, you will need to repeat this export process accordingly. ii. Importing your DIBELS data into Data Director Locate the Tab Delimited file that was just saved to your computer. Open it in Excel. An example Kindergarten file will look like this: You will need to re organize your data so that it matches the template below: 14 P age
Make sure that all headers in the template are included. Not all columns require a score. Only the columns with scores will be imported. This template can be used for all school years, all terms and all grade levels. Once your data is organized in the template above, you may save the Excel file as a Tab Delimited file by Selecting File>Save As, change the File Type to Tab Delimited, name your file for the school year, term, and grade level and select save. An example filename might be: 0809BegKdg.txt. Saving your file with a name like this will make it easier to locate when browsing to upload into Data Director. Next log into Data Director and select the ASSESSMENTS tab. Filter the assessments so that you are only viewing DIBELS for the school year and grade level you want to upload. This will make the corresponding assessment easier to find. 15 P age
In this example, we will upload DIBELS Grade K Beginning scores for school year 2008/2009. Click on the link for the corresponding assessment that you are uploading scores for. In the lefthand menu under DATA MANAGEMENT click on the link that says IMPORT SCORES. (NOTE: If you do not see this, contact Data Director to obtain permission to do this function) On the next screen, you will need to select BROWSE to locate the file that was saved earlier. Once located, select it. Be sure to remove the checkmark from the box that says: This Assessment has previously been mapped. as you will want to confirm the mapping yourself. Then select UPLOAD FILE NOW. The next screen shows you a preview of your data file and asks how the data should be used to update existing records. Select Yes, but only blank fields unless you want to overwrite existing student scores for the same school year, term, and grade. Select Next. The following screen will ask you to map your headers. Make sure that the mapped headers match the headers in your template. If not, use the drop down boxes to change them. SKIP any columns that you do not wish to import. 16 P age
Once your mapped headers match the headers from your file that you would like to import, you may select NEXT. You will have one last chance to preview data as it is mapped before selecting Looks Good, Continue. You will then get a box that prompts if you are sure you want to continue. If you are, select OK. Your data will now import. Be sure to check the log file in the Green Bar to see if you had any invalid, duplicate, or skipped students. Repeat this upload process for each school year, term, and grade level. F. Things to consider when uploading State Assessment data If this is your first time uploading state files you may need to submit your file to the Status Listing so that Data Director can make the correct Directive Import tool available to you if there is one. If there is not, you will need to use the Import Data function in the actual assessment. Any Student ID s in the files MUST match the Student ID that is uploaded into Data Director from the Four Core Files. Generally, this will be the Student UIC. If the Student ID does not match, the data will not import. You can use a VLOOKUP in Excel to replace the Student ID s. 3. Adding Local Assessment Data The key to collecting and adding local assessment data is starting with a good template that includes the Student ID s used in Data Director, descriptive headers, and descriptive file names. The template should be an Excel version of the Assessment created in Data Director to make the upload process easier. We have created many local assessment templates and corresponding assessments in Data Director but I will use a simple one for this example. i. Creating Your Template We decided to create a template that we could enter Parent/Teacher Conference attendance data into and then a corresponding assessment so that we could load the data and then compare the attendance to performance levels on other State and Local Assessments to see if parent involvement in conferences had an impact on student achievement. To create the template, I started by creating a report in Data Director called, What are my student UIC s? so that I could export it into Excel for the students that I wanted to create the template for. I can easily modify the columns and refine the data set to pull just the students that I would like in my template. To create a report like this, log into Data Director and Select the REPORTS tab. Select CREATE A NEW REPORT. Fill in the criteria to as follows, then select SAVE CHANGES. 17 P age
Under FILTERS, select DEMOGRAPHICS and the corresponding Academic Year for the students that you would like ID s for. Next select GENERAL DEMOGRAPHICS. From the drop down box select the UIC field and then click NEXT STEP. 18 P age
Next select the first option, Specify a value or range of values to search on and click NEXT STEP. Next, from the drop down menu, select, is not blank and click NEXT STEP. You will now see a report that shows the last name, first name, and UIC of all students in the building. If you would like to add additional data (Ex. Grade Level) or modify the data to show a different set of students, you can do this by selecting REFINE DATA SET in the left hand column. I like to include the grade level in my templates so let s add that. Select CLICK HERE to proceed. Next select STUDENTS as your filter and STUDENT GENERAL INFO as your data set. 19 P age
From the drop down menu, select GRADE and click NEXT STEP. Next select the first option, Specify a value or range of values to search on and click NEXT STEP. On the next screen, you may select to show all grades or you may select to show a specific grade. To show all grades, select Grade is not blank. To show a specific grade, choose that grade from the second drop down. You may choose to show grades Kindergarten through Second or some other alternative. You can use the selections from the first option to set those criteria. Once you have the data selected, click NEXT STEP. On the next screen, choose the corresponding academic year for the grade you want to view and click NEXT STEP. You will now have a report showing last name, first name, UIC, Academic Year, and Grade. Click the Excel icon to export your report to Excel. Once exported, I deleted the Academic Year column and added columns for the data that I would like to add. Now the template looks like this: 20 P age
Simple, yes, but you should get the idea. By using the report in Data Director and exporting the student names and ID s of the group of students that I want to track data for I did not have to manually enter that information and creating the template was VERY easy. Once your data is entered, save the file as a tab delimited file with a descriptive name that includes academic year of data stored, term (if applicable), type of data, and building/district (ex. 0910PTConfElem.txt). This will make the data easier to locate when you are browsing to upload it. ii. Creating Your Assessment Once you have your template, you are ready to create your assessment. To do this, log into Data Director and click on the ASSESSMENTS tab. Next, click on CREATE A NEW ASSESSMENT in the left hand menu. Fill out the Assessment Name, Description, etc as necessary. Select NEXT STEP. Fill in the Assessment Field data as applicable. Use the drop down menu to select the type of data going into the field. Select NEXT STEP. 21 P age
Your assessment is now created and you are ready to import data. iii. Importing Data in to Your Assessment Select IMPORT DATA from the left hand menu. Browse to locate your file and select UPLOAD FILE NOW. The next screen allows you to preview the data and select additional criteria on how to handle the data. Make the appropriate selections based on your needs. The below is an example of what you might select for this specific example. Select NEXT. 22 P age
On the next screen you will need to map your columns in your assessment to your headers in your template. Select NEXT. You will be given one more opportunity to preview your data. If it displays as expected, select LOOKS GOOD, CONTINUE. Your data will now be imported. Refresh the assessment screen until you see the green confirmation bar. Select to view the log to verify that no students were skipped or invalid. iv. Using the Enter/Edit Function The IMPORT DATA function works very nicely when you have a lot of data to import or if you want to keep data going into the Warehouse limited to only a few people. However, if you have a small assessment (like the Parent/Teacher Conference example) where there is little data and you are comfortable with teachers entering their own, you can enable the ENTER/EDIT function for assessments. This can be done BY ASSESSMENT by selecting the assessment and then selecting the PERMISSIONS/SHARING link in the left hand menu. Select the appropriate option for granting rights (ex. Permission Set, Individual User, etc) then select the following rights to allow entering and editing of scores: The user will now see Enter/Edit links under the Data Management group in the left hand menu. To begin entering/editing scores click on the ENTER/EDIT DATA link. You will now see a screen 23 P age
listing all students in the building. You will need to navigate through the screens and enter/edit scores for applicable students. There are SAVE and SAVE AND RETURN TO RESULTS buttons on both sides of the page and every 8 rows to encourage you to save your data often as you enter it as it does not save it automatically. When you are finished entering scores, click on SAVE AND RETURN TO RESULTS. This will return you to the assessment where you can view the data entered. 4. Final Note This guide is not meant to be inclusive of all ways that you can import/export data, create assessments/ templates, etc. It is meant to be supplemental and assist you in answering some initial questions as you begin to look at getting data into your warehouse. I would like to continue to add to this guide and make it available in later revisions. If you have suggestions for additional items that you would like to see added please feel free to contact me at Jennifer.DeGrie@lisd.us 24 P age