Developing Multipurpose Grade Reports Using Excel PivotTables

Transcription

1 Developing Multipurpose Grade Reports Using Excel PivotTables NEAIR Boston December 5, 2011 Jim Fergerson Dir. of Institutional Research and Assessment, Carleton College Northfield, Minnesota

2 Overview/Objectives Why? Ongoing concerns about grade inflation and grading patterns Why Excel? Developing reports Examples of several different reporting techniques using PivotTables (paper, interactive) Report Gallery: Examples of static and interactive multidimensional grade reports developed a single master data query

3 Does This Pattern Look Familiar? Mean End of Year Cumulative GPA Trends All Students Graduating Seniors First Year Students Mean GPA

4 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Academic Year Letter Grade Trends Satisfactory / Unsatisfactory or Credit / No Credit options A B C D F P/F + Other Source: Annual Grade Surveys. Carleton has a generous pass/fail option to encourage students to try courses in subjects outside of their comfort zone. Some courses have a pass/fail option by student choice; others are designated by the instructor.

5 We are NOT Alone! Stuart Rojstaczer. Grade Inflation at American Colleges and Universities. GradeInflation.com. March (Includes Carleton data, )

6 We are not outliers Stuart Rojstaczer. Grade Inflation at American Colleges and Universities. GradeInflation.com. March (Includes Carleton data, )

7 Names Masked Existing Report One Dimensional, Paper

8 Department Summaries Manually Re Entered into Spreadsheet

9 Multidimensional Grade Analysis is Vital! College wide snapshots tell us little; to understand the data and to make any difference, departments and instructors need much broader context We needed to replace one program that presented only one report of the data with One data model that is flexible enough to provide N dimensional views to explore the data to answer a variety of questions

10 Institutional/Department Program/Faculty Dimensions What is the grade point average and letter grade distribution? All college By department, program, or division By instructor By course and course level Class rosters, distribution of grades within a section Background material for faculty reviews Compare outcomes in a course over time (i.e. Intro Science) By other factors: gender, tenure status, rank, seniority, method of instruction, etc. Assessment and evaluation, program review

11 Student Level Dimensions What is the grade point average and letter grade distribution? By gender, race/ethnicity, socioeconomic status, legacy/1 st generation status, entering or graduating cohort, class level, course level, major vs. non major How many students in category X took a course in Y? How many and what type of students continued past the entry level course? Identification of at risk (or outstanding) students Evaluation of admissions selection Individual course patterns pseudo transcripts Complicating factors: Multiple majors, privacy issues (small N s)

12 Time Dimensions What Changes? Calendar Academic Year/Term Trends for institution, department, faculty member Categorical changes over time (Are 1 st generation students improving? etc.) Longitudinal tracking (Are individuals, specific cohorts improving during their four years?) Pre test/post test evaluations (Was there improvement in Chem 101 after adoption of new teaching method?)

13 Why Use Excel? Old program became obsolete with database upgrade. New reports are needed now, as Faculty review grading practices and want better information. A Business Intelligence (BI) tool could make reporting and distribution more efficient, but we don t have a campus wide common package. Excel is the most widely available and understood tool for those likely to use reports. Ease of exploratory data analysis and Flexibility: New reports can be easily updated directly from our administrative database OR the data warehouse.

14 Alternative Data Warehouse Connection Properties SQL code to query raw data (user modifiable) OLAP Cube Connections (pre defined by IT)

15 Working with OLAP Cubes Cubes/Value Dimensions PivotTable Fields Pros: Speed: Large amounts of data pre calculated by remote server More fields (link cubes to cubes) Uses vastly less memory: the raw data isn t brought to your desktop (2.5 MB file with OLAP vs. 43 MB file to query 10 years of raw data) OLAP = OnLine Analytical Processing Cons: IT will probably need to set OLAP cubes up and grant access Limited to pre defined and precalculated value fields Less flexibility for custom calculations and some Excel options With cubes, some calculations, such as multiple views of the same value field (ex. Grades N/%) require Excel 2010

16 Use PivotTable Filters to Select Valid Grades, Course Levels, and Terms (i.e., Parameters that should remain CONSTANT across all reports) Select only the items that are currently used, and relevant.

17 Experiment with Some Tabular Reports by Dragging in Some Changing Dimensions that You Wish to Explore Trends by Level by Department Course Detail for Instructor One Year by Term Course Roster and Student Grades for a Given Course Section

18 When You ve Built/Tested One Master PivotTable: Then Copy, Modify, Adjust, and Recycle It to Other Tabs Memory: Copies of a master PivotTable (linked to the same data source) don t duplicate the data; it s easy to refresh all Parameters: Your key parameters carry over from report to report; only minor adjustments needed. (By copying the master spreadsheet tab you re also copying default headers, footers, margins, etc.)

19 Choose the Appropriate PivotTable Options Turn autofit off, or you ll need to re size your report with each refresh Click in PivotTable, then right click to select PivotTable Options

20 Choose the Appropriate PivotTable Options Need for totals varies by report Check only if you need to drill down to detail. DO show all relevant items, so you ll have a consistent data matrix! Letter grades or other labels will sort alphabetically, and not necessarily in the desired order. You can either arrange them manually from within the PivotTable, or pre define a custom sort order in Excel Options/Advanced/Custom List.

21 Choose the Appropriate Field Setting Options Duplicate fields, different format Varies by context of your report(s) Click in PivotTable, then right click the field label to select Field Settings options Experiment! Drag to row or column labels or drag up/down within fields box to change report layout!

22 Hint: Become Familiar with the PivotTable Tools Ribbon Options Options: Design: Experiment with the Layout tools to help structure the report format this can often be easier than setting every Field Setting option individually

23 Report Option 1: Grade Average Trend Reports: Computation Area A: All Carleton GPA Trend PivotTable Here, no filters are applied except the parameters that are common to all reports

24 Computation Area B: Department GPA Trend PivotTable Change filter to create master PivotTable for next department.

25

26 Key Linkages Computation Areas Dept. Label Clean Printed Reports and Graphs Same data matrix: One page, one tab per department

27 Link PivotTables from the Computation Area to the Clean Report Using Formulas; Control for Errors Link formulas by hand, using cell references. (Excel will, by default, try to use the more complicated GETPIVOTDATA references =GETPIVOTDATA("[Measures].[Grade Avg]",$B$116,"[Term].[Year Term]","[Term].[Year Term].[Academic Year].&[ ]") Set up error trapping formulas to correct for cells with null values; replace with. Link the name of the selected department (from the PivotTable Report Filter cell) by formula as well to use as labels in the appropriate titles and subtotal cells.

28 Finalizing the Grade Average Trend Reports 1. Copy the tab of the final working Trend Report Master PivotTable (report and all print set up formats!) to new tabs (one tab for each department). 2. Update the Report Field Department filter for each Department, and rename the tabs with the department name. 3. Check for errors, inconsistencies! 4. Printing: (In this case, all reports have the same data matrix and a 1 page print area) a. SELECT ALL tabs to be printed in the desired print order. b. Print paper copies to printer, and a storage copy to Adobe Acrobat file. c. IMPORTANT! UNSELECT ALL printed tabs immediately after printing! (Any changes made to current spreadsheet will apply to the corresponding cells in any selected tabs unless this is done!) 5. Next Year: Just update the filtered date ranges in each department tab. (But I m looking for a macro to do that )

29 Compare: Carleton vs. Department Level GPA Carleton Grade Average Economics Music Psychology

30

31 Compare: Course GPA by Level within Biology

32 Compare: Course GPA by Level within Economics

33 Report Option 2: Academic Year Detail Reports by Department or Instructor 1. Compare all Carleton letter grades and grade point averages to detail for courses completed within each department, by course level 2. Provide each instructor with report with details for grades they gave in all sections, by term (including those offered outside their primary department) 3. Output: One tab, one report, print many pages, but with variable data matrix.

34 Academic Year Detail Report for Department by Course Level

35 Academic Year Detail Report for Instructor (name masked)

36 PivotTable Field Selections Department by Level Detail Instructor Sections Detail Hint: You can assign meaningful labels to field names and values to make reports more readable.

37 Set Up a Multipage PivotTable Report of Unknown Length Force a page break after the primary report item (instructor, department, etc.) by clicking on key data field label and Insert page break after each item. Caution: Page breaks aren t 100% perfect, but they work better when you also Insert a blank line after each [page break] item label. (Watch for cases where only the subtotal rows cross a page!)

38 Printing PivotTable Reports When Print Range is Unknown Do a Select All of the PivotTable data matrix, set any headers, footers, margins, repeating rows, etc. in the Page Setup menus. Important: Leave pages tall blank! Excel will calculate # of pages for a PivotTable print. (Print job may stop before finish if you force a page limit.) Recommendation: Since reports can be large, first print to Adobe Acrobat, and proof report before printing copies to paper.

39 In PivotTable Tools Options Menu (beneath File ): Report Option 3: Show Report Filter Pages Quickly let Excel generate a separate tab with a sub report for each Report Filter Item

40 Show Report Filter Pages Many sub reports are possible (i.e. one per 250+ faculty members) Unfortunately: Most custom formats in the parent report are lost (print set up, column size, graphs, etc.) This method only works for raw data reports, not OLAP cubes

41 Report Gallery: Examples Derived by Modifying the Master Report

42

43 Letter Grade and GPA Trends for a Department

44 Compare One Academic Year s Grades for All Instructors in a Department

45 Compare Selected Instructor to Carleton Letter Grades, Most Recent Academic Year

46 Lookup Instructor s Class Roster and Student Grades (ENG names masked)

47 Dean s Level Interactive Report Trends by Subject Area Conditional formatting applies a heat map to color code grade point differentials. Embedded macro allows Dean to compare trends in subject to all Carleton trends Excel 2010 adds Edward Tufte s sparkline graphs to show thumbnail trends for each subject

48 Interactive Dashboard with Slicers Slicers became available in Excel 2010 to allow more userfriendly and interactive data exploration than standard PivotTable filters

49 Student Level Cumulative GPAs by Test Score Range SAT Math and Critical Reading Range 4.00 ACT Score Range

50 Student Level Cumulative GPAs by Legacy Status and Student s Home Region st Generation No Legacy Parent Legacy New England West Middle States South Midwest

51 Issues Yet to Be Resolved Political Feedback? too early; first reports have not yet been distributed broadly Access! Who gets to see what, and for what purpose? Deans and IR can see and do analysis all Faculty can see most recent year and trends for own grades To be discussed by Deans and Faculty Enrollment and Curriculum Committee: New data will generate new questions! Can chairs see department detail? How much detail can be available for program review and assessment? Privacy issues (esp. with drill down ) Hard copy vs. Raw data

52 Technical Pushing Excel to Its Limits Lots of dead trees! Manual process Need to find better ways to automate report updates and distribution. Data Policy: Faculty can t access data behind administrative firewall Improve design and appearance of reports; better automation, labels, etc. Discovering the limits and capabilities of Excel and the data warehouse Train Dean(s) to use analytical/interactive queries Opportunity: The grade models are easily re purposed for many other related reports

53 Questions? Discussion? Special thanks for their technical support: Richard Goerwitz and Julie Latham, Information Technology Services, who designed the data warehouse and OLAP cubes Jody Friedow, Assoc. Dir. of Institutional Research and Assessment, who taught me everything there is to know about how Carleton s grades are calculated and stored