Educational Analytics Dashboard

Transcription

1 Educational Analytics Dashboard Contents 1. Introduction Accessing and running the dashboard Using the Dashboard... 4 Applications, Offers, Accepts and Enrolments... 5 National League Table Rankings... 6 Recruitment vs. Target, Entry Tariff & Protected Characteristics... 7 National Student Survey... 8 Postgraduate Taught Experience Survey... 8 Outcomes... 9 Results Navigation and Drill-through Reports Navigation Drill-through Reports UCAS Competitors Applications, Offers, Accepts and Enrolments by Course Rankings Widening Participation NSS Year on Year NSS Sector Results NSS Latest PTES Questions PTES Summary Scales Non Completions Employment DLHE Results Protected Characteristics Printing and Saving Reports Printing Saving... 26

2 1. Introduction The Educational Analytics Dashboard provides simple access to Teaching and Learning data by department at either Undergraduate or Postgraduate Taught levels of study over a number of previous academic years. It has been designed to provide both an at-a-glance summary of teaching performance at the University of Warwick, and also provide an entry portal into a number of more detailed reports that can be accessed by drilling though from the main dashboard. The Dashboard combines information from the following data sets: SITS the internal Student Records System provides data on applications, offers and enrolments as well as student characteristics and outcomes League Tables (Complete University Guide, Guardian and Times) National Student Survey (NSS) Destination of Leavers in Higher Education (DLHE) Postgraduate Taught Experience Survey (PTES) The data exposed from SITS is based on the academic census snapshot (usually taken on or around the 31 st October), which provides a consistent reference point in time. Data from other sources is released periodically throughout the year and will be included when available. Note that the underlying data is therefore relatively static and a Dashboard report will only change periodically. Where possible, performance has been benchmarked against the sector. Although sector data is widely available (e.g. from HEIDI, the Higher Education Database provided by HESA) it can be difficult to map across at the level of departments which are organisational structures specific to each institution. Accordingly, the majority of benchmarking is undertaken at a subject (or JACS code) level where direct comparisons between institutions can be made with more certainty.

3 2. Accessing and running the dashboard The Educational Analytics dashboard can be found at: If you are an existing user and are unable to access the report or use the above link, then please initially refer to: If a solution isn t found there then please a screenshot and description of the issue to: dashboards@warwick.ac.uk Existing users are able to click the first link above to access a link to the Cognos dashboard, you will then be prompted with the Cognos login screen shown below: You will need to re-enter your Warwick ITS username and password, and once you click OK you will then be presented with the following prompt screen, which allows you to choose the: 1) Level of Study 2) Department on which you wish to report: Once you have chosen a Level of Study, then a Department, click finish to run the report, and you should see the following splash screen:

4 The report should take a few seconds to load. 3. Using the Dashboard The dashboard provides slightly different single page, sectioned views of departmental performance based on which Level of Study is selected on the prompt screen. The different Level of Study views of the dashboard can be seen below: UG Dashboard View

5 PGT Dashboard View Each of the sections shown as tables and charts in the Dashboard provides a summary of one aspect of teaching and learning performance but also a link to the more detailed reporting that lies beneath. Applications, Offers, Accepts and Enrolments UG & PGT Chart The table entitled Applications, Offers, Accepts and Enrolments contains data exclusively taken from SITS, and shows data for the current and previous four academic years. As well as charts and counts, the change year on figure (Δ YonY) is shown and colour coded green for a positive change and red for a negative one. A stacked bar chart summarises the Fee Status of the student population, which is split by each of the 3 possible Regions of Domicile of the department s enrolled students.

6 Clicking on the underlined header in the table allows the user to drill-through to various drill-through reports, the options available will vary for each Level of Study(detailed in section 4 of this document): UG Drill-Throughs Enrolment Questions Applications Offers Accepts & Enrolments UCAS Competitors PGT Drill-Throughs Enrolment Questions Applications Offers Accepts & Enrolments National League Table Rankings UG Chart Only The table entitled National League Table Rankings shows data for the relevant subject rankings for the last 5 years, and also highlights the most recent score and ranking in each, with the indicator denoting the direction of change in the ranking. Clicking on the underlined header in the table allows the user to drill-through to a more detailed singlepage report on League Table Rankings. Please note that this table is not available on the PGT view of the dashboard and is substituted by a more relevant PGT table.

7 Recruitment vs. Target, Entry Tariff & Protected Characteristics UG Chart PGT Chart This table shows a variety of information regarding the incoming cohort of students in an academic year. Recruitment of Home EU, Home UK and Overseas students is shown relative to targets (where available) with the colour-coded differentials highlighting performance in both the UG and PGT charts. The Entry Tariff data is only available on the UG version of this chat and is supplied by HESA annually (although normally at least 12 months behind), and shows Warwick s average entry tariff and a colour-coded differential with the Russell Group average. The final columns in the table show the absolute number of students that meet the following Widening Participation characteristics: State are Young, Full-time, Home Undergraduates from State (ie. not Independent) Schools {on UG chart only} LowSEC are defined as Young, Full-time, Home Undergraduates from Low Socio-Economic groups (4-7 of the 9 defined socio-economic groups) {on UG chart only} LPN are Young, Full-time, Home Undergraduates from the lowest postcode areas of HE Participation (Low Participation Neighbourhoods) {on UG chart only} Mature are Home Undergraduates (both Full-time and Part-time) aged 21 or over on Enrolment {on UG chart only} Disabled are students with a registered disability {on UG and PGT charts} Gender to classify the cohort as either being Male or Female, details of other gender types can be found in the associated drill-through report disability {on UG and PGT charts} Mode of Attendance splits the cohort by whether they are Full-time or Part-time {on PGT chart only}

8 Ethnicity is comprised of the various HESA Ethnicity Codes which are then derived into 4 distinct groupings {on UG and PGT charts} (more detailed definitions are available from HESA) Clicking on the underlined header in the table allows the user to drill-through to a more detailed singlepage report on Widening Participation measures. National Student Survey UG Chart Only This table shows the results of the National Student Survey for the last five years, for the sections on Overall Satisfaction and Questions Average. The Questions Average is a calculation that takes all the scores for that department in all the sections in the NSS survey then works out an average score for all those sections, so as to provide an insight into general performance. Note that the data is based on the publicised results at Subject level to facilitate a colour-coded comparison benchmark to the Russell Group. Clicking on the underlined header in the table allows the user to drill-through to the following NSS drillthrough reports: NSS Year on Year NSS Sector Subject Latest Results Postgraduate Taught Experience Survey PGT Chart Only

9 In the PGT view of the dashboard the NSS table is replaced by the Postgraduate Taught Experience Survey (PTES) sector-wide data which is compiled annually by the Higher Education Academy. Similar to the NSS survey for UG level, the data presented is the Overall Satisfaction score as well the average of all scores in all the survey s sections. These two measures are in this case compared to the weighted average Warwick departmental score which is displayed as either a positive or negative colour-coded variance. Outcomes UG Chart PGT Chart The tables entitled Undergraduate Outcomes and Postgraduate Outcomes are the same apart from the titles, and they show data on Non-Completions (calculated from data held in SITS) and Positive Outcomes taken from the DLHE data supplied by Student Careers and Skills. As well as the calculated figures and charts, colour-coded differences to the Faculty (Δ Fac.) and the University (Δ All) averages are also shown. Clicking on the underlined header in the table allows the user to drill-through to the following dril-through reports: Employment Non Completions

10 Results UG Chart PGT Chart This final table shows a breakdown of Degree Results awarded and is slightly different for each of the Levels of Study. The UG chart calculates figures and percentages for Good Honours (Firsts and Upper Seconds) which are used in some ranking methodologies. In the PGT table there is a breakdown of Degree Results by the Award Class, (which shows percentages of Distinctions, Merits and Others) as well as counts of the types of Awards agreed for that Academic Year. Clicking on the underlined header in the table allows the user to drill-through to a more detailed singlepage Awards drill-through report on degree results at course level.

11 4. Navigation and Drill-through Reports 4.1 Navigation As noted earlier there are a number of subsidiary reports that provide more detailed information on each of the sections in the main dashboard, and these can be accessed by clicking the underlined title of each table: As you hover over the title, the pointer will change from an arrow to a hand, at which point you can either single-click the link or right-click it and use the submenu which then appears to navigate to the report of your choice. If you single-click the link and only one drill-through report is available, the report will run with no further action. If more than one drill-through report is available then you ll be presented with a list and prompted to select one of them: Pressing cancel will exit the dialog window and return you to the dashboard. It is important to note that each drill-through report will be opened in a new window (or as a new tab depending on browser behaviour), and may prompt you for further information before it can run (this will occur for example when you are drilling through to an NSS report where the report is based in the JACS subject code rather than the department).

12 4.2 Drill-through Reports The following sections provide brief details on the drill-through reports available: UCAS Competitors This report shows the top 20 UCAS competitors over the last 5 academic years for shared applicants in the chosen application department (note that these may be different to the department used for the dashboard). The following definitions are used in the report: Shared applicants are those where both Warwick and another HEI made an offer which was accepted as either Firm or Insurance Warwick Gains are those where Warwick was the Firm choice Warwick Losses are those where Warwick was the Insurance choice Net Gains are calculated as Warwick Gains Warwick Losses In the chart on the left, Warwick Gains are shown in blue and Warwick Losses in red. The second chart shows the position regarding Net Gains and is similarly colour-coded. Finally, the table on the right shows the average figures over the period. The report is sorted by the total number of shared applicants, so the most important competitors are shown at the top.

13 Applications, Offers, Accepts and Enrolments by Course UG Report PGT Report This report at UG level shows a breakdown of the data presented in the main Dashboard at Fee Status, Course and Mode of Attendance levels. The PGT report includes Postgraduate Widening Participation (WP) details, namely if the PGT students come from LowSec or LPN backgrounds. Please note that this differs from Undergraduate WP, the latter takes into account whether the student also attended a State School and was under the age of 21 at first enrolment. Both reports now show the ratio of Enrolments to Acceptances for each department at PGT and UG levels.

14 Rankings This report shows a breakdown of the National Subject Rankings data for UG level only. As well as more detailed component scores from each ranking, two tables on the right of the report show all institutions that are in the top 10 and the top 10 improving institutions (by ranking). If Warwick isn t in either list by default it will be included and highlighted for comparison.

15 Widening Participation The table at the top of this UG level only report contains a summary of the departments WP model performance, comparing actual figures with calculated targets, over the last 5 years. The model is based on a number of assumptions: the target population is distributed proportionately across each department according to the relative size of their populations the model targets and weightings are common across all academic years (the default values are taken from 11/12) the data conforms to statistical norms that permit the use of z-scores to calculate standardised metrics The model calculates a standardised WP characteristic score using the variance between the actuals and supplied targets, before weighting and combining these component scores to provide an overall total model score. The total scores are then converted to percentiles and colour-coded into eight bandings ranging from dark green (very good) to dark red (very poor). It also displays a plot of the percentile score over time, with the average value shown as dotted red line. The charts at the bottom of the report show the absolute values of applications, offers and enrolments for each of the WP characteristics over time. Non-State and Total (Home) charts have also been provided as useful additional reference information.

16 NSS Year on Year This report is only available for UG level and shows a more detailed breakdown of NSS results over the last eight years (as at 2015), for the main categories within the NSS. Colour-coded bar-charts are used to show year on year differences and variations against the Russell Group and Sector averages.

17 NSS Sector Results This report is only available for UG level and shows a sector heatmap for the chosen subject. Institutions are ranked in terms of overall satisfaction but each main category is colour coded into quintiles based on the score to highlight relative areas of strength and weakness at each institution.

18 NSS Latest This report is only available for UG level and shows a question level breakdown of the latest NSS results for the chosen subject. For each question, the subject results are profiled against the sector and Russell Group averages and also against all subjects for Warwick and the Sector. Colour-coded rankings for each question in the Sector and Russell Group are also shown, whilst the final columns show the relative subject performance.

19 PTES Questions This report is only available at PGT level and shows a question level breakdown of annual PTES results for the chosen Department. There is a breakdown of the number of students who answered each question as well as the percentage of students who agreed with each question, which is then compared to the average Warwick score for all departments that had students who answered that question. The variance is then displayed and colour-coded in the Difference column.

20 PTES Summary Scales This report is only available at PGT level and shows a summary of the latest PTES results for the chosen Department. The comparison in this report is based on the average scores received in specific JACSv2 Subjects (offered by that Department), for each Section of the survey. This is then compared to the Warwick departmental, Russell Group and Sector average scores. The report also contains information on the number of survey Returns (responses) received for a Department for each Section, over the past 5 years, along with analysis of the proportion of different response types received ranging from Agree to Disagree for example. Note that the format of PTES changed substantially in 2014 so direct comparison with prior years is not possible.

21 Non Completions This report shows a breakdown of the data presented in the main Dashboard by Type and by Course and is the same format when run for either UG or PGT level.

22 Employment This report shows a breakdown of the data presented in the main Dashboard at Course level and is the same format when run for either UG or PGT level. It also includes comparison data from HESA in a Sector Comparison table which can be drilled through to get more detailed information.

23 DLHE This report shows a summary of DLHE positive outcomes based on data held in HEIDI and can be run for UG and PGT levels of study. Note that the underlying data from HEIDI is rounded and based on JACS codes, so a direct comparison with the Employment data presented in the main Dashboard is impossible. The report is sorted by the overall average of positive outcomes (defined as further study and/or employment) with Warwick highlighted for reference.

24 Results This report shows a breakdown of the data presented in the main Dashboard at Course level, and also shows a breakdown of the Good Honours percentage (Firsts and Upper Seconds as a percentage of the total) by Fee Status, Widening Participation and Course. As with many of the other drill-through reports this too can be run at either UG or PGT level.

25 Protected Characteristics This report shows a breakdown of the Protected Characteristics data for applicants through to enrolled students. This information is summarised in the main Dashboard but and can be viewed in more detail in this report which can be run for either Undergraduate or Postgraduate Taught levels of study. Protected Characteristics include Gender, Ethnicity and Disability. There is additional detail for Disability data which is then broken down by the Fee Status for those individuals.

26 5. Printing and Saving Reports 5.1 Printing Reports must be printed from PDF to ensure the formatting of reports is maintained from on-screen to print out. To print a report: 1. Click on the view options icon (shown left) in the top right of the dashboard screen. Select view in PDF format from the drop down menu. Your screen will re-load the dashboard in PDF viewer. 2. Click on the print icon to select your printer and print the report. Please note that when printing the departmental dashboard the PDF will include a number of blank pages (these pages will be populated with additional research management information as the project progresses). Remember to select print current page before printing your report. 3. To return to HTML view, click on the view options icon and select the required format. Drill down functionality is not available in PDF format. 5.2 Saving To save reports, select the save icon report view. when in PDF view. See instructions above for accessing PDF End of Document