Quality Indicators for Learning and Teaching (QILT) Surveys and Website Briefing Paper 6 July 2015

Transcription

1 Quality Indicatrs fr Learning and Teaching (QILT) Surveys and Website Briefing Paper 6 July 2015 Alignment t Securing Success Gal 1: A Distinctly Student-Centred University Sub-bjective: 1.10 Mnitr and evaluate the quality f student learning and supprt t imprve students engagement, utcmes and satisfactin Purpse Prvide a brad verview f the Quality Indicatrs fr Teaching and Learning (QILT) surveys and website and t discuss pssible actins fr UWS in respnse t these measures. Backgrund and verview Quality Indicatrs fr Learning and Teaching (QILT) is a suite f natinal, gvernment endrsed, higher educatin surveys that are intended t prvide prspective students with infrmatin n a range f teaching and perfrmance measures, including the experiences f university students, emplyer satisfactin with graduate skills, and graduate labur market utcmes. The QILT surveys cnsist f: the Student Experience Survey (SES) (frmerly the University Experience Survey) 1 the Graduate Outcmes Survey (GOS) and GOS-L (Lngitudinal) (frmerly the Australian Graduate Survey) 2 the Emplyer Satisfactin Survey (ESS) (a newly develped survey) Attachment A prvides further detail n the purpse, scpe and data cllectin methds assciated with each survey. QILT surveys will be administered by the Scial Research Centre (SRC) under cntract t the Federal Department f Educatin. Results frm the QILT surveys will be presented n a new website which will replace the MyUniversity website, with full implementatin f the website by August The SRC will administer all surveys, including selecting and cntacting students and graduates, and publishing survey results. Higher educatin prviders will receive data frm the surveys relating t their students and graduates. Analysis The purpse f the QILT website is t prvide enhanced search and cmparisn functins fr users at bth the institutins and study area level. The SRC has advised that the fllwing infrmatin and functins will appear n the QILT website: A searchable interface where students can search fr 21 study areas r by institutin. Attachment B prvides screen shts f these functins. Infrmatin will be presented against the fllwing dmains: 1 The UES was renamed the SES t accmmdate the participatin f nn-university higher educatin prviders. The questinnaire and methdlgy fr the SES will remain unchanged frm the UES. Data cllectin fr the SES will cmmence in August the GOS will replace the current AGS survey in Nvember The GOS questinnaire will have greater alignment t the Australian Bureau f Statistics Labur Frce Survey. The GOS-L will succeed the Beynd Graduate Destinatin Survey and use the same methdlgy as the GOS. Data cllectin fr the GOS will cmmence in Nvember P age

2 Overall quality f educatin experience, teaching quality, learner engagement, learning resurces, student supprt and skill develpment frm the Student Experience Questinnaire (SEQ) Overall satisfactin, teaching and generic skills scales frm the Curse Experience Questinnaire (CEQ) Emplyment rates/utcmes, further study rates and median salary frm the GDS. Students will be able t cmpare up t six prviders with data nly t be displayed where there are at least 25 respnses in a study area. The SRC als prvides a number f fee-fr-service activities that institutins can elect t intrduce int their surveys. These services include the intrductin f additinal questins and ut f scpe ppulatins t a survey sample as well as the use f a telephne fllw-up service. These ptins are discussed in further detail belw. Table A prvides an verview f UWS s perfrmance against the sectr average fr the main items frm the CEQ, GDS and UES which are mst likely t be included n the QILT website. There may be sme differences with the emplyment data due t the change in hw sme questins in the GOS will be asked. UWS has previusly perfrmed abve the sectr average in bth the CEQ and the UES. Hwever, UWS has perfrmed belw the sectr average acrss mst measures f the GDS. Table A Ptential QILT Indicatrs CEQ Explicit Satisfactin 2013 UWS Sectr Clear Gals and Standards 67% 61% Generic Skills 81% 77% Gd Teaching 71% 67% Overall Satisfactin 84% 83% 2014 UES - % Satisfied Cmmencing UG UWS Sectr Learner Engagement 61% 60% Learning Resurces 90% 88% Skills Develpment 81% 79% Student Supprt 76% 76% Teaching Quality 82% 84% 2014 UES - % Satisfied Later Year UG UWS Sectr Learner Engagement 65% 63% Learning Resurces 86% 81% Skills Develpment 85% 84% Student Supprt 66% 68% Teaching Quality 81% 79% GDS - % Emplyed P age

3 Undergraduate Australian Permanent Resident UWS Sectr Wrking full-time, 38.1% 45.7% Wrking part-time 39.0% 33.8% Nt Wrking/Unavailable fr wrk 22.9% 20.5% GDS - % in Further Full Time Study 2013 Undergraduate Australian Permanent Resident UWS Sectr Full-time study 27.7% 20.8% Issues fr UWS Standardisatin f data cllectin and prcessing: The management f QILT surveys by the SRC is a psitive mve t ensure the standardisatin f data cllectin and prcessing acrss institutins. This apprach will ensure a level playing field and remve scpe fr prviders t use data cllectin strategies that will guarantee favurable utcmes. UWS has ensured integrity f data cllectin and reprting standards; cntinuatin f these standards by the SRC wuld be expected. Adding items t the survey instruments: Institutins have the ptin t pay t have additinal questins 3 r ppulatins 4 added t their institutinal survey fr bth the GOS and SES. There are strict timeframes fr institutins wishing t include additinal items t their survey instruments. Attachment D prvides a calendar f key deadlines. Additins t survey instruments attract a ne-ff fee and can then be used in subsequent survey cllectins fr the respective institutin making the purchase. Further infrmatin n the csts f additinal questins and ppulatins is prvided at Attachment C. UWS t encurage graduate student participatin in surveys: While the SRC has indicated that they will use a prize draw incentive prgram t encurage early cmpletin, UWS culd cnsider further internal marketing and prmtin t increase graduate awareness and cmpletin f the GOS survey. Survey fllw-up ptins: The SRC will use an nline platfrm as the primary mde f data cllectin fr the SES and GOS. Fr institutins that wuld like t achieve specific respnse rates, the SRC will allw institutins t tp-up these surveys with Cmputer Aided Telephne Interviewing (CATI) as a fee fr service activity (see Attachment C fr fees). Shuld UWS wish t pursue this service, at least ne telephne number fr each student r graduate will need t be prvided t the SRC. Institutins wishing t use CATI must advise the SRC at least three mnths prir t data cllectin. Privacy implicatins, student details and permissins: all institutins, including UWS, will be required t cnsider privacy and the prvisin f student s (persnal s fr the GOS), and telephne numbers t the SRC. Recmmendatins 3 Examples f custm questins include: (i) pen-ended items cnsisting f free-text r verbatim respnses t qualitative questins; (ii) single items, characterised by a single qualitative questin with an answer based n a yes/n respnse r a rating scale; (iii) grid item(s) that cmbine a number f single like items that share the same respnse frame. 4 Examples f ut-f-scpe ppulatins include dmestic and/r internatinal nshre middle years; ffshre undergraduate and pstgraduate; Australian higher educatin nn-accredited curses; internatinal institutins 3 P age

4 Management t ensure that students and staff are adequately infrmed and prepared fr QILT The Executive t decide whether additinal questins r ppulatins shuld be added t UWS s survey file, nting that such changes need t be prvided befre the deadlines in Attachment D. The Executive t decide whether t use a telephne nn-respnse fllw up service, particularly in the event that UWS respnse rate drps belw the benchmark figure (30 per cent), nting that the SRC will need t be ntified f this decisin three mnths prir t the next data cllectin. The Executive t agree the level f respnse rate that it seeks t achieve. The Executive t give cnsideratin t pssible cmmunicatin ptins and strategies t encurage survey cmpletin amngst graduating students. Prepared by Vineta Lpaticka Senir Plicy Officer, Higher Educatin, Plicy and Prject Office f the Deputy Vice-Chancellr, Academic Attachments Attachment A Overview f QILT Surveys Attachment B QILT Website Screen Shts Attachment C Csts f Additinal Survey Items Attachment D QILT Calendar 4 P age

5 Attachment A Overview f QILT Surveys 5 The Student Experience Survey (SES) Purpse The SES measures the satisfactin f current students curse experiences based n measures f student supprt, learner resurces, learner engagement, and teaching and develpment utcmes. Frm 2015, the Student Experience Survey (SES) will be renamed the University Experience Survey (UES). The SES will als include nn-university higher educatin prviders. Scpe and Implementatin Frm 2015 the in-scpe ppulatin fr the SES will include cmmencing and final year undergraduate higher educatin students wh are nshre in Australia. Institutins with fewer than 200 students verall, will have all f their students included in the survey. The fieldwrk perid fr the SES runs frm the first week in August fr fur weeks. Data cllectin will use an nline platfrm with a prize draw incentive prgram t encurage early cmpletin. The Graduate Outcmes Survey (GOS) and GOS-L Purpse The GOS fcuses n graduate destinatins, specifically further study r labur market. Frm , the Graduate Outcmes Survey (GOS) will replace the Australian Graduate Survey (AGS) which was administered by Graduate Careers Australia. The GOS-L is the lngitudinal cmpnent f the GOS and will cmmence in It is envisaged that the GOS-L will have similar items t the GOS and will fllw up graduates wh have cmpleted the initial GOS every year fr three years after graduatin. Develpment Institutins were cnsulted t identify use f items in the AGS and gaps in natinal perfrmance indicatr data. This feedback, tgether with a review f the literature, was used t develp a draft Graduate Outcmes Questinnaire (GOQ). The GOQ has been cgnitively tested and a full pilt test cnducted in May Data frm the pilt test will be used t refine the methdlgy. The final GOQ instrument is expected t be validated and deplyed in Nvember Scpe and Implementatin The in-scpe ppulatin fr the GOS includes undergraduate and pstgraduate students wh cmpleted their qualificatins in the six mnths prir t the data cllectin. The fieldwrk perid fr the GOS cnsists f tw cllectins per year starting frm the launch dates f Octber 31 and April 30. Data is cmbined frm the tw runds t frm the reprting requirements fr a cllectin year. Octber runds include graduates frm January t June f that calendar year and the April runds include graduates frm July and December f the 5 Infrmatin presented in this sectin was surced frm the Scial Research Centre 5 P age

6 preceding calendar year. The fieldwrk perid fr Octber 2015 cmmences the first week in Nvember fr fur weeks with similar timings ccurring early May fr the April 2016 rund. Data will be cllected thrugh an nline platfrm cnsistent with the SES and include an incentive prgram, such as a prize draw, t encurage early cmpletin. The Emplyer Satisfactin Survey (ESS) Purpse Assesses the skills (technical and generic) and wrk readiness f graduates. The ESS represents a new survey. Scpe and Implementatin The ESS was develped by the University f Sydney In 2013, a pilt ESS was cnducted, surveying the supervisrs f graduates frm fur universities acrss five fields f educatin. Tw trials f the ESS are being undertaken: A methdlgical trial in cnjunctin with the Octber 2014 AGS A cntent trial in cnjunctin with the April 2015 AGS. The ESS methdlgy was reviewed and will be trialled with the GOS in Nvember 2015 The ESS will be cnducted as an adjunct t the GOS; all emplyment supervisrs f GOS graduates are in-scpe fr the ESS. Cntact with emplyment supervisrs takes place after respnses are cllected frm the GOS. A draft questinnaire f the ESS suggests that the survey will feature 6 sectins/mdules: Screening and cnfirmatin Overall graduate preparatin Graduate attributes scale Emerging plicy issues Discipline specific issues Clse 6 P age

7 Attachment B QILT Website Screen Shts 7 P age

8 Attachment C Csts f Additinal Survey Items Custm questin csts Csts t incrprate custm items $ (AUD) exc. GST Fee estimate per custm single item r pen ended item $796 Fee estimate per custm grid item $265 Out f scpe ppulatin csts Csts t incrprate additinal ppulatins fr HEIMS sample $ (AUD) exc. GST Fee per institutin 3,348 Fee per prjected nline cmplete 1.36 Fee per Australian apprach letter 1.04 CATI Fees 2015 Item $ (AUD) exc. GST SES per cmpleted survey within Australia SES per cmpleted survey utside Australia GOS per cmpleted survey within Australia GOS per cmpleted survey utside Australia P age

9 Attachment D QILT Calendar 9 P age