Listening to Parents: Sample Focus Group and Survey Materials

Transcription

1 Listening t Parents: Sample Fcus Grup and Survey Materials Overview The Student Assessment Inventry fr Schl Districts supprts using a student-centered prcess t evaluate the assessments students currently take and then determine the minimum testing necessary t serve essential diagnstic, instructinal and accuntability purpses. The prcess shuld als lead t actins that ensure that every district-mandated assessment is f high quality, prvides the infrmatin needed fr specific schl and district purpses, and is supprted by structures and rutines s that assessment results are actually used and actin steps that will help students are taken. Parents are justifiably cncerned abut the amunt f testing their children experience in public schls. Furthermre, the layers f state, lcal and classrm testing d nt always add up t a chesive whle that infrms parents abut their child s prgress. Understanding hw parents perceive, understand and use assessments is essential t the inventry prcess and building the case fr actin. Reaching ut t parents t understand their perceptins arund assessments sends a clear signal that the district takes their cncerns abut testing seriusly and will help build mmentum leading t actin that will ultimately benefit students. This resurce is designed t guide districts in the steps necessary t design and use fcus grups and surveys t incrprate parent perspectives, experiences and expertise in the inventry prcess. It includes tw types f infrmatin. Instructins Achieve recmmends fllwing a set f steps t design, build, use and analyze parent surveys and fcus grups t effectively incrprate this infrmatin in the inventry prcess and build tward actinable recmmendatins. A set f guidance and resurces (beginning n page 2) addresses cmmn questins abut hw and when t use parent fcus grups and surveys within the inventry prcess. It als includes design cnsideratins district leaders shuld keep in mind as they develp these feedback instruments, use infrmatin gathered frm parents, and cmmunicate findings t participants and stakehlders. Districts experienced in designing fcus grups and surveys may feel cmfrtable skipping this sectin and mve directly t the prtcls, beginning n page 7. Inventry Tls. This resurce prvides parent fcus grup and survey questins that district leaders can use r adapt as part f their lcal assessment inventry prcess. Taking the Temperature n Assessments (beginning n page 7) fcuses n illuminating parent perspectives n the district s assessment prgram as a whle t build a greater understanding f the testing envirnment and help build a case fr actin. Depending n district cntext, leadership teams may decide t use fcus grups r surveys t illuminate parent perspectives, experiences and expertise. As such, this guide includes questins apprpriate fr each f these frmats. CC BY 4.0 Achieve Districts may use r adapt. If mdified, please attribute Achieve and re-title.

2 Instructins: Guidance and Resurces This guidance is intended t assist district leaders in develping a prcess fr using parent fcus grups and/r surveys as part f the assessment inventry prcess. 1. DETERMINING DISTRICT NEEDS FOR PARENT FEEDBACK BASED ON THE REFLECT AND PLAN QUESTIONS IN THE STUDENT ASSESSMENT INVENTORY FOR SCHOOL DISTRICTS At the utset f the assessment inventry prcess, district leaders may want t refer t the Reflect and Plan questins n page 4 f the Student Assessment Inventry fr Schl Districts. These questins ask districts t think abut the cntext, bjectives and structure f the inventry prcess. District leaders are then encuraged t develp a strng understanding f parent perceptins f testing thrugh the Taking the Temperature n Assessments fcus grup and survey prtcls beginning n page DECIDING WHETHER TO USE A FOCUS GROUP, SURVEY OR BOTH Districts have multiple issues t cnsider when determining whether t use fcus grups r surveys t gauge parent perceptins. Fcus grups allw fr additinal prbing and fllw-up questins. On the ther hand, fcus grups are generally mre cstly than surveys t rganize, staff and analyze, and they require access t skilled fcus grup mderatrs. Surveys allw districts t gather infrmatin frm a large number f parents in a relatively shrt perid f time, and answers can be easily aggregated. Parents may als feel mre cmfrtable cmpleting a survey than talking in a fcus grup. Hwever, surveys d nt allw fr additinal prbing questins, and respnse rates may be lw, especially if a survey takes t lng t cmplete. Districts are encuraged t carefully weigh these issues in light f resurce and timeline cnsideratins, as well as lcal cntext and needs. 3. DESIGNING A SUCCESSFUL FOCUS GROUP OR SURVEY PROCESS There are multiple design cnsideratins fr fcus grups and surveys. These cnsideratins include fcus grup and survey administratin, participatin, setting and design. The fcus grup and survey prtcls included in this resurce are penly licensed, and districts are encuraged t add and/ r adapt questins and prtcls as needed based n lcal pririties and cntext. Belw, fcus grup and survey design cnsideratins are described separately. Fcus Grup Design Cnsideratins Wh shuld administer a fcus grup? District leaders can chse t administer their wn fcus grups r wrk with an utside rganizatin with experience cnducting fcus grups. If district leaders decide t keep fcus grups in huse, it is imprtant that fcus grup mderatrs are cmfrtable speaking with parents, have a high level f rapprt with participants, and are peple with whm parents are likely t feel they can be pen and hnest. If district leaders have limited time r capacity fr cnducting fcus grups, they can cnsider using the expertise f lcal institutes f higher educatin, cmmunity-based rganizatins fcused n educatin and cnsultants with fcus grup expertise. 2

3 Wh shuld participate in a fcus grup? We encurage district leaders t make every effrt t recruit parents with diverse perspectives fr fcus grups. Districts shuld strive t select a representative sample f parents acrss multiple criteria (e.g., grade level r span; schl; r student subgrup, such as English language learner r special educatin). District leaders shuld als make effrts t ensure that fcus grups are demgraphically representative f the cmmunity, taking int cnsideratin characteristics such as race/ethnicity, gender, neighbrhd, etc. Additinally, district leaders will want t carefully cnsider the number f fcus grups that will be needed t get a detailed picture f a district s assessment landscape, as well as the number f participants that can be reasnably accmmdated in any ne fcus grup. T allw all participants multiple pprtunities t cntribute t the cnversatin, it is suggested that fcus grups nt exceed 10 individuals. It is als recmmended that the fcus grup mderatr be accmpanied by a dedicated nte-taker s that all respnses can be recrded. What are the best pprtunities t cnduct a fcus grup? Districts r cmmunity rganizatins can use a range f frmal and infrmal pprtunities t gather feedback thrugh fcus grups. There may be already-scheduled events where large numbers f parents will be present, such as back-t-schl nights, which can be cnvenient and minimally disruptive fr participants. Gathering infrmal feedback thrugh small ad-hc fcus grups is anther key pprtunity, especially fr districts with limited resurces r shrter timelines fr the inventry prcess. Hw can districts ensure that parent fcus grups are diverse and representative f the student ppulatin? Reaching a representative sample f parents fr fcus grups presents unique challenges fr districts. Districts can develp a set f strategies prir t engaging in fcus grups. First, districts are encuraged t partner with cmmunity-based rganizatins t reach ut t parents frm underrepresented grups. Secnd, when develping fcus grup participatin lists, districts shuld cmpare the demgraphics f parent participants t student demgraphics t ensure that a representative grup f parents has been reached when initial runds f fcus grups are cmpleted. Districts shuld cntinue t mnitr participant demgraphics thrughut the prcess t strive fr representativeness. Finally, districts shuld cnsider translating fcus grups prtcls int additinal languages spken by parents and students and having translatrs lead r assist with parent fcus grups where needed. Hw shuld districts prepare parents in fcus grups t talk abut testing? Parents have a wide range f experiences with and knwledge f testing, and it is imprtant fr fcus grup participants t have a cmmn understanding f assessments befre beginning the cnversatin. Sme parents may have little knwledge f the assessments students are taking, and the fcus grup is an pprtunity t build their awareness and understanding. Districts can use a variety f strategies t build this knwledge f testing. Districts might cnsider sharing a shrt definitin f testing (including different kinds f tests) with parents prir t the fcus grup cnversatin, as well as samples f the different kinds f summative, interim and frmative assessments that their children take. The inset n page 4 includes a sample descriptin f thse three cmmn types f assessment. 3

4 Distinguishing Amng Assessment Types Summative, Interim and Frmative 1 Bradly speaking, there are three assessment types summative, frmative and interim. Each type f assessment has its wn purpses and ways the infrmatin will be used. A summative assessment is generally given at the end f sme unit f instructin, such as a unit f study, curse r grade, t evaluate students perfrmance against a set f standards. These tests ften have imprtant uses such as fr accuntability and public reprting at the schl and district levels, and in sme states, fr student-level stakes such as prmtin and graduatin, r fr educatr evaluatin systems. Summative assessments are als ften used t better understand student grwth ver time, t ensure quality, cnsistency and alignment f instructin, r t prvide infrmatin t evaluate larger-scale educatinal prgrams. Frmative assessment is a prcess used by teachers and students during instructin that prvides feedback t adjust nging teaching and learning t imprve students achievement. Thus, it is administered by the teacher in the classrm fr the purpse f diagnsing where students are in their learning, where gaps in knwledge and understanding exist, and hw t help imprve student learning. The assessment is part f a classrm learning activity and linked directly t the teacher s instructin. The assessments are small scale, taking nly a few secnds r a few minutes. Furthermre, the tasks presented may vary frm ne student t anther depending n the teacher s judgment abut the need fr specific infrmatin abut a student at a given pint in time. Prviding helpful feedback, mdifying instructin t imprve the student s understanding r indicating areas fr further instructin are essential aspects f a classrm frmative assessment. Examples f frmative assessments include teachers asking students t explain the methds they used t slve a prblem r administering a shrt quiz. An interim assessment is the term fr the assessments that fall between frmative and summative assessment. Interim assessments (1) evaluate students knwledge and skills relative t a specific set f standards, typically within a limited time frame, and (2) are designed t infrm decisins at the classrm level and schl r district level. Cnsequently, while interim assessments may be given at the classrm level t prvide infrmatin t the teacher, the results can be reprted at a brader level than thse f frmative assessments. These assessments may serve a variety f purpses, including predicting a student s likelihd f achieving a certain scre n a statewide summative assessment, evaluating a lcal assessment prgram, r identifying gaps in a student s learning. Many f the assessments currently in use that are labeled benchmark r predictive fall within this definitin f interim assessments. 1 This descriptin is adapted frm The Rle f Interim Assessments in a Cmprehensive Assessment System (Achieve, 2008). 4

5 Survey Design Cnsideratins Hw shuld surveys be used t generate the mst useful infrmatin? District leaders shuld address several cnsideratins as they design surveys as part f the inventry prcess. Chse nly the questins that are mst likely t generate the infrmatin needed. In priritizing sme questins ver thers, be sensitive t the time needed t cmplete the survey. Chse questin types that best match needs and capacity t analyze data. A variety f questin types are included in these sample surveys, all with different implicatins fr data analysis. Fr instance, pen-ended questins have the benefit f enabling respndents t prvide their persnal feedback, but they can be time cnsuming t analyze when lking at hundreds f respnses. Cnversely, multiple-chice respnses are very easy t cde fr data analysis but d nt typically give respndents an pprtunity t prvide in-depth feedback. After identifying the questins that wuld be mst helpful, edit them as needed t ensure that they fit the lcal cntext and will resnate with respndents. Phrases in brackets within the template belw indicate places where the language needs t be custmized r remved prir t being released publicly. These areas shuld be updated with the apprpriate infrmatin. Decide whether t ask respndents t prvide the name f their child s schl r ther identifying infrmatin as part f this survey. If these types f questins are included, it is highly recmmended that they be clearly listed as ptinal and be placed at the end f the survey s individuals are fully aware f what will be shared prir t prviding their infrmatin. Prir t implementing the survey, try ut the survey items n a small number f ptential survey participants t get their feedback n the questins asked, clarity f questins and length f time the survey tk. Wh shuld administer a survey? Surveys can be administered by a district r third-party rganizatin. District leaders may want t administer their wn surveys if they have prir experience develping and administering them, have had gd respnse rates with previus surveys, and/r have sufficient capacity and expertise. District leaders may want t cnsider using an external, third-party rganizatin t administer a survey if sufficient resurces exist t hire an utside firm with expertise, the district has little experience designing and administering its wn surveys, and/r privacy and annymity f results are significant cncerns. Wh shuld participate in a survey? Depending n district size, resurces and capacity, district leaders can chse t either survey all parents (census survey) r select a representative sample f parents acrss multiple criteria (e.g., grade level r span; schl; r student subgrup, such as English language learner r special educatin). Districts shuld als strive t cllect surveys frm a demgraphically representative sample f parents, taking int cnsideratin characteristics such as race/ethnicity, gender, neighbrhd, etc. (see belw). If initial survey results are nt representative f the district s student demgraphics, districts shuld cnsider targeting additinal individuals frm underrepresented grups with fllw-up survey requests. Hw can districts ensure that survey respnses are frm a diverse and representative sample f parents? Similar t fcus grups, reaching a representative sample f parents thrugh a survey presents unique challenges fr districts. First, districts shuld cnsider translating parent surveys int additinal languages spken by parents and students. 5

6 Secnd, if the survey is paper based, districts can use a range f frmal and infrmal pprtunities t gather survey respnses, in much the same way as fcus grups described abve. Fr example, back-t-schl nights and ther scheduled events where large numbers f parents will be present are natural pprtunities t have parents cmplete surveys. Fr nline surveys, districts shuld be aware that nt all parents have internet access and prvide multiple pprtunities fr example, at schls and libraries fr parents t cmplete surveys. Third, districts may want t cnsider partnering with cmmunity-based rganizatins t reach ut t parents frm underrepresented grups. Finally, when initial runds f surveys are cmpleted, districts shuld cmpare the demgraphics f parent participants t student demgraphics t ensure that a representative grup f parents has been reached. 4. NEXT STEPS: ANALYSIS, USE AND COMMUNICATION OF RESULTS After districts have cnducted all parent fcus grups and/ r surveys, infrmatin gathered frm these surces can be analyzed and incrprated int the Inventry Table (see pages 8 and 9 f the Student Assessment Inventry fr Schl Districts). It is recmmended that districts als cmmunicate results f the fcus grups and surveys t participants (and depending n timing and cntext, the public) t demnstrate that parents vices are being heard in the inventry prcess. Analyzing and using results. Fcus grups and surveys are part f the inventry prcess and will help infrm the Inventry Table, verall analysis and recmmendatins. Please refer t pages 5 6 f the Student Assessment Inventry fr Schl Districts. Analyzing the respnses may raise additinal questins abut parent perceptins and experiences and impact n students. Fr example, d particular assessments mentined by parents fall within a specific grade level r grade band, r are these assessments taken by a specific subgrup f students? Are there cncerns abut the amunt f testing in particular grades r subjects? This infrmatin frm parents may als help infrm the district abut cmmunicating the results f particular tests t parents, as well as infrm systemic cmmunity needs arund assessment and data literacy. Cmmunicating the results t participants and the public. It is imprtant that district leaders cmmunicate t fcus grup and survey participants that their respnses and suggestins are being heard. Examples f cmmunicatin t participants culd include a letter t participants with a summary f what was learned thrugh the sessins r survey, an invitatin t a district r bard meeting during which results will be discussed, and a timeline fr mving frm feedback t recmmendatins t actin. District leaders may als want t cnsider releasing a summarized set f findings frm the fcus grups and surveys t the brader public as an interim step during the inventry prcess. Additinal Resurces These additinal resurces may be helpful in designing and administering surveys and fcus grups. Achieve (2012). Feedback Lps fr Cmmn Cre Standards Implementatin: Survey Tls fr State Educatin Leaders. TOSHAREv2.pdf Duke Initiative n Survey Methdlgy. Pew Research. Questinnaire Design. 6

7 Taking the Temperature n Assessments This sectin, which includes sample fcus grup and survey questins, fcuses n illuminating students perspectives n the district s assessment prgram as a whle t build a greater understanding f the testing envirnment. It is recmmended that these questins be asked f students tward the beginning f the prcess, prir t the Inventry Table being cmpleted. Cnducting fcus grups r administering surveys early in the prcess helps make the case fr why an inventry prcess is needed and will lead t cncrete actin t ensure that every assessment being used is useful and f high quality. The prtcls and questins listed belw are penly licensed, and districts are encuraged t add and/r adapt questins and prtcls as needed based n lcal pririties and cntext. TAKING THE TEMPERATURE ON ASSESSMENTS: SAMPLE FOCUS GROUP QUESTIONS INTRODUCTION AND WARM-UP A warm-up perid fr fcus grups is imprtant t help participants understand the gals fr the sessin, create a cmmn understanding arund hw participants and the mderatr will interact, and build cmfrt amng participants and with the mderatr. Belw are a suggested set f grund rules and an intrductry exercise, fllwed by sample fcus grup questins. Nte t mderatr: Prir t cnducting this fcus grup, it is imprtant t be clear with participants abut the types f assessments fr which the district wuld like feedback. Will there be discussin nly f assessments that are given acrss multiple classrms r schls as part f a district/ schl assessment prgram, r is the district als interested in teacher-develped classrm assessments? This shuld be discussed as part f the intrductin and warm-up. Add and/r adapt questins t best meet yur lcal needs and cntext. Intrductin f mderatr and fcus grup guidelines: > We re ging t be talking abut testing in public schls. This will help the district better understand the vlume, quality and use f tests given in the district. > The district will lk at all f its testing thrugh the perspective f students. It is imprtant t 7

8 understand hw different kinds f students experience testing. That is why we have invited parents f students frm different grades, schls and backgrunds t participate in these fcus grups. > This fcus grup is part f a larger prcess t examine many kinds f tests given t students. Thrugh this prcess, the district will make decisins abut which tests are high quality and necessary fr students t take and which tests can be mdified r eliminated. > One persn shuld speak at a time. > There are n wrng answers t any f these questins. We are interested in hearing yur perspectives as parents. > We value cnfidentiality. We will nt assciate any feedback that cmes ut f this fcus grup with a particular individual. > Everyne will have a chance t speak. If yu have nt had an pprtunity t prvide yur perspective, I may call n yu. > Please turn ff r silence yur cell phnes. > Are there additinal guidelines the grup wuld like t add? Disclsure f nte-taking: > A nte-taker is present t make sure that we get all f yur feedback. The nte-taker will nt assciate cmments with names. Parking lt: > There is a parking lt chart/bard/area where we can put ideas r thughts that cme up in ur discussin that are imprtant but may nt be related t the purpse f this grup. We want t capture thse imprtant thughts, but we als want t keep fcused n the purpse f ur meeting. These ideas r thughts will be shared with apprpriate individuals fllwing the meeting. Cmmunicating results f fcus grups: > The results f the fcus grups will be summarized, and yu will receive a summary by {date}. If we missed any key pints yu raised during this cnversatin, please let us knw. Intrductin t testing (see page 4 fr mre infrmatin): > Types f tests given t students > Examples f tests given in the district > What questins d yu have abut testing befre we begin? Intrductry exercise (pssible ptins): > Hw lng have yu lived in this cmmunity? > What kinds f activities are yur kids invlved in? > What d yu like t d fr fun? 8

9 FOCUS GROUP QUESTIONS 1. What grade(s) are yur child(ren) in? Which schl(s) d they attend? 6. What are sme examples f tests that yu have fund useful in mnitring yur child s prgress in schl? Why have they been useful? (Mderatr prmpt: results are timely and clear; test measures imprtant cncepts, predicts later perfrmance r readiness fr cllege and careers) 2. What kinds f tests des yur child take? (Mderatr can use specific test names r prmpt: classrm tests designed by teachers such as quizzes, chapter tests r finals; tests given t all students in a schl r district; state-mandated tests) 3. D yu think students in this district are given t many tests, nt enugh tests r abut the right number f tests thrughut the schl year? Why? 7. What are sme examples f tests that yu have nt fund useful in mnitring yur child s prgress in schl? Why have they nt been useful? (Mderatr prmpt: results are nt timely r clear; test desn t measure imprtant cncepts, is nt helpful in predicting later perfrmance r readiness fr cllege and careers) 8. What tests, if any, wuld yu suggest the district cntinue t administer? Why? 4. Frm whm d yu receive infrmatin abut testing? (Mderatr prmpt: schl principal, teachers, district staff, ther parents, students, the media) 9. What tests, if any, wuld yu suggest the district cnsider eliminating r changing significantly? Why? (Mderatr prmpt: perfrmance based assessments, mre diverse item types, mre r less time spent n tests) 5. What are yu hearing abut testing frm ther parents? What are yu hearing frm yur children? 10. Are there any ther suggestins yu want t make t the district as it reviews its testing prgram? 9

10 TAKING THE TEMPERATURE ON ASSESSMENTS: SAMPLE SURVEY QUESTIONS Sample language t intrduce the survey: This survey asks questins abut testing given in this district. It is part f a larger prcess t examine all administered tests in the district and make decisins abut which tests are high quality and necessary fr students t take and which tests can be mdified r eliminated. Yur respnses t this survey are very imprtant and will help shape recmmendatins fr the future f the district s testing prgram. Finally, yur respnses t this survey are cnfidential we d nt ask fr any persnally identifying infrmatin. If yu have questins abut this survey, please cntact { }. 1. What grade(s) are yur child(ren) in? {district shuld preppulate with grades} Nte t survey designer: Prir t administering this survey, it is imprtant t be clear with respndents abut the types f assessments fr which the district wuld like feedback. Is the primary fcus n assessments given acrss multiple classrms r schls as part f a district/schl assessment prgram, r is the district als interested in teacher-develped classrm assessments? It may be helpful t prvide a descriptin f different types f assessments (see page 4 fr sme sample language).the district shuld als explain hw results f the survey will be used, whether district stakehlders will be presented with a summary f findings and hw these findings will be used t infrm actin. 4. What level f cncern have yu heard abut the time spent n testing frm ther parents? N cncern 2. D yu think students in this district are given t many tests, nt enugh tests r abut the right number f tests thrughut the schl year? Nt enugh tests Abut the right number f tests Sme cncern A lt f cncern 5. What cncerns abut testing in the district, if any, d yu have? {pen-ended item} T many tests 3. D yu think the time spent n testing in this district is t much, t little r abut right? T little Abut right T much 6. Frm whm d yu receive infrmatin abut testing? Schl principal Teachers District staff Other parents Students Media (televisin, radi, internet) Other (please describe): 10

11 7. What are sme examples f tests that yu have fund helpful in understanding yur child s prgress? {the district can preppulate this item with a list f assessments given in the district if knwn} Des nt give me ideas n hw t help my child Other 8. In general, what aspects f these tests make them helpful? Cnnected t what my child is learning in the classrm Results are available sn after taking the test Scres and infrmatin prvided t me are easy t understand Prvides ideas n hw t help my child Results shw me hw my child s scres cmpare t the scres f ther students in his r her schl, district and/r state Other 9. What are sme examples f tests that yu have nt fund helpful fr understanding yur child s prgress? {the district can preppulate this item with a list f assessments given in the district if knwn} 11. What tests, if any, wuld yu suggest the district cntinue t use as it des tday? {pen-ended item OR district can list specific assessments} 12. What tests, if any, wuld yu suggest the district think abut eliminating? {pen-ended item OR district can list specific assessments} 13. Are there subjects r areas yu think the district shuld be testing where it is currently nt ding s? {pen-ended item} 14. Are there any ther suggestins yu want t make t the district as it reviews its testing prgram? {pen-ended item} 10. In general, what aspects f these tests make them nt helpful? D nt understand cnnectin t what my child is learning in class I d nt receive results quickly Scres and infrmatin prvided t me are difficult t understand 11