PISA FOR SCHOOLS How is my school comparing internationally? Andreas Schleicher Director for Education and Skills OECD Madrid, September 22 nd
PISA in brief Over half a million students representing 28 million 15-year-olds in 65 countries/economies took an internationally agreed 2-hour test Goes beyond testing whether students can reproduce what they were taught to assess students capacity to extrapolate from what they know and creatively apply their knowledge in novel situations Mathematics, reading, science, problem solving, financial literacy Total of 390 minutes of assessment material and responded to questions on their personal background, their schools and their engagement with learning and school Parents, principals and system leaders provided data on school policies, practices, resources and institutional factors that help explain performance differences.
PISA in brief Key principles Crowd sourcing and collaboration PISA draws together leading expertise and institutions from participating countries to develop instruments and methodologies guided by governments on the basis of shared policy interests Cross-national relevance and transferability of policy experiences Emphasis on validity across cultures, languages and systems Frameworks built on well-structured conceptual understanding of academic disciplines and contextual factors Triangulation across different stakeholder perspectives Systematic integration of insights from students, parents, school principals and system-leaders Advanced methods with different grain sizes A range of methods to adequately measure constructs with different grain sizes to serve different decision-making needs e.g. PISA for Schools Productive feedback to fuel improvement at every level of the system.
4 PISA 2012 Sample Question 2 Helen the Cyclist Helen has just got a new bike. It has a speedometer which sits on the handlebar. The speedometer can tell Helen the distance she travels and her average speed for a trip. Helen rode 6 km to her aunt s house. Her speedometer showed that she had averaged 18 km/h for the whole trip. Which one of the following statements is correct? A. It took Helen 20 minutes to get to her aunt s house. (answer code: pisa2a) B. It took Helen 30 minutes to get to her aunt s house. (answer code: pisa2b) C. It took Helen 3 hours to get to her aunt s house. (answer code: pisa2c) D. It is not possible to tell how long it took Helen to get to her aunt s house. (answer code: pisa2d)
5 PISA 2012 Sample Question 2 Helen the Cyclist Correct Answer: A. It took Helen 20 minutes to get to her aunt s house. This item belongs to the change and relationships category. This involves understanding fundamental types of change and recognising when they occur in order to use suitable mathematical models to describe and predict change. SCORING: Description: Mathematical content area: Context: Process: Calculate time travelled given average speed and distance travelled Change and relationships Personal Employ
Shanghai-China Singapore Hong Kong-China Korea Chinese Taipei Macao-China Japan Liechtenstein Switzerland Estonia Netherlands Finland Canada Poland Vietnam Germany Belgium Austria Ireland Denmark Australia Czech Republic Slovenia New Zealand France United Kingdom Iceland OECD average Latvia Norway Luxembourg Portugal Spain Italy Russian Federation Slovak Republic Sweden Lithuania United States Hungary Israel Croatia Greece Serbia Turkey Bulgaria Romania United Arab Emirates Kazakhstan Chile Thailand Malaysia Uruguay Montenegro Mexico Albania Qatar Costa Rica Brazil Argentina Tunisia Jordan Peru Colombia Indonesia 6 PISA 2012 Sample Question 2 Percent of 15-year-olds who scored Level 3 or Above 100 90 80 70 60 50 40 30 20 10 0
Mean score 580 570 560 550 High mathematics performance Shanghai-China performs above this line (613) Chinese Taipei Singapore Hong Kong-China Korea Average performance of 15-year-olds in Mathematics Fig I.2.13 540 530 520 510 500 490 480 470 460 450 440 430 420 410 Poland Belgium Germany Austria Slovenia New Zealand Denmark Czech Republic France Luxembourg Latvia Portugal Spain Slovak Republic United States Hungary Israel Greece Romania Chile Macao-China Japan Liechtenstein Switzerland Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia Serbia Turkey Bulgaria U.A.E. Kazakhstan Thailand Malaysia Mexico Low mathematics performance 12 countries perform below this line
High mathematics performance Chinese Taipei Singapore Hong Kong-China Korea Average performance of 15-year-olds in mathematics Strong socio-economic impact on student performance Poland Belgium Germany Austria Slovenia New Zealand Denmark Czech Republic France Luxembourg Latvia Portugal Spain Slovak Republic United States Hungary Israel Macao-China Japan Liechtenstein Switzerland Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia Socially equitable distribution of learning opportunities Greece Romania Chile Serbia Turkey Bulgaria U.A.E. Kazakhstan Thailand Malaysia Mexico Low mathematics performance
2012 Singapore Chinese Taipei Korea Hong Kong-China Strong socio-economic impact on student performance 26 Slovak Rep. 24 Japan Switzerland Liechtenstein Netherlands Estonia Poland Belgium Canada Finland Germany Viet Nam Denmark Austria New Zealand Australia Slovenia Ireland Czech Rep. Iceland 22France 20 18 16 14 12 10 8 6 4 UK Luxembourg Latvia Norway Portugal Italy US Russian Fed. Spain Lithuania Sweden Hungary Croatia Israel Macao-China Socially equitable distribution of learning opportunities 2 0 Chile Bulgaria Romania Greece Turkey Serbia United Arab Emirates Malaysia Kazakhstan Thailand Mexico
Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy impact on student Japan performance Korea Luxembourg Mexico Netherlands Slovak Rep. New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US Strong socio-economic Netherlands Estonia Poland Belgium Canada Finland Germany Denmark Austria New Zealand Australia Slovenia Ireland Czech Rep. Iceland France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Chile Switzerland Israel Greece Turkey Korea Japan Mexico 2012 Socially equitable distribution of learning opportunities
Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US
Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Luxembourg Mexico Netherlands Slovak Rep. New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US Shanghai Singapore Singapore Switzerland Korea Japan Netherlands Estonia Poland Belgium Canada Finland Germany Denmark Austria New Zealand Australia Slovenia Ireland Czech Rep. Iceland France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey Chile Mexico 2003-2012
13 Fostering resilience The country where students go to class matters more than what social class students come from
Mexico Chile Greece Norway Sweden Iceland Israel Italy United States Spain Denmark Luxembourg Australia Ireland United Kingdom Hungary Canada Finland Austria Turkey Liechtenstein Czech Republic Estonia Portugal Slovenia Slovak Republic New Zealand Germany Netherlands France Switzerland Poland Belgium Japan Macao-China Hong Kong-China Korea Singapore Chinese Taipei Shanghai-China 300 325 350 375 400 425 450 475 500 525 550 575 600 625 650 675 14 PISA mathematics performance by decile of social background Source: PISA 2012
Hong Kong-China Korea + Liechtenstein Macao-China + Japan Switzerland Belgium - Netherlands - Germany Poland + Canada - Finland - New Zealand - Australia - Austria OECD average 2003 - France Czech Republic - Luxembourg Iceland - Slovak Republic Ireland Portugal + Denmark - Italy + Norway - Hungary United States Sweden - Spain Latvia Russian Federation Turkey Greece Thailand Uruguay - Tunisia Brazil Mexico Indonesia Percentage of top performers in mathematics in 2003 and 2012 Fig I.2.23 % 40 30 2012 2003 Across OECD, 13% of students are top performers (Level 5 or 6). They can develop and work with models for complex situations, and work strategically with advanced thinking and reasoning skills 20 10 0
18 Why care about advanced skills? % 25 Evolution of employment in occupational groups defined by PIAAC problem-solving skills 20 15 10 Employment of workers with advanced problem-solving skills 5 0-5 -10-15 Employment of workers with medium-low problem-solving skills (PIAAC) Employment of workers with poor problem-solving skills -20 Source:PIAAC 2011
19 Math teaching math teaching PISA = reason mathematically and understand, formulate, employ and interpret mathematical concepts, facts and procedures
Viet Nam Macao-China Shanghai-China Turkey Uruguay Greece Hong Kong-China Chinese Taipei Portugal Brazil Serbia Bulgaria Singapore Netherlands Japan Argentina Costa Rica Lithuania Tunisia New Zealand Czech Republic Israel Korea Latvia Qatar Italy United States Estonia Ireland Australia Mexico United Arab Emirates Norway Malaysia Kazakhstan United Kingdom Romania OECD average Albania Colombia Indonesia Sweden Belgium Peru Thailand Denmark Russian Federation Canada Slovak Republic Hungary Germany Croatia Luxembourg Montenegro Chile Poland Finland Austria Slovenia France Switzerland Jordan Liechtenstein Spain Iceland Index of exposure to word problems Focus on word problems Fig I.3.1a 2.50 2.00 1.50 1.00 Formal math situated in a word problem, where it is obvious to students what mathematical knowledge and skills are needed 0.50 0.00
Sweden Iceland Tunisia Argentina Switzerland Brazil Luxembourg Ireland Netherlands New Zealand Costa Rica Austria Liechtenstein Malaysia Indonesia Denmark United Kingdom Uruguay Lithuania Germany Australia Chile OECD average Slovak Republic Thailand Qatar Finland Portugal Colombia Mexico Peru Czech Republic Israel Italy Belgium Hong Kong-China Poland France Spain Montenegro Greece Turkey Slovenia Viet Nam Hungary Bulgaria Kazakhstan Chinese Taipei Canada United States Estonia Romania Latvia Serbia Japan Korea Croatia Albania Russian Federation United Arab Emirates Jordan Macao-China Singapore Shanghai-China Ireland Index of exposure to formal mathematics Focus on conceptual understanding Fig I.3.1b 2.50 2.00 1.50 1.00 0.50 0.00
Japan Hong Kong-China Luxembourg Norway Czech Republic Iceland Korea Indonesia Thailand Mexico Denmark Liechtenstein Italy Austria Macao-China Turkey Belgium Canada Portugal Poland Spain OECD average 2003 Switzerland Brazil United States Greece Slovak Republic Netherlands Russian Federation Hungary Ireland New Zealand Australia Uruguay Sweden Latvia France Finland Germany Tunisia Mean index change In most countries and economies, the disciplinary climate in schools improved between 2003 and 2012 Fig IV.5.13 Change between 2003 and 2012 in disciplinary climate in schools 0.5 0.4 0.3 Disciplinary climate improved 0.2 0.1 0-0.1-0.2 Disciplinary climate declined -0.3
Mean mathematics performance OECD average Mean index of mathematics self-efficacy Countries where students have stronger beliefs in their abilities perform better in mathematics Fig III.4.5 650 Shanghai-China 600 550 500 450 400 350 Singapore Hong Kong-China Korea Chinese Taipei Japan Macao-China Switzerland Netherlands Estonia Finland Canada Liechtenstein Belgium Poland Germany Viet Nam Denmark Slovenia New Zealand Latvia Italy Portugal Austria Australia Russian Fed. Hungary Croatia Luxembourg Greece Slovak Republic Spain Turkey Israel Sweden Norway Serbia Lithuania Czech Republic U.A.E. United Kingdom Thailand Malaysia Romania Iceland Chile Bulgaria Kazakhstan Ireland United States Montenegro France Costa Rica Brazil Uruguay Mexico Albania Argentina Tunisia Colombia Qatar Jordan Indonesia Peru R² = 0.36 300-0.60-0.40-0.20 0.00 0.20 0.40 0.60 0.80 1.00 1.20
Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements Fig IV.1.16 School autonomy for curriculum and assessment x system's level of posting achievement data publicly Score points 478 476 474 472 470 468 466 464 School data public Less school autonomy School data not public More school autonomy
Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements Fig IV.1.16 School autonomy for curriculum and assessment x System's extent of implementing a standardised policy Score points 485 480 475 470 465 460 455 Less school autonomy Central mathematics standards No mathematics standards More school autonomy
Malaysia Singapore Korea Abu Dhabi (UAE) Finland Mexico Alberta (Canada) Flanders (Belgium) Netherlands Australia England (UK) Romania Israel United States Chile Average Norway Japan Latvia Serbia Bulgaria Denmark Poland Iceland Estonia Brazil Italy Czech Republic Portugal Croatia Spain Sweden France Slovak Republic Percentage of teachers Mean mathematics performance, by school location, after accounting for socio-economic status 26 Teachers' perceptions of the value of teaching Fig II.3.3 Percentage of lower secondary teachers who "agree" or "strongly agree" that teaching profession is a valued profession in society 100 90 80 70 60 50 40 30 20 10 0 Above-average performers in PISA
Share of mathematics top performers 27 Countries Mean mathematics where teachers performance, believe by school their profession location, is valued show after higher accounting levels of for student socio-economic achievement status Fig II.3.3 Relationship between lower secondary teachers' views on the value of their profession in society and the country s share of top mathematics performers in PISA 2012 45 40 Singapore 35 30 Korea 25 Japan Flanders (Belgium) R 2 = 0.24 r= 0.49 20 15 10 5 0 Poland Netherlands Estonia France Australia Czech Republic England (UK) Slovak Republic Italy Iceland Portugal Norway Israel Sweden Spain Denmark Latvia United States Croatia Serbia Bulgaria Romania Chile Brazil Alberta (Canada) Finland Mexico 0 10 20 30 40 50 60 70 80 Percentage of teachers who agree that teaching is valued in society
PISA for Schools and PISA PISA and PISA for Schools measure the skills needed for future life of 15 years around the world PISA PfS Shows how well a country is performing COMPARABLE Shows how well a school is performing
PISA for Schools - Objectives Provide information about how schools are performing Provide a backdrop for setting goals and planning improvements How are students performing in maths, science and reading - in an international context? How conducive is the school environment and student motivation to learning? How do these contextual factors shape learning? What levels do we want our students to reach? The benchmark is no longer national standards alone. What can be learnt from higher-performing school and school systems?
PISA for Schools instruments and data Cognitive test: reading. mathematics and science Student questionnaire: Sociodemographic factors and students attitudes School questionnaire: school characteristics
PISA for Schools in Spain Pilot 2013-2014 First administration 2015-2016
Results from PISA for Schools Understand the data provided in the school report My school results Identifying areas to work on in the future Planning Improvements
Reading performance
Socio-economic background
Disciplinary climate
Proficiency levels
What schools use the assessment for PISA for Schools enables schools to Benchmark internationally Measure students real-life skills Local and global peer learning Drive practice shifts
Thank you for your attention!