RUHR. The Interplay of Standardized Tests and Incentives ECONOMIC PAPERS. An Econometric Analysis with Data from PISA 2000 and PISA 2009 #356

Transcription

1 RUHR ECONOMIC PAPERS Christoph Helbach The Interplay of Standardized Tests and Incentives An Econometric Analysis with Data from PISA 2000 and PISA 2009 #356

2 Imprint Ruhr Economic Papers Published by Ruhr-Universität Bochum (RUB), Department of Economics Universitätsstr. 150, Bochum, Germany Technische Universität Dortmund, Department of Economic and Social Sciences Vogelpothsweg 87, Dortmund, Germany Universität Duisburg-Essen, Department of Economics Universitätsstr. 12, Essen, Germany Rheinisch-Westfälisches Institut für Wirtschaftsforschung (RWI) Hohenzollernstr. 1-3, Essen, Germany Editors Prof. Dr. Thomas K. Bauer RUB, Department of Economics, Empirical Economics Phone: +49 (0) 234/ , Prof. Dr. Wolfgang Leininger Technische Universität Dortmund, Department of Economic and Social Sciences Economics Microeconomics Phone: +49 (0) 231/ , Prof. Dr. Volker Clausen University of Duisburg-Essen, Department of Economics International Economics Phone: +49 (0) 201/ , Prof. Dr. Christoph M. Schmidt RWI, Phone: +49 (0) 201/ , Editorial Offi ce Joachim Schmidt RWI, Phone: +49 (0) 201/ , Ruhr Economic Papers #356 Responsible Editor: Volker Clausen All rights reserved. Bochum, Dortmund, Duisburg, Essen, Germany, 2012 ISSN (online) ISBN The working papers published in the Series constitute work in progress circulated to stimulate discussion and critical comments. Views expressed represent exclusively the authors own opinions and do not necessarily reflect those of the editors.

3 Ruhr Economic Papers #356 Christoph Helbach The Interplay of Standardized Tests and Incentives An Econometric Analysis with Data from PISA 2000 and PISA 2009

4 Bibliografische Informationen der Deutschen Nationalbibliothek Die Deutsche Bibliothek verzeichnet diese Publikation in der deutschen Nationalbibliografie; detaillierte bibliografische Daten sind im Internet über: abrufbar. ISSN (online) ISBN

5 Christoph Helbach 1 The Interplay of Standardized Tests and Incentives An Econometric Analysis with Data from PISA 2000 and PISA 2009 Abstract Since their first implementation in 2000, the PISA studies have attracted public attention and spurred the demand for institutional changes in schooling systems. The introduction of standardized student tests and of incentives for schools and teachers are notable examples of such institutional changes. This paper examines the effects of these particular developments. Identification is based on within-country variation between PISA 2000 and PISA The results indicate that comparing schools by means of standardized student test results is a promising measure, while evaluating teachers this way decreases the overall performance of a schooling system. The discussion provides possible explanations for these ambiguous findings. JEL Classification: I20, I21, I29 Keywords: Education economics; incentives; standardized tests; PISA July University of Duisburg-Essen. I am grateful to Jeannette Brosig-Koch, Matthias Giesecke, Timo Heinrich, Laura Helbach, Klemens Keldenich, Miriam Krieger, and seminar participants at University of Duisburg-Essen for helpful comments. All correspondence to: Christoph Helbach, Faculty of Economics and Business Administration, University of Duisburg-Essen, Universitätsstr. 12, Essen, Germany, christoph.helbach@ibes.uni-due.de.

6 1.Introduction Various studies have shown the remarkable impact of schooling quality on economic growth 1 and duringthelastdecadegovernmentstheworldoverhavemadeschooleducationpoliciesafocusof their attention. The Economist (2011) refers to the latter as The great schools revolution and emphasizes the crucial role that the PISA studies have played in this context by providing both internationally comparable data and eyeopening analyses. Since the first PISA implementation in 2000 researchers have used these rich datasets to identify key elements of successful education policies.oneofthemostimportantpolicyimplicationsissummarizedbyhanushek(2006,p.866) whostatesthat [ ]evidence[ ]suggeststhatpureresourcepoliciesthatdonotchangeincentives areunlikelytobeeffective. Therearetwoimplicationsfromthisconclusion. First, incentives may serve as a useful tool to improve an educational system s outcome. On principle, every agent who is involved in educational production can be incentivized. Recent research, however, has focused on incentives for students, teachers, and schools. 2 While the literatureonstudentincentivesyieldsratherinconclusiveresults, 3 directincentivesforschoolsandin particularforteacherstendtoaffectstudents outcomespositively.thisholdstrueforstudiesinthe United States (Figlio & Kenny, 2007), in Israel (Lavy, 2002, 2009), in India (Muralidharan & Sundararaman,2011)aswellasbetweencountries(Wößmann,2011).However,somepapersshow that teachers respond too well to incentives and consequently disregard nonincentivized goals. Reback(2008),Jacob(2005),Eberts,Hollenbeck&Stone(2002),andGlewwe,Ilias&Kremer(2010) find that teachers focus only on the incentivized target group and the success of incentivized measures.goalsthatarenotpartoftheincentiveschemebutmaystillbedesirablefromsociety s pointofviewmaynotbenefitormayevensufferfromtheincentiveschemes. Second, the institutional framework of a schooling system may matter more than the mere availability of resources. Among the institutions that have attracted analysts attention are accountabilitysystemssuchascurriculumbasedexternalexitexams.figlio&loeb(2011)discussthe incentive effect of such systems on schools if the schools future development implicitly or explicitly depends on the performance measures. Furthermore, external exams may provide a guidelineforagentsintheeducationsystemandthusreduceinsecurityofstudents,teachers,and schools. 4 Crosscountry evidence suggests that centralized exit exams indeed affect student s outcomespositively(bishop,1997;wößmann,lüdemann,schütz&west,2009).jürges&schneider 1 SeeHanushek&Wößmann(2008)forasurvey. 2 Ofcourse,teachersarepartoftheschoolandincentivesforschoolsandteachersmayconsequentlynotbe separable.pleaserefertothehypothesesandtheanalysisoftheestimationresultsforafurtherdiscussionon thistopic. 3 Whilegirlsrespondtoincentivesinsomecases(Angrist&Lavy,2009;Kremer,Miguel&Thornton,2009),in otherstheydonot(fryerjr.,2011).forboystheconsequencesofexplicitincentivesseemevenlesspromising (Angrist&Lavy;FryerJr.).InparticularthestudybyFryerJr.pointsouthowmuchthereactionofstudentsto incentivesmaydependonthedetailsoftherespectivesetup. 4 AnotheraspectishighlightedbyDePaola&Scoppa(2010)forthecaseofcentralizedexitexams.Thesecan ontheonehandserveasanincentivedevicefortheschools.ontheotherhand,standardizedtestsprovidea less dispersed measure of graduates abilities for employers. Consequently, students themselves could be incentivizedbysuchaninstitutionastheyanticipatethatprospectiveemployersputmoreweightonresults fromstandardizedexamsthanfromnonstandardizedones. 4

7 (2010)aswellasJürges,Schneider,Senkbeil&Carstensen(2012)confirmthispositivetendencyby analyzingvariationbetweengermanstates.however,theyalsoconcludethatcentralizedexitexams can reduce students interest, at least in mathematics. Furthermore, a meta study by Holme, Richards, Jimerson & Cohen (2010) casts some doubt on the robustness of the positive evidence mentioned above. While Figlio & Loeb share those doubts in particular when it comes to reading scores,theyconcludethat schoolaccountabilityimprovesaveragestudentperformanceinaffected schools,atleastingeneral (p.410). Whilethisliteraturesuggeststhatbothteacherandschoolincentivesaswellasaccountabilitycan improveaschoolingsystem,newevidencebyhanushek,link&wößmann(2011)pointstotheneed foramoredetailedanalysis.theycombinethedatafromvariouspisastudiesand,basedonwithin countryvariation,findthatinstitutions suchasschoolautonomyintheirpaper mayworkwellin certaincountriesandcultureswhiletheyarecounterproductiveinothers.furthermore,theauthors discussthepositiveinteractionofautonomywithaccountabilityandthuspointouttheimportance oftheinterplayofvariousinstitutions. InthepresentpaperdatafromPISA2000andPISA2009arecombinedtoanalyzetheinterplayof accountabilityandteacher/schoolincentives.theestimationsrevealthatstandardizedtestsperse donothaveaconsistentlysignificantimpactonpisascores.itisshown,however,thatitisimportant how the results of these tests are used. While the combination with teacher evaluation yields a significant decrease of PISA scores, it seems useful if schools as a whole are compared and rated basedontheresultsofstandardizedtests. The paper proceeds as follows. The following section introduces the econometric approach, discussesthekeyvariables,andstatesthehypotheses.insection3and4theestimationresultsare reportedanddiscussed.section5concludes. 2.Econometricapproach ThispaperisbasedonthedatasetsofPISA2000andPISA2009thatarecombinedintoonedataset toanalyzeinstitutionaleffectsofwithincountryvariationonpisascores. 5 Thereareseveralreasons for this approach. First, the abovementioned literature on the effect of schooling quality on economic growth has shown the importance of PISA outcomes as schooling quality measures. Second,therichPISAdatasetsincludevariousstudentandschoollevelvariablesthatcanbeusedas controlswhenanalyzingtheeffectsofstandardizedtestsandteacher/schoolincentivesonstudent outcomes.third,thefocusofpisaalternatesbetweenreading,mathematics,andscienceliteracy.in 2009 reading was the first focus to be repeated, which results in very similar school and student questionnairesthatsimplifythemergingprocessofthedata.fourth,thealmosttenyearsthatlay between the studies provide enough time for potentially PISAtriggered schooling reforms to affectstudentperformance. 5 ForadetaileddescriptionofthePISAdatasetsandstudyframeworkpleaserefertoAdams&Wu(2002)and OrganisationforEconomicCooperationandDevelopment(1999,2009). 5

8 2.1Datapreparation ThedatasetsofPISA2000andPISA2009arereducedtostudentsfromcountriesthattookpartin both waves, aligned with regards to the variable names and codings, and finally merged into one school and one student dataset. 6 Within these datasets missing values are imputed following the approach described by Wößmann et al. and using Stata files by Royston (2009). As a first step, missingvaluesforelementaryvariablesarereplacedbythegroupmedian,i.e.theschoolmedianfor studentvariablesandthecountrymedianoftherespectivewaveforschoolvariables. 7 Second,the variables to be imputed are regressed on these elementary variables, country dummies and a dummyvariableindicatingthepisastudyacasestemsfrom.third,themissingvaluesarepredicted bythecoefficientsoftheseestimations.tomakesurethatnoneoftheresultsofthispaperisdriven bytheimputedvalues,everysubsequentregressionincludesonedummyvariableperexplanatory variable that indicates whether the respective value is imputed. In addition, interaction terms betweenthesedummyvariablesandtherespectivevariableitselfareincluded. 8 Theseprocedures resultinadatasetof490,226studentsfrom38countries. 2.2Keyvariablesandhypotheses Three variables from the PISA school questionnaire are of primary interest here. The respective questionsfrompisa2009(organisationforeconomiccooperationanddevelopment,oecd,2008) readasfollows(extract): Question15 Generally, in your school, how often are students in <national modal grade for 15yearolds> assessedusingthefollowingmethods? a) Standardisedtests Question16 Inyourschool,areassessmentsofstudentsin<nationalmodalgradefor15yearolds>usedforany ofthefollowingpurposes? d) Tocomparetheschoolto<districtornational>performance f) Tomakejudgementsaboutteachers effectiveness The corresponding questions from the PISA 2000 school questionnaire are very similar. For the subsequentanalysistheanswersarecodedasindicatorvariableswithvalueoneifstandardizedtests arecarriedout,iftheyareusedtocompareschools,andiftheyareusedtomakejudgementsabout teachers effectiveness,respectively. 6 Furthermore,studentsfromAustria,Canada,France,andKoreaareremovedfromthedatasetduetomissing keyvariablesandstudentweights. 7 Elementaryvariablesincludee.g.age,gender,grade,migrationbackground,numberofbooksathome,and countryoftestforstudentsandshareofpublicfunding,studentteacherratio,size,shortageofcomputers,and countryoftestforschools.elementaryvariablesaremissingforlessthan3%ofthestudentsandlessthan 10.7%oftheschools. 8 Forcountrylevelvariablestherespectivecountrylevelaveragesoftheindicatorvariablesareconstructedand used for the estimation. However, the key results of this paper do not depend on the inclusion of these variables. 6

9 Asidefromthedirectinterpretationofthevariable,theuseofstandardizedtestsfortheassessment of15yearoldscanaswellbeinterpretedasanindicatorforthetendencytousecentralizedexamsin aschoolingsystemingeneral.panelaoffigure1showsthatthereisastrongtrendtoincreasethe useofstandardizedtestsatthecountrylevel,withonly6of38countriesreducingtheaverageuseof this assessment method. This trend is significant (p=0.000, Wilcoxon signedrank test). From the point of view of a standard economic theory, this trend should only result in changes of PISA performanceiftwoconditionsaresatisfied.first,thecontentofthestandardizedtestsneedstobe relatedtotheskillsassessedbypisa.second,agentsintheschoolingsystemneedtobemotivated throughincentives explicitorimplicit toaimathigherstudentscoresinstandardizedtests.inthis studyitisassumedthatsuchincentivesaregivenatleasttosomeextentforschoolsandteachersif theanswerstoquestions16d)and16f),respectively,are yes.whilethedesign andthusthe power of these incentives remain unclear, it seems reasonable to assume that a school that is compared to other schools based on the results of standardized student assessments feels more bound to set standards than a school for which this is not the case. Similarly, a teacher whose effectivenessjudgementdependsonhisstudents performanceinstandardizedtestsisassumedto complymorewiththesetstandardsthanacolleagueforwhomthisisnotthecase.thevariablesare thussubsequentlyreferredtoasschoolincentivesandteacherincentives,respectively.panelbandc offigure1displaythecountryleveldevelopmentoftheincentivevariablesbetween2000and2009. Both variables reveal a significant upwards shift (p=0.024 and p=0.000, respectively, Wilcoxon signedranktest). Figure1:Weightedcountrylevelaveragesofkeyvariablesin2000and2009 A B C Notes:Weightedcountrylevelaveragesin2000(horizontalaxis)and2009(verticalaxis):Usageofstandardizedtestsas assessments(panela),usageofassessmentsforteacherevaluation(panelb)andforschoolcomparison(panelc).in addition,the45degreereferencelineisdisplayed. Notably,thedevelopmentsoftheincentivevariablesare(weakly)significantlycorrelatedwiththe developmentsinthe use ofstandardizedtests. 9 InPortugal,Poland,andGermanyforinstancean increaseintheuseofstandardizedtestsbetween2000and2009camewithanincreaseintheusage of student assessments for school comparison. Despite these examples, the variables are not per definition linked to each other, as can be seen from cases like Denmark and Latvia where school 9 Aspearmanrankcorrelationatcountrylevelyieldsacoefficientof0.410(0.282)andapvalueof0.01(0.09) forthecorrelationofstandardizedtestsandschoolincentives(teacherincentives)

10 incentivesincreaseddespiteaconstantlyhighlevelofstandardisedexams.consequently,interaction termsforthevariablefromquestion15andtheincentivevariablesareconstructedandincludedin thesubsequentanalysis.thecountrylevelaveragesoftheseinteractionvariablesaredisplayedin Figure2. Figure2:Weightedcountrylevelaveragesofinteractiontermsofkeyvariablesin2000and A B Notes:Weightedcountrylevelaveragesin2000(horizontalaxis)and2009(verticalaxis):Usageofstandardizedtestsas assessmentsinteractedwithusageofassessmentsforteacherevaluation(panela)andusageofassessmentsforschool comparison(panelb),respectively.inaddition,the45degreereferencelineisdisplayed. Asdiscussedbefore,researchclearlyindicatesthatstandardizedtestsaswellasteacherandschool incentivestendtopositivelyinfluencepisaperformance.thisleadstothefirsthypothesis: H1: Standardizedtests,teacherincentives,andschoolincentivesallhaveapositiveimpact onpisascores. Thepresentpaperisthefirsttoexplicitlyanalyzetheinterplayofstandardizedtestsandincentives. As expressed by H1, it is concluded from the literature that incentives affect agents behaviour. Undertheassumptionthatthecontentofthestandardizedtestsisrelatedtothecontentexamined bypisa,thiseffectshouldbeevenlargerifobjectivecriteriasuchasstandardizedtestsareusedas performancemeasures.thisleadstothesecondhypothesis: H2: The combination of school and teacher incentives with standardized tests yields a furtherincreaseofpisaperformance. Ontheotherhand,hypothesisH2mayberejectedifthestandardizedtestsarenotorpoorlyrelated topisaassessments. 2.3Regressionmodelsandidentification Similar to the framework of Hanushek et al., the analysis of this paper is based on the following educationproductionfunction,subsequentlyreferredtoasthegeneralmodel: 8

11 where the achievement T of a student i in country c at a point of time t is the results of various inputs.theseinputsstemfromthestudentandherfamily(f),herschool(s),therespectivecountry s policies(p),andanerrorterm().notehoweverthatacleandistinctionbetweenschoolproperties andcountrypoliciesisdifficultorevenimpossible.bothpublicandprivateschoolsaredependenton lawsandguidelinesandthusevenbasicschoolpropertiessuchaslocationorsizemightbesubjectto changes for political reasons. For the estimations of the paper at hand the general model is convertedtothefullmodelthatreadsasfollows: Here, time invariant country properties such as robust policies or economic circumstances are captured by country fixed effects (D c ). Time fixed effects are controlled for by D t. The PISA data provide rich background information on students and schools. The full model includes student variablesongender,age,familyandmigrationbackground,parentaleducation,andnumberofbooks at home. On the school level, various measures e.g. on general school properties, autonomy, equipmentandstaff,assessment,oradmissionareincludedascontrols.inaddition,gdppercapita isincludedatthecountrylevel(heston,summers&aten,2011). 10 The key variables for this paper are measured at school level and are as such properties of a particularschool.identificationinthefullmodelisthenbasedontheindependenceoftheerrorterm fromtherespectivekeyvariable,whichcannotbetakenforgranted.onthecontrary,itseemslikely thatstudentselectionintoschoolsiscorrelatedwithoneorseveralofthekeyvariables.ingermany forinstancecentralizedexamsfor15yearoldshavebeenimplementedinhalfofthefederalstates between2000and2009.however,noteverystudenthastosittheseexams.particularly,insome states students who plan to proceed to the final secondaryschool examinations (Abitur) the prerequisitefortertiaryeducation donotsitthecentralexamsattheageof Anotherpotential selectionbiascouldariseifschoolsarenotobligedtopublishresultsfromstandardizedtestsbutcan dosovoluntarily.insuchascenarioitispossiblethatonlyresultsfromgoodschoolsarepublished andcanbecomparedbyparents.ifparentsthendecidetosendtheirchildrentotheschoolsthat providethisinformation,theseschoolsmaygetthechancetochoosemoretalentedstudents.these exemplaryselectioneffectsarelikelytobiastheestimation.toavoidthesebiasestheidentification strategyofhanusheketal.isimplemented: Key variables are measured as weighted averages at the country level and included in the regressions.identificationisthenbasedonthevariationofcountrylevelaveragesbetween2000and 2009.Figure1indicatesthatthisvariationissubstantialandthuscanbeusedtoidentifytheeffects of the interplay between standardised exams and school and teacher incentives. This model is subsequently referred to as full model key. 12 Note however, that this approach may lead to a differentkindofendogeneityproblem.ifchangesinthekeyvariablesatthecountrylevelcomewith 10 LiechtensteinisexcludedfromtheregressionduetomissingGDPinformation. 11 IamgratefultoAndreaSchwermerfromtheStandingConferenceoftheMinistersofEducationandCultural AffairsoftheLänderintheFederalRepublicofGermany(Kultusministerkonferenz)forthispointer. 12 Theindicatorvariablesformissingvaluesarealsoaveragedatthecountrylevelandincludedinthemodel both as such and interacted with the respective key variable averages. For a descriptive overview of the countrylevelaveragesofthekeyvariablespleaserefertotheappendix. 9

12 changesinnonobservablevariablestheestimatedcoefficientsmayagainbebiased.thiswouldbe thecaseforexampleifotherreformstookplacebetween2000and2009.toreducethisbiasother schoollevel variables are as well included in addition as country level averages. Even though this approachmaysufferfromveryfewdegreesoffreedomandmayleavesomevariablesunobserved,it seems likely to tackle large parts of the endogeneity problem as various possible reforms are controlledfor.amongothers,measuresonschoolautonomy(takenfromhanusheketal.),budget, andadmissioncriteriaareincludedatthestatelevelintheseregressions.thismodelissubsequently referredtoasfullmodelall. 13 Allmodelsareestimatedbyleastsquaresregressionsweightedwithsamplingprobabilities.Robust standarderrorsareclusteredatcountrylevel. 3.Estimationresults TheresultsoftheestimationsarereportedinTable1. 14 Asafirstobservationitcanbenotedthatthe twomodelsyieldqualitativelysimilarresults.despitethefewdegreesoffreedomthefullmodelall estimationstentativelyconfirmtheresultsofthefullmodelkeyregressions.thediscussionisthus mainlybasedonthelatter. Table1:ImpactofkeyvariablesonPISAscores fullmodelkey fullmodelall Dependentvariable RL ML SL RL ML SL Standardizedtests Schoolincentives Standardizedtestsx schoolincentives Teacherincentives Standardizedtestsx teacherincentives (50.65) ** (75.98) *** (77.99) (62.45) * (71.56) (44.65) * (86.66) *** (94.91) ** (57.47) *** (68.60) 88.86** (40.92) (75.09) (76.78) ** (49.55) *** (57.84) (26.08) ** (47.88) *** (56.72) (36.11) (39.82) (29.76) 88.26* (44.35) *** (55.82) 96.05*** (34.41) *** (42.77) (33.26) ** (65.31) *** (71.39) (45.58) * (55.29) Observations 489, , , , , ,922 Countries Notes:Impactofthekeyvariablesmeasuredascountrylevelaveragesonreadingliteracy(RL),mathematicalliteracy (ML),andscientificliteracy(SL).Estimatesoffullmodelkeyandfullmodelallbyleastsquaresweightedwithsampling probabilities.controlsincludecountryfixedeffects,timefixedeffects,controlsformissingvalues,gdppercapita,school levelvariables(e.g.size,location,staff,funding,autonomy,admissionrules),andstudentlevelvariables(e.g.gender, age, education of parents, migration background). Robust standard errors clustered at country level in parentheses. Significancelevels:***1%,**5%,*10%. 13 Controllingformissingvalueaveragesatthecountrylevelisnotpossibleinthismodelbecauseoftoofew degreesoffreedom.furthermore,studentsfromalbania,andperuareexcludedinthismodelduetomissing valuesatthecountrylevel. 14 Forthecompleteresultspleaserefertotheappendix. 10

13 The results shown in Table 1 translate into the marginal effects of school incentives and teacher incentivesillustratedinfigures3and4. 15 Thegraphsshowthatchangesinschoolincentivestendto increasepisascoresiftheincentivesarelinkedtohighlevelsofassessmentstandardization.onthe otherhand,suchchangesreducepisascoresiflinkedtolowlevelsofassessmentstandardization. These effects are most pronounced and widely significant for reading and mathematical literacy whiletheyarenotsignificantforscientificliteracy.asforteacherincentives,theresultsarecontrary. WhileevaluatingteachersviastudentassessmentsingeneralimprovesPISAscores(significantlyso for mathematical and scientific literacy), connecting evaluations to standardized tests is counterproductiveforallthreeliteracies. 16 Figure3:Marginaleffectsofschoolincentivesasafunctionofstandardizedtesting RL ML SL Notes:Marginaleffectsofschoolincentives(solidline)onreadingliteracy(RL),mathematicalliteracy(ML),and scientificliteracy(sl)asafunctionofstandardizedtesting.thedashedlines( )display95%confidenceintervals. Notethatleavingtheinteractiontermsoutoftheregressionsentanglestheresultsandyieldswidely insignificanteffectsforstandardizedexams,positiveeffectsofschoolincentivesandnegativeeffects of teacher incentives (cf. Table A5 in the Appendix). Altogether, these results lead to a partial rejectionofallstatedhypotheses. Quantitatively, the large coefficients for the variables, particularly for the interaction terms, underline their importance. It should be noted, however, that the variation within countries and betweenpointsoftimethatisusedforidentificationintheseregressionsdoesnotcoverthewhole possiblerange.theinteractiontermsforexampledonotchangebetween2000and2009bymore than 0.54 in absolute value in any country. The medians of changes in the interactions of standardized tests with teacher incentives and school incentives are 0.06 and 0.13, respectively. Thesechangesimplyanaveragedecrease(increase)ofPISAscoresby12.1%(29.8%)ofastandard deviationasaresultofcombiningstandardizedtestswithteacherincentives(schoolincentives). 15 Brambor, Clark & Golder (2006) underline the importance of considering marginal effects in interaction modelsandprovideusefultoolsfordoingso. 16 Totestwhethertheresultsaredrivenparticularlybyloworhighperformingstudents,thesampleissplitat the countrybywave median for each literacy category and full model key is then reestimated for both subsamples. The results are very similar to those shown in Figures 3 and 4, although somewhat less pronouncedforthesubsamplecomprisingthetop50%ofthestudents(cf.appendix,figuresa1toa4). 11

14 Figure4:Marginaleffectsofteacherincentivesasafunctionofstandardizedtesting RL ML SL Notes:Marginaleffectsofteacherincentives(solidline)onreadingliteracy(RL),mathematicalliteracy(ML),and scientificliteracy(sl)asafunctionofstandardizedtesting.thedashedlines( )display95%confidenceintervals. 4.Discussion TheresultsshowthatstandardizedtestspersedonotconsistentlyincreasePISAoutcomes. 17 Itis thencrucialtoanalyzewhatthetestresultsareusedfor.thekeytounderstandingthepresented resultsisthequestionwhyalinkbetweenstandardizedtestsandcomparing(andthusincentivizing) schoolsisfruitfulwhilealinkwithevaluating(andthusincentivizing)teachersisnot. Thepertinentliteratureindicatesconsistentlythatteachersrespondtoincentives.However,thisalso includespossiblesideeffectsofteacherincentives.jacob&levitt(2003)forexamplefindindication forcheatingbyteachersandadministrationdespitestandardizedstudenttesting.theyshowthatthe prevalenceofcheatingstronglyrespondstochangesinincentives.inadditiontosuchunintended behaviouralconsequencesofincentives,theresultsathandcouldalsobeexplainedifthecontentof the standardized tests were not closely related to the competence based measures of PISA. If teachers then focus too much on their students performance in the incentivized test PISA performancemaysuffer.againstthebackgroundofresearchsupportingtherelevanceofpisaand what it measures (Hanushek & Wößmann, 2008; Schleicher, 2007) and the present paper, this interpretation should lead to an adjustment of the standardized tests or a detachment of their resultsandteacherincentives. A third and complementary explanation for the overall negative effect of teacher incentives is provided by the method used in this paper. Identification here is based on policy changes within countriesratherthanvariationbetweencountries(asitisusede.g.bywößmann).thus,historymay playarole.forexample,theselfselectionofpeopleintotheteachingbodyislikelytobedrivennot onlybytheprofessionitselfbutalsobyaspectssuchasjobsecurity.dohmen&falk(2010)findthat teachersingermanyaremoreriskaversethantheaveragegermanpopulation.suchpreferences 17 Notethatthisresultdoesnotnecessarilycontradictresultsfromotherresearch.First,otherstudiesoften use curriculum based external exit exams as accountability measure. They may thus include an additional positiveeffectthroughstudentincentivestoperformwellintheseexams(cf.footnote4).furthermore,some oftheregressionspresentedherealsorevealasignificantlypositiveeffect. 12

15 are likely to influence the effect of incentives. In a laboratory experiment in Germany, Helbach & Keldenich (2012) find that prospective teachers respond differently (i.e. more weakly) than other studentstoincentiveschemevariations.eventhoughthisinterpretationbyitselfisunlikelytoleadto negative effects of incentives it is possible that it reduces the positive incentive effects that are shownintable1andthuscontributestothenegativeoveralleffectofteacherincentives. In contrast toteachers,schoolsasa wholemaynotbeable toinfluencethestyleand contentof teachingtothesameextent.insteaditislikelythatincentivizedschoolsareabletoimprovegeneral measures such as atmosphere, learning environment, and general discipline (e.g. truancy). Claes, Hooghe&Reeskens(2009)provideevidencethatschoolsplayamajorroleinreducingtruancyrates. These general improvements are likely to improve schooling outcomes independently of the respectivecurriculum.however,schoolsareonlyincentivizedtoexerttherequiredeffortiftheyare held accountable for their performance. The results show that standardized tests can serve as an accountabilitymeasurewhileotherstudentassessmentscannot(cf.table1).thisseemsreasonable asresultsfromnonstandardizedtestsmaynotconvincethepublicofthegoodorbadqualityofa school. Furthermore, as schools may be able to affect the framing of such results, they have no incentivetosustainhighschoolingquality.instead,theymayevenbeencouragedtodeveloptools fordisguisingdeficits. 5.Conclusion In Finland, the average results of the central exit exams (ylioppilastutkinto) of every Finnish high school are published every year by various media. The Supreme Administrative Court (Korkein HallintoOikeus) of Finland underlined the importance of these publications by judging that the resultsneedtobemadeavailabletothepublicinelectronicformaswell(korkeinhallintooikeus, 2007). 18 Itisthiskindofcomparisonthatimposespressure andhenceincentives ontheschools. Atthesametime,teacherscanberewarded andhenceincentivized byindividualcashbonuses that are not necessarily based on their students performance in centralized exams. Based on the previousdiscussion,thisisasoundincentivemixthatmaybeonereasonwhyfinlandhasproven successfulineverypisastudysofar. The analysis has shown that school comparisons are a main channel through which standardized teststakeeffect.itisthuspromisingthatbothinstitutionsarestronglycorrelated.incontrast,the correlationoftheuseofteacherincentivesandstandardizedtestsseemscounterproductiveinthe light of the analysis while teacher incentives in general remain a way to achieve better schooling outcomes. 18 Therankingof2011isavailableforexampleonthewebsiteofHelsinginSanomat,thelargestnewspaperin Finland(HelsinginSanomat,2011,2012). 13

16 References Adams,R.J.&Wu,M.(2002).PISA2000technicalreport.Paris:OECD. Angrist,J.&Lavy,V.(2009).Theeffectsofhighstakeshighschoolachievementawards:evidence fromarandomizedtrial.americaneconomicreview,99(4), Bishop,J.H.(1997).Theeffectofnationalstandardsandcurriculumbasedexamsonachievement. AmericanEconomicReview,87(2), Brambor, T., Clark, W. R. & Golder, M. (2006). Understanding interaction models: improving empiricalanalyses.politicalanalysis,14(1), Claes,E.,Hooghe,M.&Reeskens,T.(2009).Truancyasacontextualandschoolrelatedproblem:a comparativemultilevelanalysisofcountryandschoolcharacteristicsoncivicknowledgeamong14 yearolds.educationalstudies,35(2), De Paola, M. & Scoppa, V. (2010). A signalling model of school grades under different evaluation systems.journalofeconomics,101(3), Dohmen,T.&Falk,A.(2010).Yougetwhatyoupayfor:incentivesandselectionintheeducation system.economicjournal,120(546),f256 F271. Eberts,R.,Hollenbeck,K.&Stone,J.(2002).Teacherperformanceincentivesandstudentoutcomes. JournalofHumanResources,37(4), Figlio, D. & Loeb, S. (2011). School accountability. In E. A. Hanushek, S. J. Machin, & L. Wößmann (Eds.),HandbookoftheEconomicsofEducation(Vol.3,pp ).Amsterdam:Elsevier. Figlio,D.N.&Kenny,L.W.(2007).Individualteacherincentivesandstudentperformance.Journalof PublicEconomics,91(56), Fryer Jr., R. G. (2011). Financial incentives and student achievement: evidence from randomized trials.quarterlyjournalofeconomics,126(4), Glewwe,P.,Ilias,N.&Kremer,M.(2010).Teacherincentives.AmericanEconomicJournal:Applied Economics,2(3), Hanushek, E. A. (2006). School resources. In E. A. Hanushek & F. Welch (Eds.), Handbook of the EconomicsofEducation(Vol.2,pp ).Amsterdam:Elsevier. Hanushek, E. A., Link, S. & Wößmann, L. (2011). Does school autonomy make sense everywhere? panelestimatesfrompisa.nationalbureauofeconomicresearchworkingpaperseries, Hanushek,E.A.&Wößmann,L.(2008).Theroleofcognitiveskillsineconomicdevelopment.Journal ofeconomicliterature,46(3), Helbach,C.&Keldenich,K.(2012).Teachinginthelab:financialincentivesintheeducationprocess. WorkingPaper.RuhrEconomicPapers,

17 HelsinginSanomat.(2011).Lukioidenmenestysparemmuusjärjestyksessäkevään2011 ylioppilaskirjoituksissa.retrievedmarch5,2012,from Helsingin Sanomat. (2012). International edition about. Retrieved February 29, 2012, from Heston, A., Summers, R. & Aten, B. (2011). Penn world table version 7.0. Center for International ComparisonsofProduction,IncomeandPricesattheUniversityofPennsylvania. Holme, J. J., Richards, M. P., Jimerson, J. B. & Cohen, R. W. (2010). Assessing the effects of high schoolexitexaminations.reviewofeducationalresearch,80(4), Jacob,B.A.(2005).Accountability,incentivesandbehavior:theimpactofhighstakestestinginthe ChicagoPublicSchools.JournalofPublicEconomics,89(5 6), Jacob,B.A.&Levitt,S.D.(2003).Rottenapples:aninvestigationoftheprevalenceandpredictorsof teachercheating.quarterlyjournalofeconomics,118(3), Jürges,H.&Schneider,K.(2010).Centralexitexaminationsincreaseperformance...buttakethefun outofmathematics.journalofpopulationeconomics,23(2), Jürges, H., Schneider, K., Senkbeil, M. & Carstensen, C. H. (2012). Assessment drives learning: the effect of central exit exams on curricular knowledge and mathematical literacy. Economics of EducationReview,31(1), KorkeinHallintoOikeus.(2007).KHO:2007:39.RetrievedMarch5,2012,from Kremer,M.,Miguel,E.&Thornton,R.(2009).Incentivestolearn.ReviewofEconomicsandStatistics, 91(3), Lavy, V. (2002). Evaluating the effect of teachers group performance incentives on pupil achievement.journalofpoliticaleconomy,110(6), Lavy, V. (2009). Performance pay and teachers effort, productivity, and grading ethics. American EconomicReview,99(5), Muralidharan,K.&Sundararaman,V.(2011).Teacherperformancepay:experimentalevidencefrom India.JournalofPoliticalEconomy,119(1), Organisation for Economic Cooperation and Development. (1999). Measuring student knowledge andskillanewframeworkforassessment.paris:oecd. Organisation for Economic Cooperation and Development. (2008). Programme for international studentassessment:schoolquestionnaireforpisa2009.paris:oecd. 15

18 OrganisationforEconomicCooperationandDevelopment.(2009).PISA2009assessmentframework keycompetenciesinreading,mathematicsandscience.paris:oecd. Reback, R. (2008). Teaching to the rating: school accountability and the distribution of student achievement.journalofpubliceconomics,92(56), Royston,P.(2009).Multipleimputationofmissingvalues:furtherupdateofice,withanemphasison categoricalvariables.thestatajournal,9(3), Schleicher, A. (2007). Can competencies assessed by PISA be considered the fundamental school knowledge15yearoldsshouldpossess?journalofeducationalchange,8(4), TheEconomist.(2011).Thegreatschoolsrevolution:reformingeducation.17September.Retrieved fromhttp:// Wößmann,L.(2011).Crosscountryevidenceonteacherperformancepay.EconomicsofEducation Review,30(3), Wößmann,L.,Lüdemann,E.,Schütz,G.&West,M.R.(2009).Schoolaccountability,autonomyand choicearoundtheworld.cheltham:edwardelgarpublishing. 16

19 Appendix TableA1:Weightedcountrylevelaveragesofkeyvariables Standardizedtest Teacherincentives Schoolincentives Albania Argentina Australia Belgium Brazil Bulgaria Chile CzechRepublic Denmark Finland Germany Greece HongKongChina Hungary Iceland Indonesia Ireland Israel Italy Japan Latvia Liechtenstein Luxembourg Mexico Netherlands NewZealand Norway Peru Poland Portugal Romania RussianFederation Spain Sweden Switzerland Thailand UnitedKingdom UnitedStates

20 TableA2:Weightedcountrylevelaveragesofinteractiontermsofkeyvariables Standardizedtestsxteacherincentives Standardizedtestsxschoolincentives Albania Argentina Australia Belgium Brazil Bulgaria Chile CzechRepublic Denmark Finland Germany Greece HongKongChina Hungary Iceland Indonesia Ireland Israel Italy Japan Latvia Liechtenstein Luxembourg Mexico Netherlands NewZealand Norway Peru Poland Portugal Romania RussianFederation Spain Sweden Switzerland Thailand UnitedKingdom UnitedStates

21 TableA3:Furtherresultsoffullmodelkeyregressions fullmodellkey Dependentvariable RL ML SL Studentlevelvariables Female 23.84***(1.83) 16.91***(2.18) 5.24***(2.02) Age(inmonths) 0.91***(0.3) 0.96***(0.34) 0.63***(0.3) Grade 35.85***(2.53) 34.85***(2.52) 30.54***(2.38) Familymemberslivingathome Mother 22.9***(3.18) 22.9***(2.07) 23.51***(1.37) Father 2.54**(2.34) 6.38**(2.54) 6.08**(2.77) Brother(s) 4.40(1.99) 3.09(1.88) 6.01(1.7) Sister(s) 2.16(2.26) 2.79(2.91) 5.02(2.21) Educationofmother ISCED0.95***(3.49) 11.28***(2.59) 6.52***(4.15) ISCED1 8.3***(3.02) 10.17***(2.68) 6.23***(3.65) ISCED2 8.04**(3.2) 7.40**(2.91) 6.40**(4.17) ISCED3b/3c 1.78(2.11) 2.38(2.56) 1.44(3.9) ISCED3a/4 2.41(2.35) 0.13(1.97) 1.33(3.63) Highereducation 7.15***(1.68) 6.01***(2.15) 9.69***(2.4) Educationoffather ISCED0 8.44(4.47) 1.82(3.59) 9.17(4.98) ISCED1 1.36(3.92) 0.32(2.8) 4.32(3.96) ISCED2 2.32(3.53) 1.37(2.7) 4.19(3.48) ISCED3b/3c 0.71**(3.37) 5.87**(2.38) 0.84**(3.28) ISCED3a/4 5.66***(3.32) 8.18***(2.52) 3.24***(2.94) Highereducation 11.76***(2.41) 16.98***(2.58) 11.88***(2.85) Numberofbooksathome ***(9.08) 55.82***(8.09) ***(12.11) ***(8.72) 73.74***(8.37) ***(11.77) ***(8.85) ***(8.78) ***(11.92) > ***(9.12) ***(9.18) ***(12.11) Migrationbackground Studentbornintestcountry 4.94(2.77) 0.74(2) 1.29(2.61) Motherbornintestcountry 1.61*(3.23) 5.16*(2.97) 6.10*(2.26) Fatherbornintestcountry 7.30***(2.86) 8.38***(2.29) 9.21***(2.48) Testlanguageusedathome 7.66(5.14) 1.62(4.62) 5.92(5.56) Schoollevelvariables Shareofpublicfunding 0.05**(0.04) 0.07**(0.03) 0.09**(0.04) Studentteacherratio 0.05(0.05) 0.06(0.05) 0.05(0.06) Schoolsize 0.01**(0) 0.01**(0) 0.01**(0) Schoollocation Smalltown 3.28(2.29) 1.69(3.05) 2.40(3.03) Town 9.72**(2.68) 6.25**(2.93) 5.92**(3.07) City 14.36(3.94) 8.23(4.9) 7.15(4.38) Largecity 15.28(5.68) 8.21(6.41) 6.38(5.76) 19

22 TableA3(continued) fullmodellkey Dependentvariable RL ML SL Shortageassessmentofprincipal Scienceteachers Alot 0.82(3.33) 4.73(4.09) 2.62(2.95) Tosomeextent 4.58(1.98) 3.19(2.13) 3.56(1.4) Little 3.57(2.19) 4.45(2.67) 3.77(1.75) Mathteachers Alot 2.41(5.42) 6.75(6.12) 5.19(5.24) Tosomeextent 0.91(3.4) 1.95(4.99) 0.54(2.86) Little 2.95*(1.44) 2.78*(1.58) 2.61*(1.52) Languageteachers Alot 3.72(5.37) 5.48(4.91) 3.44(4.64) Tosomeextent 2.34(2.94) 1.91(3.29) 1.36(3.12) Little 3.09(2.34) 2.93(2.4) 3.21(2.94) Supportingstaff Alot 1.02(1.99) 1.98(1.75) 1.81(1.93) Tosomeextent 0.91(2.01) 1.94(1.97) 1.36(1.95) Little 0.45(1.95) 0.12(1.99) 0.85(2.38) Laboratorymaterial Alot 5.5**(2.64) 6.46**(3.12) 5.7**(2.78) Tosomeextent 7.44**(2.11) 7.12**(2.66) 5.42**(2.14) Little 2.81(1.64) 1.50(2.03) 1.13(1.73) Instructionalmaterial Alot 0.78(3.74) 3.89(3.72) 4.87(4.81) Tosomeextent 0.81(2.78) 2.36(2.87) 1.48(2.27) Little 2.57(2.52) 3.15(2.51) 2.62(2.19) Computers Alot 1.08(2.52) 0.03(2.14) 0.38(2.58) Tosomeextent 6.36**(2.14) 5.99**(2.48) 5.78**(2.06) Little 3.75**(1.92) 4.06**(1.93) 3.75**(1.87) Librarymaterial Alot 9.17***(2.27) 10.27***(2.42) 11.53***(2.43) Tosomeextent 1.59*(1.83) 2.67*(1.53) 3.80*(1.67) Little 2.19**(1.3) 3.54**(1.33) 5.05**(1.75) Audiovisualresources Alot 2.64*(2.08) 4.37*(2.43) 2.21*(1.84) Tosomeextent 1.11(1.88) 3.23(2.06) 0.40(1.35) Little 0.49(1.89) 0.26(1.33) 1.19(1.52) Assessmentsareusedfor Informingparents 6.37(2.93) 3.45(3) 6.19(4.23) Decidingaboutstudent'scarreer 2.02(2.24) 1.87(2.63) 1.38(2.73) Groupingstudents 0.07(2.76) 0.38(2.62) 1.06(2.5) Comparingschooltoothers 2.05(1.97) 1.74(2.4) 1.64(2.18) Monitoringschool 1.55(2.42) 0.06(2.38) 0.24(2.04) Judgingteachers'effectiveness 2.65(1.86) 2.73(1.78) 2.90(2.11) 20

23 TableA3(continued) fullmodellkey Dependentvariable RL ML SL Autonomy,responsibilityfor Selectingteachers 2.03***(1.68) 4.34***(1.56) 2.12***(1.52) Firingteachers 0.47(2.7) 2.35(2.71) 0.89(2.16) Teachers'startingsalary 8.27**(1.98) 5.52**(2.23) 6.92**(1.65) Teachers'salaryincreases 2.94(3.13) 2.13(2.86) 2.94(2.52) Formulatingschoolbudget 1.66(1.39) 0.13(1.51) 0.36(1.51) Allocatingschoolbudget 5.42***(2.19) 5.14***(1.85) 6.79***(2.67) Disciplinarypolicies 8.19**(5.29) 7.11**(3.31) 9.59**(4.8) Assessmentpolicies 0.55(2.49) 1.78(2.9) 1.49(3.02) Studentadmission 5.78**(2.4) 5.58**(2.47) 6.81**(2.69) Textbooks 2.09(2.49) 1.14(2.39) 4.30(2.77) Coursecontent 0.96(1.82) 0.62(1.68) 2.18(1.9) Courseoffers 2.77**(1.5) 3.72**(1.83) 4.08**(1.66) Studentsareassessedby Standardizedtests 3.52(2.35) 2.98(2.25) 3.24(2.42) Teacherdevelopedtests 4.87(3.97) 2.06(4.19) 4.52(6.27) Teachers judgementalratings 1.29(1.33) 1.64(1.45) 1.27(1.11) Student<portfolios> 1.89*(0.94) 2.22*(1.22) 2.41*(1.22) Projects/Homework 4.81(6.32) 7.18(6.96) 5.65(5.16) Criteriaforadmission Residence Always 1.72(3.06) 1.18(2.78) 1.06(2.99) Sometimes 0.46(1.41) 0.86(1.99) 1.22(1.54) Student'srecordofperformance Always 12.42***(2.28) 12.06***(2.18) 12.40***(2.16) Sometimes 3.46(2.78) 3.19(2.3) 3.12(2.82) Recommendationoffeederschools Always 4.13(3.21) 4.48(3.61) 3.60(3.85) Sometimes 1.27(1.71) 1.36(2.41) 0.10(2.32) Parents endorsementofphilosophy Always 3.79(1.99) 1.99(1.76) 1.74(1.94) Sometimes 0.06(1.27) 0.65(1.31) 1.06(1.38) Student'sinterestinprogramme Always 7.44***(1.96) 8.15***(2.83) 6.12***(2.33) Sometimes 1.67(1.66) 1.79(2.38) 0.25(2.09) Familymemberofcurrentstudent Always 1.28(2.29) 2.41(1.53) 2.07(2.22) Sometimes 0.99(1.65) 2.17(1.86) 2.18(2.25) Learningofstudentsishinderedby Teachers'lowexpectations Alot 11.19***(4.28) 11.74***(3.97) 12.24***(5.63) Tosomeextent 9.00**(3.05) 9.04**(3.46) 8.69**(3.37) Little 2.39(1.86) 1.33(2.41) 2.20(2.08) 21

24 TableA3(continued) fullmodellkey Dependentvariable RL ML SL Studentabsenteeism Alot 21.17***(2.53) 24.47***(2.53) 19.73***(3.05) Tosomeextent 16.53***(2.16) 19.65***(2.5) 17.59***(2.24) Little 5.20***(2.25) 8.17***(2.46) 6.82***(2.15) Poorstudentteacherrelations Alot 6.01(3.39) 2.67(5.1) 6.19(5.67) Tosomeextent 5.40**(1.88) 5.74**(2.12) 5.64**(2.16) Little 1.61(1.5) 1.57(1.89) 2.60(1.35) Disruptionofclasses Alot 19.28***(5.49) 16.56***(5.58) 17.87***(4.95) Tosomeextent 16.99***(3.91) 16.82***(4.04) 17.51***(3.89) Little 8.58**(3.86) 8.62**(3.75) 9.05**(3.42) Teachersnotmeetingstudents'needs Alot 14.61***(4.32) 11.82***(4.18) 15.27***(4.35) Tosomeextent 8.96***(2.25) 9.81***(2.28) 8.62***(2.44) Little 7.50***(2.44) 6.37***(2.13) 6.26***(2.17) Teacherabsenteeism Alot 2.74(4.25) 4.24(3.43) 5.67(3.36) Tosomeextent 1.30(2.67) 2.66(2.14) 2.44(2.67) Little 0.84**(1.63) 2.92**(1.3) 2.47**(1.43) Studentsskippingclasses Alot 13.4***(5.25) 11.77***(4.16) 13.44***(3.97) Tosomeextent 14.71***(4.3) 13.55***(3.84) 13.51***(3.31) Little 8.89**(3.65) 9.38**(3.93) 8.93**(3.23) Lackofrespect Alot 0.04(2.26) 0.56(2.67) 4.62(2.7) Tosomeextent 0.63(2.87) 1.86(2.37) 3.23(2.77) Little 0.05(1.76) 0.26(1.21) 1.27(1.34) Staffresistingchange Alot 0.07(3.76) 1.07(4.41) 2.38(3.77) Tosomeextent 4.41(2.06) 4.27(2.62) 6.35(2.36) Little 3.72(1.64) 2.83(1.8) 3.67(1.88) Studentsusingdrugs Alot 5.66(5.58) 3.43(4.37) 6.89(5.96) Tosomeextent 0.69(3.86) 0.92(3.5) 0.59(3.82) Little 2.55*(1.77) 2.47*(1.22) 3.11*(1.68) Teachersbeeingtostrict Alot 7.28(3.48) 7.87(4.72) 3.22(3.57) Tosomeextent 4.85*(2.6) 6.36*(3.69) 4.38*(2.92) Little 0.97(1.43) 1.26(2.02) 0.32(1.73) Studentsbullyingstudents Alot 4.72(2.74) 0.39(2.31) 1.88(2.05) Tosomeextent 5.11*(4.08) 5.06*(2.96) 5.16*(3.74) Little 0.43(2.08) 1.31(2.47) 1.80(2.52) 22

25 TableA3(continued) fullmodellkey Dependentvariable RL ML SL Studentsnotbeeingencouraged Alot 5.09(3.4) 4.99(3.42) 6.62(2.31) Tosomeextent 1.14(2.02) 1.14(1.95) 1.63(2.32) Little 0.46(1.83) 0.82(1.95) 0.82(1.57) Countrylevelvariables Standardizedtests 25.01(50.65) 32.01(44.65) 88.86**(40.92) Schoolincentives **(75.98) *(86.66) 56.30(75.09) Standardizedtestsx schoolincentives ***(77.99) ***(94.91) (76.78) Teacherincentives 63.26(62.45) **(57.47) **(49.55) Standardizedtestsx teacherincentives *(71.56) ***(68.60) ***(57.84) GDPpercapita 0.002(0.002) 0.002(0.002) 0.001(0.002) Observations 489, , ,736 Countries Notes:Impactofthekeyvariablesmeasuredascountrylevelaveragesonreadingliteracy(RL),mathematicalliteracy (ML), and scientific literacy (SL). Estimates of full model key by least squares weighted with sampling probabilities. Controls include country fixed effects, time fixed effects, and controls for missing values. Robust standard errors clusteredatcountrylevelinparentheses.significancelevels:***1%,**5%,*10%. 23

26 TableA4:Furtherresultsoffullmodelallregressions fullmodellall Dependentvariable RL ML SL Studentlevelvariables Female 24.18***(1.86) 16.70***(2.23) 5.08***(2.06) Age(inmonths) 0.90***(0.31) 0.97***(0.34) 0.62***(0.31) Grade 36.01***(2.68) 34.98***(2.64) 30.78***(2.47) Familymemberslivingathome Mother 24.49***(3.05) 23.61***(1.89) 23.93***(1.16) Father 2.98**(2.35) 6.62**(2.55) 6.14**(2.8) Brother(s) 5.02(1.95) 3.70(1.85) 6.36(1.69) Sister(s) 2.79(2.24) 3.40(2.87) 5.45(2.15) Educationofmother ISCED1.46***(3.49) 12.01***(2.56) 6.76***(4.38) ISCED1 9.08***(2.96) 10.80***(2.52) 6.52***(3.76) ISCED2 9.38**(3.17) 8.16**(2.77) 6.54**(4.18) ISCED3b/3c 2.84(1.89) 3.72(2.18) 1.76(3.74) ISCED3a/4 1.18(2.27) 0.95(1.97) 1.31(3.7) Highereducation 6.24***(1.65) 4.86***(2) 9.23***(2.53) Educationoffather ISCED0 9.34(4.49) 2.54(3.74) 9.15(5.19) ISCED1 1.75(3.92) 0.80(2.89) 4.03(4.12) ISCED2 2.94(3.51) 0.77(2.7) 3.96(3.64) ISCED3b/3c 0.36**(3.38) 4.97**(2.35) 0.79**(3.43) ISCED3a/4 5.19***(3.31) 7.93***(2.57) 3.68***(3.06) Highereducation 11.36***(2.5) 16.54***(2.78) 12.17***(2.99) Numberofbooksathome ***(9.23) 53.32***(6.81) ***(11.44) ***(8.76) 71.11***(6.42) ***(10.73) ***(8.89) 99.99***(6.75) ***(10.77) > ***(9.17) ***(7.14) ***(10.89) Migrationbackground Studentbornintestcountry 4.74(2.75) 0.66(1.98) 1.09(2.55) Motherbornintestcountry 1.61*(3.18) 5.17*(2.94) 6.17*(2.25) Fatherbornintestcountry 7.03***(2.84) 8.09***(2.26) 9.10***(2.48) Testlanguageusedathome 7.42(5.07) 1.33(4.56) 5.47(5.47) Schoollevelvariables Shareofpublicfunding 0.06**(0.04) 0.08**(0.03) 0.08**(0.04) Studentteacherratio 0.03(0.06) 0.04(0.05) 0.04(0.06) Schoolsize 0.01**(0) 0.01**(0) 0.01**(0) Schoollocation Smalltown 2.59(2.3) 1.41(3.04) 2.18(3.06) Town 9.10**(2.69) 5.74**(2.95) 5.29**(3.09) City 13.20(4.06) 7.11(4.9) 6.06(4.32) Largecity 14.44(5.76) 7.16(6.44) 5.56(5.81) 24