The Short-Term Effectiveness of a Remedial Mathematics Course: Evidence from a UK University

Transcription

1 D I S C U S S I O N P A P E R S E R I E S IZA DP No The Shor-Term Effeciveness of a Remedial Mahemaics Course: Evidence from a UK Universiy Giorgio Di Piero February 2012 Forschungsinsiu zur Zukunf der Arbei Insiue for he Sudy of Labor

2 The Shor-Term Effeciveness of a Remedial Mahemaics Course: Evidence from a UK Universiy Giorgio Di Piero Wesminser Business School and IZA Discussion Paper No February 2012 IZA P.O. Box Bonn Germany Phone: Fax: iza@iza.org Any opinions expressed here are hose of he auhor(s) and no hose of IZA. Research published in his series may include views on policy, bu he insiue iself akes no insiuional policy posiions. The Insiue for he Sudy of Labor (IZA) in Bonn is a local and virual inernaional research cener and a place of communicaion beween science, poliics and business. IZA is an independen nonprofi organizaion suppored by Deusche Pos Foundaion. The cener is associaed wih he Universiy of Bonn and offers a simulaing research environmen hrough is inernaional nework, workshops and conferences, daa service, projec suppor, research visis and docoral program. IZA engages in (i) original and inernaionally compeiive research in all fields of labor economics, (ii) developmen of policy conceps, and (iii) disseminaion of research resuls and conceps o he ineresed public. IZA Discussion Papers ofen represen preliminary work and are circulaed o encourage discussion. Ciaion of such a paper should accoun for is provisional characer. A revised version may be available direcly from he auhor.

3 IZA Discussion Paper No February 2012 ABSTRACT The Shor-Term Effeciveness of a Remedial Mahemaics Course: Evidence from a UK Universiy * Whils in he US here is a growing debae abou he effeciveness of remedial universiy courses, his issue is less quesioned in he UK. Using a regression disconinuiy approach and daa from a large School of a pos-1992 UK universiy, we esimae he effec of remediaion on suden oucomes. We find no evidence ha aending a mah remedial program improves suden performance in he firs year. This finding is consisen and complemens ha of a recen sudy by Lagerlöf and Selzer (2009), which is based on daa from a pre-1992 UK universiy. Taken ogeher, hese resuls may call for a review of he remediaion policy offered a universiy level in he UK. JEL Classificaion: A22, I20 Keywords: regression disconinuiy design, remedial mahemaics, suden performance Corresponding auhor: Giorgio Di Piero Universiy of Wesminser Wesminser Business School Deparmen of Economics and Quaniaive Mehods 35 Marylebone Road NW1 5LS London Unied Kingdom G.D.I.Piero@wesminser.ac.uk * Financial suppor received by he Economics Nework of he Higher Educaion Academy is graefully acknowledged.

4 1. INTRODUCTION Higher educaion has undergone significan expansion in recen decades in several OECD counries. Expansion has been paricularly dramaic in he UK where he number of sudens enrolled a higher educaion insiuions (HEIs) increased from abou 400,000 in he early 1960s o jus less han 2,400,000 in Several commenaors see his massive increase in enrolmen as a key sep for providing educaional opporuniy for people from less-advanaged backgrounds. Addiionally, graduaes are also considered o be a crucial componen of a knowledge-based economy. However, one of he concerns relaed o his impressive growh is ha a large number of sudens acceped ino higher educaion may no be fully prepared for his level of sudy. They may no have adequae academic skills in crucial areas such as mahemaics, reading and wriing. In an aemp o address his problem, colleges and universiies have se up remedial courses 2. These courses are ofen specifically designed for firs-year sudens and heir aim is o help o bridge any gaps beween sudens knowledge and he requiremens for heir degree courses. However, despie he need, remedial universiy programs are a conroversial opic. Criics argue ha hese programs should no exis given ha unprepared sudens should no have been allowed o have access o universiy educaion. Providing remedial educaion a higher educaion level may also have considerable derimenal effecs. I may resul in a reducion in academic sandards and his, in urn, may make degrees less valuable over ime. Similarly, i is also possible o claim ha a large number of unprepared 1 Daa come from differen issues of he Times Higher Educaion Supplemen. 2 A survey of vice-chancellors in 2004 showed ha nearly half of all UK universiies (i.e. 48 percen) se up remedial classes in English and mah because of he perceived deerioraing qualiy of undergraduaes (The Telegraph, 18 h July 2004). 3

5 sudens may demoralize faculy (Pis e al., 1999). Furhermore, remedial courses are very cosly for sudens, HEIs and sociey a large. Sudens are ofen required o pay uiion fees for courses associaed wih credis ha do no coun owards graduaion. HEIs ypically inves a significan proporion of heir resources in programs aimed a helping sudens who have academic deficiencies. Taxpayers may also be relucan o bear he cos of remediaion as hey may quesion he need o pay wice for he same learning opporuniy, firs in secondary educaion and hen in universiy educaion. Finally, and perhaps mos imporanly, several opponens of remedial courses call heir efficacy ino quesion (Jenkins and Boswell, 2002). A major difficuly in asceraining he effeciveness of remedial educaion lies in he endogeneiy of remedial programs. Whils schools/universiies assign sudens o remedial programs primarily on he basis of observables, selecion ino remedial educaion may also be driven by unobservables. Given ha sandard regression echniques are able o accoun for he effec of he former bu no of he laer on suden performance, hey do no idenify he rue effec of remediaion. Specifically, he remediaion esimae may be biased downward as i picks up he negaive influence exered by hese unobserved characerisics on suden performance. In his paper, we analyze he effeciveness of a mah remedial course adoped by a School of a pos-1992 UK universiy 3. A regression disconinuiy (RD) design is used in order o address he issue of selecion on unobservables ino remedial educaion. Specifically, he RD design explois he disconinuiy induced by he rules used for 3 Universiies in England can be divided ino wo broad caegories on he basis of heir hisorical background: pre-1992 universiies and pos-1992 universiies. Whils he former group comprises hose insiuions wih universiy saus prior o 1992, he laer includes mosly former polyechnics ha received universiy saus when he Furher and Higher Educaion Ac of 1992 came ino force. 4

6 he assignmen o remedial classes. HEIs ypically use a diagnosic placemen es o assess firs-year sudens enering skill levels. While sudens whose es score is below a given cuoff are assigned o remedial courses, hose who achieve a score above his cuoff are no referred o remedial classes. The idea of he RD design is o compare he average oucome of sudens whose resul is jus above he minimum score requiremen wih he average oucome of heir peers who have scored jus below he cuoff. Given ha hese wo groups of sudens are basically hough o have he same observed (and unobserved) academic abiliy, he former group represens a good proxy of wha would have happened o sudens if hey had no aended remedial courses. The inuiion behind he RD design is ha i emulaes a random experimen a he cuoff poin (Lesik, 2008). The probabiliy of aending remedial classes can be inerpreed as being random for hose sudens whose score is in he close neighborhood of he cuoff. Our work adds o previous research on effeciveness of remedial educaion a universiy/college level in hree main aspecs. Firs, very few sudies focus heir aenion on he UK (one excepion is he paper by Lagerlöf and Selzer (2009), which is discussed in he nex Secion). Papers on he US (ypically based on experiences from Communiy Colleges) end o dominae he empirical lieraure in his area and, whils hese sudies are insrucive, i would be raher hazardous o make inference exclusively based on hem. Second, no only are here few UK-based sudies, bu hese end o examine pre-1992 universiies. However, remediaion is likely o be less relevan in hese insiuions given ha heir admission sandards are generally higher 4 4 Jerrim (2008) shows ha average sudens universiy enry score is subsanially higher in pre-1992 universiies relaive o pos-1992 universiies. 5

7 and he majoriy of sudens have radiional enry qualificaions (i.e. A-levels) 5. By conras, one would expec remediaion o play a more imporan role in pos-1992 universiies where mos sudens show lower academic abiliy and have non-sandard enry qualificaions (e.g. Advanced General Naional Qualificaion, Access courses and foundaion years) 6. Coffield and Vignoles (1997) repor ha he proporion of sudens wih alernaive enry qualificaions is considerably higher in pos-1992 universiies, when compared o pre-1992 universiies. The laer are relucan o vary heir enry requiremens as hey fear ha his change would resul in admiing less able sudens. Third, o he bes of our knowledge, his is he firs paper ha invesigaes wheher he iming of remediaion maers. The effeciveness of remedial educaion is measured agains suden performance in a mah-based course ha is compulsory for boh remedial and nonremedial sudens. While some of he remedial sudens aend he remedial program prior o his course, for ohers boh run a he same ime 7. In his paper, we analyze wheher his difference in iming affecs he effeciveness of remedial educaion 8. The remainder or he paper is organized as follows. Secion 2 briefly reviews he exising lieraure on he effeciveness of remedial programs offered a universiy/college level. Secion 3 describes he remediaion policy adoped by he School under examinaion. I comprises informaion abou he mah diagnosic es 5 Lagerlöf and Selzer (2009) look a he experience of a pre-1992 universiy and heir sample includes only sudens who ook A-levels prior o universiy. 6 Anoher disinc feaure of pos-1992 universiies is ha hey comprise many sudens from ehnic minoriy groups (Shiner and Modood, 2002; Tolley and Rundle, 2006). This is due no only o minoriy sudens having average lower enry scores and being more likely o have non-radiional enry qualificaions han whies, bu also o oher facors such as lower geographical mobiliy. Ehnic minoriy applicans end o favor insiuions in heir home region even hough hey could apply o more disan universiies ha provide a beer mach wih heir academic abiliy. 7 Remedial sudens are randomly allocaed o remedial programs aking place eiher before or a he same ime as he mah-based course. 8 Anoher advanage of our sudy is he sample size. We use a relaively large sample considering ha i only comprises daa from one insiuion. For insance, our sample is much larger han he one employed by Lagerlöf and Selzer (2009) and hence we expec he esimaes o be more precise. 6

8 and how sudens are assigned o remedial educaion. Secion 4 depics he RD design and how i is implemened in his sudy. Secion 5 presens he daa and ess he validiy of he RD design in our conex. Secion 6 repors and discusses he empirical resuls. Secion 7 concludes. 2. RELATED LITERATURE Alhough here are a large number of sudies looking a he effeciveness of universiy and college remedial courses, only few of hem, ypically published since 2005, employ a rigorous mehodology ha properly conrols for he endogeneiy bias associaed wih remediaion. These sudies, however, have produced conflicing resuls. On he one hand, some papers show ha remediaion has a posiive impac on educaional oucomes. Beinger and Long (2009) analyze he effecs of remediaion a four-year colleges in Ohio. To accoun for he endogeneiy bias of remediaion, hey use an insrumenal variable (IV) sraegy ha relies on boh variaion in remedial placemen policies and he imporance of proximiy in college choice. Their resuls sugges ha mah and English remediaion decreases he probabiliy of dropping ou of universiy and increases he likelihood of earning a degree. Furhermore, in line wih expecaions, remediaion is found o be more beneficial o sudens on he margin of requiring remediaion relaive o he general sample. A similar resul is obained by Moss and Yeaon (2006) who use an RD design in an aemp o examine he effeciveness of a remedial English course offered by a large, mulicampus communiy college. No only do hey find ha remediaion significanly improves English academic achievemen, bu hey also conclude ha sudens who are mos in need of remedial educaion receive he mos benefis. Addiional evidence 7

9 supporing he effeciveness of remedial educaion is provided by Lesik (2006) who analyses daa from a single, four-year meropolian universiy locaed in he Norheas of he US. Employing an RD design and an IV sraegy, she shows ha paricipaion in a developmenal mahemaics program increases he likelihood of successfully compleing a college-level mah course on he firs enry. On he oher hand, he effeciveness of remedial educaion has been quesioned by oher papers. Using longiudinal adminisraive daa from Texas and employing an RD design, Marorell and McFarlin (2010) find ha here is lile empirical evidence suggesing ha remedial courses have a beneficial impac on suden oucomes. Calcagno and Long (2008) analyse daa from 28 communiy colleges in Florida and conclude ha, whils remediaion has some benefis in erms of promoing early persisence in college, i does no have a saisically significan impac on degree compleion. They employ an RD design ha explois he fac ha remedial placemen in Florida mainly depends on he score obained in he Florida College Enry Level Placemen Tes (CPT). Finally, he scarce evidence from he UK seems also o indicae ha he link beween remediaion and suden oucomes is prey weak. Lagerlöf and Selzer (2009) analyze he effeciveness of a mah remedial course aended by a subse of sudens in he economics deparmen a a Briish universiy. Their sudy, which relies on a difference-in-differences (DiD) mehodology, finds ha paricipaion in his remedial course does no have a significan effec on sudens performance in economics subjecs. By conras, secondary school grades in mahemaics appear o be srong predicors of suden academic oucomes. 8

10 3. BACKGROUND In his paper, we assess he effeciveness of remediaion by looking a he experience of a large School of a pos-1992 UK universiy where all sudens enrolling for he firs ime are required o ake an on-line muliple-choice diagnosic es in mah 9. They can ake i remoely a anyime beween approximaely hree weeks before he inducion sars and he end of his session. Sudens are old no ge any help from anyone in compleing he es. This is because heir resul will deermine wheher hey need addiional help wih quaniaive skills and, if his is he case, hey will be enrolled on an appropriae course. Sudens have 30 minues o complee he es, can only aemp i once and have no informaion on he pass mark. The mah screening es is designed o assess basic algebraic skills including simplifying algebraic expressions and fracions, manipulaing roos and exponens and facoring and expanding polynomials. The maerial of his es is aligned wih he conen of hose firs-year courses ha have a mah componen. This permis faculy o se up a predeermined cuoff score on he diagnosic es. While sudens whose score on he screening mah es is below a given cuoff value are auomaically enrolled on a remedial mah course, hose who scored equal or above his cuoff are exemped from i. There are no excepions o his rule. One should noe ha, hough he mah course is remedial in naure, is successful compleion confers he same amoun of degree-couning credis 10 as any of he oher 9 Sudens who have no aken his es canno be formally regisered. 10 This is very imporan since in he US remedial courses generally do no confer degree-couning credis. 9

11 firs-year courses 11. This means ha sudens failing he placemen sandardized es have one less opional course o choose from during heir firs year of sudy relaive o heir peers who have passed he es. Conferring credis is a crucial feaure for remedial courses as i may increase heir effeciveness. Several sudies (see, for insance, Deil-Amen and Rosenbaum, 2002) show ha sudens who ake several remedial courses wihou earning credi for a degree are more likely o drop ou han oher sudens. The former sudens may develop a low level of self-confidence in heir abiliy which, in urn, may have a negaive effec on heir educaional oucomes. In his sudy, he effeciveness of he mah remedial course is measured by comparing he performance of remedial and nonremedial sudens in a mah-based course, which all firs-year sudens are required o ake. Such a course comprises wo pieces of assessmen. The firs componen, which accouns for 30% of he oal course score, consiss of a wrien repor submied by he suden a he end of he eaching period. The second componen, which is worh 70% of he oal course score, is a final examinaion ha sudens ake a he end of he academic year. The academic year is divided ino wo eaching semesers and he mah remedial course and he compulsory firs-year mah-based course are run in boh semesers. Sudens can eiher ake he wo courses concurrenly (eiher boh in he firs semeser or boh in he second semeser) or hey can ake he former in he firs semeser and he laer during he second semeser. The final imeable is randomly generaed by a sofware package used by he School. The conen of hese wo courses, heir assessmen regime and heir eaching saff and mehod have all no changed during 11 Sudens are required o ake eigh courses during he firs year, i.e. four in he firs semeser and four in he second semeser. 10

12 he hree academic years under examinaion. This means ha suden performance in hese courses is comparable over his period. 4. IDENTIFICATION STRATEGY In his secion we skech he RD design employed o esimae he effec of remedial educaion on suden academic oucomes. A sharp RD design (see, for insance, Ou, 2010; Filmer and Schady, 2011) is used given ha assignmen o remedial educaion is a deerminisic and disconinuous funcion of he score on he diagnosic mah es. While sudens scoring below a predeermined cuoff are pu ino a remedial mah course, hose whose score is equal or higher o he cuoff are exemped from he remedial course. One simple way of esimaing he effec of remediaion is o compare he average oucome of he group jus above he cuoff wih he average oucome of he group jus below he cuoff. However, his procedure may lead o very imprecise esimaes of he remediaion effec given ha ypically barely passers and barely failers make up only a small sample of he suden populaion. Therefore an equivalen bu more efficien mehod is o employ all available daa and a conrol funcion approach (Peerson-Lidbom, 2008). This involves regressing he oucome of ineres (i.e. suden performance in he compulsory firs-year mah-based course) agains a polynomial in he assignmen variable (i.e. score on he diagnosic mah es); ha is, he conrol funcion, and a dichoomous indicaor for remedial and nonremedial saus. Hence our esimaion equaion is: y i = α ) + + α REM i + f ( SCOREi + η ε (1) 0 1 i where y i is he performance of suden i in cohor in he compulsory firs-year mah-based course; REM i is a dummy variable ha akes he value 1 if suden i in cohor is assigned o he mah remedial course, and zero oherwise; f ( SCORE i ) is 11

13 an unknown smooh funcion of he score obained by suden i in cohor in he mah diagnosic es; η is a cohor-specific inercep and ε i is an error erm. The inclusion of f ( SCORE i ) as a regressor is expeced o capure any correlaion beween REM i and ε i, and herefore α 1 will be an unbiased esimae of he effec of remediaion on suden performance in he compulsory firs-year mah-based course 12. Alhough, in principle i is no necessary o include addiional covariaes in Equaion (1), heir inroducion can be jusified on wo grounds. Firs, i allows for variance reducion and his is imporan given ha he RD echnique has large sampling variabiliy. Second, he inclusion of hese addiional covariaes can also be hough as a check for he plausibiliy of he idenificaion sraegy. An underlying assumpion of our RD approach is ha, condiional on f ( SCORE i ), he only source of disconinuiy in suden performance a he cuoff score should be failing he mah screening es. Therefore if his assumpion is me, one would expec he presence of conrol variables no o significanly affec he effec of remediaion. Incorporaing he covariaes, he esimaion equaion becomes he following: y i = ' α 0 + α1rem i + f SCOREi ) + α 2SEM 2i + X i ψ + η + ( ε (2) i where SEM 2 i is a dummy variable ha akes he value 1 if suden i in cohor ook he compulsory mah-based course in he second semeser, and zero oherwise 13 ; X i 12 As we have no daa on aendance of he mah remedial course, wha we idenify is he impac of being enrolled on his course. This means ha α 1 is an inen-o-rea parameer. As observed by Bandiera e al. (2010), suden enrolmen is a more ineresing policy parameer for universiies relaive o suden aendance. Whils he former can be easily measured and manipulaed by hem, his does no hold for he laer. 13 The raionale for including he second semeser dummy follows. Given ha he final examinaion for he compulsory firs-year mah-based course akes place a he end of he academic year for sudens who aended his course in he firs semeser as well as for hose who ook i in he second semeser, (oher hings being equal) one would expec he laer group o perform, on average, beer han he 12

14 is a vecor ha includes a se of conrol variables (i.e. age, gender, naionaliy, ehniciy and socio-economic background 14 ) for suden i in cohor. We consider wo differen indicaors of suden performance in he compulsory firsyear mah-based course: 1) a binary variable ha akes he value 1 if he suden has successfully compleed he course, and zero oherwise; 2) he overall score (ou of 100) received by he suden in he course. The raionale for including he firs measure is ha firs-year sudens may have lile incenives for geing high scores on heir examinaions as scores in firs-year subjecs do no coun owards he final degree classificaion (his occurs in mos UK universiies). Because in he academic year he cuoff score on he mah screening es is one-poin lower han ha in he academic years and , his was redefined by subracing he cohor-specific cuoff score from each suden s score. 5. DATA AND TESTS OF THE VALIDITY OF THE RD DESIGN Our daa comprise suden-level observaions for hree consecuive cohors of firsyear sudens in he academic years , and From his sample, we exclude remedial sudens who did no ake he exam for he remedial mah course (i.e. 49 sudens). In addiion, we remove from he sample hose sudens who did no paricipae in any of he wo assessmens of he compulsory mah-based former one. The long delay beween he end of he course and he examinaion may in fac have a derimenal effec on suden achievemen. 14 We use occupaion of paren or guardian as a proxy for socio-economic background. 15 However, in he academic year he number of quesions included on he mah diagnosic es were wo less han ha included in he academic years and

15 course (i.e. 145 sudens). This leaves a sample of 1,521 sudens 16. Columns 1, 2 and 3 of Table 1 repor he descripive saisics of our dependen and independen variables for he whole sample, nonremedial sudens and remedial sudens, respecively. Approximaely 33.3% of firs-year sudens have failed he diagnosic es and hence have been assigned o he mah remedial course. Unforunaely, we do no have informaion on he socio-economic background of many sudens. However, in order o maximize he sample size, we keep hese observaions and creae a dummy variable indicaing missing saus. Column 4 of Table 1 presens he difference in means beween remedial and nonremedial sudens. In line wih he findings obained by similar sudies (see, for insance, Aewell e al., 2006), remedial sudens are more likely o be older, more likely o come from an ehnic minoriy group and more likely o have a low socio-economic background relaive o nonremedial sudens. Inser Table 1 abou here As oulined in Secion 1, he RD design assumes ha, in he neighborhood of he cuoff on he mah diagnosic es, remedial and nonremedial sudens have similar observed characerisics. To check for his assumpion, Table 2 presens means for remedial sudens whose adjused score on he mah diagnosic es is no more han wo poins away from he cuoff (Column 1), nonremedial sudens whose similar score is up o wo poins above he cuoff (Column 2), and he difference in means beween hese wo groups (Column 3). This bandwidh is chosen as i comprises a roughly comparable number of sudens on eiher side of he cuoff. I also includes a sufficien number of sudens o obain meaningful resuls. Our resuls show suppor 16 The number of sudens included in each cohor is 562 ( ), 553 ( ) and 406 ( ). 14

16 for he RD assumpion. The samples jus below and jus above he cuoff are found no o be differen in erms of observed characerisics. Inser Table 2 abou here As oulined in Secion 4, anoher assumpion of our RD design is ha he only source of disconinuiy a he cuoff score is failing he mah screening es. To es for his assumpion, we run a se of regressions, where each suden characerisic (raher han he oucome of ineres) is regressed agains alernaive specificaions of he conrol funcion f ( SCORE i ) and our remediaion dummy. The resuls from hese regressions, which are depiced in Table 3, show ha here are no disconinuiies in predeermined characerisics around he cuoff. Inser Table 3 abou here Anoher hrea o idenificaion in our RD design sems from he possibiliy ha he adjused score on he mah diagnosic es is manipulaed by he School. Such a possibiliy, however, seems very limied. The School has no incenive o keep he number of remedial sudens low as no addiional resources are invesed in remedial programs. As argued in Secion 3, remedial sudens have one less opional course o choose from as successful compleion of he mah remedial course confers he same amoun of degree-couning credis as any of he oher firs-year courses. Addiionally, as he mah diagnosic es is a muliple choice es, his makes manipulaion even less likely. In spie of hese argumens, following he approach of Carrell e al. (2011), in Figure 1 we look a wheher here are sharp breaks in he disribuion of he adjused score on he mah diagnosic es near he cuoff. In paricular, we are ineresed o see 15

17 wheher i has a peak in zero. We es for his by using linear and quadraic funcions o approximae he disribuion of our assignmen variable and find ha in boh cases he observed number of sudens a he cuoff is prediced by he overall disribuion, wihou any significan deparure. Inser Figure 1 abou here Before urning in he nex Secion o he esimaes of Equaions (1) and (2), in Figure 2 we depic he relaionship beween he adjused score on he mah diagnosic es and suden performance in he compulsory firs-year mah-based course, using pooled daa from he hree cohors. This Figure shows he raw average suden oucome a each value of he adjused score, as well as predicions from a linear funcion of SCORE i. Whils in panel A of Figure 2 suden performance is proxied by he overall score on he mah-based course, in panel B a simple binary oucome (pass or fail) is used. Looking a boh panels, in line wih our expecaions, suden performance is an increasing funcion of he adjused score on he mah diagnosic es. However, one may noe ha in boh panels here seems o be no disconinuiy in suden performance a he cuoff based on he mah diagnosic es. Addiionally, alhough graphically i appears ha he disribuion of he scores is fairly linear, in he nex Secion we will es wheher our RD esimaes are robus across specificaions ha include higher order polynomials of SCORE i. Inser Figure 2 abou here 6. EMPIRICAL RESULTS Table 4 presens regression disconinuiy esimaes of he effec of remediaion on suden performance in he firs-year compulsory mah module. Whils for he binary 16

18 suden oucome variable (i.e. pass or fail) we employ he maximum likelihood probi echnique o esimae he models, for he coninuous suden oucome variable (i.e. overall score) OLS models are esimaed. Following Lee and Card (2008), sandard errors are clusered a he adjused score (on he mah diagnosic es) level as his is appropriae in RD frameworks where he assignmen variable is discree. Addiionally, given ha a big concern in RD design is wheher he conrol funcion is correcly specified, we employ four alernaive specificaions of f ( SCORE i ). Columns 1 hrough 4 depic esimaes of he regression disconinuiy parameer α1 from Equaion (1) where we use linear, quadraic, cubic and quaric conrols for he adjused score on he mah diagnosic es, respecively. In all hese specificaions his parameer has a negaive sign and i is no saisically significan a he convenional levels. Therefore hese esimaes no only show ha he remediaion effec is indisinguishable from zero, bu also ha such a finding is no sensiive o differen specificaions of he conrol funcion. Inser Table 4 abou here We hen perform anoher sensiiviy check by adding conrol variables o our specificaions. Columns 5 horough 8 of Table 4 repor esimaes of α1 from Equaion (2) where linear, quadraic, cubic and quaric polynomials of SCORE are included, respecively. The resuls confirm he appropriaeness of he use of he RD framework in his conex. They show ha our RD esimaes are no significanly affeced by he inclusion of covariaes 17. i 17 The esimaes repored in he firs row of Table 4 are broadly consisen wih hose obained when suden performance is measured by he score on he end of course repor. These esimaes are available upon reques from he auhor. 17

19 As an addiional sensiiviy check, Table 5 presens esimaes of he remediaion effec when we use a disconinuiy sample (i.e. i only includes daa ha are ± 2 poins from he cuoff). Columns 1 and 2 depic esimaes from specificaions wihou and wih conrols, respecively. In hese specificaions no conrols for he adjused score on he mah diagnosic es are included, under he assumpion ha among hose sudens sufficienly close o he cuoff, assignmen o remedial educaion is basically random. Again, all he esimaed RD coefficiens have a negaive sign and hey are no saisically significan a he usual levels, hough he size and he sandard errors are larger han hose repored in Table 4. Inser Table 5 abou here Nex we exend our empirical analysis by addressing he issue of he possible heerogeneiy of remediaion across sudens. Specifically, we examine wheher aking he mah remedial course before or a he same ime as he compulsory firsyear mah-based course has any effec on he effeciveness of remediaion. Theoreical argumens can be pu forward o suppor boh hypoheses. On he one hand, one may argue ha i would be beer for sudens o aend he mah remedial course firs. This gives hem an opporuniy o build on wha hey have learn in he remedial classes. Addiionally, sudens would have he ime for proper digesion of remedial maerial and mauring of ideas and undersanding. On he oher hand, however, here are also reasons supporing he hypohesis ha aking hese courses simulaneously is beneficial. In so doing, for insance, here would be no need for sudens o spend ime in revisiing earlier lessons. Sudens may also be more ready o learn as hey have an immediae opporuniy o apply he skills acquired in he remedial course o anoher course. 18

20 To es wheher aking he mah remedial course a he same ime as he compulsory firs-year mah-based course affecs he effeciveness of remediaion, we resric our sample o hose remedial sudens who have aken hese courses in he same semeser (eiher boh in he firs semeser or boh in he second semeser) plus all nonremedial sudens. Columns 1 hrough 4 of Table 6 18 show he regression disconinuiy esimaes for his subsample. Resuls are presened for RD specificaions ha include linear, quadraic, cubic and quaric conrols for he adjused score on he mah screening es. All hese esimaes do no provide suppor for he proposiion ha aking he mah remedial course simulaneously wih he compulsory firs-year mahbased course makes remediaion effecive. Inser Table 6 abou here Especially in ligh of he resuls discussed above, we now look a he quesion wheher i would be bes for sudens o ake he mah remedial course prior o he compulsory firs-year mah-based course. Hence we include in our sample hose remedial sudens who have aended he mah remedial course in he firs semeser and he compulsory firs-year mah-based course in he second semeser plus hose nonremedial sudens who have aken he laer in he second semeser (conrol group). Also in his case, he esimaes, which are repored in Columns 5 hrough 8 of Table 6, provide no evidence ha remediaion works. For he overall score on he compulsory firs-year mah-based course, he esimaes have a negaive sign and hey are no saisically significan a he convenional levels across all specificaions. For he binary oucome (pass or fail), hough he regression disconinuiy parameer is weakly saisically significan (i.e. a he 10% level) in specificaions comprising linear and 18 Esimaes presened in Table 6 remain largely unchanged once covariaes are added o all specificaions. 19

21 quadraic conrols for he adjused score on he mah diagnosic es, i becomes saisically insignifican when higher order polynomials of f ( SCORE i ) are included. Finally, we explore he possibiliy ha remediaion may affec suden performance in differen firs-year courses han he mah-based one. The raionale for his is ha, since he remedial program makes i easier for sudens o undersand he maerial of he mah-based course, hey may decide o reallocae heir ime and effors o oher subjecs. This indirec effec may be paricularly relevan given ha, as oulined above, scores on firs-year courses do no coun owards he final degree classificaion. While Columns 1 hrough 4 of Table 7 presen esimaes of he effec of remediaion on suden performance in a compulsory firs-year economics-based course, Columns 5 hrough 8 depic similar esimaes for a compulsory firs-year human resources course 19. The regression disconinuiy parameer is never saisically differen from zero, which implies ha no only does remediaion have no discernible effec on suden performance in he mah-based course, bu also in oher firs-year compulsory courses. Inser Table 7 abou here 7. CONCLUSIONS In his paper, we have analyzed he shor-erm effeciveness of a mah remedial course adoped by a School of a pos-1992 UK universiy. Whils here is a significan body of US-based lieraure looking a he impac of remedial programs on suden achievemen a universiy/college level, much less evidence is available for he UK. This is unforunae given ha in he las decades in he UK he unprecedened 19 Esimaes presened in Table 7 remain largely unchanged once covariaes are added o all specificaions. 20

22 expansion of paricipaion in higher educaion has been accompanied by an increase in he number of sudens who are perceived o be unprepared o sudy a his level. They ofen lack academic skills in crucial areas such as mahemaics, reading and wriing. Addiionally, as hese sudens are more likely o come from an ehnic minoriy group and from a disadvanaged background, remedial courses may play an imporan role in improving heir educaional opporuniies. Using a regression disconinuiy approach o accoun for he endogeneiy bias caused by he poenial correlaion beween aending remedial classes and unobserved characerisics negaively affecing suden achievemen, we have found no empirical evidence suggesing ha remediaion is effecive. Specifically, our findings indicae ha aending he mah remedial course does no improve suden performance in a mah-based course. Addiionally, our empirical resuls show ha he effeciveness of remediaion does no depend on is iming. Taking he mah remedial course a he same ime or prior o he compulsory mah-based course does no change he conclusion menioned above. Finally, our esimaes do no suppor he proposiion ha he mah remedial program improves suden performance in compulsory firsyear courses differen from he mah-based one. No only are our findings in line wih hose of a recen sudy by Lagerlöf and Selzer (2009), which is based on daa from anoher UK higher educaion insiuion, bu hey also complemen each oher. Remediaion does no seem o work neiher in a pos universiy nor in a pre-1992 universiy. While Lagerlöf and Selzer (2009) find ha remediaion is ineffecive for he group of remedial sudens as a whole, our sudy suggess ha he same conclusion holds for hose remedial sudens whose score on 21

23 he mah diagnosic es is close o he cuoff 20. Therefore, aken ogeher, he resuls from hese wo sudies may underscore he need o review remediaion policy offered a universiy level in he UK. While effors have o be made o improve universiy remediaion policy in he UK, fewer resources are likely o be available owards his given he curren financial climae in he higher educaion secor. In ligh of he proposed wihdrawal of public subsidies (December 2010), UK universiies could poenially face funding cus up o 40% over he nex four years. While universiies are puing forward heir plans on areas where expendiure can be reduced, ineffecive remedial programs appear o be quie an obvious candidae. 20 In conras o DiD esimaes, RD esimaes are no applicable o he enire suden populaion as hey only esimaes remediaion effecs around a narrow inerval (he local effec is esimaed). 22

24 REFERENCES Aewell, P., Lavin, D., Domina, T. and Levey, T. (2006). New evidence on college remediaion, Journal of Higher Educaion, Vol.77, pp Bandiera, O., Larcinese V. and Rasul, I. (2010). Heerogeneous class size effecs: new evidence from a panel of universiy sudens, Economic Journal Vol. 120, pp Beinger, E. P. and Terry Long, B. (2009). Addressing he needs of underprepared sudens in higher educaion: Does college remediaion Work?, Journal of Human Resources, Vol. 44, pp Calcagno, J. C., and Terry Long, B. (2008). The impac of possecondary remediaion using a regression disconinuiy approach: Addressing endogenous soring and noncompliance, Working Paper No , NBER. Carrell, S. E., Hoeksra, M. and Wes, J. E. (2011). Does drinking impair college performance? Evidence from a regression disconinuiy approach, Journal of Public Economics, Vol. 95, pp Coffield, F., and Vignoles, A. (1997). Widening paricipaion in higher educaion by ehnic minoriies, women and alernaive sudens, Repor No 5, The Naional Commiee of Inquiry ino Higher Educaion. Newcasle, Deparmen of Educaion. Deil-Amen, R. and Rosenbaum, J. E. (2002). The uninended consequences of sigma-free remediaion, Sociology of Educaion, Vol. 75, pp Filmer, D. and Schady, N. (2011). Does more cash in condiional cash ransfer programs always lead o larger impacs on school aendance, Journal of Developmen Economics, Vol. 96, pp Imbens, G. W. and Lemieux, T. (2008). Regression disconinuiy designs: A simple guide, Journal of Economerics, Vol. 142, pp Jenkins, D. and Boswell, K. (2002). Sae Policies on Communiy College Remediaion: Findings from a Naional Survey, Denver: Educaion Commission of he Saes, Cener for Communiy College Policy. Jerrim, J. (2008). Wage expecaions of UK sudens: how do hey vary and are hey realisic?, S3RI Applicaions and Policy Working Paper A08/08, Souhampon Saisical Sciences Research Insiue. Lee, D S. and Card, D. (2008). Regression disconinuiy inference wih specificaion error, Journal of Economerics, Vol. 142, pp Lagerlöf, J. N. M., and Selzer, A. J. (2009). The effecs of remedial mahemaics on he learning of economics: evidence from a naural experimen, Journal of Economic Educaion, Vol. 40, pp

25 Lesik, S. A. (2008). Evaluaing developmenal educaion programs in higher educaion, Issue 4, ASHE/Lumina Policy Brief. Lesik, S. A. (2006). Applying he regression disconinuiy design o infer causaliy wih non-random assignmen, Review of Higher Educaion, Vol. 30, pp Marorell, P, and McFarlin I. (2010). Help or hindrance? The effecs of college remediaion on academic and labor marke oucomes, Working Paper, School Texas Projec, Texas Educaion Research Cener a UT. Moss, Brian G., and Yeaon, W. H. (2006). Shaping policies relaed o developmenal educaion: an evaluaion using he regression-disconinuiy design, Educaional Evaluaion and Policy Analysis, Vol. 28, pp Ou, D. (2010). To leave or no o leave? A regression disconinuiy analysis of he impac of failing he high school exam, Economics of Educaion Review Vol. 29, pp Peersson-Lidbom, P. (2008). Do paries maer for economic oucomes? A regression-disconinuiy approach, Journal of he European Economic Associaion, Vol. 6, pp Pis, J. M., Whie, W. G., and Harrison, A. B (1999). Suden academic underpreparedness: Effecs on faculy, Review of Higher Educaion, Vol. 22, pp Shiner, M. and Modood, T. (2002). Help or hindrance? Higher educaion and he roue o ehnic equaliy, Briish Journal of Sociology of Educaion, Vol. 23, pp Tolley, J. and Rundle, J. (2006). A review of black and minoriy ehnic paricipaion in higher educaion, Aimhigher, London. 24

26 Table 1: Descripive saisics All sudens (1) Nonremedial sudens (2) Remedial sudens (3) Remedial sudens vs nonremedial sudens (4) Mean Mean Mean Difference in means Overall score on he compulsory firs-year mah-based course (16.375) (15.752) (15.890) *** (0.860) Successfully compleed compulsory firsyear mah-based course (0.354) (0.299) (0.428) *** (0.189) Adjused score on he mah diagnosic es (4.283) (2.442) (2.453) *** (0.133) Remediaion (0.471) Compulsory firs-year mah-based course aken in he second semeser (0.487) (0.490) (0.481) (0.026) Female ** (0.500) Age< or = (0.492) Ehniciy Whie (0.410) Black (0.322) Bangladeshi/Indian/Pakisani (0.412) Chinese (0.144) Oher Asian (0.210) Oher ehnic group or Unknown (0.487) Naionaliy UK naionaliy (0.498) Oher EU naionaliy (0.438) Non-EU naionaliy (0.452) Socio-economic background High socio-economic background (0.375) (0.500) (0.485) (0.413) (0.273) (0.399) (0.161) (0.206) (0.496) (0.492) (0.452) (0.460) (0.499) (0.500) (0.406) (0.394) (0.436) (0.099) (0.217) (0.453) (0.498) (0.403) (0.434) (0.027) *** (0.027) (0.022) 0.111*** (0.017) 0.057** (0.022) ** (0.008) (0.011) *** (0.026) 0.133*** (0.027) *** (0.024) ** (0.025) (0.380) (0.365) (0.020) Inermediae socio-economic background (0.306) (0.305) (0.309) (0.017) Low socio-economic background (0.357) (0.342) (0.384) 0.045** (0.019) Unknown socio-economic background (0.494) (0.493) (0.497) (0.027) Observaions 1,521 1, Noes: 1) In Columns 1, 2 and 3 sandard deviaions are in brackes. In Column 4 sandard errors are in brackes. 2)*** denoes saisically significan a 1%; ** denoes saisically significan a 5%; * denoes saisically significan a 10%. 25

27 Table 2: Descripive saisics for he disconinuiy sample (± 2 poins from he cuoff on he mah diagnosic es) Nonremedial sudens (1) Remedial sudens (2) Remedial sudens vs nonremedial sudens (3) Mean Mean Difference in means Female (0.501) (0.498) (0.045) Age< or = (0.491) (0.489) (0.044) Ehniciy Whie (0.426) (0.414) (0.038) Black (0.309) (0.370) 0.057* (0.031) Bangladeshi/Indian/Pakisani (0.429) (0.441) (0.039) Chinese (0.165) (0.102) (0.012) Oher Asian (0.247) (0.208) (0.020) Oher ehnic group or Unknown (0.469) (0.460) (0.042) Naionaliy UK naionaliy (0.501) (0.045) (0.499) Oher EU naionaliy (0.429) (0.409) (0.038) Non-EU naionaliy (0.434) (0.431) (0.039) Socio-economic background High socio-economic (0.400) (0.363) (0.034) background Inermediae socio-economic background Low socio-economic background Unknown socio-economic background Compulsory firs-year mahbased course aken in he second semeser (0.297) (0.331) (0.028) (0.386) (0.382) (0.034) (0.501) (0.499) (0.045) (0.460) (0.489) (0.043) Observaions Noes: 1) In Columns 1 and 2 sandard deviaions are in brackes. In Column 3 sandard errors are in brackes. 2)*** denoes saisically significan a 1%; ** denoes saisically significan a 5%; * denoes saisically significan a 10%. 26

28 Figure 1: Disribuion of observaions of he adjused score on he mah diagnosic es near he cuoff Frequency coun adjused score on he mah diagnoisc es 27

29 Figure 2: Suden performance in he compulsory firs-year mah-based course and he adjused score on he mah diagnosic es Panel A: overall score on he compulsory mah-based course overall score on he compulsory mah-based course adjused score on he mah diagnosic es Panel B: probabiliy of passing he compulsory mah-based course probabiliy of passing he compulsory mah-based course adjused score on he mah diagnosic es Noes: 1) The figure presens raw average suden oucome a each value of he adjused score, along wih he fied values of a linear regression calculaed wihin each of he wo segmens. 2) In boh panels he size of he symbols reflecs he number of sudens in each value of he adjused score. 28

30 Table 3: Regression disconinuiy esimaes on pre-deermined suden characerisics Adjused score on he mah screening es specificaion Linear Quadraic Cubic Fourh-order Female (0.120) Age< or = (0.121) Whie (0.134) Black (0.154) Bangladeshi/Indian/Pakisani (0.132) Chinese (0.290) Oher Asian (0.203) Oher ehnic group or Unknown (0.124) UK naionaliy (0.120) Oher EU naionaliy (0.131) Non-EU naionaliy (0.129) High socio-economic background (0.142) Inermediae socio-economic background (0.161) Low socio-economic background (0.143) Unknown socio-economic background (0.120) (0.121) (0.123) (0.137) (0.154) (0.133) (0.339) (0.204) (0.126) (0.121) (0.134) (0.131) (0.143) (0.165) (0.143) (0.121) (0.151) (0.153) (0.172) (0.186) (0.165) (0.372) (0.252) (0.160) (0.153) (0.170) (0.166) (0.179) (0.206) (0.178) (0.152) (0.155) (0.157) (0.175) (0.206) (0.169) (0.477) (0.262) (0.161) (0.156) (0.172) (0.169) (0.183) (0.209) (0.183) (0.155) Noes: 1) Each enry is a separae regression. 2) Each cell repors esimaes on he coefficien of he remediaion dummy in a probi regression of a given suden characerisic on he conrol funcion f ( SCORE i ). 29

31 Table 4: Regression disconinuiy esimaes of he effec of remediaion on suden performance in he compulsory firs-year mah-based course Overall score (OLS) Pass or fail (Probi) (1) (2) (3) (4) (5) (6) (7) (8) (1.661) (1.490) (1.949) (2.323) (1.651) (1.475) (1.832) (2.138) (0.032) (0.032) (0.039) (0.043) (0.030) (0.029) (0.034) (0.037) Adjused Linear Quadraic Cubic Quaric Linear Quadraic Cubic Quaric score on he mah screening es specificaion Conrols N N N N Y Y Y Y Obs 1,521 1,521 1,521 1,521 1,521 1,521 1,521 1,521 Noes: 1) Each enry is a separae regression. 2) Sandard errors are repored in brackes and are clusered a he adjused score (on he mah diagnosic es) level. 3) All regressions include cohor specific effecs. 4) OLS coefficiens and probi marginal effecs are repored. 5) The conrols include gender, age, naionaliy, ehniciy, socio-economic background and he semeser dummy. 30

32 Table 5: Regression disconinuiy esimaes of he effec of remediaion on suden performance in he compulsory firs-year mah-based course (disconinuiy sample) Overall score (OLS) Pass or fail (Probi) (1) (2) (2.302) (1.976) (0.044) (0.039) Conrols N Y Obs Noes: 1) Each enry is a separae regression. 2) Sandard errors are repored in brackes and are clusered a he adjused score (on he mah diagnosic es) level. 3) All regressions include cohor specific effecs. 4) OLS coefficiens and probi marginal effecs are repored. 5) The conrols include gender, age, naionaliy, ehniciy, socio-economic background and he semeser dummy. 31

33 Table 6: Regression disconinuiy esimaes of he effec of he iming of remediaion on suden performance in he compulsory firs-year mah-based course Timing of remediaion Overall score (OLS) Pass or fail (Probi) Adjused score on he mah screening es specificaion Mah remedial course aken a he same ime as he compulsory firs-year mah-based course Mah remedial course aken prior o he compulsory firs-year mah-based course (1) (2) (3) (4) (5) (6) (7) (8) (1.871) (0.047) (1.929) (0.047) (2.192) (0.045) (2.317) (0.044) (2.169) * (0.039) (2.077) * (0.042) (2.731) (0.057) (2.477) (0.050) Linear Quadraic Cubic Quaric Linear Quadraic Cubic Quaric Obs 1,268 1,268 1,268 1, Noes: 1) Each enry is a separae regression. 2) Sandard errors are repored in brackes and are clusered a he adjused score (on he mah diagnosic es) level. 3) All regressions include cohor specific effecs. 4) OLS coefficiens and probi marginal effecs are repored 5)*** denoes saisically significan a 1%; ** denoes saisically significan a 5%; * denoes saisically significan a 10%. 32

34 Table 7: Regression disconinuiy esimaes of he effec of remediaion on suden performance in oher compulsory firs-year courses Economics-based course Human resources course (1) (2) (3) (4) (5) (6) (7) (8) Overall score (OLS) (0.947) (1.001) (0.918) (0.838) (0.965) (0.965) (1.055) (0.896) Pass or fail (Probi) (0.016) (0.016) (0.020) (0.021) (0.020) (0.020) (0.022) (0.022) Adjused Linear Quadraic Cubic Quaric Linear Quadraic Cubic Quaric score on he mah screening es specificaion Obs 1,401 1,401 1,401 1,401 1,450 1,450 1,450 1,450 Noes: 1) Each enry is a separae regression. 2) Sandard errors are repored in brackes and are clusered a he adjused score (on he mah diagnosic es) level. 3) All regressions include cohor specific effecs. 4) OLS coefficiens and probi marginal effecs are repored. 33