Mismatch in Law School by Jesse Rothstein, Princeton University and NBER Albert Yoon, Northwestern University CEPS Working Paper No.

Transcription

1 Msmatch n Law School y Jesse Rothsten Prnceton Unversty and NBER Alert Yoon Northwestern Unversty CEPS Workng Paper No. 123 Feruary 2006 Astract: An mportant crtcsm of affrmatve acton polces n admssons s that they may hurt mnorty students who are therey nduced to attend selectve schools. We use two comparsons to dentfy so-called msmatch effects n law schools wth consstent results. Black students attan etter employment outcomes than do whtes wth smlar credentals. Any msmatch effects on graduaton and ar exam passage rates are confned to the ottom quntle of the enterng credentals dstruton where selecton as s an mportant potentally confoundng factor. Elte law schools use of affrmatve acton thus does not appear to generate msmatch effects. Acknowledgements: We thank Rck Ael Bll Bowen Tom Kane Larry Katz Jde Nzele Max Schanzenach and semnar partcpants at NBER UCSB the Northwestern Unversty School of Law Faculty Workshop and the Ramon Areces Foundaton for helpful comments and suggestons. We are extremely grateful to the Andrew W. Mellon Foundaton and the Prnceton Center for Economc Polcy Studes for fnancal support. Jesse Rothsten: Industral Relatons Frestone Lrary Prnceton NJ 08544; jrothst@prnceton.edu Alert Yoon: School of Law 357 E. Chcago Ave. Chcago IL 60611; alerthyoon@law.northwestern.edu

2 I. Introducton Affrmatve acton polces n college admssons have een legally and poltcally contentous snce ther ncepton n the late 1960s. Insttuted wth the goal of offsettng the legacy of dscrmnaton aganst lacks these polces grant preferences n admssons to lack and other underrepresented mnorty applcants. The Unted States Supreme Court has repeatedly held that race-conscous polces are a permssle response to the compellng state nterest n dversty. 2 Pulc and academc deate over affrmatve acton polces has focused at least as much on ther mpacts on lack students academc and laor market outcomes as on other students educatonal experences (see e.g. Arcdacono 2005). Crtcs have argued that affrmatve acton hurts lack students y admttng them to schools for whch they are unprepared and that these students would do etter learn more e more lkely to graduate avod psychologcal damage etc. f they were requred to attend schools more approprate for ther qualfcatons (Summers 1970; Sowell 1978; Thernstrom and Thernstrom 1997). Although legally rrelevant ths clam s central to the poltcal evaluaton of affrmatve acton: Support for preferences would almost certanly evaporate f they were shown to make lack students worse off. Evdence regardng the so-called msmatch hypothess has een extremely lmted (Holzer and Neumark 2000). Sustantal dfferences etween lack and whte college completon rates are sometmes presented as ndcatons of msmatch (Thernstrom and Thernstrom 1997; Herrnsten and Murray 1994; D Souza 1991) though these may smply reflect large lack-whte gaps n enterng students qualfcatons (Kane 1998). 2 Regents of Unv. of Cal. v. Bakke 438 U.S. 265 (1978) and Grutter v. Bollnger 539 U.S. 306 (2003). 1

3 Msmatch mght e characterzed as a negatve peer effect: A student s outcomes wll declne f the average qualfcatons of her classmates rse too hgh aove her own (Loury and Garman 1995). Bowen and Bok (1998) fnd that mnorty students at elte colleges despte havng qualfcatons well elow ther whte classmates experence etter long-run outcomes than do smlarly qualfed mnorty students at less selectve colleges. More recently however Sander s (2004) analyss of dfferences n outcomes etween lack and whte law students leads hm to conclude that msmatch effects are sustantal. 3 In ths paper we use Sander s data ut apply more drect emprcal strateges to assess the msmatch hypothess as t apples to law schools. The key challenge n estmatng msmatch s the measurement of counterfactual outcomes. Namely what would have happened to lack students admtted to selectve schools had they not receved admssons preferences? We use two smple comparsons each drectly related to the hypothess of nterest. Frst we compare students attendng the most selectve law schools wth students of the same race wth smlar admssons credentals at less-selectve schools. Second we compare lack students wth whte students. 4 Black students on average attend much more selectve schools than do whtes wth smlar admssons credentals. If selectvty has a negatve effect on educatonal outcomes ths should appear as a negatve lack effect on those outcomes. We expect that our two strateges are ased n opposte drectons the etweenselectvty comparson provdng an upward-ased estmate of the selectvty effect (so 3 Sander s analyss and conclusons have een the suject of vehement crtcsm (Ayres and Brooks 2005; Chamers et al. 2005; Dauer 2005; Wlkns 2005; Ho 2005) to whch Sander (2005a; 2005) responded. As we dscuss n the Appendx Sander s results are drven y large etween-race gaps n law school grades and a strong wthn-race assocaton etween grades and later outcomes lke graduaton and ar exam passage rates. 4 Ths s essentally the comparson used y Sander (2004). Both our approach and hs assume that oservale credentals summarze the gap n potental outcomes etween lack and whte enterng students. Sander addtonally requres assumptons aout the role of law school GPAs n medatng the effects of credentals on later outcomes whch we are ale to avod and whch the data appear to reject. 2

4 understatng msmatch) and the etween-race comparson a downward-ased estmate. The results should therefore racket the true effect. At the outset t s mportant to emphasze the lmted overlap etween the dstrutons of admssons credentals of lack and whte law students: Usng Sander s (2004) ndex of Law School Admssons Test (LSAT) scores and undergraduate grade pont averages the 95 th percentle lack student s at only the 54 th percentle of the whte dstruton; and the 5 th percentle whte student s at the 61 st percentle of the lack dstruton. Any msmatch analyss must dstngush the effect of nterest from the effects of these dfferences n credentals. We use reweghtng technques that do not requre us to parameterze the effects of enterng credentals. Our comparson etween more- and less-selectve schools offers no support for the msmatch hypothess. Although school selectvty has a sustantal negatve effect on class rank for oth lack and whte students ths does not translate nto later outcomes: Students attendng more selectve schools are more lkely to complete ther programs are equally lkely to pass the ar exam and earn hgher salares after graduaton. Results of our second comparsons are more mxed. Even after adjustng for dfferences n enterng credentals lack students are sgnfcantly less lkely to graduate or pass the ar than are whte students. Black law graduates however otan etter jos than do ther whte counterparts. Underperformance on the frst two dmensons s concentrated at the very ottom of the enterng credentals dstruton and among students n the upper four quntles of the admssons ndex dstruton lacks perform as well or etter than whtes on every outcome except class rank. Although three quarters of lacks n our sample fall n the ottom quntle nference aout msmatch n ths range s hazardous: Law school admssons rates for applcants wth 3

5 such poor credentals are very low partcularly among whtes. If etter prepared students are more lkely to e admtted e.g. f lower-ranked law schools evaluate applcatons for ndcatons of preparedness that we cannot oserve dfferental sample selecton could generate mean performance dfferences etween admtted whte and lack students even n the asence of msmatch effects. We nterpret our results as demonstratng that there are no msmatch effects on the ar passage rates of the most qualfed lack students. For students n the ottom quntle of the qualfcatons dstruton the data are consstent ether wth msmatch effects on graduaton and ar passage or wth dfferental selecton so do not support strong conclusons. By contrast there are clearly no msmatch effects on lack employment outcomes n any part of the credentals dstruton. The structure of the paper s as follows: Part II argues that legal educaton s a partcularly attractve arena n whch to study msmatch. Part III dscusses the msmatch hypothess. We defne the term dstngush t from related phenomena and dentfy varous formulatons of the hypothess. Part IV ntroduces our dentfcaton strateges and our emprcal approach. Part V descres the Bar Passage Study (BPS) data. Parts VI and VII presents results of our two strateges and Part VIII concludes. II. Why Law Schools? Though our focus on law school admssons s largely a functon of data avalalty the msmatch hypothess s partcularly plausle n ths arena. Students must pass a ar exam to practce law and the sujects covered y the exam receve lttle emphass n elte law schools currcula. Lower-ranked less-selectve schools tend to devote more of ther currcula to exam preparaton (Edwards 1992; Whte 1993). Attendng a more selectve 4

6 school thus reduces the amount of ar preparaton students receve perhaps reducng ther success rates on the exam. Moreover students grades partcularly those earned n the frst year of law school when courses are typcally exam-ased and graded on strct curves are mportant qualfcatons for opportuntes durng law school (e.g. law revew memershp and good summer nternshps) and after graduaton. Even wthout msmatch effects a student can e expected to rank lower wthn her class the more selectve the school she attends. Unless employers take school selectvty nto account when evaluatng jo applcants class ranks ths mechancal rank effect could carry over nto employment outcomes. 5 There are also analytc advantages to a focus on legal educaton. Frst the ar exam provdes an ojectve measure of students accomplshment early n ther post-school careers. Bar exams are the same for all prospectve lawyers n a state are graded wthout regard to the test-taker s ethncty or educatonal ackground and are outsde of law schools drect control. 6 Other outcome measures are suject to crtcsm: Colleges may lower ther gradng and graduaton standards to mask racal achevement gaps (Mansfeld 2001); salares of recent graduates may not correlate wth the desralty of ther jos as some prestgous jos (e.g. judcal clerkshps) have low current salares ut promse hgher long-run ncome; and drect measures of jo qualty may e affected y affrmatve acton n hrng. Though we also present results on graduaton employment and salares we focus on ar exam passage as our prmary outcome measure. 5 On the other hand there s a relatvely strct herarchy of law schools and some employers may consder jo canddates only from hghly-ranked schools; ths could create a selectve-school premum that s unrelated to achevement. 6 The exam vares across states ut natonally only 83% of students pass the exam on the frst try (authors calculatons from Natonal Conference of Bar Examners 1995). 5

7 A second analytcal advantage s that law school admssons decsons depend strongly on applcants numercal qualfcatons. Both of our emprcal strateges requre assumptons that a treatment and a comparson group would have acheved the same outcomes condtonal on oservales had ther treatment status een the same. Ths s mplausle f treatment depends to a great extent on unoserved characterstcs applcaton essays letters of recommendaton extracurrcular actvtes that may e correlated wth potental outcomes. Whle these factors nfluence law school admssons they receve sustantally less weght relatve to numercal qualfcatons than n undergraduate admssons. LSAC (1992) reports counts of applcatons and admssons for the cycle y LSAT and GPA range from each law school that provded data. Usng data from 17 schools ranked n the U.S. News and World Report (2005) top 25 we estmated a grouped prot model for admssons decsons as a functon of the cell-mean LSAT score and undergraduate GPA. Margnal effects from ths model are reported n Column 1 of Appendx Tale A. A sngle extra LSAT pont or a 0.07 ncrease n undergraduate GPA measured on the tradtonal 4- pont scale rases admssons proaltes for average applcants y aout 1.6 percentage ponts. Estmates for undergraduate admssons at the most selectve quntle of colleges reported y Kane (1998 Tale 12-2) and reproduced as Column 2 of Appendx Tale A ndcate much smaller effects: A SAT ncrease equvalent (n standard devaton terms) to 1 LSAT pont mproves admssons proaltes y only 0.9 percentage ponts whle a

8 pont ncrease n GPA s worth only 0.4 percentage ponts. 7 Admssons rates dffer dramatcally etween the samples however confoundng comparsons of margnal effects at the mean. The ottom rows show the margnal effects for students wth admssons proaltes of 0.5. Test scores and grades are 3.2 and 7.6 tmes as mportant respectvely to law school admssons as to undergraduate admssons. The potental endogenety of school selectvty to unoserved alty s thus a relatvely mnor concern n law schools. III. Defnng Msmatch The msmatch hypothess s that some students would otan etter outcomes hgher grades hgher graduaton and ar passage rates and etter jos at hgher salares f they attended less selectve schools than f they attended more selectve schools and that the mnorty students affected y affrmatve acton preferences fall nto ths group. Elmnaton of affrmatve acton would thus mprove outcomes of the students who would e dsplaced nto less selectve schools. Two closely related clams are not n our understandng what s commonly meant y msmatch. Frst t mght e that the most selectve law schools have negatve treatment effects on all students. In ths case the deal polcy mght elmnate the selectve law schools or alter ther currcula to more closely resemle those seen at less compettve schools. We fnd the unversal msmatch hypothess mplausle. Consderaton of ths hypothess however suggests that an approprate analytcal framework for msmatch must allow for heterogeneous treatment effects of attendng selectve law schools. 7 Kane s model ncludes controls for race famly ncome and parental educaton whch ours does not. To the extent that colleges gve preferences for demographc characterstcs that are negatvely correlated wth SATs and GPAs excluson of these controls would lkely reduce Kane s estmated SAT and GPA effects. 7

9 A second hypothess contends that some applcants receve negatve treatment effects from attendng law school rrespectve of the school they attend and would e etter off pursung other careers. Only a t over half of law school entrants from the ottom decle of the admssons qualfcatons dstruton pass the ar exam wthn 5.5 years of enterng law school and t s plausle that students n ths range should not go to law school. 8 To the extent that affrmatve acton s responsle for admssons of applcants wth extremely poor prospects for ar passage then elmnaton of preferences mght eneft these applcants. Evaluaton of ths clam requres data on students alternatves f they do not attend law school as an applcant may prefer even a low proalty of ecomng a lawyer to her other optons. We focus on msmatch among law schools whch can plausly e oserved n ar passage rates and other ndcators of law school success rather than on msmatch etween law school and other career choces. It s the effect of attendng a hghly selectve school relatve to a less selectve school that s at the center of most dscussons of msmatch. It s mportant also to note that naïve comparsons that do not take careful account of dfferences n credentals among students may e extremely msleadng. Students attendng less selectve law schools acheve much worse outcomes on average than do those attendng more selectve schools and lack students outcomes are generally nferor to those of whtes. These gaps cannot easly e attruted to msmatch effects however as each of these comparsons s etween groups that dffer sustantally n ther enterng credentals (Kane 1998). An mportant advantage of our analytcal approach s that we do 8 It s also plausle that students who never practce law nevertheless eneft from attendng law school. When we analyze employment outcomes elow we nclude students who faled or dd not take the ar exam so long as they graduated from law school. 8

10 not rely on assumptons aout the partcular parametrc form of the relatonshp etween enterng credentals and student outcomes. Fnally there s reason to thnk that selectvty effects may e more negatve for lack students. Whle a whte student wth anomalously low credentals for her school may e ale to avod notce the vslty of race may make ths mpossle for lack students. Oservers wth ncomplete nformaton aout partcular students alty may form lower estmates for memers of groups that eneft from admssons preferences (Murray 1994; Steele 1990; Sowell 2004). 9 The resultng low expectatons may e self-fulfllng for lack students as socalled stereotype threat has een clamed to worsen lack performance n contexts where racal performance dfferences are salent (Steele and Aronson 1998). The Socratc method of nstructon wdely used n law schools may further aggravate these effects y focusng classroom attenton on lack students as representatves of ther race (Guner et al. 1994). The relevant selectvty effects for the msmatch hypothess are those on lack students. Our frst comparson etween more- and less-selectve schools provdes separate estmates of average selectvty effects on lacks and on whtes. Our second comparson etween lack and whte students dentfes average selectvty effects on lacks. Notaton We adopt a potental outcomes framework. For smplcty suppose that law schools come n only two types more- (s=1) and less-selectve (s=0). Let p s e the outcome that student would otan f he or she attended a school of type s. The treatment effect of a more-selectve school on student s τ p 1 -p 0. Let e a race ndcator (wth =1 9 Justce Thomas wrtes n hs Grutter dssent (539 U.S ) that The majorty of lacks are admtted to the [Unversty of Mchgan] Law School ecause of dscrmnaton and ecause of ths polcy all are tarred as undeservng. Ths prolem of stgma does not depend on determnacy as to whether those stgmatzed are actually the enefcares of racal dscrmnaton. 9

11 ndcatng a lack student) and let e a vector of oservale admssons qualfcatons. The average treatment effect of selectvty on ( ) students s M( ) E[τ = =]. In ts weakest form the msmatch hypothess states that some students are hurt y selectvty: Msmatch verson 1: For some τ < 0. Ths s untestale wthout further structure as for any ndvdual student only p = s p 1 + (1- s )p 0 can ever e oserved. A slghtly stronger verson states that some ex ante dentfale categores of students have negatve average treatment effects of selectvty: Msmatch verson 2: For some ( ) M( ) < 0. A fnal verson of the hypothess states that the average lack student who s treated y affrmatve acton polces s msmatched. Ths s the clam that seems to motvate most dscussons of msmatch. 10 Let s noaa {0 1} e the selectvty of the law school that the student would attend n the asence of racal preferences. Msmatch verson 3: E[ τ =1 s > s noaa ]< 0. The effect of affrmatve acton on mean outcomes of lack students wth noaa characterstcs s E[ ( s s ) τ = ] 1. Ths can e wrtten as the product of = admssons preferences and the msmatch effect plus a term reflectng the selectvty of affrmatve acton enefcares wth respect to ther treatment effects (1) E noaa [( s s ) τ = 1 = ] noaa = δ( 1 ) M( 1 ) + cov ( s s ) ( τ = 1 = ) 10 An example s Sander s (2004) clam that affrmatve acton reduces the numer of lack lawyers. Sander s concluson also requres that any lack students who would not attend law school n the asence of affrmatve acton pass the ar exam at suffcently low rates that they are outweghed y those who fal the exam due to affrmatve acton-nduced msmatch. 10

12 where δ( )=E[s -s noaa = =] s the effect of affrmatve acton on ( ) students dstruton across schools. IV. Oservale Implcatons and Methods To evaluate the ntermedate verson of the msmatch hypothess we requre estmates of M( ) throughout the ( ) dstruton. For the strong verson we also requre estmates of δ( ) and an assumpton aout the covarance term n (1). We assume unconfoundedness (Rosenaum and Run 1983; Run 1978): Admssons offces at selectve schools whch presumaly seek to dentfy students wth ether a large p 1 or a large τ oserve no more aout students qualfcatons than do we and student noaa matrculaton decsons are smlarly gnorale. Thus (p 0 p 1 ) (s s ). Ths elmnates the second term of (1): Average treatment effects are the same for students who attend selectve schools as for oservatonally dentcal students who attend unselectve schools. We approxmate a race-lnd admssons rule y that oserved among whte students and we further assume that lack applcaton and matrculaton decsons would mrror those of whte students wth smlar qualfcatons n the asence of affrmatve acton (Krueger Rothsten and Turner 2005): δ(0 )=0 and δ(1 )=E[s =1 =] - E[s =0 =]. 11 It s helpful to decompose M( ) nto components dervng from the susets of ( ) students attendng selectve and unselectve law schools: 11 In practce a race-lnd admssons rule wll lkely produce E[s noaa =] ntermedate etween E[s =1 =] and E[s =0 =] so our expresson for δ(1 ) overstates the mpact on lack enrollment rates of movng to race-lnd admssons. However lacks make up only 8% of whtes and lacks n law school so the approxmaton δ(0 )=0 s lkely reasonaly accurate and the overstatement small. 11

13 12 (2) ( ) [ ] ( ) ( ) ( ) [ ] ( ) ( ) ( ) ( ) p p E s p p E s M c c + = where ( ) [ ] s s p E s p s = = = = s the average outcome among ( ) students attendng type-s schools and ( ) [ ] s s p E s p s c = = = = 1 s the counterfactual average outcome had these students attended schools of the other type. Our two strateges nvolve dfferent assumptons aout ths counterfactual. A. Wthn-race comparsons etween more- and less-selectve schools Our frst strategy s smple: We use students oserved attendng less selectve schools as the counterfactual for students of the same race and oserved qualfcatons at more selectve schools. The dfference etween the two types of schools n oserved outcomes of students wth the same ( ) s (3) ( ) [ ] [ ] [ ] [ ] [ ] [ ] [ ] ( ) ( ) ( ) ( ) ( ). p p M s p E s p E s E τ s p E s p E s p E s p E D c = = = + = = = = = = = = The unconfoundedness assumpton s that [ ] [ ] p E s p E j j = for j= Wth t ( ) ( ) s p s p c 1 = for each (s ) and ( ) ( ) M D =. Ths assumpton s unlkely to hold exactly: Law school admssons offces can oserve student qualfcatons that are not reported n our data sets ut that may e correlated wth potental outcomes. Ths would mply ( ) ( ) p p c 0 1 > and D( ) > M( ). Gven the evdence aove that law school admssons proaltes depend 12 Note that ths rules out the Roy model of self selecton where s s ncreasng n τ. To the extent that students ratonally choose from among ther avalale optons to maxmze ther outcomes we wll not fnd evdence of msmatch. Of course n ths case affrmatve acton merely enlarges lack students choce sets so cannot hurt them.

14 heavly on varales ncluded n we expect that the dfference s relatvely small. If so estmates of the msmatch effect otaned from ths comparson wll provde a reasonaly tght upper ound for the true effect. B. Comparsons etween whtes and lacks Our second strategy compares the outcomes of lack students to those of whte students wth smlar admssons qualfcatons. The average outcome among ( ) students can e wrtten as (4) E[p = =] = E[p 0 + s τ = =] 0 p + s * M = ( ) ( ) ( ) C ( ) + where Z ( ) ndcates E[Z = =] and C ()=cov[s (p 1 -p 0 ) = =]. The whte-lack gap n mean outcomes among students wth qualfcatons s then (5) p 0 = = ( ) p1( ) = 0 0 ( p0 ( ) p1 ( )) + ( s 0 ( )* M( 0 ) s1( )* M( 1 )) + ( C 0 ( ) C 1( )) 0 Δp ( ) + Δs ( )* M( 1 ) + s ( )( M( 0 ) M( 1 )) + ΔC ( ) 0 where Δf() ndcates f 0 ()-f 1 (). The frst term n the fnal lne of (5) s the whte-lack dfference n outcomes f everyone attended unselectve schools. The thrd term s the dfference etween the selectve school effect on whte students and that on lack students multpled y the whte enrollment rate at selectve schools. The fnal term s the whte-lack dfference n the covarance etween selectvty and the selectve school treatment effect; t s postve f selectve schools do a etter jo of dentfyng and admttng whte students wth postve treatment effects than they do of dentfyng postve-τ lack students. We assume that each of these s zero: 0 (6) p ( ) = M( 0 ) M( 1 ) = ΔC ( ) = 0 Δ. The whte-lack outcome dfference s then 13

15 (7) p ( ) = Δs ( ) * M( 1 ) Δ the average selectvty effect on lack students multpled y the dfference etween lack and whte enrollment rates at selectve schools. If lack students wth characterstcs would acheve worse outcomes at selectve than unselectve schools M(1 ) < 0 and Δp()<0. Note that the dfference n enrollment rates Δ s ( ) s the negatve of the estmator of the affrmatve acton preference δ() proposed earler. The key assumpton mplct n our approach s that lack students outcomes would mrror those of whte students wth the same admssons credentals f the two groups were smlarly dstruted across schools. Ths assumpton s almost certanly volated n ways that lead us to overstate the degree of msmatch. At the undergraduate level a long lterature estalshes that lack students underperform whte students wth the same enterng credentals even when they attend the same colleges (see e.g. Rothsten 2004; Young 2001). A smlar pattern appears to hold n law school (Wghtman 2000; Wghtman and Muller 1990; Anthony and Lu 2003; Powers 1977). It thus seems lkely that Δp 0 () < 0. The sgn of ΔC() s less ovous ut gven a reduced oservales threshold for lack applcants admssons offces may exert more effort to dentfy unoserved correlates of future success for lack than for whte applcants whch could produce ΔC()<0. Fnally lack underperformance suggests that a dffcult law school currculum may overmatch lack students more than t does whtes wth smlar oservale characterstcs producng M(0 ) - M(1 ) > 0 and s ( )( M( 0 ) M( 1 )) 0. We thus vew p( ) 0 < for the true msmatch effect M ( ) Δs( ) 1. Δ as a lower ound There s another mportant source of as n our estmates of msmatch at the lowest values. We compare lack and whte students who are oserved attendng law schools; 14

16 potental students who were at rsk for attendng law school ut dd not actually attend are not n our sample. Below we show that low- applcants are qute lkely to receve zero admssons offers partcularly when they are whte. There s lkely selecton on unoservales wthn race: Those low- students who are admtted to some law school lkely have etter unoserved qualfcatons than are those who are rejected everywhere. If 0 so our sample estmates of p ( ) are upward-ased relatve to what would e seen f outcome measures were avalale for all race- students wth qualfcatons. The lack-whte dfference n sample selecton s a drect consequence of the use of affrmatve acton at the margn of admsson to the least-selectve law schools. Because lack selecton rates are so much hgher than those of whtes p 0 1 ( ) s less ased than s p 0 0 ( ). Our estmates of p( ) Δ are therefore upward-ased (assumng that selecton s postve) partcularly at low values where the dfferental selecton s most extreme. We do not attempt to correct our estmates for ths. Instead we present estmates of msmatch oth ncludng and excludng the lowest values. When we exclude the lowest- porton of our sample we otan estmates of the effect of affrmatve acton on the most qualfed lack students. C. Methods All of our analyses allow for an artrary nonlnear relatonshp etween an ndex of enterng credentals (LSAT scores and undergraduate GPAs) and potental outcomes. We accomplsh ths va a form of matchng re-weghtng data from a comparson group so that the ndex dstruton wthn that group matches that n the group of nterest (DNardo Fortn and Lemeux 1996; Hahn 1998; Hrano Imens and Rdder 2003). 15

17 Our frst strategy s ased on the dfference etween mean outcomes of s=1 and s=0 students of race and qualfcatons (8) D ( ) p( 1 ) p( 0 ) We estmate the p ( s ) =. functons as kernel means usng an Epanechnkov kernel. We use ootstrap resamplng for nference re-estmatng the kernel means n each ootstrap sample and reportng pontwse 90% confdence ntervals for D( ). D To otan the average treatment effect on race- students we estmate () D( ) f ( ) d (where f () s the densty of for race- students) as a dfference n re-weghted means: (9) Dˆ () = s = p ( f ( ) f ( )) f ( ) f ( ) 1 1 s = 1 1 s = p ( f ( ) f ( )) f ( ) f ( ) 0 0 s = 0 0 f s () s the densty of among race- students at type-s schools. We use a kernel densty estmator for oth f () and f s () agan usng an Epanechnkov kernel. There s samplng error n our densty estmates that carres over nto the reweghtng factors. We compute standard errors for Dˆ () y ootstrappng the entre process (ncludng the densty estmaton). Our approach amounts to a decomposton of the selectve/unselectve gap n mean outcomes nto a component due to dfferences n dstrutons and a remander that we treat as an estmate of the average selectvty effect for race- students (see e.g. DNardo Fortn and Lemeux 1996 and Frpo Fortn and Lemeux 2005) Ths s equvalent to a propensty score (Rosenaum and Run 1983; Deheja and Waha 2002) estmator of the average treatment effect for race- students n whch the propensty score s a nonparametrc functon of. The propensty score can e wrtten as N 1 f 1 ()/N f () where N s s the total numer of race- students at type-s schools and N =N 0 +N 1. 16

18 Our second strategy uses whte students as the counterfactual for lack students wth the same. We present kernel mean estmates of p 0 ( ) and ( ) estmates Δ p( ) p 1 ; the dfference the treatment effect of eng lack for students wth credental. We estmate the average treatment effect on the treated Δ ( ) f ( ) = Δp 1 d as another dfference n reweghted means ths tme etween the actual lack mean outcome and a reweghted whte mean that makes the whte dstruton dentcal to that of lacks: (10) ( f 1( ) f ( )) 1 f ( ) f ( ) N 1 p = 0 0 = p Δˆ =. s = An mportant advantage of ths approach s that we do not requre a perfect measure of the admssons preferences enjoyed y lack students; as we dscuss elow we have only lmted nformaton aout the selectvty of the schools that students n our data attend. V. Data Our data come from the Law School Admssons Councl s (LSAC) Bar Passage Study (BPS; Wghtman ) whch attempted to survey all students enterng accredted law schools n the fall of Survey responses were matched wth admnstratve records contanng students undergraduate grade pont averages (UGPAs) and Law School Admssons Test (LSAT) scores and wth nformaton from law schools and state ar assocatons aout law school and ar passage outcomes through July The BPS contans nformaton on over students aout 62% of 1991 enterng students. 14 We focus on lack and whte students yeldng an analyss sample of of whch 7.6% are lack. A suset of BPS respondents was chosen for follow-up surveys the 14 Most non-response was ndvdual: 163 of 172 accredted law schools partcpated n the study (Wghtman 1999). 17

19 last of whch took place four to sx months after students scheduled graduaton. 15 Our sample for analyses of employment outcomes s The BPS data are exceedngly rch provdng a reasonale approxmaton to a full cohort of law school students. Our analyss s lmted however y a few decsons made y LSAC n ts aggressve protecton of respondents prvacy. Most mportantly the BPS data group law schools nto sx clusters and report only the cluster that each student attended. Clusters are ntended to group smlar schools along dmensons lke sze cost selectvty tuton level and mnorty representaton. In practce t s dffcult to rank the clusters y selectvty. We characterze two clusters (laeled Elte and Pulc Ivy n the BPS data though nether lael s qute accurate; see Wghtman 1993) as more selectve and the remanng clusters as less selectve acknowledgng the sustantal measurement error n ths classfcaton (Sander 2005). A second data lmtaton relates to measurement of grade pont averages (GPAs). Law school admssons frequently use adjusted undergraduate GPAs that account for dfferences n gradng standards etween colleges. The BPS however reports unadjusted GPAs and contans no nformaton aout the undergraduate nsttuton. As a result our undergraduate GPA varale s mperfectly related to that actually used n admssons. Oddly the BPS law school GPA varales are lmted n the opposte way: Only GPAs standardzed wthn law schools are reported. These varales are est seen as measures of class rank rather than of asolute performance. It would not e surprsng to fnd large 15 Samplng proaltes for the follow-up surveys depend on race school cluster (see elow) and school racal composton. Unfortunately the BPS data contan no nformaton aout the last so do not permt constructon of nverse proalty weghts. The codeook recommends weghtng y race whch would not yeld a representatve sample. We weght y the nteracton of race and school cluster to match the full-sample dstruton. Wth these weghts the susample s representatve of lack students ut overrepresents whte students from schools wth a hgh fracton lack wthn each cluster. 18

20 msmatch effects on class rank as these are a nearly mechancal consequence of the effect of students own credentals on achevement. Fnally the ar exam measures are lmted n two ways. Frst states vary sustantally n the dffculty of ther ar exams ut the BPS does not report whch state s exam a student took much less whch the student consdered. Second n 14 states successful attempts at the exam are oserved ut faled attempts are not asng oserved success rates. To avod ths as we focus on whether a student passed the ar n any state durng the 2.5 years for whch data were collected rrespectve of whether where or how many tmes the exam was attempted. Falures on ths outcome thus nclude students who chose careers that dd not requre admsson to the ar as well as those who took the exam and faled. 16 Summary statstcs for lack and whte students n the BPS data are reported n Tale 1. The frst several rows descre admssons credentals. LSAT scores ranged from 10 to 48 n 1991 wth a mean of 33.3 and a standard devaton of aout 7 among applcants. Matrculants have a hgher mean (36.8) and a smaller standard devaton (5.6). The lackwhte LSAT gap s over 1.5 standard devatons and the UGPA gap s aout one S.D. 17 We comne these two varales nto a sngle ndex (8) UGPA LSAT 10 Index = 400 * * Aout 16% of study partcpants (7% of those oserved to graduate from law school) were not matched to any ar exam data ndcatng ether that they dd not take the exam at all or that they faled the exam n a state that dd not report faled attempts. 17 Among all 1991 law school applcants the LSAT gap s 1.33 standard devatons (Wghtman 1997 Tale 3) though the standard devaton s larger for ths group; the asolute LSAT gap s slghtly gger among applcants than among BPS respondents. 18 Ths ndex s taken from Sander (2004) who descres t as a close approxmaton to the weghts used n law school admssons. A prot model for attendance at the most selectve cluster of law schools yelds nearly dentcal weghts as does a model predctng law school GPAs wthn clusters. None of our results are senstve to the susttuton of ndces ased on weghts from ether of these models. 19

21 The lack-whte gap n ths ndex s nearly 1.7 standard devatons. We present most of our results n terms of percentle scores computed from the ndex usng ts dstruton wthn the BPS sample. The average percentle score for lack BPS respondents s 15; for whtes t s percent of lacks ut only 13 percent of whtes score n the ottom quntle. Fgure 1A dsplays the densty of admssons ndex scores among lack and among whte students n the BPS whle Fgure 1B dsplays cumulatve dstruton functons of percentle scores for each group. The second group of rows n Tale 1 reports the dstruton of students across the sx clusters of law schools. We order clusters y ther ndex score rankng; the clusters have mean ndces of and 578. Blacks and whtes are more smlarly dstruted across clusters than one mght expect gven the large gaps n enterng credentals a reflecton of affrmatve acton n admssons. Fgure 2 dsplays the fractons of whte and lack students at each ndex percentle who attend schools n clusters 1 and 2 ( elte and pulc vy respectvely n the BPS documentaton). Although ths fgure ncorporates the effects of any dfferences n applcaton ehavor unoserved admssons credentals or matrculaton decsons t s clear that admssons preferences must e qute large: Prot models for enrollment at schools n clusters 1 and 2 ndcate that lack students have enrollment proaltes resemlng those of whte students wth ndex scores 165 ponts (1.6 standard devatons) hgher. The fnal rows of Tale 1 present statstcs for our outcome measures. The frst row shows the GPA durng the frst year of law school whch as mentoned aove s est thought of as a long-taled measure of class rank. 19 The next two rows show law school 19 Analyses of the cumulatve law school GPA avalale for students who completed law school and standardzed n the same way yeld smlar results. 20

22 graduaton and eventual ar passage. 20 Fnally we consder several employment-related outcomes measured from the post-graduaton survey. 21 The frst s a measure of whether the respondent s employed full tme. Second for those who are employed we form an ndcator for whether the current work settng s a judcal clerkshp (14.0% of respondents) an academc jo (0.8%) a prosecutor s offce (4.4%) a pulc defender s offce (2.3%) or a large law frm (20.0%) each of whch we take to e a postve employment outcome for lawyers. 22 Fnally the BPS reports the annual salary whch we report n oth levels and logs. 23 Black students GPAs graduaton rates and ar passage rates are sgnfcantly worse than those of whte students. Dfferences n employment measures are much smaller and do not take a consstent sgn. A. Selecton nto law school An ssue that wll ecome mportant elow s the posslty of selectvty n whch law school applcants ultmately matrculate. Fgure 3 shows the fracton of whte and lack applcants for the 1991 enterng cohort who appear n the BPS sample. Applcants wth low ndex scores are notaly underrepresented partcularly among whtes. These dfferences n representaton appear largely to reflect admssons decsons. Unlke colleges even the least selectve law schools have compettve admssons and only 20 Students who stopped out of the survey are counted as havng mssng graduaton status. They (along wth dropouts) are ncluded n the ar passage varale. 21 We nclude only students who graduated from law school n our employment analyses as response rates to the follow-up survey were extremely low for non-graduates. Among graduates the response rate was 74% overall ut only 65% for lacks a dfference that does not appear to e attrutale to dfferences n enterng credentals. Cauton s therefore requred n nterpretaton of our analyses of employment outcomes. 22 Ths varale s set to zero for lawyers n md-sze or smaller frms (29.7%) solo practce (3.1%) government agences and legslatve offces (7.5%) pulc nterest (2.5%) usness/fnance (4.0%) other law related (3.8%) and non-law-related work other than academcs (7.9%). 23 Salares are reported n 8 ns (less than $20000 $20-$30000 $70-$80000 and greater than $80000). We assgn each respondent to the log of the mdpont of the relevant n usng $15000 for the lowest n and $ for the hghest; respondents who are not employed full-tme are excluded from the salary analyses though ths has lttle mpact on the results. 21

23 56% of the applcants from the BPS cohort were admtted to any law school (Barnes and Carr 1992; see also Wghtman 1997). Fgure 4 shows the fracton of applcants who were admtted to at least one school y enterng credentals and race. 24 Whte students whose credentals would have placed them n the ottom quarter of the BPS dstruton were more lkely to e rejected than to e accepted. Black admsson rates were well aove one half everywhere aove the ffth percentle and are doule or more those of smlarlyqualfed whtes through much of the lower part of the dstruton. These dfferental admsson rates y race suggest that etween-race comparsons may e suject to sample selecton as. It s somewhat plausle that mean latent outcomes are smlar for whte and lack applcants wth dentcal ndex scores. But admssons offces have more nformaton than just the LSAT score and UGPA and students who were admtted lkely had etter essays recommendaton letters or other qualtatve qualfcatons than dd oservaly (to us) dentcal students who were rejected from all the schools to whch they appled. If admssons offces dstngush among students on the ass of characterstcs that are correlated wth potental outcomes and f these offces set a lower threshold for lack than for whte applcants then comparsons usng the BPS data are ased n favor of whtes. On the ass of Fgure 3 t seems reasonale to assume that selecton as s not a major prolem for lack-whte comparsons at ndex values aove aout the 30 th percentle. Restrctng our analyss n ths way would come at a sustantal cost as t would exclude 85% of lack students n the BPS sample. We nstead use the 20 th percentle as a compromse 24 The data for ths fgure (and the denomnator for the prevous fgure) are from Barnes and Carr s (1992) report of the numer of applcants n each of 90 LSAT-undergraduate GPA cells separately y race and the numer admtted to at least one school. We assgn to each cell the mean ndex percentle of BPS respondents n that cell. 22

24 threshold that rsks lmted selecton as n order to enlarge the sample sze and we report estmates for oth the full sample and the top four quntles. B. Smple regresson estmates Tale 2 presents regresson versons of our two comparsons frst etween students attendng more- and less-selectve schools (columns 1 and 2) and second etween lacks and whtes (columns 3 and 4). Each specfcaton controls for quadratcs n the LSAT and the undergraduate GPA and for an nteracton etween the two. The frst par of columns ndcates that attendng a more selectve school proxed throughout y the elte and pulc vy clusters has a postve effect on all outcomes ut law school grades and employment though ths s generally nsgnfcant and the pont estmate for ar passage s very slghtly negatve n the smaller lack sample. Column 3 ndcates that lack students earn much lower GPAs than do whtes wth smlar qualfcatons. Blacks also graduate and pass the ar at lower rates ut acheve etter employment outcomes. Column 4 restrcts the sample to students wth admssons qualfcatons aove the ottom quntle. The negatve lack effect on graduaton and ar passage s asent here ndcatng that lack underperformance s concentrated at the ottom of the qualfcatons dstruton. These models nclude relatvely sparse parameterzatons of the relatonshp etween students enterng qualfcatons and ther eventual outcomes. If they are ms-specfed the estmated effects are ased n unknown drectons. The next two sectons present reweghtng analyses that are roust to artrarly nonlnear relatonshps etween the ndex and outcomes. In Secton VI we explore the selectve-unselectve comparson and n Secton VII we turn to the lack-whte comparson. 23

25 VI. Results: Wthn-race comparsons across clusters Fgure 5A dsplays the fracton of students passng the ar y race ndex percentle and school selectvty and Fgure 5B dsplays the selectve-unselectve gap y race and percentle. 25 Ths gap s generally close to zero and s postve at many ponts. Integratng over the wthn-race dstrutons we estmate that the lack ar passage rate would e 3.9 percentage ponts lower f all lacks attended selectve schools than f all attended unselectve schools though ths estmate s nsgnfcantly dfferent from zero. For whtes the correspondng fgure s 1.1 percentage ponts hgher also nsgnfcant. The thrd set of rows n Tale 3 presents ths result n taular form. Wthout accountng for dfferences n ndex dstrutons lacks attendng more selectve schools pass the ar at a rate 15.6 percentage ponts hgher than do lacks attendng less selectve schools. When the two susamples are reweghted to have the same ndex dstruton as the overall lack sample however we otan the -3.9 fgure cted aove. Thus more than 100% of the raw selectve-unselectve gap n lack ar passage rates can e attruted to dfferences n enterng credentals. 26 The other rows of Tale 3 present mean treatment effects on our remanng outcomes efore and after reweghtng. Unsurprsngly selectve schools have large negatve effects on law school GPAs (class ranks) for oth races ut these do not carry over 25 Graphs are trmmed at the 3 rd and 97 th percentle of the wthn-race or pooled dstrutons whchever s more restrctve. Thus for lacks the graphs are shown for percentle scores etween 3 and 69; for whtes etween 8 and The choce of a ase dstruton for reweghtng s analogous to the choce of a ase group n a Oaxaca-Blnder decomposton (Oaxaca and Ransom 1999). Our use of the overall ndex dstruton s akn to use of the grand mean of as the ase for a Oaxaca-Blnder decomposton and provdes us wth estmates of the average treatment effect on lack students. 24

26 consstently to the more cardnal outcomes. 27 The estmated selectvty effect s postve and sgnfcant for whtes for three of the other fve outcomes postve and nsgnfcant for one and sgnfcantly negatve for full-tme employment. For lacks only the postve estmate for salary s sgnfcant and the other pont estmates show no clear pattern. Another way to express these results s as estmates of the mpact of affrmatve acton on lack students average outcomes. Fgure 2 shows lack and whte students proaltes of attendng selectve law schools at each admssons ndex percentle. Elmnaton of affrmatve acton can e approxmated as forcng lack students to the oserved whte profle. Tale 4 reports the estmated effect of ths on lack mean outcomes under the contnued assumpton that selectvty s uncorrelated wth latent outcomes condtonal on race and the ndex. Where Tale 3 reported average treatment effects for lacks ( reweghted estmates n Column 3) the estmates n Tale 4 can e seen as local average treatment effects for affrmatve acton complers (Angrst and Imens 1994). Elmnaton of affrmatve acton would worsen lack outcomes along every dmenson ut LGPA sgnfcantly so for graduaton rates and salares. VII. Results: Comparsons etween whtes and lacks Due to affrmatve acton lacks have the opportunty to attend more selectve law schools than do whtes wth smlar enterng credentals even f we cannot always oserve selectvty. Our second strategy takes advantage of ths comparng average outcomes of whtes and lacks rrespectve of cluster wth the same enterng credentals. 27 A gven ndvdual s rank wthn her school s dstruton s y defnton decreasng n the school s selectvty. Thus f there s any relatonshp etween students ncomng and outgong ranks we wll see negatve selectve school effects on outgong rank. For comparson when we repeat the analyss from Tale 3 for an ncomng rank varale computed y normalzng the varale wthn cluster we estmate selectvty effects of for lacks and for whtes. 25

27 Fgure 6A shows average ar passage rates y race and ndex percentle n the BPS data and Fgure 6B shows the whte-lack gap at each percentle n the common support. In the upper four ffths of the dstruton lack students generally perform aout as well as whte students wth smlar credentals and sometmes outperform them ut n the lower quntle where passage rates are lower for oth races lacks sustantally underperform whtes. Aout three quarters of lack law school students are n the ottom quntle so when we average the gaps n Fgure 6B over the lack dstruton the lack effect on ar passage rates s negatve and sgnfcant. Ths s presented n the left-hand columns of Tale 5 n the thrd set of rows for ar passage: A t over half of the raw lack-whte gaps n ar passage and law school graduaton are attrutale to dfferences n enterng credentals. A smaller porton of the GPA gap s attrutale to oservales consstent wth a larger msmatch effect on class rank than on other outcomes. Reweghted lack-whte dfferences n full-tme employment and salares are small and nsgnfcant ut lacks are much more lkely to otan good jos than are whtes wth smlar credentals. 28 As noted earler whte-lack comparsons at the ottom of the ndex dstruton may e ased y endogenous matrculaton. Columns 4-6 of Tale 5 present whte-lack gaps among students n the upper four-ffths of the dstruton. Only a quarter of lack students fall nto ths category ut estmates are nevertheless reasonaly precse. On average lack students n ths sugroup pass the ar at rates only 1.7 percentage ponts (s.e. 2.0) lower than do smlarly-qualfed whte students. Employment outcomes are unformly etter for lacks than for whtes even n the raw data ut partcularly after reweghtng. The 28 The good jos effect s drven y the prosecutor and large frm categores: Blacks are much more lkely than whtes wth the same credentals to e n these categores. Blacks are much less lkely to work at small frms or n solo practce and somewhat more lkely to work for government agences. 26

28 only ndcaton of msmatch s n GPAs where lack underperformance s worse than t was n the full sample. Ths suggests that the sample restrcton has not elmnated lack-whte gaps n admssons prospects so any msmatch effects should e apparent even n the susample. The lack-whte comparson whch s lkely ased toward overstatng msmatch thus ndcates zero or a small negatve effect on the ar passage rates of lack students n the top four quntles. In the ottom quntle the estmated negatve effects are sustantally larger consstent ether wth large msmatch effects or wth selecton ases dervng from the excluson of many whte applcants not admtted to any law school. The lack effect on employment outcomes s postve n the top four quntles and zero or postve n the ottom quntle despte any selecton as. To convert our estmates of p( ) Δ nto estmates of the msmatch functon M(1 ) we dvde y the affrmatve acton effect on selectvty. We explore two measures of selectvty. Frst we treat our dchotomous measure clearly a poor proxy as true selectvty. Fgure 2 showed the lack-whte gap n enrollment at schools n the two most selectve clusters whch we take as an estmate of - Δ s ( ). Dvdng the gap n ar passage rates from Fgure 6B y (the negatve of) ths affrmatve acton effect on admsson yelds an estmate of the msmatch functon M(1 ). Ths s graphed as the sold lne n Fgure 7. As mght e expected from the prevous results ths msmatch functon s large and negatve n the ottom quartle of the qualfcatons dstruton ut s close to zero elsewhere. The pont estmates of M(1 ) are clearly ncorrect as they ndcate that selectve schools depress graduaton rates of the least qualfed students y much more than 100 percentage ponts. Ths reflects lmtatons of our proxy for admssons preferences s ( ) whch s partcularly poor at the ottom of the dstruton where preferences at less 27 Δ