GAO POSTSECONDARY EDUCATION. Student Outcomes Vary at For-Profit, Nonprofit, and Public Schools. Report to Congressional Requesters

Transcription

1 GAO United Sttes Government Aountbility Offie Report to Congressionl Requesters Deember 2011 POSTSECONDARY EDUCATION Outomes Vry t For-Profit, Nonprofit, nd Publi Shools GAO

2 Contents Letter 1 Limited Reserh Suggests tht For-Profit Shool s Generlly Hve Different Outomes thn Nonprofit or Publi Shool s For-Profit Shool Grdutes Generlly Hd Lower Pss Rtes thn Grdutes from Other Shools on Liensing Exms We Reviewed Ageny Comments Appendix I Sope nd Methodology 12 Appendix II List of Studies nd Ongoing Reserh Inluded in Our Literture Review 24 Appendix III Detiled Anlysis of Liensing Exm Pss Rtes 26 Appendix IV Briefing Slides 53 Appendix V GAO Contt nd Stff Aknowledgments 91 Tbles Tble 1: Studies Inluded in GAO s Report, By Outomes 24 Tble 2: RN (Bhelor s Degrees): Number of Progrms nd First Time Tkers, by Setor 26 Tble 3: RN (Bhelor s Degree): Pss Rte nd Men Progrm Pss Rte, by Setor 27 Tble 4: RN (Assoite s Degree): Number of Progrms nd First Time Tkers, by Setor 28 Tble 5: RN (Assoite s Degree): Pss Rte nd Men Progrm Pss Rte, by Setor 28 Tble 6: LPN: Number of Progrms nd First Time Tkers, by Setor 29 Pge i GAO Outomes

3 Tble 7: LPN: Pss Rte nd Men Progrm Pss Rte, by Setor 30 Tble 8: Rdiogrphy: Number of Progrms nd Tkers, by Setor 32 Tble 9: Rdiogrphy: Pss Rte nd Men Progrm Pss Rtes, by Setor 32 Tble 10: EMT: Number of Progrms nd Tkers, by Setor 34 Tble 11: EMT: Pss Rte nd Men Progrm Pss Rtes, by Setor 34 Tble 12: Prmedi: Number of Progrms nd Tkers, by Setor 36 Tble 13: Prmedi: Pss Rte nd Men Progrm Pss Rtes, by Setor 36 Tble 14: Surgil Tehnologist: Number of Progrms nd Tkers, by Setor 38 Tble 15: Surgil Tehnologist: Pss Rte nd Men Progrm Pss Rtes, by Setor 38 Tble 16: Mssge Therpist: Number of Progrms nd Tkers, by Setor 39 Tble 17: Mssge Therpist: Pss Rte nd Men Progrm Pss Rtes, by Setor 40 Tble 18: Lw: Number of Progrms nd Tkers, by Setor 43 Tble 19: Lw: Pss Rte nd Men Progrm Pss Rtes, by Setor 43 Tble 20: Cosmetology: Number of Progrms nd Tkers, by Setor 45 Tble 21: Cosmetology: Pss Rte nd Men Progrm Pss Rte, by Setor 46 Tble 22: Funerl Diretors: Number of Progrms nd Tkers, by Setor (All Progrms) 49 Tble 23: Funerl Diretors: Number of Progrms nd Tkers, by Setor (Assoite s Progrms Only) 50 Tble 24: Funerl Diretors: Pss Rte nd Men Progrm Pss Rtes, by Setor (All Progrms) 50 Tble 25: Funerl Diretors: Pss Rte nd Men Progrm Pss Rtes, by Setor (Assoite s Progrms Only) 51 Pge ii GAO Outomes

4 Figures Figure 1: Nursing: Number of Progrms by Pss Rte Rnge, Figure 2: Rdiogrphy: Number of Progrms by Pss Rte Rnge, Figure 3: EMT Bsi: Number of Progrms by Pss Rte Rnge, Figure 4: Prmedi: Number of Progrms by Pss Rte Rnge, Figure 5: Surgil Tehnologists: Number of Progrms by Pss Rte Rnge, 2010 Figure 6: Mssge Therpist: Number of Progrms by Pss Rte Rnge, Figure 7: Lw: Number of Progrms by Pss Rte Rnge, Figure 8: Cosmetology: Number of Progrms by Pss Rte Rnge, Figure 9: Number of Progrms by Pss Rte Rnge on Sienes Setion, (Assoite s Progrms Only) Pge iii GAO Outomes

5 Abbrevitions ABFSE BPS CIP Edution EMT ERIC IPEDS LPN NPSAS NSLDS NTIS OIG RN Amerin Bord of Funerl Servie Edution Beginning Postseondry s Longitudinl Study Clssifition of Instrutionl Progrms Deprtment of Edution Emergeny Medil Tehniin Edution Resoures Informtion Center Integrted Postseondry Edution Dt System Liensed Prtil Nurse Ntionl Postseondry Aid Survey Ntionl Lon Dt System Ntionl Tehnil Informtion Servie Offie of Inspetor Generl Registered Nurse This is work of the U.S. government nd is not subjet to opyright protetion in the United Sttes. The published produt my be reprodued nd distributed in its entirety without further permission from GAO. However, beuse this work my ontin opyrighted imges or other mteril, permission from the opyright holder my be neessry if you wish to reprodue this mteril seprtely. Pge iv GAO Outomes

6 United Sttes Government Aountbility Offie Wshington, DC Deember 7, 2011 Congressionl Requesters Institutions of higher edution, inluding for-profit, nonprofit, nd publi shools, reeive billions of dollrs eh yer from the Deprtment of Edution (Edution) to help students py for shool. 1 In the shool yer, Edution provided $132 billion in grnts nd lons to students under federl student id progrms, up from $49 billion in the shool yer. 2 However, reltively little informtion is vilble bout the qulity of edution being provided by these shools. 3 Mesuring the qulity of edutionl progrms (i.e., how muh knowledge or skill students gin) is diffiult. Beuse few diret mesures re vilble, indiret outome mesures, suh s grdution nd student lon defult rtes, re often used. Although no single outome n be used to fully mesure something s omplex s edutionl qulity, looking t multiple outome mesures (e.g., grdution rtes, pss rtes on liensing exms, employment outomes, nd student lon defult rtes) n shed light on the qulity of edution provided by shools. hrteristis re lso importnt to onsider when ompring edutionl outomes t shools in different setors (for-profit, nonprofit, 4 nd publi). Avilble dt indite tht for-profit shools enroll higher proportion of low-inome, minority, nd nontrditionl students who fe hllenges tht n ffet their edutionl outomes. s with these hrteristis tend to hve less positive edutionl outomes 1 For the purposes of this report, we refer to privte for-profit shools s for-profit shools nd privte nonprofit shools s nonprofit shools. 2 These progrms re uthorized by Title IV of the Higher Edution At of 1965, s mended. For the purposes of this report, we define federl student id progrms s finnil id progrms uthorized under Title IV of the Higher Edution At. Federl student id spending dt beginning in the shool yer re more relible thn dt from previous yers. 3 The federl government relies on rediting genies reognized by Edution to ensure edutionl qulity, but reditors ollet vrying types of dt on student outomes. Individul shools my lso ollet dt on vriety of student outomes. 4 The term student hrteristis refers to demogrphi hrteristis suh s gender, re, nd inome, s well s to other hrteristis, suh s prior edution nd delyed postseondry shool enrollment. Pge 1 GAO Outomes

7 thn other students for number of resons. For exmple, students who re low-inome, minority, or older generlly hve lower grdution rtes thn other students regrdless of setor. 5 Consequently, student outomes t different types of shools n be ssoited with differenes in student hrteristis, s well s shool type. Aounting for differenes in student hrteristis s muh s possible llows for more meningful omprisons between types of shools nd better understnding of the shool s role in produing student outomes. This n be done in different wys, suh s using sttistil models or ompring outomes for similr groups of students or grdutes. To respond to your interest in student outomes t different types of shools, this report ddresses the following questions. 1. Wht does reserh show bout grdution rtes, employment outomes, student lon debts, nd defult rtes for students t forprofit shools ompred to those t nonprofit nd publi shools, tking differenes in student hrteristis into ount? 2. How do pss rtes on liensing exms for seleted ouptions ompre mong grdutes of for-profit, nonprofit, nd publi shools? We used the following methodologies to develop our findings (see pp. I for detiled disussion of our sope nd methodologies). We begn by ontting representtives from severl higher edution ssoitions representing shools in ll three setors to obtin their perspetives on key issues disussed in this report. To identify omprtive reserh on outomes tht ontrols for student hrteristis, we onduted literture serh nd rigorously reviewed the dt nd methodologies used by externl reserhers nd only reported findings tht were bsed on sound methods nd relible dt. For most outomes we reviewed, we relied primrily on studies using dt from Edution s Beginning Postseondry s Longitudinl Study (BPS), whih trks ntionlly representtive smple of first-time students for 6 yers. BPS grdution rtes re 5 Reserh shows tht being ril or ethni minority my be ssoited with less positive edutionl outomes in prt beuse ertin minorities re more likely to hve risk ftors (suh s hving prent who did not finish high shool) tht n ffet edutionl hievement. Pge 2 GAO Outomes

8 more representtive of first-time students thn grdution rtes from other dt soures beuse they inlude prt-time nd trnsfer students. BPS lso ollets self-reported informtion on ernings nd employment sttus, s well s extensive dt on student hrteristis. Some of the grdution rte studies inluded in our review used dt from Edution s Integrted Postseondry Edution Dt System (IPEDS). IPEDS ptures detiled enrollment dt from ll shools prtiipting in federl student id progrms; however, IPEDS grdution rtes inlude only full-time, first-time students nd exlude signifint number of other students (e.g., those who ttend prttime or trnsfer to nother shool). Beuse of this limittion, we gve greter weight in our report to studies using BPS dt to lulte grdution rtes; however, studies using IPEDS dt hd similr results. Studies in our review tht nlyzed debt levels used dt from Edution s Ntionl Postseondry Aid Survey (NPSAS), whih ollets detiled dt on finnil id nd student lons for lrge, ntionlly representtive smple of students. One study in our review nlyzed shool defult rtes using dt from the Ntionl Lon Dt System (NSLDS), whih is Edution s entrl dtbse for federl student id lons nd grnts. We found reltively smll number of studies tht ompred student outomes ross setors nd ounted for differenes in student hrteristis (see pp. II for the list of studies inluded in our literture review). This body of reserh lso hs ertin limittions. For exmple, while BPS hs some of the best vilble dt on outomes nd student hrteristis, it does not represent the experiene of more reent students. 6 Further, while two studies in our review onduted regression nlyses tht ount for multiple student hrteristis simultneously (whih llows for more rigorous omprison), other studies nlyzed subgroups of students, ounting for single hrteristi t time. Despite these limittions, we believe tht the studies inluded in our 6 Beuse the most reent ohort of students strted during the shool yer, BPS does not inlude outomes for students who enrolled more reently. Pge 3 GAO Outomes

9 review provide insight on the omprtive outomes of students ttending different types of shools. To ompre the performne of grdutes from for-profit, nonprofit, nd publi shools on professionl liensing exms, we nlyzed pss rtes for seleted exms for first-time test tkers. 7 For this nlysis, we foused on shools tht prtiipte in federl student id progrms. We seleted ouptions in whih pssing n exm ws generlly required nd signifint work experiene ws not required prior to tking the exm. We lso used Edution dt to selet ouptions tht (1) hd progrms in multiple setors, inluding the for-profit setor, nd (2) hd suffiiently lrge numbers of students grduting from these progrms. When possible, we used exms offered by ntionl orgniztions to mximize the number of sttes in our nlysis. We exluded from our nlyses sttes tht did not require the exm in n ouption. For ouptions tht use stte or multiple exms, we used Edution dt to selet four sttes in whih the numbers of grdutes nd the distribution of grdutes ross setors provided the best hne to detet ny sttistilly signifint differenes tht might exist between setors. Results for individul sttes re not generlizble to other sttes nd it is possible tht setor omprisons in other sttes would show different results or would differ s to whether the results were sttistilly signifint. Beuse demogrphi informtion on test tkers ws generlly not vilble, diretly ontrolling for the hrteristis of test tkers in our nlyses ws not possible. However, beuse our nlysis of liensing exm pss rtes fouses on outomes for progrm grdutes, it my prtilly mitigte the effet of differenes in student hrteristis on exm results sine some hrteristis, suh s re, ge, nd inome, re ssoited with lower grdution rtes. We ssessed the relibility of the dt from eh test inluded in our nlyses by interviewing representtives knowledgeble bout the dt, reviewing relevnt dt nd relted doumenttion, nd onduting dditionl nlyses. We determined tht these exm dt were suffiiently relible for the purpose of ompring pss rtes ross 7 We use the term liensing exm to refer to exms tht re required to work in speifi ouption, even though some of these exms re tehnilly ertifition exms. Differenes between setors re sttistilly signifint unless otherwise noted. Pge 4 GAO Outomes

10 setors. For more detiled results from our nlyses of liensing exm pss rtes, see ppendix III. We onduted our work from November 2010 to Deember 2011 in ordne with generlly epted government uditing stndrds. Those stndrds require tht we pln nd perform the udit to obtin suffiient, pproprite evidene to provide resonble bsis for our findings bsed on our udit objetives. We believe tht the evidene obtined provides resonble bsis for our findings. On November 10, 2011, we briefed ogniznt ongressionl stff on the results of this study nd this report formlly onveys the informtion provided during these briefings. (See pp. IV for the slides we used to brief the requesters.) Limited Reserh Suggests tht For- Profit Shool s Generlly Hve Different Outomes thn Nonprofit or Publi Shool s The smll number of vilble studies tht ounted for seleted student hrteristis, suh s gender, re, or inome, suggests tht student outomes vry by type of shool. outomes inlude grdution rtes, employment outomes, student lon debt, nd defult rtes. Grdution Rtes Two studies show tht for-profit shool students hd higher grdution rtes for ertifite progrms, similr grdution rtes for ssoite s degree progrms, nd lower grdution rtes for bhelor s degree progrms thn students t nonprofit nd publi shools. 8 For exmple, one study found tht 36 perent of low-inome students who strted t for-profit shools ompleted ertifite, ompred to 6 perent t 2-yer 8 We reported grdution rte findings for ertifite, ssoite s degree nd bhelor s degree progrms from two studies tht used BPS dt. For bhelor s degree progrms, we lso reviewed severl studies using IPEDS dt, whih hd similr findings. The term grdution rte refers to students who omplete higher edution progrm nd reeive degree, diplom, ertifite, or other forml wrd. Pge 5 GAO Outomes

11 publi shools. 9 In ontrst, 3 perent of low-inome students who strted t for-profit shools ompleted bhelor s degree, ompred to 49 perent t 4-yer publi shools nd 13 perent t 2-yer publi shools. 10 Employment Outomes An ongoing study suggests tht students who strted t for-profit shools hd similr nnul ernings, but higher rtes of unemployment ompred to students who strted t nonprofit nd publi shools. For exmple, students who strted t for-profit shools during the shool yer nd were no longer enrolled fter 6 yers were more likely to hve been unemployed for more thn 3 months, ompred to students who strted t nonprofit nd publi shools. Lon Debt Three studies show tht higher proportion of bhelor s degree reipients from for-profit shools took out student lons nd tht they generlly hd higher totl student lon debt thn bhelor s degree reipients from nonprofit nd publi shools. For exmple, one study shows tht, mong low-inome students who grduted in , the perentge who borrowed ws greter t for-profit shools (99 perent) thn t nonprofit nd publi shools (83 nd 72 perent, respetively). Defult Rtes Two studies show tht for-profit shools hve higher defult rtes thn 4- yer publi shools, but the results re mixed when ompring for-profit shools to other types of shools. One ongoing study shows tht for-profit 9 This study does not differentite between 2- nd 4-yer for-profit shools, nor does it ontrol for the type of progrm student strts in or whether student trnsfers to higher degree progrm. Grdution rtes re for the highest degree ttined within 6 yers. As result, students who strt in ertifite progrm nd omplete n ssoite s degree re inluded in the ssoite s degree grdution rte. Similrly, students who strt in n ssoite s degree progrm nd omplete bhelor s degree will be inluded in the bhelor s degree grdution rte. BPS dt show tht few, if ny, students t forprofit nd nonprofit shools strt in ertifite progrms nd omplete n ssoite s degree within 6 yers, while smll perentge of students t publi shools do so. 10 We inluded 2-yer publi shools in our nlysis beuse some students who strted t these shools my hve trnsferred to 4-yer shool to omplete bhelor s degree progrm. BPS dt show tht few, if ny, students t for-profit shools strt in ssoite s degree progrms nd omplete bhelor s degree progrm within 6 yers, while smll perentge of students t nonprofit nd publi shools do so. Pge 6 GAO Outomes

12 shools hd higher proportion of students defult on their student lons thn 4-yer nonprofit shools nd 2-yer nonprofit nd publi shools, while the other study did not find ny sttistilly signifint differenes between for-profit shools nd these other types of shools. 11 For-Profit Shool Grdutes Generlly Hd Lower Pss Rtes thn Grdutes from Other Shools on Liensing Exms We Reviewed Exm Results On 9 of the 10 liensing exms we reviewed, grdutes of for-profit shools generlly hd lower pss rtes over the period. The nine liensing exms for whih grdutes of for-profit shools generlly hd lower pss rtes were for Registered Nurses (RN), Liensed Prtil Nurses (LPN), Rdiogrphers, Emergeny Medil Tehniins (EMT), Prmedis, Surgil Tehnologists, Mssge Therpists, Lwyers, nd Cosmetologists. On some exms, the differenes ross setors were sttistilly signifint, but reltively smll. For exmple, 85 perent of grdutes erning bhelor s degree from for-profit nursing progrms pssed the RN exm, ompred to 87 perent of suh grdutes from nonprofit shools. While we were unble to lulte overll pss rtes on the 10th exm (for Funerl Diretors), seprte nlyses of the two setions of the exm suggest tht grdutes of for-profit shools hd similr or better pss rtes thn grdutes of nonprofit nd publi shools. 12 While for-profit grdutes s group 11 One study used NSLDS dt to lulte defult rtes nd IPEDS enrollment dt to ontrol for seleted student hrteristis. While the grdution rtes lulted in IPEDS exlude prt-time nd trnsfer students, IPEDS enrollment dt inlude these students. The other study used BPS dt to lulte defult rtes. In generl, lk of sttistil signifine n men tht there is no tul differene or tht the smple sizes re too smll to detet ny differenes. 12 It ws not possible to ompre the overll performne of grdutes on the Funerl Diretor exm beuse dt on the ombined pss rte for the two setions of the exm (Arts nd Sienes) were not vilble. Pge 7 GAO Outomes

13 generlly hd lower pss rtes, some individul for-profit shools hd reltively high pss rtes. For exmple, 9 of the 40 for-profit shools in our nlysis of the rdiogrpher exm hd pss rtes of 100 perent in Limittions Severl experts nd higher edution ssoition offiils sid tht liensing exm pss rtes re one resonble mesure of shool qulity. However, exm pss rtes lso hve some limittions when used for this purpose. For exmple, reltively few postseondry grdutes overll tke liensing exms, s mny ouptions do not require liense. Further, pss rtes on liensing exms only mesure the performne of students who both omplete progrm nd tke the exm. Dt were not vilble to ompre the totl number of students who begin progrm with those who tke the exm. Consequently, high pss rte my not provide omplete informtion bout the qulity of progrm if lrge number of enrolled students do not finish progrm or do not tke liensing exm. 14 Differenes in student popultions my lso ffet pss rtes. While fousing on grdutes n mitigte the effet of differenes in student hrteristis, it my not ompletely eliminte the impt of these hrteristis on test results. Nevertheless, the federl government hs strong interest in ensuring tht shools tht reeive federl student id funds re ppropritely prepring grdutes for ny required liensing exms. Ageny Comments We provided drft opy of this report to Edution for review nd omment. Edution did not hve omments on the report. In ddition, we shred relevnt setions of the drft report with (1) the uthors of studies inluded in our literture review nd (2) the sttes nd entities tht 13 In most ses, the pss rte dt provided by testing entities did not identify individul shools. As result, it ws not possible to ondut further nlyses on shool hrteristis tht might be ssoited with higher pss rtes. 14 For exmple, shool my enroll 100 students in n edutionl progrm. If 75 students do not omplete the progrm or hoose not to tke required liensing exm, only 25 of the initil 100 students will tke the exm. If ll 25 pss the exm, the progrm will hve 100-perent pss rte. This shool will hve the sme pss rte s shool tht enrolled 100 students, who ll ompleted the progrm, took the exm, nd pssed. Pge 8 GAO Outomes

14 provided liensing exm dt to us. We inorported their tehnil omments s pproprite. We re sending opies of this report to relevnt ongressionl ommittees, the Seretry of Edution, nd other interested prties. In ddition, this report will be vilble t no hrge on GAO s website t If you or your stffs hve ny questions bout this report, plese ontt me t (202) or sottg@go.gov. Contt points for our Offies of Congressionl Reltions nd Publi Affirs my be found on the lst pge of this report. GAO stff who mde key ontributions to this report re listed in ppendix V. George A. Sott Diretor, Edution, Workfore, nd Inome Seurity Issues Pge 9 GAO Outomes

15 Congressionl Requesters The Honorble Rihrd J. Durbin Mjority Whip United Sttes Sente The Honorble Tom Hrkin Chirmn The Honorble Mihel B. Enzi Rnking Member Committee on Helth, Edution, Lbor, nd Pensions United Sttes Sente The Honorble Lmr Alexnder United Sttes Sente The Honorble John Kline Chirmn The Honorble George Miller Rnking Member Committee on Edution nd the Workfore House of Representtives The Honorble Virgini Foxx Chirwomn The Honorble Rubén Hinojos Rnking Member Subommittee on Higher Edution nd Workfore Trining Committee on Edution nd the Workfore House of Representtives The Honorble Robert Andrews Rnking Member Subommittee on Helth, Employment, Lbor, nd Pensions Committee on Edution nd the Workfore House of Representtives Pge 10 GAO Outomes

16 The Honorble Timothy Bishop House of Representtives The Honorble Alee L. Hstings House of Representtives Pge 11 GAO Outomes

17 Appendix I: Sope Appendix I: Sope nd Methodology To help us identify dt soures on student outomes nd reent reserh ompring student outomes ross setors (for-profit, nonprofit, nd publi), we interviewed offiils from the Deprtment of Edution (Edution) nd Edution s Offie of Inspetor Generl (OIG). We lso spoke with 10 higher edution experts nd reserhers, s well s representtives from 6 higher edution ssoitions, 6 postseondry shool reditors, nd 8 stte genies tht oversee postseondry institutions. 1 We lso reviewed relevnt federl lws nd regultions. To ddress our objetives, we (1) onduted strutured literture review of reent studies ompring seleted postseondry student outomes t for-profit nd nonprofit nd/or publi shools nd (2) olleted nd nlyzed pss rte dt for seleted liensing exms for first-time test tkers from for-profit, nonprofit, nd publi shools, fousing on shools tht prtiipte in federl student id progrms. We onduted this performne udit from November 2010 through Deember 2011 in ordne with generlly epted government uditing stndrds. Those stndrds require tht we pln nd perform the udit to obtin suffiient, pproprite evidene to provide resonble bsis for our findings bsed on our udit objetives. We believe tht the evidene obtined provides resonble bsis for our findings bsed on our udit objetives. Literture Review To identify reent reserh on omprtive postseondry student outomes t for-profit, nonprofit, nd publi shools, we onduted strutured literture review. We serhed numerous bibliogrphi dtbses inluding Edution Resoures Informtion Center (ERIC), ProQuest, Edution Journls, PsyINFO, Ntionl Tehnil Informtion Servie (NTIS), EonLit, nd WorldCt for rtiles or studies published from Jnury 2000 through July 2011 tht used dt from 2000 or lter. 2 1 We spoke with the following higher edution ssoitions: Amerin Assoition of Collegite Registrrs nd Admissions Offiers, Amerin Assoition of Community Colleges, Assoition of Privte Setor Colleges nd Universities, Ntionl Assoition of College Admissions Counselors, Ntionl Assoition of Independent Colleges nd Universities, nd Ntionl Cleringhouse. 2 Our serh lso inluded the following dtbses: Congressionl Reserh Servie; Wilson Soil Sienes Abstrts; Soil SiSerh; Soiologil Abstrts; Soil Servies Abstrts; Ademi OneFile; PoliyFile; Sttistil Insight; Eletroni Colletions Online; PpersFirst; ArtileFirst; Disserttion Abstrts Online; PAIS Interntionl; PASCAL; nd British Edution Index. Pge 12 GAO Outomes

18 Appendix I: Sope nd Methodology We employed vriety of serh strtegies to identify reserh relted to student outomes suh s grdution rtes, employment outomes, ernings, student lon indebtedness, nd defult rtes. In ddition to serhing the bibliogrphi dtbses, we identified studies through ittions in previously identified work nd through review of severl higher edution news publitions. We lso sked higher edution ssoitions, reserhers, nd Edution offiils to identify ny relevnt studies nd inluded suh studies in our review. We defined studies brodly to inlude published peer-reviewed journl rtiles; ongoing studies submitted to journls for forml publition by demi reserhers; unpublished studies by higher edution ssoitions, demi reserhers, nd other experts; nd studies issued or ommissioned by different higher edution ssoitions, reserhers, the Congressionl Reserh Servie, or Edution. In order to fous on studies tht ompred postseondry student outomes t for-profit nd nonprofit nd/or publi shools, we exmined ll initil serh results nd restrited our forml review to studies meeting the following riteri: foused on the U.S. student popultion; used t lest some dt olleted in 2000 or lter; ddressed t lest one of the following student outomes: grdution rtes, ernings, employment outomes, student lon debt, nd defult rtes; ompred outomes of for-profit shools with outomes of nonprofit nd/or publi shools; nd ontined originl nlysis ontrolling for t lest one student hrteristi (e.g., re, gender, nd ge). We identified 32 studies tht met these sreening riteri. For eh of these studies, we onduted rigorous review of the reserh methodology, inluding the reserh design; objetives; dt soure; nlyses onduted; nd ny pplible dt-relted or methodologil limittions. As result of this review, we exluded 21 studies due to methodologil limittions nd retined 11 for our nlyses (see pp. II for list of the 11 studies). Seven of the 11 remining studies inluded informtion on grdution rtes, 1 study foused on employment outomes, 4 studies inluded informtion on totl student lon Pge 13 GAO Outomes

19 Appendix I: Sope nd Methodology indebtedness, nd 2 studies inluded informtion on student lon defult rtes t shools. All of these studies inluded omprtive nlyses of student outomes t for-profit nd nonprofit nd/or publi shools, while ontrolling for t lest one student hrteristi, nd ll were determined to be methodologilly sound. 3 Liensing Exm Pss Rte Anlysis To identify potentil ouptions for our nlyses, we reviewed informtion from the Bureu of Lbor Sttistis Ouptionl Outlook Hndbook nd Edution dt on the lrgest fields of study by enrollment. We lso spoke with nd reviewed informtion from representtives t ntionl redentiling orgniztions, stte liensing bodies, testing ompnies, nd other entities involved in ouptionl liensing, to lern more bout whih ouptions require prtitioners to pss n exm, whih sttes require prtitioners to pss speifi exms, nd the vilbility of exm dt. To identify edutionl progrms of study ssoited with our potentil list of ouptions, we reviewed Edution s Clssifition of Instrutionl Progrms (CIP) odes. We initilly onsidered more thn 20 ouptionl fields. 4 We restrited our nlysis of liensing exm dt to ouptions tht met the following riteri. 5 Prtitioners re generlly required to pss liensing exm in order to work. 6 3 The studies in our literture review did not seprtely nlyze outomes for students t minority-serving institutions. 4 Ouptions tht we onsidered inluded: funerl diretors/emblmers, osmetology, ulinry rts/ooking, tehing, lw/ttorneys, legl ssisting/prlegl, riminl justie/lw enforement/orretions, dentl ssisting, dentl hygienists, medil ssisting, EMT/prmedis, rdiogrphy, surgil tehnology, ultrsound/sonogrphy, nursing, nurse ssistnts/home helth ides, dietetis/nutrition, mssge therpy, ounting, rel estte, plumbing, nd eletriins. 5 We use the term liensing exm to refer to exms tht re required to work in speifi ouption, even though some of these exms my tehnilly be ertifition exms. 6 Surgil tehnologists who work in helth re fility, suh s hospitl or mbultory surgil enter, must generlly pss n exm to work in the two sttes inluded in our nlysis. While surgil tehnologists who work in physiins offies re not neessrily required to pss the exm, knowledgeble individuls told us tht the signifint mjority of surgil tehnologists work in helth re filities. Surgil tehnologists who work for the federl government or were trined by the U.S. militry or the U.S. Publi Helth Servie re lso exempt from the testing requirement in these sttes. Pge 14 GAO Outomes

20 Appendix I: Sope nd Methodology Obtining liense does not require signifint work experiene before tking the liensing exm. 7 Pssing liensing exm is generlly not requirement to grdute from progrm of study. 8 To ensure tht we seleted progrms with suffiient numbers of grdutes ross setors, we used CIP dt to determine the number of students ompleting eh progrm in eh setor in shool yer As we identified potentil ouptions nd progrms nd spoke with representtives from stte nd other liensing entities, we further refined our list by eliminting ouptions where vilble dt would not llow us to both (1) relibly identify the type of shool t whih test tkers ompleted n edutionl progrm nd (2) relibly distinguish first time test tkers from repet test tkers. 10 Liensing exms in the following 10 ouptions met our riteri nd the ssoited exm entities greed to provide us with dt: Registered Nurse (RN), Liensed Prtil Nurse (LPN), Rdiogrpher, Emergeny Medil Tehniin (EMT), Prmedi, Surgil Tehnologist, Mssge Therpist, Lwyer, Cosmetologist, nd Funerl Diretor. We generlly olleted liensing exm pss rte dt for first-time test tkers for lendr yers 2008, 2009, nd 2010 to llow us to identify pss rte ptterns nd ount for ny dt nomlies tht might our in single yer. To prevent the identifition of individul shools or students, we did not report speifi pss rtes if there were less thn five progrms in 7 Some ouptions we initilly onsidered, suh s plumber nd eletriin, typilly require long pprentieships prior to tking liensing exm. We exluded suh ouptions sine pssge of liensing exm might reflet skills quired during n pprentieship rther thn from n edutionl progrm. 8 An Edution offiil told us tht some tehing progrms hve historilly required students to pss liensing exm to grdute, so pss rtes would lwys be 100 perent nd therefore not resonble mesure of progrm qulity. 9 While CIP ompletions dt do not diretly orrespond to the ext number of liensing exm test tkers in ny field, we used these dt s proxy for tul test tker dt to sertin whih progrms of study nd orresponding ouptions were worth pursuing. 10 We foused on first-time test tkers beuse we believe their results re more losely ssoited with the qulity of the progrm they ompleted, sine they re less likely to hve hd intervening experienes sine ompleting their shooling. Pge 15 GAO Outomes

21 Appendix I: Sope nd Methodology setor over the time period, unless the dt were publily vilble. Dt Soures for Pss Rtes on Liensing Exms Inluded in Our Anlyses When possible, we seleted liensing exms offered by ntionl orgniztions to mximize the number of sttes in our nlysis. 11 To ensure tht ntionl dt inluded in our nlyses were onsistent nd equivlent, we restrited our nlysis to ntionl liensing exms where single exm with ntionlly set pss sore ws used. We inluded in our nlyses only sttes tht required pssing the liensing exm to prtie in the ouption. 12 We obtined pss rte dt for ntionl exm for seven ouptions RN, LPN, rdiogrpher, EMT, prmedi, surgil tehnologist, nd funerl diretor. RN nd LPN. We nlyzed liensing exm dt from the Ntionl Counil of Stte Bords of Nursing for first-time exm tkers from LPN progrms, ssoite s degree RN progrms, nd bhelor s degree RN 13 progrms. For eh of these degrees, we olleted dt on less thn 2- yer progrms, 2-yer progrms, nd 4-yer progrms. All sttes require RNs nd LPNs to pss these exms in order to prtie. 14 Rdiogrpher. We olleted liensing exm dt from the Amerin Registry of Rdiologi Tehnologists for its rdiogrphy tehnologist exm. We obtined dt for 34 sttes tht require rdiogrphers to pss this exm in order to prtie in the stte. EMT nd Prmedi. We olleted liensing exm dt from the Ntionl Registry of Emergeny Medil Tehniins for its bsi EMT nd prmedi exms. We obtined dt for 32 sttes tht required 11 The Distrit of Columbi is ounted s stte in reporting on the number of sttes from whih liensing dt were olleted. 12 To determine whih sttes require prtitioners to pss speifi exms, we spoke with nd reviewed informtion from representtives t ntionl redentiling orgniztions, stte liensing bodies, testing ompnies, nd other entities involved in ouptionl liensing. 13 Some sttes use the term Liensed Votionl Nurse (LVN) insted of LPN. LVNs must pss the sme exm s LPNs. 14 Nursing dt lso inlude progrms in Amerin Smo, Gum, the Northern Mrin Islnds, Puerto Rio, nd the Virgin Islnds. Pge 16 GAO Outomes

22 Appendix I: Sope nd Methodology EMTs to pss this bsi EMT exm nd for 38 sttes tht require prmedis to pss this prmedi exm in order to prtie in the stte. Surgil Tehnologist. We olleted liensing exm dt from the Ntionl Bord of Surgil Tehnology nd Surgil Assisting for its surgil tehnologist exm. 15 While no stte requires surgil tehnologists to be liensed, two sttes do require most surgil tehnologists to pss this exm in order to prtie in the stte. We obtined dt for these two sttes. Funerl Diretor. We olleted liensing exm dt from the Interntionl Conferene of Funerl Servie Exmining Bords, In. for its Ntionl Bord Exmintion. These dt were for grdutes of the 56 Amerin Bord of Funerl Servie Edution (ABFSE) redited shools, whih re loted in 32 sttes. While not ll sttes require pssing this exm to prtie s funerl diretor, ll students in ABFSEredited progrms re required to tke the exm in order to grdute. 16 Therefore, we determined tht this exm llowed for resonble omprison of progrm qulity ross setors. The exm onsists of two setions Arts nd Sienes whih my be tken together or t different times. We report pss rtes for eh setion seprtely beuse the Interntionl Conferene of Funerl Servie Exmining Bords does not lulte ombined pss rte. Seven of the shools tht ABFSE redits offer bhelor s degree progrms in ddition to or insted of ssoite s degree progrms; however, the Interntionl Conferene of Funerl Servie Exmining Bords nnot distinguish between test tkers from ssoite s degree nd bhelor degree progrms. In order to ensure fir omprison ross setors, we olleted nd nlyzed the dt both inluding nd exluding shools tht offer bhelor s degree progrm. The findings were generlly similr for both nlyses with respet to the reltionship between test tkers from for-profit shools nd 15 We only report surgil tehnologist pss rtes for 2010 beuse one of the two sttes inluded in our review did not require surgil tehnologists to pss this exm until Aording to the Interntionl Conferene of Funerl Servie Exmining Bords, generlly individuls re required to tke, but not pss, the exm to grdute. One of the ABFSE-redited progrm requires pssing the exm to grdute; however, we obtined exm dt for ll students who took the exm, not just grdutes, so this requirement did not impt our nlysis. Pge 17 GAO Outomes

23 Appendix I: Sope nd Methodology those from nonprofit nd publi shools. 17 There were only four for-profit shools inluded in our nlysis; however, beuse shool pss rte dt re vilble publily, we mde n exeption to our rule of not reporting on setors with less thn five progrms, whih is ment to protet the identity of individul shools. For the three remining ouptions, we olleted dt from seleted sttes for stte nd/or multiple ntionl exms epted for liensing 18 purposes. For eh ouption, we used Edution s CIP dt to identify the four sttes in whih the numbers of grdutes nd the distribution of grdutes ross setors provided the best hne to detet ny sttistilly signifint differenes tht might exist between setors. However, in some ses, we were unble to obtin dt from one of the top four sttes, so we olleted dt from the stte tht ws the next most likely to llow us detet differenes ross setors. 19 We generlly inluded test tkers from shools tht were onsidered in-stte by the sttes in our nlysis. Results for individul sttes re not generlizble to other sttes nd it is possible tht setor omprisons in other sttes would show different results or would differ s to whether the results were sttistilly signifint. Mssge Therpist. We olleted mssge therpy liensing exm pss 20 rte dt for shools in Florid, New York, North Crolin, nd Ohio. New York nd Ohio use their own stte exms. During the time period for whih we olleted dt, Florid nd North Crolin both epted 17 Results when inluding bhelor s degree progrms in our nlysis were slightly different in one yer it eliminted the sttistilly signifint differene between test tkers from for-profit shools nd those from publi shools on the sienes setion of the exm in See ppendix III for dt inluding nd exluding shools offering bhelor s degrees. 18 For stte liensing exms, sttes ould hve different requirements, but eh individul stte hd to use one or omprble liensing exms for everyone who is liensed to prtie in speifi ouption within the stte. 19 For osmetology, for exmple, Georgi ws mong the four sttes whih best met our riteri; however, we were unble to ollet dt from this stte, so we used North Crolin s our lterntive. 20 We report mssge therpy exm pss rtes for for Florid, North Crolin, nd New York. Ohio offers its exm in June nd Deember; we were only ble to obtin dt for the Deember exm in 2008, so the pss rte reported for Ohio is for the seond hlf of 2008 nd ll of 2009 nd Pge 18 GAO Outomes

24 Appendix I: Sope nd Methodology pssge of exms from either of two different testing entities nd we report pss rtes seprtely for the seprte exms. 21 We do not report speifi pss rtes for nonprofit mssge therpist progrms beuse there were fewer thn five suh progrms in New York nd Ohio, nd none in Florid or North Crolin over the time period. Br Exm for Lwyers. We olleted publily vilble br exm pss rte dt from Cliforni, Florid, Georgi, nd South Crolin, but eliminted South Crolin beuse first-time nd repet test tkers ould 22 not be seprtely identified. There were fewer thn five shools in severl setors in these sttes; however, beuse shool pss rte dt re publily vilble, we mde n exeption to our rule of not reporting on setors with less thn five progrms, whih is ment to protet the identity of individul shools. 21 Although we tried to void sttes in whih more thn one exm ws used, we inluded dt from both Florid nd North Crolin beuse we were ble to obtin omplete dt on eh exm epted in these sttes. Both sttes ept exms from two testing entities: (1) the Federtion of Stte Mssge Therpy Bords, whih offers the Mssge nd Bodywork Liensing Exmintion, nd (2) the Ntionl Certifition Bord for Therpeuti Mssge nd Bodywork, whih offers the Ntionl Certifition Exmintion for Therpeuti Mssge nd the Ntionl Certifition Exmintion for Therpeuti Mssge nd Bodywork exms. We determined tht ombining the results of both exms offered by the Ntionl Certifition Bord ws methodologilly sound fter interviewing offiils t the Ntionl Certifition Bord, who told us tht the ontent of the two exms ws lrgely identil. Results for these two exms re reported individully in ppendix III. The Ntionl Certifition Bord provided results from the English lnguge version of its exms. A bord offiil told us tht the bord offers Spnish lnguge version of its exms, but test tker volume is very low. 22 These dt were olleted from the websites of the Stte Br of Cliforni, the Florid Bord of Br Exminers, the Supreme Court of Georgi, Offie of Br Admissions, nd the South Crolin Supreme Court, Offie of Br Admissions. In the sttes for whih we nlyzed dt, the br exm is offered twie eh yer, in Februry nd July. We olleted dt from both exms nd ombined the Februry nd July results in our nlysis. In most sttes, only grdutes of shools redited by the Amerin Br Assoition (ABA) re eligible to tke the br exm, but in Cliforni, students from ABAredited, Cliforni-redited, nd Cliforni-unredited lw shools re eligible to tke the br exm nd prtie in the stte. Two of the three Cliforni for-profit lw shools in our dt set were unredited; the third ws ABA-redited. Seven of the 22 nonprofit shools were redited by Cliforni, but not the ABA; the rest were ABAredited. All of the publi shools were ABA-redited. Pge 19 GAO Outomes

25 Appendix I: Sope nd Methodology Cosmetologist. We olleted osmetology liensing exm pss rte dt from Cliforni, Florid, North Crolin, nd Texs. 23 To obtin liense in these sttes, individuls must pss both written nd prtil portion of the exm. Only test tkers who pssed both the written nd prtil portions of the exm on their first ttempt re inluded in the perent of first-time test tkers who pssed the exm. In Cliforni, North Crolin, nd Texs, ndidtes n pply for generl osmetology liense, whih llows them to perform wide rnge of osmetologyrelted tivities, or more speifi liense, suh s mniure or esthetiin liense, whih hve their own liensing exms. 24 In Florid, only one osmetology liensing exm ws offered. 25 In the sttes tht offer multiple exms, we olleted dt on eh exm, but only reported pss rtes on the lrgest exm by test tker volume. 26 We did not report speifi pss rtes for nonprofit osmetology progrms beuse Cliforni nd North Crolin did not hve ny nonprofit osmetologist progrms nd Florid nd Texs eh hd less thn five nonprofit osmetologist progrms over the time period. We ssessed the relibility of liensing exm dt for eh exm in our nlysis by interviewing representtives t eh entity from whih we olleted dt nd reviewing doumenttion relted to the dt systems nd the olletion, storge, nd proessing of dt, when vilble. We determined tht ll dt inluded in our report re suffiiently relible for the purpose of ompring pss rtes ross setors. 23 We olleted dt from the Cliforni Stte Bord of Brbering nd Cosmetology, the Florid Deprtment of Business nd Professionl Regultion, Bord of Cosmetology, the North Crolin Bord of Cosmeti Art Exminers, nd the Texs Deprtment of Liensing nd Regultion. 24 Esthetiins speilize in skin re therpy nd perform tretments suh s fils nd wxing. 25 Beuse Florid ould not relibly identify the shool from whih test tkers grduted for its 2008 exm dt, we olleted only 2009 nd 2010 dt from Florid. 26 When ll osmetologist-relted exms were nlyzed together, the pss rte of grdutes of for-profit shools ws generlly lower thn tht of grdutes of publi shools. However, there were few sttistilly signifint differenes in pss rtes when the less ommon tests were exmined individully, possibly due to smll numbers of test tkers (see pp. III for dt on eh individul exm). Pge 20 GAO Outomes

26 Appendix I: Sope nd Methodology Dt Anlysis For eh ouption, liensing exm dt were olleted t either the progrm level or individul test tker level. Entities providing progrm level dt identified first-time nd repet test tkers for us. 27 In some ses, the entity providing the dt did not wnt to provide dt in wy tht would llow us to identify speifi shool s pss rte. In suh ses, we sent the entity the list of shools with their setor identified, nd the entity repled the shool nme with generi, setor-speifi identifier suh s publi shool 1, publi shool 2, et. As result, further nlysis with respet to individul shool hrteristis ws not possible. For test tker level dt, we identified the first time n individul took n exm using the exm dtes provided, nd ompiled shool level reords bsed on the shool nme or unique identifier ssoited with eh shool. We determined the setor of eh shool using informtion from Edution s Integrted Postseondry Edution Dt System (IPEDS) 28 dtbse. This llowed us to fous on shools tht prtiipte in federl student id progrms. However, it is possible tht we were unble to mth some shools to setor beuse the nme provided did not mth losely enough to the shool nme listed in IPEDS. Additionlly, IPEDS ontins smll number of shools tht do not prtiipte in federl student id progrms. It is possible tht smll number of nonprtiipting shools re ptured in our nlysis if they offer progrms relted to the ouptions for whih we olleted liensing dt. After we grouped the shools by setor for eh liensing exm, we used SAS softwre to lulte liensing exm pss rtes nd men shool 27 In some ses, only dt on first-time test tkers ws provided. 28 Generlly, the entity from whih we olleted dt provided list of shools with progrms from whih grdutes were eligible to tke the exm. However, in some ses we obtined list of pplible shools from publily vilble soures. For the br exm, we obtined list of pplible shools from eh stte s br website. For the rdiogrphy, Texs osmetology, nd Funerl Diretor exms, we obtined list of pplible shools from the Amerin Registry of Rdiologi Tehnologists, the Texs Deprtment of Liensing nd Regultion, nd the Amerin Bord of Funerl Servie Edution s websites, respetively. Pge 21 GAO Outomes

27 Appendix I: Sope nd Methodology pss rtes for first-time test tkers for eh exm for eh setor. 29 We onduted pproprite tests to ssess the sttistil signifine of differenes in student pss rtes nd men shool pss rtes ross setors (see pp. III for overll setor nd men shool pss rte dt nd shool pss rte distribution dt). 30 We presented overll setor pss rtes rther thn men shool pss rtes in our briefing to void hving shools with smll number of test tkers disproportiontely influene setor omprisons. In ddition, using the student (rther thn the shool) s the unit of nlysis resulted in lrger omprison groups, whih inresed the likelihood of deteting ny sttistilly signifint differenes tht might exist between setors. Generlly, there were not substntil differenes between the overll setor pss rtes nd the men shool pss rtes. 31 In some ses, setor differenes in student pss rtes were sttistilly signifint, but differenes in the men shool pss rtes were not. This my be due to the ft tht nlyses of men shool pss rtes re bsed on fewer observtions thn nlyses of overll setor pss rtes. Limittions of the Anlysis There re some limittions relted to using liensing exm pss rtes s n inditor of shool qulity. First, lthough experts nd higher edution ssoition offiils told us tht liensing exm pss rtes re one resonble mesure of shool qulity, reltively few postseondry shool grdutes tke liensing exms beuse mny ouptions do not 29 To lulte men shool pss rtes, we ounted the number of shool progrms in our dt. There re hllenges when mthing shools with Edution s IPEDS dt bse nd ounting the number of shool progrms. Some shools hve multiple brnhes nd mpuses nd n be inluded in Edution s dt s either single shool or multiple shools. As result, mthing shool progrms nd ounting the number of progrms involved some judgment. 30 Although we hve dt for the popultion of students nd shools tking speifi tests in eh yer, we expet some rndom flutution in the popultion over time. Aordingly, we did not tret pss rte informtion s fixed popultion dt, but insted we used sttistil tests to determine whether the differenes we observed exeeded wht we would expet to see with rndom flutution. We used t-tests t the 95 perent onfidene level. A 95 perent onfidene level for t-tests implies tht we would hve less thn 5 perent hne of observing the differenes tht ourred by hne. 31 Almost 80 perent of the time, differenes between overll setor pss rtes nd men shool pss rtes were within 5 perentge points of eh other. In the remining ses, differenes of more thn 5 perentge points ourred most frequently for lw progrms in Cliforni, mssge therpy progrms in Florid nd North Crolin, nd osmetology progrms in North Crolin nd Texs. Pge 22 GAO Outomes

28 Appendix I: Sope nd Methodology require liense or ertifition. Therefore, this nlysis is limited to speifi progrms for whih grdutes require liensure nd does not provide informtion on the qulity of other types of postseondry progrms. In ddition, relible dt were not vilble to estimte the number of students who begin progrms likely to led to speifi ouptions requiring liense; s result, we ould not ompre the number of students who begin progrm to those who omplete the progrm or to those who tke the relevnt liensing exm. Therefore, shool ould hve high liensing exm pss rte, but ould lso hve high drop-out rte if the students lest likely to pss the exm did not omplete the progrm. Additionlly, shool ould hve high exm pss rte if those grdutes lest likely to pss the exm deided not to tke it. Although student hrteristis, suh s re nd inome, hve generlly been found to be orrelted with student outomes, dt were generlly not vilble on the hrteristis of liensing exm test tkers. As result, ontrolling for these ftors in our nlysis ws not possible. Exm pss rtes my be ffeted by the extent to whih shools in one setor serve higher proportion of nontrditionl or disdvntged students. Similrly, shools tht ttrt better prepred students my hve higher liensing exm pss rtes, whih my not be diret funtion of the qulity of the edution provided. Although fousing on outomes for grdutes n mitigte the impt of student hrteristis, it my not ompletely eliminte the effet of these hrteristis on test results. Lstly, the number of shools nd students for some of the exms in our nlysis ws quite smll. For exmple, there ws only one for-profit lw shool in Florid nd only one for-profit, two nonprofit, nd two publi lw shools in Georgi. Similrly, there were only 5 for-profit prmedi progrms ompred to 368 publi prmedi progrms (see pp. III for dt on the number of progrms nd test tkers for eh exm). Pge 23 GAO Outomes