GEP Testing, Spring 2013

Transcription

1 GEP Testing, Spring 2013 Contents Overview... 2 Summary of results & models page... 3 Demographics... 4 levels... 6 Appendix I: List of Institutions of comparison that have administered ETS PP..10 Appendix II: Comparison of NCSU ETS PP test takers with other ETS PP test takers.. 14 Appendix III: Description of proportional weighting.15 Appendix IV: Mapping of ETSPP and NCSU content..16

2 2 GEP Testing, Spring 2013 OVERVIEW Assessment of general education proficiency took place in Spring 2013 semester at North Carolina State University. The assessment, the ETS Profile (ETS PP), was given to both first year freshmen and graduating seniors. The freshman and seniors cohorts were selected for comparison in order to begin to understand the extent to which students have attained general education competencies. Methods: ETS Profile (36 questions, multiple choice, 40 minutes) Supplemental questions added to measure effort/motivation. Used Student Opinion Survey (Sundre & Moore, 2002) On campus, multiple dates/times. Administered by Office of Assessment staff First year freshmen tested January 2013 (recruited through HES courses as a course requirement) Graduating seniors assessed April 2013 (recruited through , multiple venues; voluntary participation with incentives) Exploratory Analysis: Compared mean scores of NC State students to the mean scores provided by ETS of other institutions who have given the ETS PP to similar student populations (see Appendix II). Our samples were not, on their own, representative of population. As such, we created proportional weights post hoc to accommodate for this (see Appendix III). A matching process was used to identify similar freshman and seniors on the basis of race, gender, college, SAT verbal and SAT math (e.g. a White, freshman male in Engineering with a 650 verbal SAT and 700 math SAT would be matched with a White senior male in Engineering with a similar SAT score). 1 Compared the difference in mean scores between the matched freshmen and seniors. GLM regression models were run for freshmen overall, freshmen scaled scores, senior overall, senior scaled scores, and seniors and freshmen overall score combined. Future Plans Increase sample size for sufficient break down by college Use truly longitudinal data when possible (2016 graduating seniors) Add variable years (since matriculating/since completed GEP coursework) to attempt to account for AP credits, etc., in senior group. Key Findings: The overall score difference between freshman and seniors (weighted) was 9 points, unweighted 7 points. Both freshmen and seniors have a majority not proficient in critical thinking The overall score difference between freshman and seniors using matched pairs was 10.8 (95% confidence interval, 6.6 to 15.0) SAT and effort were the predictors common to all three models. GPA was a significant predictor for freshman and the combined model. Gender was only significant in the senior model (males performed better than females), but not in the combined model Transfer student mean score was 456 compared to non-transfer mean score 462 (weighted 464). However, significant variation in transfer student population and small N should be considered. 1 It should be noted that in the models and matched pairs, there was often missing student demographic data from the year 2009 (the year that the SIS transition occurred).

3 3 SUMMARY OF RESULTS Figure 1: Mean Scores and Difference in matched pairs Population Overall Mean Score (scale ) Mean Score on Effort Scale Critical Thinking Reading Natural Science Social Science Humanities Mathematics Writing Freshmen (N= 580) 455 (same as weighted mean) ) ) ) ) ) ) ) Seniors (N=214) Transfer Seniors (N=38) 464* (462* unweighted) 461 (p=0.076) ) (p=0.103) ) (p=0.301) ) (p=.395) ) (p=.246) ) (p=0.108) ) (p=0.368) ) (p=0.062) Difference in scores for matched freshmen and seniors 10.6 * (se.2.08) 3.4* (se.0.73) 3.1* (se.0.83) 2.3* (se.0.67) 2.9* (se.0.84) 3.3* (se.0.84) 2.3* (se.0.59) 1.1 (se.0.60) Proportional weights were created by gender, race, and college. P-values test the difference between transfer and non-transfer seniors. *Indicates significant difference SUMMARY OF MODELS Due to missing data, these models do not include transfers. Missing data (generally SAT scores) also resulted in fewer students included in some of the models. Freshmen: Overall (N=547) For freshmen, the predictors selected in a stepwise procedure for overall ETS PP score were GPA, SAT, and effort. Higher GPA SAT and greater self-reported effort on the ETS PP predicted higher overall scores. Gender, race, and college were not significant predictors. See page 6 of the full report for the full model and discussion. Seniors: Overall (N=186) For seniors, the predictors selected in a stepwise procedure for overall ETS PP score were gender, SAT and effort. Males performed better than females and higher SAT scores and self-reported effort predicted higher scores on the ETS PP. Race, college and GPA were not significant predictors. See page 10 for the full model and discussion. Combined Model: Overall (N=733) Seniors performed better than freshmen. When combining freshman and senior data for analysis, we found that, controlling for all other factors, being a senior predicted a performance on the ETS PP of 5 points higher than freshman (5% improvement- 100 point scale). Other significant predictors in the combined model selected by a stepwise process were SAT, GPA and effort. Matched pairs: Overall (N=106) When seniors and freshman were matched by characteristics such as gender, race, college, and SAT score, the difference in score from freshman to senior year was 10.8 points.

4 4 Demographics Race Freshman Senior Sample Population Sample Population Non-Transfer Transfer Senior Total Non-Transfer Transfer Senior Total N % N % N % N % N % N % N % N % NRA % % 6 2.8% 1 2.8% 7 2.8% % % % Unknown 8 1.4% % 7 3.3% 1 2.8% 8 3.2% % % % Hispanic % % 7 3.3% % % % % % American Indian or Alaska Native 1 0.2% % 0 0.0% 0 0.0% 0 0.0% % 1 0.2% % Asian % % % 1 2.8% % % % % Black or African American % % % 0 0.0% % % % % Native Hawiian or Other Pacific Islander 0 0.0% 4 0.1% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% White % % % % % % % % Two or More % % 4 1.9% 2 5.6% 6 2.4% % % % Total Gender Freshman Senior Sample Population Sample Population N % N % Non- Transfer Transfer All Non-Transfer Transfer All N % N % N % N % N % N % Female % % % % % % % % Male % % % % % % % % Total

5 5 College Freshman Senior Sample Population Sample Population N % N % Non- Transfer Transfer All Non-Transfer Transfer All N % N % N % N % N % N % CALS % % % % % % % % Design % % 4 1.9% 0 0.0% 4 1.6% % % % Education % % % 3 8.3% % % % % Engineering % % % % % % % % CNR % % 7 3.3% 2 5.6% 9 3.6% % % % CHASS % % % % % % % % PAMS % % % 2 5.6% % % % % Textiles % % 8 3.8% 2 5.6% % % % % Management % % % % % % % % FYC % % 6 0.2% 2 0.4% 8 0.2% Total Sample Freshman Non- Transfer Sample Senior Senior Mean Population Freshman Nontransfer Population Seniors Transfer GPA and SAT* Transfer SAT GPA 3.34* * 3.19* * Senior Means * Indicates significant difference

6 6 Levels Freshmen READING Reading Level 1 Frequency Percent Proficient % Reading Level 2 Frequency Percent Proficient % Critical Thinking Level 3* Frequency Percent Proficient % Not Proficient % Not Proficient % Not Proficient % Proficient % Proficient % Proficient % Subscale Score Percentile (ETS Comparison group, N= 37) Reading 78 th percentile; Critical thinking 76 st percentile Grand Total 580 Grand Total 580 Grand Total 580 Writing Writing Level 1 Frequency Percent Proficient % Writing Level 2 Frequency Percent Proficient % Writing Level 3* Frequency Percent Proficient % Not Proficient % Not Proficient % Not Proficient % Proficient % Proficient % Proficient % Grand Total 580 Grand Total 580 Grand Total 580 Subscale Score Percentile (ETS Comparison group, N= 37) Writing 86 th percentile

7 7 Mathematics Mathematics Level 1 Frequency Percent Proficient % Mathematics Level 2 Frequency Percent Proficient % Mathematics Level 3* Frequency Percent Proficient % Not Proficient % Not Proficient % Not Proficient % Proficient % Proficient % Proficient % Grand Total 580 Grand Total 580 Grand Total 580 Subscale Score Percentile (ETS Comparison group, N= 37) Math 89 th percentile Humanities, Social Science, Natural Science** Subscale Area Subscale Score Percentile (ETS Comparison group, N= 37) Humanities Social Science Natural Science 62 th percentile 65 th percentile 65 th percentile

8 8 SENIORS READING Reading Level 1 Frequency Percent Proficient Reading Level 2 Frequency Percent Proficient Critical Thinking Level 3* Frequency Percent Proficient Not Proficient Not Proficient Not Proficient Proficient Proficient Proficient Total Total Total Subscale Score Percentile (ETS Comparison group, N= 57) Reading 75 th percentile; Critical thinking 63 rd percentile WRITING Writing Level 1 Frequency Percent Proficient Writing Level 2 Frequency Percent Proficient Writing Level 3* Frequency Percent Proficient Not Proficient Not Proficient Not Proficient Proficient Proficient Proficient Total Total Total Subscale Score Percentile (ETS Comparison group, N= 57) Writing 89 th percentile MATHEMATICS Mathematics Level 1 Frequency Percent Mathematics Level 2 Frequency Percent Mathematics Level 3 Frequency Percent Subscale Score Percentile (ETS Comparison group, N= 57) Mathematics

9 9 Proficient Proficient Proficient 95 th Not Proficient Not Proficient Not Proficient percentile Proficient Proficient Proficient Total Total Total Humanities, Social Science, Natural Science** Subscale Area Subscale Score Percentile (ETS Comparison group, N= 37) Humanities Social Science Natural Science 58 th percentile 70 th percentile 54 th percentile *Competency at which a majority of our students are not at least marginally proficient ** levels are not provided by ETS. They note that, Materials included in the ETS Profile reading sections, graphs, tables, and artistic designs are concerned with issues, themes and ideas from the humanities, social sciences and natural sciences. The questions related to these materials for example, the questions that follow a reading section measure a students ability to read carefully, to make judgments about the clarity, correctness, or organization of the materials as pieces of writing, and to think critically about the issues and arguments presented. Mathematical data are presented largely although not exclusively in a mathematical form independent of academic content. The questions do not explicitly ask for recall of information learned in specific courses in the humanities, social sciences or natural sciences, they assume however that a student who has taken general education courses in these areas will be better able to read the materials and answer the accompanying questions than a student who has not taken such courses. In essence, Reading, Writing, Critical Thinking and Mathematics are the content measured on the ETS PP and many of the questions that seek to measure these skills are framed as Humanities, Social Science, or Natural Science questions.

10 10 Appendix I: List of Institutions of comparison that have administered ETS PP Freshmen RI/RII/doctoral <30 hours (N= 37) Freshmen ALL institutions <30 hours (N=273) Arizona State University - Tempe Clark Atlanta University Clemson University Colorado State University- Global Campus Florida Agricultural and Mechanical University Florida International University Indiana State University Indiana University - System Office Long Island University - CW Post Mississippi State University Missouri University of Science and Technology Morgan State University Northern Arizona University Oklahoma State University Saint Philips College Sam Houston State University Spalding University Texas A&M University - San Antonio Texas A&M University-Commerce Thomas Edison State College University of Akron, The University of Alabama at Birmingham University of Alaska Fairbanks University of Cincinnati University of Colorado - Denver University of Delaware University of Illinois University of Mississippi University of Missouri - Columbia University of North Carolina at Greensboro University of North Texas - Dallas University of South Carolina - Columbia University of Southern Mississippi University of Tennessee - Knoxville University of Tulsa Walden University Wayne State University Adventist University of Health Sciences Aiken Technical College Alabama A&M University Alabama State University Albertus Magnus College Alcorn State University Alice Lloyd College American Intercontinental University American Public University American Sentinel University Anderson University - South Carolina Antioch University McGregor Arizona State University - Tempe Art Institute of Austin Art Institute of Charlotte, The Art Institute of Dallas Art Institute of Houston Art Institute of Raleigh-Durham Asbury University Ashford University Athens State University Atlantic Cape Community College Ave Maria University Belhaven College (MS) Bellarmine University Benedict College Bennett College for Women Bethel University Black Hawk College Bloomsburg University Blue Mountain College Bluffton University Brescia University Brewton-Parker College Burlington County College Cairn University California University of Pennsylvania Calvary Bible College Camden County College Campbell University Capital University Central Arizona College Central Bible College Clarion University of Pennsylvania Eastern University Eckerd College Edison State College Elizabeth City State University Endicott College Everest College Excelsior College Ferris State University Ferrum College Fisk University Florida Agricultural and Mechanical University Florida Gateway College Florida International University Florida State College at Jacksonville George Fox University (OR) Governors State University Grambling State University Greenville Technical College Henderson State University Hesser College High Point University Houston Baptist University Huston-Tillotson University Indiana State University Indiana University - System Office Jefferson State Community College Johnson C. Smith University Kaplan University Kaskaskia College Keiser University Keystone College LaGrange College Lake Land College Lake Superior State University Lamar University Lander University Laramie County Community College Laredo Community College LeTourneau University Liberty University Limestone College Lindenwood University Lock Haven University of PA Mount Vernon Nazarene University Neumann University New England College of Business & Finance New Jersey City University Newberry College Norfolk State University Northeast State Technical Community College Northern Arizona University Northwest Missouri State University Northwestern College (MN) Norwich University Ohio Christian University Oklahoma State University Ottawa University- Phoenix Palm Beach Atlantic University Palm Beach State College Parker University Patrick Henry College Patten University Pfeiffer University Pima Community College Pontifical College Prairie View A&M University Presentation College Prince George's Community College Providence College Quinnipiac University Raritan Valley Community College Rasmussen College Regent University Reinhardt College Rio Salado College River Parishes Community College Roane State Community College Robert B. Miller College Saint Augustines University Saint Leo University Saint Mary of the Woods University Saint Mary's University Saint Peter's College Saint Philips College Sam Houston State University Troy University Tusculum College University of Akron Wayne College University of Akron, The University of Alabama at Birmingham University of Alaska Fairbanks University of Central Missouri University of Central Oklahoma University of Charleston University of Cincinnati University of Colorado University of Colorado - Denver University of Delaware University of Houston - Victoria University of Illinois University of Maine - Presque Isle University of Maryland - Eastern Shore University of Maryland - University College University of Mississippi University of Missouri - Columbia University of Mobile University of North Carolina at Greensboro University of North Florida University of North Texas - Dallas University of Northern Iowa University of Phoenix University of South Alabama University of South Carolina - Aiken University of South Carolina - Columbia University of South Florida - St. Petersburg University of Southern Indiana University of Southern Mississippi University of Tennessee - Chattanooga University of Tennessee - Knoxville University of Tennessee at Martin University of Texas at Tyler University of the Cumberlands

11 11 SENIORS RI/RII/Doctoral (N=57) Andrews University Biola University Bowie State University Capella University Clark Atlanta University Clemson University Colorado State University- Global Campus Florida Agricultural and Mechanical University Florida International University Indiana State University Indiana University - Bloomington Indiana University - System Office instituto Tecnologico y de Estudios Superiores de Long Island University - CW Post Mississippi State University Montana State University Morgan State University New Jersey Institute of Technology North Carolina A&T State University Northern Arizona University Oglala Lakota College Clark Atlanta University Clayton State University Clemson University Coastal Carolina University Coconino Community College College of Charleston College of Southern Maryland College of the Ozarks Colorado State University- Global Campus Colorado State University-Pueblo Columbia College (MO) Community College of Baltimore County Community College of the Air Force Concordia College Concordia University Chicago Coppin State University Covenant College Craven Community College Daytona State College Dean College Del Mar College Delaware County Community College Denison University DeVry University Dickinson State University Dordt College Dyersburg State Community College East Stroudsburg University Abilene Christian University Adventist University of Health Sciences Aiken Technical College Alabama State University Albany State University Albertus Magnus College Amberton University American Public University American Sentinel University Anderson University - South Carolina Andrews University Angelo State University Antioch University McGregor Aquinas College (MI) Armstrong Atlantic State University Art Institute of Atlanta, The Art Institute of Charlotte, The Art Institute of Dallas Art Institute of Houston Art Institute of Pittsburgh, The Asbury University Ashford University Ashland Community and Technical College Long Island University - CW Post Maharishi University of Management Mansfield University Maranatha Baptist Bible College Marietta College Martin Community College Massachusetts Maritime Academy McDowell Technical Community College McNeese State University Metropolitan Community College Miami International University of Art and Design Midland University Midway College Milligan College Minnesota State University Moorhead Mississippi State University Missouri Southern State University Missouri State University Missouri University of Science and Technology Montcalm Community College Morehouse College Morgan State University Motlow State Community College Mott Community College Fisk University Florida Agricultural and Mechanical University Florida International University Florida State College at Jacksonville Forman Christian College Fort Hays State University Framingham State College Francis Marion University Franklin University George Fox University (OR) Georgetown College Glendale Community College God's Bible School and College Governors State University Grambling State University Greenville Technical College Guilford College Hesser College High Point University Holy Family University Houghton College Houston Baptist University SENIORS ALL Institutions (N= 379) Seminole State College of Florida South Arkansas Community College South Carolina State University South College South University Southeast Missouri State University Southeastern University Southern Utah University Southern Vermont College Southern Wesleyan University Southwest Baptist University Southwestern Christian College Southwestern College Spalding University Spelman College St. Johns River State College State College of Florida, Manatee- Sarasota Stephen F. Austin State University Stevenson University Strayer University Surry Community College Talladega College Tarleton State University Texas A&M University - Corpus Christi Texas A&M University - San Antonio Texas A&M University Kingsville Texas A&M University-Commerce Thomas Edison State College Thomas More College Touro College North Carolina Wesleyan College North Greenville University Northeast State Technical Community College Northeastern Illinois University Northeastern Oklahoma A&M College Northern Arizona University Northern Michigan University Northwest Missouri State University Norwich University Oakland City University Oglala Lakota College Ohio Christian University Oklahoma Baptist University Oklahoma State University Ottawa University Ottawa University- Phoenix Pace University - New York Pacific Union College Palm Beach Atlantic University Palm Beach State College Parker University Patrick Henry College University of the Ozarks University of Tulsa University of West Alabama Vanguard University of Southern California Victor Valley Community College Victoria College Volunteer State Community College Walden University Walters State Community College Warren County Community College Washburn University Wayland Baptist University Wayne State University Webber International University West Coast University West Kentucky Community and Technical College West Virginia University Western Oklahoma State College Western Texas College Western Wyoming Community College Wilkes University William Carey University Winthrop University Young Harris College Trevecca Nazarene University Troy University Troy University - Global Truman State University Tusculum College University of Akron, The University of Alabama at Birmingham University of Alabama in Huntsville University of Alaska Fairbanks University of Arkansas - Little Rock University of Central Arkansas University of Central Missouri University of Central Oklahoma University of Charleston University of Cincinnati University of Colorado University of Colorado - Denver University of Delaware University of Houston - Downtown University of Illinois University of Kansas University of Louisiana - Lafayette

12 12 Oklahoma State University Pace University - New York Saint Philips College Sam Houston State University Seattle University Spalding University Tennessee State University Texas A&M University - San Antonio Texas A&M University-Commerce Thomas Edison State College University of Akron, The University of Alabama at Birmingham University of Alabama in Huntsville University of Alaska Fairbanks University of Cincinnati University of Colorado - Denver University of Delaware University of Illinois University of Louisiana - Lafayette University of Massachusetts Lowell University of Mississippi University of Missouri - Columbia University of Missouri - Kansas City University of Nevada University of North Carolina at Greensboro University of North Texas - Dallas University of South Carolina - Columbia University of South Florida - Sarasota- Manatee University of South Florida - Tampa University of South Florida Polytechnic University of Southern Mississippi University of Tennessee - Knoxville University of Tulsa Walden University Wayne State University Wilmington University Athens State University Austin Peay State University Ave Maria University Baldwin Wallace University Bauder College Becker College Belhaven College (MS) Bellarmine University Bemidji State University Bennett College for Women Bethel College Bethel University Biola University Bloomsburg University Blue Mountain College Bluffton University Bowie State University Bradley University Brenau University Brewton-Parker College Bridgewater College Bryan College Cairn University California University of Pennsylvania Camden County College Campbell University Capella University Cazenovia College Central Bible College Central Christian College of the Bible Charleston Southern University Charter Oak State College Cheyney University of Pennsylvania Claflin University Clark Atlanta University Clayton State University Clemson University Coastal Carolina University Cochise College College of Charleston College of Coastal Georgia College of New Jersey, The College of Southern Maryland College of the Ozarks Colorado Mesa University Colorado State University- Global Campus Colorado State University-Pueblo Columbia College (MO) Columbia State Community College Community College of Baltimore County Community College of the Air Force Concordia College Concordia University (CA) Concordia University Chicago Howard Payne University Humboldt State University Huston-Tillotson University Indiana State University Indiana University - Bloomington Indiana University - System Office instituto Tecnologico y de Estudios Superiores de Jacksonville State University Jamestown College Jefferson State Community College John Tyler Community College Judson College Kaplan University Kean University Keiser University Keystone College Kutztown University of Pennsylvania LaGrange College Lake Superior State University Lamar University Lambuth University Lander University Laramie County Community College Laredo Community College LDS Business College Lee University LeTourneau University Lewis-Clark State College Liberty University Limestone College Lindenwood University Lock Haven University of PA Long Island University - CW Post Louisiana College Luther Rice University Madisonville Community College Maharishi University of Management Mansfield University Maranatha Baptist Bible College Marian University (IN) Marietta College Martin Community College Mary Baldwin College Massachusetts Maritime Academy McNeese State University Metropolitan Community College Metropolitan State College of Denver Miami International University of Art and Design Midland University Midway College Mineral Area College Minnesota State University Moorhead Paul Quinn College Pfeiffer University Philander Smith College Pima Community CollegePoint Loma Nazarene University Point University Presentation College Providence College Queens University of Charlotte Quinnipiac University Rasmussen College Regent University Reinhardt College Rio Salado College Rivier University Roane State Community College Robert B. Miller College Rocky Mountain College Saint Augustines University Saint Leo University Saint Mary of the Woods University Saint Philips College Sam Houston State University San Diego Christian College Schreiner University Seattle University Seminole State College of Florida Shawnee State University Shenandoah University Shepherd University South Carolina State University South College South University Southeast Missouri State University Southeastern University Southern Adventist University Southern Catholic College Southern Illinois University Carbondale Southern Illinois University Edwardsville Southern Utah University Southern West Virginia Community and Technical Col Southwest Baptist University Southwest Tennessee Community College Southwest Texas Junior College Southwestern Assemblies of God University Southwestern College Spalding University Spelman College Spring Hill College St. Petersburg College St. Vincent's College Winthrop University University of Maine - Fort Kent University of Maine - Presque Isle University of Maryland - Eastern Shore University of Maryland - University College University of Massachusetts Dartmouth University of Massachusetts Lowell University of Memphis University of Mississippi University of Missouri - Columbia University of Missouri - Kansas City University of Mobile University of Nevada University of North Carolina at Greensboro University of North Carolina at Wilmington University of North Florida University of North Texas - Dallas University of North Texas - Denton University of Northern Iowa University of Phoenix University of Pikeville University of South Alabama University of South Carolina - Aiken University of South Carolina - Columbia University of South Carolina - Upstate University of South Florida - Sarasota- Manatee University of South Florida - St. Petersburg University of South Florida - Tampa University of South Florida Polytechnic University of Southern Indiana University of Southern Mississippi University of Tennessee - Chattanooga University of Tennessee - Knoxville University of Tennessee at Martin University of Texas at Tyler University of Texas Pan American University of the Cumberlands University of the Ozarks University of Tulsa University of Wisconsin - Platteville University of Wisconsin - Stevens Point Vanguard University of Southern California Victoria College Victory University Volunteer State Community College Walden University Walters State Community College Washburn University

13 13 Coppin State University Covenant College Craven Community College Dallas Baptist University Darton College Daytona State College Delaware Valley College Denison University DeVry University Dickinson State University Dordt College Durham Technical Community College Dyersburg State Community College East Stroudsburg University Eastern Mennonite University Eastern New Mexico University Eckerd College ECPI University Edinboro University of Pennsylvania Endicott College Everest College Everglades University Excelsior College Felician College - Lodi Ferris State University Fisher College Minnesota State University, Mankato Mississippi College Mississippi State University Mississippi Valley State University Missouri Southern State University Missouri State University Missouri Western State University Montana State University Montana Tech of the University of Montana Montclair State University Morgan State University Motlow State Community College Mott Community College Mount Olive College Mount Vernon Nazarene University National College Neumann University New Jersey City University New Jersey Institute of Technology New Orleans Baptist Theological Seminary Newberry College Nicholls State University Norfolk State University North Carolina A&T State University Worcester State College State Fair Community College Stephen F. Austin State University Sterling College Strayer University Sullivan University Surry Community College Talladega College Tarleton State University Taylor University Tennessee State University Tennessee Wesleyan College Texas A&M University - Central Texas Texas A&M University - Corpus Christi Texas A&M University - San Antonio Texas A&M University - Texarkana Texas A&M University Kingsville Texas A&M University-Commerce Texas Wesleyan University Thomas Edison State College Thomas More College Three Rivers Community College Toccoa Falls College Touro College Wayland Baptist University Wayne State University Webber International University West Kentucky Community and Technical College Western Connecticut State University Western International University Western Oklahoma State College Western Wyoming Community College Wiley College Wilkes University Wilmington University Wingate University Winston-Salem State University

14 14 Appendix II: Comparison of NCSU ETS PP test takers with other ETS PP test takers nationally Freshmen Overall Compared to other freshmen in ETS Comparison group 2, our freshman 3 overall score was in the 92nd percentile. Compared to all freshman who had taken the ETS PP our students were in the 97 th percentile. o Scaled Scores Freshmen Compared to ALL institutions (N=273) Critical thinking 90 th percentile Reading 89 th percentile Writing 96 th percentile Math 97 th percentile Humanities 79 th percentile Social Science 86 th percentile Natural Sciences 89 th percentile o Scaled Scores Freshman Compared to ETS Comparison Group (N=37) Critical thinking 76 st percentile Reading 78 th percentile Writing 86 th percentile Math 89 th percentile Humanities 62 th percentile Social Science 65 th percentile Natural Sciences 65 th percentile Senior Overall Compared to other seniors in ETS Comparison group our seniors overall score was in the 89 th percentile. Compared to all seniors who had taken the ETS PP our students were in the 96 th percentile. o Scaled Scores Seniors Compared to ALL institutions (N=379) Critical thinking is 81 st Reading is 86 th Writing is 93 rd Math is 98 th Humanities is 78 th Social Sciences is 85 th Natural Sciences is 78 th o Scaled Scores Seniors Compared to ETS Comparison Group (N=57) Critical thinking 63 rd percentile Reading is 75 th Writing is 89 th Math is 95 th Humanities 58 th Social Science is 70 th Natural Sciences is 54 th 2 The ETS comparative data report conflates RI, RII and doctoral universities into a single category for comparison. This is their convention and since they do not distinguish between the three, the term in this report ETS Comparison group refers to all other RI, RII and doctoral institutions which have administered the ETS PP. 3 Here, freshmen does not refer to incoming freshmen, but rather freshmen with <30 credit hours. This is the cohort we tested and ETS allows for the distinction in comparison groups between freshmen with 0 credit hours (incoming) and those with <30 semester hours or <45 quarter hours.

15 15 Appendix III: Description of proportional weighting We used a weighting formula by proportion to adjust for any proportional differences between the sample and population. Calculating Weights (post-stratification weighting): w ijkl is the weight of the i th observation in race j, gender k, and college l. Calculating weighted average:

16 Appendix IV: Mapping of ETSPP and NCSU content 16 Humanities NCSU Subject & Definition ETS PP proficiency definition Mapping between NCSU and ETS The humanities comprise the subjects and disciplines that Humanities: The humanities materials include selections (poetry, N/A use various models of rational inquiry to understand fiction, and nonfiction prose) from classical literature in translation human nature and experience, organization and change in and from American and British literature. The materials also include human societies, the nature of the world, and rational discussion of key terms in humanities. Visual elements are inquiry itself. An education in the humanities requires presented, as well as discussions of painting, music, philosophical analysis and interpretation of significant works, gaining an issues, and materials by and about women and minorities. The exposure to a variety of methodologies, and learning to content is drawn from material typically encountered in introductory apply these in written exposition. An education in the basic courses in literature, art, music, and philosophy humanistic disciplines is necessary to become a citizen with a broad knowledge of human cultures and with wellconsidered moral, philosophical, aesthetic, and intellectual convictions. Usability? Not useable Category Objectives: Each course in the Humanities category of the General Education Program will provide instruction and guidance that help students to: 1. Engage the human experience through the interpretation of human culture and 2. Become aware of the act of interpretation itself as a critical form of knowing in the humanities; and 3. Make academic arguments about the human experience using reasons and evidence for supporting those reasons that are appropriate to the humanities. Natural sciences The natural sciences pursue basic questions about the workings of the universe, and the richness, variety and interconnectedness of the world around us. Students today are exposed to an increasing volume of information, from a large variety of sources, in diverse and changing formats. Training in the natural sciences is essential to help students develop skills to distinguish between testable and un-testable ideas, recognize scientifically valid tests of theories, and understand how information relates to those tests. By studying the natural sciences, students learn to reason both inductively and deductively, develop and test scientific hypotheses, and understand the value and limitations of scientific studies. The development and application of new technologies require scientifically literate citizens who can understand technological issues and evaluate the role of science in society's debate of those issues. Natural Sciences: The natural sciences materials are often interdisciplinary in nature; that is, they present topics and issues that might be discussed in a general education course in biology, chemistry, or physics. In addition, the materials may present experimental findings in ways that are familiar to a student who has taken any kind of general education science course. Where materials are not interdisciplinary, topics and issues in biology will receive more emphasis than those in chemistry and physics. The materials includes both reading selections and graphs and/or charts. N/A Not useable Category Objectives: Each course in the Natural Sciences will provide instruction and guidance that help the student to: 1. use the methods and processes of science in testing hypotheses, solving problems and

17 17 making decisions; and 2. make inferences from and articulate, scientific concepts, principles, laws, and theories, and apply this knowledge to problem solving. Social Sciences Rationale: The study of social sciences enables students to understand individual and collective human behavior by exploring meaning within a variety of social, cultural, political, and economic contexts, analyzing the structures within which human goals are established and human choices are made, and applying theoretical and empirical models to specific cases. Category Objectives: Each course in the Social Sciences category will provide instruction and guidance that help students to: Social Sciences: The social sciences materials present topics and issues that might be discussed in a general education course in history, economics, political science, psychology, anthropology, sociology, or indeed any general education social science course. The materials are often interdisciplinary in their approach to a topic; that is, they draw on more than one of the social science fields in their presentation of a topic. Social sciences topics of particular relevance to women and minorities are represented in the materials, which include both reading selections and graphs and/or tables. Questions include reading selections, graphs, and charts. N/A Not useable 1. Examine at least one of the following: human behavior, culture, mental processes, organizational processes, or institutional processes; and 2. Demonstrate how social scientific methods may be applied to the study of human behavior, culture, mental processes, organizational processes, or institutional processes; and 3. Use theories or concepts of the social sciences to analyze and explain theoretical and/or real-world problems, including the underlying origins of such problems. Writing Written communication is the purposeful development, expression, and revision of ideas in writing for specific audiences. Effective written communication follows appropriate genre conventions and may include a combination of text and other media. Students will be able to: Recognize that writing is purposeful and varies according to context Understand that writing is a complex, nonlinear process Understand that writing often builds on the work of others and that strategies for locating and criteria for evaluating information vary according to context Recognize when to acknowledge the source of others texts or ideas in their own writing Be aware of a variety of technologies to compose, produce and publish their work Level 1 Level 2 Level 3 Recognize agreement among basic grammatical elements (e.g. nouns, verbs, pronouns and conjunctions) Recognize appropriate transition words Recognize incorrect word choice Order sentences in a paragraph Order elements in an outline Incorporate new material into a passage Recognize agreement among basic grammatical elements (e.g. nouns, verbs, pronouns, and conjunctions) when these elements are complicated by intervening words or phrases Combine simple clauses into single, more complex combinations Recast existing sentences into new syntactic combinations Discriminate between appropriate and inappropriate use of parallelism ETS PP s Discriminate between appropriate and inappropriate use of idiomatic language comprises a small part of NCSU s description of recognizing that writing is purposeful and varies according to context ETS PP focuses primarily on mechanics. There is very limited connection between outcomes.

18 18 Quantitative Literacy Reading/Critical Thinking A habit of mind, competency, and comfort in working with numerical data. Individuals with strong QL skills possess the ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations. They understand and can create sophisticated arguments supported by quantitative evidence, and they can clearly communicate those arguments in a variety of formats (using words, tables, graphs, mathematical equations, etc., as appropriate). Students will be able to: Explain information presented in mathematical forms (e.g. equations, graphs, diagrams, tables, words) Convert relevant information into various quantitative forms (e.g. equations, graphs, diagrams, tables, words) Attempt and successfully complete all appropriate calculations for the task at hand Make judgments and draw appropriate conclusions based on the quantitative, logical and statistical analysis of data, while recognizing the limits of this analysis Critical thinking is the active, persistent, and careful consideration of a belief or form of knowledge, the grounds that support it, and the conclusions that follow. It involves analyzing and evaluating one s own thinking and that of others. Students will be able to: Raise vital questions and problems, and formulate these clearly and precisely Gather and assess relevant information Level 1 Level 2 Level 3 Level 1 Level 2 Discriminate between appropriate and inappropriate use of idiomatic language Recognize redundancy Discriminate between correct and incorrect constructions Recognize the most effective revision of a sentence Solve word problems that would most likely be solved by arithmetic and do not involve conversion of units or proportionality Solve problems involving the informal properties of numbers and operations, often involving the Number Line, including positive and negative numbers, whole numbers and fractions (including conversion of common fractions to percent, such as a conversion ¼ to 25%) Solve problems requiring a general understanding od square roots and the squares of numbers Solve a simple equation or substitute numbers into an algebraic expression Find information from a graph Solve arithmetic problems with some complications, such as complex wording, maximizing or minimizing and embedded ratios Simplify algebraic expressions, perform basic translations and draw conclusions from algebraic equations and inequalities Interpret a trend represented in a graph or choose a graph that reflects a trend Solve problems involving sets Solve word problems that would be unlikely to be solved by arithmetic; the answer choice are either algebraic expressions or are numbers that do not lend themselves to back solving Solve problems involving difficult arithmetic concepts such as exponents and roots other than squares and squares roots and percent s of increase or decrease Generalize about numbers, e.g.; identify the values of (x) for which an expression increases as (x) increases Solve problems requiring an understanding o the properties of integers, rations numbers, etc. Interpret a graph in which the trends are to be expressed algebraically or in which one of the following is involved: exponents and roots other than squares and square roots, percent of increase or decrease Solve problems requiring insight or logical reasoning Recognize factual material explicitly presented in a reading passage Understand the meaning of particular works or phrases in the context of the reading passage Synthesize material from different sections of a passage Recognize valid inferences derived from material in the passage Identify accurate summaries of a passage or of Levels 1 and 2 of the ETS PP contain outcomes tied to NCSU s (AACU) outcomes for calculation and interpretation. Level 3 of ETS PP measures outcomes tied to NCSU s outcomes for calculation, interpretation, representation and application/analysis. Most of Level 2 and 3 are loosely tied to the NCSU outcomes: Gather and assess relevant information Reach wellreasoned conclusions There is a sufficient connection between NCSU and ETS PP outcomes. There is a sufficient connection between NCSU and ETS PP outcomes. This is usable, though not perfect match.

19 19 Reach well-reasoned conclusions and test them against appropriate criteria and standards Openly consider alternative systems of thought or points of view Effectively communicate to others the analysis of and/or proposed solutions to questions or problems. Level 3 significant sections of a passage Understand and interpret figurative language Discern the main idea, purpose, or focus of a passage or a significant portion of a passage Evaluate competing casual explanations Evaluate hypotheses for consistency with known facts Determine the relevance of information for evaluating an argument or conclusion Determine whether an artistic interpretation is supported by evidence contained in a work Recognize the salient features or themes in a work of art Evaluate the appropriateness of procedures for investigating a question of causation Evaluate data for consistency with known facts, hypotheses or methods and test them against appropriate criteria and standards Openly consider alternative systems of thought or points of view Materials included in the ETS Profile reading sections, graphs, tables, and artistic designs are concerned with issues, themes and ideas from the humanities, social sciences and natural sciences. The questions related to these materials for example, the questions that follow a reading section measure a students ability to read carefully, to make judgments about the clarity, correctness, or organization of the materials as pieces of writing, and to think critically about the issues and arguments presented. Mathematical data are presented largely although not exclusively in a mathematical form independent of academic content. The questions do not explicitly ask for recall of information learned in specific courses in the humanities, social sciences or natural sciences, they assume however that a student who has taken general education courses in these areas will be better able to read the materials and answer the accompanying questions than a student who has not taken such courses. In essence, Reading, Writing, Critical Thinking and Mathematics are the content measured on the ETS PP and many of the questions that seek to measure these skills are framed as Humanities, Social Science, or Natural Science questions.