1 Online Developmental Mathematics Instruction in Colleges and Universities: An Exploratory Investigation Taylor Martin Nicole Forsgren Velasquez Jason Maughan ABSTRACT 1 Mathematics proficiency is critical for students continued success in school, access to postsecondary education, and preparation for future employment. As much as 40 percent of incoming college students are unprepared, and many are placed in developmental mathematics courses. Unfortunately, many of these students do not pass these courses. In this study we employed exploratory analyses with a large dataset to examine patterns of course activity by students, and found that students often skip difficult material. We also compared patterns of content delivery, defined by instructors, in developmental mathematics courses and intermediate mathematics courses and found that developmental courses devote much more time to basic concepts than intermediate courses. Utah State University, Department of Instructional Technology & Learning Sciences, EDUC 211, 2830 Old Main Hill, Logan, UT 84321, Fax: , Utah State University, Management Information Systems, Huntsman School of Business, 3515 Old Main Hill, Logan, UT 84322, USA. Cell: , Utah State University 1 Cite the paper: Martin, T., Velasquez, N.F., Maughan, J. (April, 2014). Online Developmental Mathematics Instruction in Colleges and Universities: An Exploratory Investigation. Paper presented at the American Educational Research Association Annual Conference, Philadelphia, PA. 1. OBJECTIVES Mathematical ability is critical for students continued success in school, access to postsecondary education, and preparation for future employment. In the United States, strong performance in middle school mathematics classes are essential to access the full range of mathematics classes in high school, which in turn are required for entry into many four-year colleges. People who succeed in mathematics in high school and college have better employment prospects than others and can expect to earn more. The organizations where they are employed also benefit from having skilled effective employees. In addition, mathematical literacy is a growing need in our increasingly technological society. Many students come to college unprepared for success; depending on the college, as much as 40 percent of these students are placed in developmental courses their freshman year. Developmental mathematics is a college course, generally aimed to remediate from areas missed in high school mathematics to prepare students for higher-level college mathematics. Unfortunately, many students do not complete or pass their developmental courses; college completion rates within six years is less than 60 percent at many colleges . In addition, even if they do succeed in these courses, they are not necessarily prepared to succeed in more advanced courses [1, 10]. Researchers and policymakers have therefore questioned the effectiveness of remedial or developmental coursework. This paper employs exploratory analyses with a large dataset to examine patterns of activity in the class, which inform hypotheses regarding the lack of success in developmental mathematics. 2. THEORETICAL FRAMEWORK Several hypotheses regarding the lack of success in developmental mathematics have been put forward. One is that the way developmental mathematics courses are taught is not likely to be successful . What evidence there is suggests that many of these courses are taught in the same lecture and practice format that many of these students experienced in their education prior to college . First, if this method did not help these students learn the material in developmental mathematics when they attempted to learn it before, it is unlikely repetition would help. Second, the evidence we have suggests that students have learned the material by rote and do not show deep understanding .
2 Another approach being researched involves providing support structures such as tutoring, mentoring, and other services to improve postsecondary success [4, 3]. However, these support services require human resources, which can be costly. In addition, in some communities the need for these services may exceed the supply (e.g., rural areas). With technological advances, there are new ways to provide similar support to students in need of remediation and students in developmental courses that are more cost effective and do not require support to be in the form of a physical person. Another theory regarding why students may not be succeeding has to do with affective factors involved in mathematics engagement. For example, a student s own belief about his/her ability to do a task or efficacy overall in a topic can influence performance [2, 5, 8]. Many students believe that doing well at a task or field is innate, while others see it as a feature of how hard one works. The idea of stereotype threat has been put forward as an explanation (e.g., ). Research has demonstrated that how a person believes the society thinks they will succeed in a field can affect their performance. Finally, beliefs about how effort contributes to success also affect performance (e.g., ). It is likely that all of these explanations contribute to the result of many students not passing developmental mathematics. In this paper, we examine an additional potential contributing factor: whether the content assigned and completed could prepare a student to succeed in higher level college mathematics courses. The size of our dataset provides us a novel lens on this question (N approximately 40,000 college students). We conducted an exploratory study of the role of content in preparation by investigating the content assigned by instructors and the degree to which this content was completed by students. We then use data visualizations to compare the structure of course content between developmental math and intermediate math courses. 3. DATA SOURCES AND MATERIALS Participants Data were collected from an online course management site for college mathematics courses developed by an educational textbook company. All postsecondary instructors who use one of the textbooks included in the site have access to it. Our dataset is comprised of a random selection of courses from the and school years. It includes 37,468 students in 1,603 university courses at 276 institutions. The dataset only includes information related to course activities (i.e., demographic information was not available). There was no compensation associated with this study. Instructors and students participated with the online course management site as normal. Materials The online course management site allows instructors to select assignments for their students, and provides students with a platform for completing homework assignments and taking quizzes and tests. For our analyses of student completion, we focus on the math exercises (called items) that appear on homework assignments, because this provides the best view of student-driven content coverage throughout a course. For our analyses of course content coverage, we focus on the items that appear on exams, because this provides the best view of instructor-driven content progression. Procedure Assignment of students and teachers into classes was not controlled, but the result of college registration. Participants interacted with the online course site as they normally would. 4. DATA ANALYSIS AND RESULTS Variables The following variables were coded for use in this study: Items. We define an item as a math exercise assigned to a student to complete as part of their homework (in Analysis 1) or on an exam (in Analysis 2). In the context of homework problems, students can repeat items to aid in practice and learning. Skipped items. Items that appear on a homework assignment for a course, but which a student in that course skipped. These items correspond to a topic in the course. Topic. The mathematical content area of an item, as defined by the book publisher. Analysis 1 We begin by creating graphs to visually depict the items that instructors have assigned, yet students have skipped. We see that in general, all topic areas have a high skip rate. One learning objective has 100% skip rate, for extra practice problems that appear in an Appendix of the textbook. The data for 45 topics appearing in our analysis is given in Table 1. Based on this analysis, we see that the topics with the lowest skip rate are: Real Numbers and Algebraic Expressions (38%) Review of Real Numbers (44%) Exponents, Polynomials, and Polynomial Functions (48%) Multiplying and Dividing Fractions (51%) The topics with the highest skip rate are: Appendix: Additional Geometry Questions (100%) Study Skills (82%) Quadratic Equations (82%) Roots and Radicals (80%) Further Algebraic Topics (77%)
3 Topic Items Items % Assigned Skipped Skipped Appendix: Additional Geometry Questions % Study Skills % Quadratic Equations % Roots and Radicals % Further Algebraic Topics % Geometry and Measurement % Ratio, Proportion, and Triangle Applications % Solving Systems of Linear Equations % and Inequalities Factoring Polynomials % Rational Expressions % Statistics and Probability % Fractions and Mixed Numbers % Measurement % Introduction to Algebra % Graphing and Introduction to Statistics % Percent % Appendices % Geometry % Decimals % Sequences, Series, and the Binomial Theorem % Solving Equations and Problem Solving % Exponents and Polynomials % Rational Exponents, Radicals, % and Complex Numbers Quadratic Equations and Functions % Integers and Introduction to Solving Equations % Math-Reading Connections % Systems of Equations % Further Geometric Topics % Graphing % Signed Numbers % Graphing Equations and Inequalities % Conic Sections % Ratio and Proportion % Adding and Subtracting Fractions % Exponential and Logarithmic Functions % The Whole Numbers % Orientation Questions for Students % Equations, Inequalities, and Problem Solving % Real Numbers and Introduction to Algebra % Graphs and Functions % Multiplying and Dividing Fractions % Exponents, Polynomials, % and Polynomial Functions Review of Real Numbers % Real Numbers and Algebraic Expressions % Table 1: Items Assigned and Skipped by Topic Across All Students
4 This analysis suggests that the topic areas that are easier and appear early in the course are skipped less by students. Therefore, we extend our analysis to investigate the coverage of topics in developmental math and compare this to the coverage of topics in intermediate math. Analysis 2 We next examine Developmental Math and Intermediate Math courses delivered online. The data for developmental math included 354 courses, and the data for intermediate math included 306 courses. As stated earlier, to investigate the coverage of content across courses, we focus on items that appear on exams. The first heat map in Figure 1 provides a graphical representation of data noting areas of topic delivery, with highest concentration in purple, and shows the topical content covered in developmental math across all 354 courses (based on the data available for analysis). Note that the x-axis is time, with the beginning of the course at the left. The y-axis is course content, with book chapters progressing from 1 to 12. Based on this heat map, we see that content coverage increases as the course progresses, but that early topics are never left behind and appear on exams throughout the course. Figure 2: Test Item Content Coverage: Intermediate Math These heat maps suggest there are differences in the way developmental math and intermediate math courses are delivered in universities and colleges. 5. SIGNIFICANCE Research suggests that the way in which developmental mathematics is currently taught may not be appropriate for some students. Our exploratory analyses suggest that this may be for two reasons. First, students appear to be skipping exercises in advanced topic areas. Second, instructors appear to be dwelling on early topic coverage, which may take time away from learning more advanced content as the course progresses. This focus on basic mathematics topics and less coverage of advanced topics could be a strong contributor to why students are not prepared to succeed when they progress to more advanced courses, even if they pass their developmental mathematics course. Figure 1: Test Item Content Coverage: Developmental Math The second heat map, shown in Figure 2, shows the topical content covered in intermediate math across all 306 courses (based on the data available for analysis). Again, the x-axis is time, with the beginning of the course at the left. The y- axis is course content, with book chapters progressing from 1 to 12. Based on this heat map, we see some differences from the developmental math course. First, we see the use of pre-tests (covering content in chapters 1-8), indicated by the data markings in a line at the left, which broadly cover content up to the 8 on the y-axis. We also see that while some early content is still covered in the courses, there is much less emphasis placed on early content, and once mastered, the courses move on to more advanced topics. Early content is covered again at the end of the course, probably on a comprehensive final. This may suggest interventions, such as additional encouragement to complete exercises as the content becomes more difficult, or recommender systems for instructors for how to cover the more difficult material. Future research should investigate the pattern of topic assignment to determine if this is due to the lack of mastery early in the course or to instructors preferences to continually review material. References  T. Bailey. Challenge and opportunity: Rethinking the role and function of developmental education in community college. New Directions for Community Colleges, 2009(145):11 30,  A. Bandura. Perceived self-efficacy in cognitive development and functioning. Educational psychologist, 28(2): ,  E. P. Bettinger, A. Boatman, and B. T. Long. Student supports: Developmental education and other academic programs. The Future of Children, 23(1):93 115, 2013.
5  K. F. Butcher and M. G. Visher. The impact of a classroom-based guidance program on student performance in community college math classes. Educational Evaluation and Policy Analysis, 35(3): ,  C. S. Dweck. Self-theories: Their role in motivation, personality, and development. Psychology Press, Philadelphia,  M. Eppler, C. Carsen-Plentl, and B. Harju. Achievement goals, failure attributions, and academic performance in nontraditional and traditional college students. Journal of Social Behavior and Personality, 15(3): ,  W. N. Grubb and R. D. Cox. Pedagogical alignment and curricular consistency: The challenges for developmental education. New Directions for Community Colleges, 2005(129):93 103,  E. A. Linnenbrink and P. R. Pintrich. Motivation as an enabler for academic success. School Psychology Review, 31(3),  C. M. Steele. A threat in the air: How stereotypes shape intellectual identity and performance. American psychologist, 52(6):613,  J. W. Stigler, K. B. Givvin, and B. J. Thompson. What community college developmental mathematics students understand about mathematics. MathAMATYC Educator, 1(3):4 16, 2010.
THE CARNEGIE FOUNDATION FOR THE ADVANCEMENT OF TEACHING Problem Solution Exploration Papers WHAT COMMUNITY COLLEGE DEVELOPMENTAL MATHEMATICS STUDENTS UNDERSTAND ABOUT MATHEMATICS James W. Stigler, Karen
U.S. Postsecondary Faculty in 2015 Diversity In People, Goals And Methods, But Focused On Students January 2015 Preface A confluence of social, technical, economic, and other factors have created the demand
Table of Contents THE MASTERY LEARNING MANUAL CHAPTER 1 INTRODUCTION TO MASTERY LEARNING Introduction by Carla Ford 1 Perspective on Training: Mastery Learning in the Elementary School 2 CHAPTER 2 AN OVERVIEW
Toward a More Comprehensive Conception of College Readiness 2007 David T. Conley Prepared for the Bill & Melinda Gates Foundation, March 2007 Table of Contents Introduction. 5 An Operational Definition
Bachelor of Science in Business Human Resource Management The Bachelor of Science in Business Human Resource Management is a competency-based program that enables students to earn a Bachelor of Science
Bachelor of Science in Business Management The Bachelor of Science in Business Management is a competencybased program that enables leaders and managers in organizations to earn a Bachelor of Science degree.
New Approaches to Online Accounting Education Mark Myring, Ph.D. Department of Accounting Miller College of Business Ball State University email@example.com 765-285-5100 Jennifer P. Bott, Ph.D., SPHR Department
Bachelor of Science in Marketing Management The Bachelor of Science in Marketing Management is a competencybased program that enables marketing and sales professionals to earn a Bachelor of Science degree.
Bachelor of Science in Accounting The Bachelor of Science in Accounting is a competency-based program that enables professionals in accounting to earn a Bachelor of Science degree. The Accounting degree
Final report Research into the impact of Project Maths on student achievement, learning and motivation Jennifer Jeffes Emily Jones Máirín Wilson Emily Lamont Suzanne Straw Rebecca Wheater Anneka Dawson
Master of Arts in Mathematics Education for Grade 5 12 Teachers The Master of Arts in Mathematics Education (5 12) is a competencybased degree program that prepares already licensed teachers both to be
The Accuracy and Validity of Online Homework Systems by Kathleen Malevich MS Candidate: Program in Applied and Computational Mathematics Advisor: Carmen Latterell Department of Mathematics and Statistics
common core state STANDARDS FOR Mathematics Appendix A: Designing High School Mathematics Courses Based on the Common Core State Standards Overview The (CCSS) for Mathematics are organized by grade level
RESEARCH OVERVIEW / APRIL 2013 What We Know About Online Course Outcomes Online Higher Education Is Expanding Rapidly Since 2010, online college course enrollment has increased by 29 percent. Currently,
PREDICTORS OF SUCCESS FOR COMMUNITY COLLEGE DEVELOPMENTAL MATHEMATICS STUDENTS IN ONLINE, HYBRID, AND TRADITIONAL COURSES A Dissertation by GEORGE HARRISON HENDRICKS, III Submitted to the Graduate School
AGENCY/PHOTOGRAPHER Barriers to College Attainment Lessons from Chicago Jenny Nagaoka, Melissa Roderick, and Vanessa Coca, The Consortium on Chicago School Research at The University of Chicago January
Sahin A., Cavlazoglu, B., & Zeytuncu, Y. E. (2015). Flipping a College Calculus Course: A Case Study. Educational Technology & Society, 18 (3), 142 152. Flipping a College Calculus Course: A Case Study
Criminal Justice 207: Program Evaluation Research Comparison of CCE and Ground Programs Spring 2014 Project Director/Instructor: Yvette Farmer, Ph.D. Graduate Students (in alphabetical order): Margaret
UNDERGRADUATE PROGRAM SELF-STUDY BACHELOR OF ARTS DEGREE/ BACHELOR OF SCIENCE DEGREE IN PSYCHOLOGY Prepared by: The Undergraduate Program Committee Email Dr. Shannon Scott at SScott@twu.edu February, 2012
Promoting Success In Online College Courses: Understanding Interactions Between Course Structure And Learner Readiness Cathleen A. Kennedy University of California, Berkeley Presented at the Annual Meeting
Journal of Information Technology Education Volume 8, 2009 A Comparison of Student Perceptions of their Computer Skills to their Actual Abilities Donna M. Grant, Alisha D. Malloy, and Marianne C. Murphy
The Connecticut Plan: Academic and Personal Success for Every Middle and High School Student SECTION ONE: Organizing Concepts Reform of Connecticut s high schools was first identified as a major policy
BSc (Hons) in Applied Accounting and Research and Analysis Project INFORMATION PACK Submission Periods 29 & 30 2014-2015 CONTENTS 1) About Oxford Brookes University... 1 2) Eligibility... 2 3) Conversion
Creativity in Online Courses: Perceptions of MBA Students Alma Mintu Wimsatt Professor of Marketing Texas A & M University Commerce Commerce, TX, USA alma_wimsatt@tamu commerce.edu Theresa Sadler Instructor
Teacher Professional Development: It s Not an Event, It s a Process Sandra H. Harwell, Ph.D. Vice President, Professional Development CORD 2003 CORD Published and distributed by: CORD P.O. Box 21689 Waco,
Educate~ Vol.7, No.1, 2007, pp. 39-47 Research Paper The relationship between goals, metacognition, and academic success by Savia A. Coutinho (firstname.lastname@example.org) Northern Illinois University, United States
Findings from the Michigan School Readiness Program 6 to 8 Follow Up Study October, 2007 Elena Malofeeva, Marijata Daniel-Echols, and Zongping Xiang High/Scope Educational Research Foundation 600 North
Learning Outcomes of Information Literacy Instruction at Business Schools Brian Detlor DeGroote School of Business, McMaster University, 1280 Main Street West, Hamilton, ON, Canada, L8S 4M4. E-mail: email@example.com
Best and Innovative Practices in Higher Education Assessment April 2013 In the following report, Hanover Research investigates innovative practices in higher education assessment. Trends and future directions
Are Middle School Counseling Programs Meeting Early Adolescent Needs? A Survey of Principals and Counselors Suzanne McCotter & Sarah Cohen This article explores the needs of middle school students in regards