1 Assessing the Costeffectiveness of Online and Blended Learning PACE Seminar December 6th, 2012 Fiona Hollands, Ph.D. Center for Benefit-Cost Studies of Education Center for Technology and School Change Teachers College, Columbia University
2 Overview of Presentation What do we mean by cost-effectiveness of online/blended learning? Review of existing research on costs of online/blended learning Cost considerations specific to online/blended learning School of One, blended learning math program for middle school students, as an example to demonstrate ingredients approach to costing programs
3 A definition of cost-effectiveness What do people mean when they claim that something is costeffective? Given a target objective, e.g., increasing by 10% the # of students meeting state math standards: o What is the lowest cost alternative (educational program/intervention) to achieving the 10% objective? o OR o If the budget is already fixed, which affordable alternative is expected to get the highest percentage of students to meet the math standards?
4 For online/blended learning the two parts to the cost-effectiveness question are usually Compared with traditional face-to-face (f2f) classroom teaching: o 1) Does online/blended learning cost less, the same or more? o 2) Is online/blended learning more, less or just as effective in promoting a specified outcome?
5 For online/blended learning interventions, costs can be separated into: Up-front development costs vs. running costs Replication costs experienced by sites adopting the intervention o Prerequisites, e.g., wireless connectivity o New costs, e.g., licensing fees o Fixed vs. variable costs o Gross vs. marginal costs
6 Research on costs of online/blended education Bates (2005) estimated the cost of an online UBC course at around US$660 per student or $6,600 for an MA degree. No recent higher ed. cost studies or comparison with f2f. Anderson et al. (2006) estimated costs of K-12 virtual schooling to be about the same as regular brick and mortar schools when similar services are being provided, excl. transportation and capital costs.
7 Research on costs of online/blended education Cavanaugh - estimated average costs for fulltime students at K-12 virtual schools in 2008 at $4,310 per student vs. average per pupil expenditure in 2006 of $9,138 (NCES). Battaglino, Haldemann, & Laurans (2012) o K-12 Virtual schools costs: $5,500 - $7,100 vs. $10,000 traditional schools o Blended learning costs: $7,600 $10,200.
8 Cost drivers for online/blended learning Battaglino, Haldemann, & Laurans (2012) Labor Content development and/or acquisition Technology and Infrastructure School Operations Student Support Services
9 School of One (So1) So1 is a blended learning math program for 6th - 8th graders developed by the NYC DoE Students spend 70 minutes a day learning math in a variety of modalities both virtual and live So1 computer program or Learning Algorithm tracks individual student progress, determines what skills (s)he has mastered and what still needs work, and creates a daily playlist of math activities for each individual student.
10 Development costs for So1 estimated at $8mm over 2 years Technology devpt. costs for Learning Algorithm and capacity to house 5,000 math lessons that can be completed and assessed online Estimated cost $5 mm over 2 years Curriculum devpt: Panel of math experts to develop math skills map and review 25,000 lessons Estimated cost $150,000 Content: 5,000 math lessons purchased from 50 different vendors and adapted for So1 system. Estimated cost $0.5mm Team of 12 educational technology professionals working with software developer to develop system and interface with schools Estimated cost $1.2mm/yr ($2.4mm total)
11 Prerequisite resources at replication sites: estimated costs per annum Math teachers 4 math sections (25-33 students each) operating at once with 4 certified teachers plus 2 student teachers o Estimated cost $380,000 (expect to be lower outside of NYC) Wireless connectivity: $50,000 for entire school amortized over 5 years Estimated cost $10,000 access for entire school Estimated cost $25,000
12 Estimated replication costs Assumes 6 teachers, pre-existing wireless connectivity, for all students cost amortized over 5 years cost amortized over 3 years Construction Initial p.d. Digital content manager/tech support Hardware So1 licensing Ongoing p.d. TOTAL Virtual tutor option Cost per annum for school of 480 $40,000 $8,000 $80,000 $36,667 $150,000 $10,000 $324,667 $324,000 Cost per student $83 $17 $167 $76 $313 $21 $677 $675 TOTAL with virtual tutors $648,667 $1,352
13 Is So1 Cost-Effective? Several studies of effectiveness have been conducted. None have conclusively shown that academic achievement for students using So1 is better than for comparison students experiencing traditional f2f teaching study by Cole, Kemple & Segeritz Comparing So1 with traditional teaching, the extra cost of $1,352 per student above and beyond existing school costs, is not an efficient use of resources for improving academic achievement.
14 Are online college level courses cost-effective? No published documentation on costs of a specific online/blended program/course compared directly with f2f. Obom & Cummings (2007): Johns Hopkins online students learned content as well as f2f students. Courses not cheaper than f2f due to fixed costs such as buildings/faculty (Falk, 2012 pers. comm). Costs per student can be lower if scale can be increased significantly (MOOCs) But if effectiveness drops due to lower completion rates/ less learning, may not be cost-effective What is the desirable outcome: greater access or greater achievement?
15 Conclusions Significant cost savings are possible when online learning is used to replace f2f instruction, due to o o o increasing student/teacher ratios elimination of non-instructional services faster completion of same content/skills (e.g. online credit recovery) In some cases the costs are just being deferred elsewhere, e.g., families, colleges. Little documentation of effectiveness of K-12 online/blended learning wrt improving academic outcomes Little documentation of costs of specific interventions
16 Useful readings Anderson, A., Augenblick, J., DeCesare, D., & Conrad, J. (2006). 20/20 Costs and Funding of Virtual Schools: Augenblick, Palaich & Associates. Retrieved from: Bates, A.W. (2005). Technology, E-learning and Distance Education (2nd ed.). New York: Routledge. Battaglino, T.B., Haldemann, M., & Laurans, E. (2012) The Costs of Online Learning Washington DC: Thomas B. Fordham Institute. Retrieved from: Cavanaugh, C. (2009). Getting Students More Learning Time Online: Distance Education in Support of Expanded Learning Time in K-12 Schools Washington, DC. Center for American Progress. Retrieved from:
17 Useful readings Cole, R., Kemple, J.J., & Segeritz, M.D. (2012) Assessing the Early Impact of School of One: Evidence from Three School-Wide Pilots. The Research Alliance for New York City Schools. Retrieved from: Levin, H.M., & McEwan, P. J (2001). Cost-effectiveness analysis: Methods and Applications. (2 nd ed.) Thousand Oakes, Sage Publications. Rice, J.K. (2012). Review of the Costs of Online learning. National Education Policy Center. Retrieved from: U.S. Department of Education, Office of Educational Technology. (2012) Understanding the Implications of Online Learning for Educational Productivity, Washington, D.C. Retrieved from:
B l e n d e d Learning B l e n d e d Learning B l e n d e d Learning the best of both worlds How the integration of face-to-face and digital learning can lead to greater educational equity, opportunities,
UMUC S ONLINE MBA PROGRAM: A CASE STUDY OF COST-EFFECTIVENESS and the IMPLICATIONS FOR LARGE-SCALE PROGRAMS Tana Bishop and Claudine SchWeber University of Maryland University College "Ingredients" approach
Building Multiple Pathways to a High School Diploma: A Cost Study of Non-Traditional Academic Options Overview Over the last two decades, Massachusetts has emerged as a national and global education leader.
Putting a Price Tag on the Common Core: How Much Will Smart Implementation Cost? By Patrick Murphy and Elliot Regenstein with Keith McNamara Foreword by Chester E. Finn, Jr. and Amber M. Winkler May 2012
MOOCs: Expectations and Reality Full Report Fiona M. Hollands, Ph.D. Devayani Tirthali, Ed.D. Center for Benefit-Cost Studies of Education Teachers College, Columbia University Table of Contents Acknowledgements...
Online Learning Needs in Technology Education Jim Flowers Introduction The number of distance education courses, degree programs, and enrollment in the US nearly doubled from 1995 to 1998 (US Department
Kent State University s Cloud Strategy Table of Contents Item Page 1. From the CIO 3 2. Strategic Direction for Cloud Computing at Kent State 4 3. Cloud Computing at Kent State University 5 4. Methodology
Credit Recovery INFORMATIONAL BRIEF: A look at credit recovery programs across the United States Prepared by the New York Comprehensive Center Educational Technology Team July 2012 Maxwell Mileaf Anushka
The State of Online Learning in California: A Look at Current K-12 Policies and Practices September 54 2006 The State of Online Learning in California: A Look at Current K-12 Policies and Practices September
June 2008 Promising Practices in online learning Using Online Learning for At-Risk Students and Credit Recovery PROMISING PRACTICES IN ONLINE LEARNING Using Online Learning for At-Risk Students and Credit
Evaluating the Effectiveness of Technology in Our Schools ACT POLICY REPORT RICHARD J. NOETH BORIS B. VOLKOV EVALUATING THE EFFECTIVENESS OF TECHNOLOGY IN OUR SCHOOLS ACT Policy Report Richard J. Noeth
IDB WORKING PAPER SERIES No. IDB-WP-476 The Effects of Shared School Technology Access on Students Digital Skills in Peru Germán Bet Julián Cristia Pablo Ibarrarán January 2014 Inter-American Development
Journal of Teacher Education, Vol. 54, No. 4, September/October 2003 10.1177/0022487103255985 ARTICLE EXAMINING TEACHER TECHNOLOGY USE IMPLICATIONS FOR PRESERVICE AND INSERVICE TEACHER PREPARATION Michael
The costs of charter and cyber charter schools Updated January 2014 Research and policy implications for Pennsylvania school districts Education Research & Policy Center 400 Bent Creek Blvd., Mechanicsburg,
The Relationships Between School Poverty and Student Achievement in Maine David L. Silvernail James E. Sloan Chelsea R. Paul Amy F. Johnson Erika K. Stump Maine Education Policy Research Institute University
February 2010 Promising Practices in online learning A Parent s Guide to Choosing the Right Online Program PROMISING PRACTICES IN ONLINE LEARNING A Parent s Guide to Choosing the Right Online Program Written
Online Learning Tools and Technologies Tony Bates, President and CEO, Tony Bates Associates Ltd, Vancouver, B.C., Canada Abstract Online learning has become the standard form of delivery for distance education
Why Do Some Schools Get More and Others Less? An Examination of School-Level Funding in New York City Amy Ellen Schwartz Institute for Education and Social Policy Wagner and Steinhardt Schools New York
February 2010 PROmisinG PRACtiCes in Online learning A Parent s Guide to Choosing the Right Online Program PROMISING PRACTICES IN ONLINE LEARNING A Parent s Guide to Choosing the Right Online Program Written
More than Robots: An Evaluation of the FIRST Robotics Competition Participant and Institutional Impacts Prepared by: Alan Melchior, Faye Cohen, Tracy Cutter, and Thomas Leavitt Center for Youth and Communities
National Charter School Study 2013 National Charter School Study 2013 Edward Cremata, M.A. Technical Appendix Devora Davis Original 16-State Update Kathleen Dickey, M.S. Database Manager Kristina Lawyer,
Online Students: Relationships between Participation, Demographics and Academic Performance J. Coldwell, A. Craig, T. Paterson and J. Mustard Deakin University, Geelong, Victoria, Australia email@example.com
State-Funded PreK Policies on External Classroom Observations: Issues and Status Policy Information Report This report was written by: Debra J. Ackerman Educational Testing Service Author contact: firstname.lastname@example.org
ONLINE K-12 SCHOOLING IN THE U.S. UNCERTAIN PRIVATE VENTURES IN NEED OF PUBLIC REGULATION Gene V Glass and Kevin G. Welner University of Colorado Boulder October 2011 National Education Policy Center School
Online Learning at Public Universities: Building a New Path to a College Degree A joint project of The Learning House, Inc. and the American Association of State Colleges and Universities Online Learning
Teaching in the World of Virtual K 12 Learning Challenges to Ensure Educator Quality July 15, 2011 Teaching in the World of Virtual K 12 Learning: Challenges to Ensure Educator Quality A final report prepared
State Test Score Trends Through 2007-08, Part 6 Has Progress Been Made in Raising Achievement for English Language Learners? Center on Education Policy April 2010 Has Progress Been Made in Raising Achievement
Implementing Online Learning Labs in Schools and Districts: Lessons From Miami-Dade s First Year November 2011 S Developed by SRI International with funding from the Bill & Melinda Gates Foundation. This