California elearning Census: Increasing Depth and Breadth

April 2014 California elearning Census: Increasing Depth and Breadth California Learning Resource Network 1100 H Street Modesto, CA 95354 Brian Bridges 209.238.1420 Fax: 209.238.4223 A Statewide Education Technology Service of the California Department of Education Administered by the Stanislaus County Office of Education

Executive Summary! The California Learning Resource Network (CLRN,) a statewide education technology service of the California Department of Education and administered by the Stanislaus County Office of Education, began conducting the California elearning Census in 2012 to better understand how full-time virtual and blended learning are evolving in California. On February 1st, 2014, CLRN distributed the survey to 1014 California public school districts and 796 direct-funded charters. We received 569 responses accounting for 31% of the 1810 districts and charters. 53% of districts and direct-funded charters reported having students participate in virtual or blended learning, up from 46% in 2013. 21% reported they were discussing or planning to implement online or blended learning. The census counted 174,632 virtual and blended students in 2013-2014, a 39% increase since last year. While the virtual student population has remained stable since 2012, the number of blended students skyrocketed this year, increasing 49% since 2013 and 74% since 2012. The blended median population rose 80%. While the top 25% of districts and charters continue to contribute a significant proportion of the total elearning population, the three-year trend lines show that elearning adoption is broadening across a greater percentage of schools and that the number of participating students at each school is steadily rising. elearning adoption varies between charter schools and school districts as well as between elementary and high schools. While the 2012 census indicated adoption was fairly consistent between each of these groups, the 2014 census found that 60% of charter schools embraced virtual and/or blended learning as compared with 48% of traditional districts. However, traditional districts account for the majority (67%) of California s blended learning population, while charter schools contribute 82% to the virtual population. While the blended learning population grew 49% this year, the majority of that growth has occurred in charter schools. Since 2012, blended learning in traditional districts has grown 43% while charters have experienced a 287% increase. Although blended learning activity is rapidly increasing, each type of institution seems to rely on specific models. Within elementary districts, Rotation is the dominant choice with 76% of districts, while the other three models languish around the 10% level. In high school and K-12 districts and charters, A La Carte and Enriched Virtual are the preferred choice by 59% and 53% of all districts. A La Carte dominates in districts while Enriched Virtual dominates in charter schools. However, Rotation and Flex seem to be more popular in traditional districts with 33% and 37% penetration. Districts and charters are becoming more sophisticated when selecting courseware. While their predominate criteria is price (79%), districts are investigating an increasing number of publishers and sampling a greater number of courses before purchasing. In 2014, 56% of districts and charters utilized multiple publishers, compared with 46% last year. Asked what they would do differently if they could start over, 17% would be more systemic in their initial planning, 33% would have been much more careful in their course selection, and 11% of districts wished they had prepared their staff better for the transition into online or blended learning. An increasing number of districts and charters are providing professional development to their online and blended teachers, averaging 26 hours. However, the content provided to teachers primarily focuses on the technical aspects of the learning management system and not the pedagogical aspects of teaching and learning online. Finally, 58% of districts and charters feel their virtual and blended programs have resulted in greater student engagement and half reported increased course completion rates. California elearning Census: Increasing Depth and Breadth Page 2

Introduction The California Learning Resource Network (CLRN), a statewide education technology service of the California Department of Education and administered by the Stanislaus County Office of Education, reviews online courses, open educational resources (OER) and supplemental electronic learning resources for their alignment to the Common Core State Standards and to California s other content standards. CLRN certifies online courses for their alignment to both the content standards and to inacol s Standards for Quality Online Courses. CLRN created the California elearning Census in 2012 to better understand how full-time virtual and blended learning are evolving in California. Beginning on February 1 st, CLRN contacted 1014 California public school districts and 796 direct-funded charters, which are charter schools that receive their allocation of state funds directly from the state. The survey asked superintendents and principals about online and blended learning enrollments, blended learning models in use, professional development, and course selection. This report reviews CLRN s California elearning Census data and makes comparisons to the 2012 and 2013 censuses. Survey questions are provided in Appendix A and CLRN s description, mission, and partnership with the University of California may be found in Appendix B. The author thanks Michael Horn and Heather Staker from the Clayton Christensen Institute for their input as well as John Watson and Amy Murin from the Evergreen Education Group for their suggested additions to the census question set. California elearning Census In response to elearning s dramatic growth and in an effort to promote high-quality online and blended learning, CLRN collaborated with the Evergreen Education Group to develop the California elearning Census. CLRNS original goals were to: 1. Collect accurate virtual and blended learning populations 2. Report the blended models in use 3. Understand the distribution of course publishers in California 4. Inform the California state profile in Keeping Pace 2014 On February 1, 2014, CLRN sent an initial letter to superintendents of K-12 public school districts and direct-funded charters in California (N = 1810) containing the purpose of the census with an invitation to participate. The letter contained a link to the electronic survey site and contact information. Respondents were given until April 1, 2014 to complete the electronic survey. Follow-up emails were sent to superintendents every two weeks. There were a total of 569 responses for a 31% response rate. California elearning Census data may be published in the California profile of the 2014 Keeping Pace with K-12 Online Learning: An Annual Review of Policy and Practice, which will report both elearning trends and student participation from each state, in October 2014. California elearning Census: Increasing Depth and Breadth Page 3

Definitions While CLRN partnered with the Evergreen Education Group to create the census question set, Michael Horn and Heather Staker participated by sharing their updated blended learning definitions prior to publication. The census also invited districts and charters to share their blended learning program with the Clayton Christensen Institute. Full-Time Virtual School- Students take all their courses online away from school. Students do not visit a physical campus, except on a limited basis. Also referred to as Online Learning. Blended Online learning- Online learning that typically takes place at school, where students have some control over time, place, path and/or pace. elearning- For this report, elearning refers to any form of virtual, online, or blended learning. Rotation model- Students rotate, on a fixed schedule in a course, between learning online and learning from a face-to-face teacher. Rotation includes teachers who Flip their class. To count use of supplemental and/or Internet resources as blended, students must rotate between them and a classroom on a fixed schedule within an individual course. Flex model- Students take a majority of their courses online at school in an individually customized, fluid schedule and on-site teachers or paraprofessionals provide support. A La Carte model- Students choose to take one or more online courses to supplement their schedules and the teacher of record is online. Note: As the 2013 census concluded, the Clayton Christensen Institute, which created the blended learning definitions, changed the Self-Blend model to A la Carte. Enriched Virtual model-independent study students take all their online courses at home but visit a physical campus to meet with a teacher. What is not virtual or blended learning-participation in supplemental electronic activities or technology-rich activities that don t fit the previous definitions. California elearning Census: Increasing Depth and Breadth Page 4

Findings CLRN created the California elearning Census in 2012 to discover the enrollments in online and blended learning in K-12 public school districts and direct-funded charters and conducted the census in 2012 through 2014. A total of 569 districts and direct-funded charters responded this year, representing 31% of the 1810 districts and charters. By contrast, participation rates were 30% in 2012 and 29% in 2013. This year, 53% indicated their students were learning online (see Figure 1), a dramatic uptick from last year s census (46%). Additionally, 21% of districts and directed funded charters who were not learning online indicated they were discussing or planning to implement online learning. Because online and blended learning are being adopted at different rates and in different ways in elementary and high schools, we ve disaggregated data between elementary districts and charters (K-5 and K-8) as well as between unified and high school districts and charters (K-12 and 9-12). Growth occurred in two areas: 1) the second and third quartiles and 2) charter schools. From Figure 16 on page 14, notice that in 2012, 25% of districts and charters (the top quartile) accounted for 90% of the elearning population. By 2014, the top quartile contributed 83% of the total while the second and third quartiles increased in quantity. With the 80% increase this year in median populations for blended students, it seems clear that districts and charters are becoming more comfortable with elearning and are expanding their offerings. That 56% of districts are purchasing courseware from at least two publishers, compared with 46% in 2013, seems to confirm that districts and charters are increasing their elearning participation. N=569 responses Yes No Are you a direct-funded charter? 40% 60% Do students in your district or direct-funded charter school learn online? 53% 47% Is your charter or district currently discussing or planning to implement online learning? 21% 79% Figure 1: Select questions from California elearning Census, 2014. Note: Not all respondents answered all questions so the % may not exactly be a % of N. Districts/Charters Currently elearning Districts/Charters Planning to elearn Figure 2: Districts and direct-funded charters that are participating in online or blended learning. Figure 3: Districts and direct-funded charters who are discussing or planning to implement online or blended learning California elearning Census: Increasing Depth and Breadth Page 5

Virtual and Blended Learning by Grade Span elearning involvement continues to expand within all grade levels. While 53% of all districts and charters are involved in elearning, virtual and blended learning are adopted at different rates and in different modes at elementary and unified districts. In 2012, 16% of elementary districts reported students were learning online, increasing to 19% and 28% this year. However, in unified and high school districts, 78% learned online in 2014, up from 68% in 2012. Figure 4: Districts and direct-funded charters indicating they are participating in online or blended learning. Online and Blended Learning Adoption by Institution elearning adoption varies between charter schools and school districts as well as between elementary and high schools. While the 2012 census indicated adoption was fairly consistent between the two, the 2013 census indicated a divergence with 53% of charters compared to 44% of districts supporting elearning. That gap has continued to widen in 2014 with 60% of charter schools embracing online and/or blended learning as compared with 48% of traditional districts. Figure 5: Online and blended learning adoption, comparing districts with direct-funded charters. California elearning Census: Increasing Depth and Breadth Page 6

Online and Blended Learning Population by Institution However, traditional school districts account for the majority (67%) of California s blended learning population, while direct-funded charter schools contribute 82% of the full-time virtual, or online learning population. Blended learning still occurs at charter schools and districts contribute to the full-time virtual population but we re uncertain whether the different rules afforded charter schools account for their stronger virtual population. Figure 6: Online and blended learning adoption, comparing districts with direct-funded charters. Planning for Online or Blended Learning If districts and direct-funded charters answered that they were not participating in online or blended learning, we asked if they were currently discussing or planning to implement it in the future. Overall, 21% shared that they were in the planning stages. However, K-12 and 9-12 districts and charters are more engaged in investigating online learning than elementary districts. Just 13% of K-8 districts and charters that are not elearning, say they are planning to implement it, while 44% of the unified and high school districts and charters are planning to implement online and blended learning. Figure 7: Districts and direct-funded charters indicating they are discussing or planning to implement online or blended learning, disaggregated by grade span. California elearning Census: Increasing Depth and Breadth Page 7

Planning Comments District and direct-funded charters who responded they were currently discussing or planning to implement online learning were asked to share any programs or models they are planning to pilot or implement. The comments below provide further insight into the elearning s evolution and districts comments related to their research, planning, piloting, and implementation. Planning Discussing the benefits of using a lab to share the teaching/learning experience to improve instruction Looking at blended learning for credit recovery only. Looking at programs that focus on blended learning. Considering an enriched virtual model for some courses in the upcoming 2014-15 school year. Our continuation high school is looking in to capturing more attendance for those students that are over 18 but still do not have a high school diploma. Implementation expected for the 2014-15 school year. Perhaps as a form of intervention, tutorials or credit recovery. Teachers at the middle school level have discussed using the Flipped Model classroom. However, it is in its very early stages of implementation. Teachers are exploring models and working out the details. It is very possible that implementation will not happen until next year. There are some early discussions about our alternative high school having an online or blended component. This is far enough down the road that we have not as yet discussed details. We are currently examining the benefits of having some of our alternative students register for online classes (high school and/or community college) to supplement their high school course work taken in the community school setting. We are discussing the implementation of a school wide blended model for learning. We are exploring a limited pilot for one school in the future. We are in the initial stages and have not determined what models to consider just yet. We are looking at options for our GATE students (Stanford EPGY) to be able to participate in advanced learning opportunities via the Internet, and possibly at interventions for our struggling students. We are looking into enrichment and remediation software in both math and literacy for 2014-2015. We may also consider offering full online World Language courses for our middle school students. We are researching online courses for independent studies, and Rosetta Stone for language. We currently use DreamBox as a supplement and are investigating other options. We have not found a model to pilot, but the discussion is in its infancy We have not selected a specific program yet. We would like to implement a blended learning program. We are considering to provide laptops to all or students and provide online curriculum in all core subject areas. We will still follow a traditional schedule and most of the instruction will be delivered by teachers in the classroom. Classroom assignments, homework, assessments and interventions will mostly be delivered online. Models Planned to Pilot or Implement Blended Online Learning-Rotation-Flipping the classroom. We will be encouraging more of our teachers to implement "Flipping the Classroom". Contingent on ability to identify accessible software for elearning for students with severe visual impairments and the ability to purchase hardware and software. California elearning Census: Increasing Depth and Breadth Page 8

Currently, we have teachers piloting some home-grown flipped learning projects for middle school math, while others are using Google Docs as blended learning tools, Edmodo as an LMS, and Google Sites. We do not yet have plans to take blended learning to the next level-- that is, to formalize these elements as official, regular and ongoing and systemic components of coursework. Online learning - pilot for Fall 2014 Salida USD has three K - 5 campuses, one 6-8 campus and one K-8 Independent Study Charter program. The independent study charter program would fall under the category of Enriched Virtual Learning - two classes are offered online HSS and Science. Study island in a blended model of home instruction We are implementing blended learning in classrooms. We are at the very beginning stages. We are in the process of launching our LMS, Desire2Learn. We currently have a 1:3 computer to student ratio. We have a variety of online learning activities infused in our education. We can be called a blended learning school with the amount of technology and the amount of use. We do not utilize "full-time" but we do use Rosetta Stone for foreign language. We expect to use the lab rotation model in the future. We have used the A La Carte model in the past, and use it when needed with individual students. The online programs have been added in the past, and will likely be added in the future, to assist us in meeting the needs of every student. We offer on line courses now however, the teacher is in the classroom not online. We offer a course in which student may select from a variety of courses to supplement their current course load. The course offerings are from Columbia College (Columbia, CA) and from BYU. We plan to implement Apex Learning for credit recovery this summer (2014) and to pilot UC Scout for AP coursework in Environmental Science with labs conducted by our chemistry instructor in Fall 2014. California elearning Census: Increasing Depth and Breadth Page 9

Grade levels An important goal of the census was to estimate the number of California K-12 students participating in online and blended learning. Figure 8 summarizes the percentage of elementary, middle, and high schools are involved in online or blended learning. Not surprisingly, the data show that much higher percentages of students are enrolled in online and blended learning in the upper levels with the majority at the high school level. When asked in which grade levels students were learning online, 74% indicated grades 9-12, a 4% drop from 2013. However, participation in grades 6-8 increased 8% to 57%, while participation in grades K-5 increased 5% to 34%. Figure 8: Distribution of online and blended learning among grade levels. Overall numbers of students participating in elearning Based on census data from 569 districts and direct-funded charters (31% of all districts and charters), the total number of students participating in fully virtual and blended learning in California in 2013-2014 was 174,632 students. An additional 38,745 students participated in either fully virtual and/or blended learning during the summer of 2013. School Year 2013-14 Virtual: 24,043 Blended 150,589 Total: 174,632 Summer 2013 38,745 California elearning Census: Increasing Depth and Breadth Page 10

Comparing Online and Blended Population Trends Actual population numbers only increased for blended learning during the past year, while virtual student populations seem to have stabilized. We re not surprised, as Michael Horn had previously predicted that virtual populations wouldn t account for more than 10% of the overall elearning population. However, it s also possible that state regulations have restricted districts opportunities to offer virtual schooling. Starting with 19,820 full-time online (virtual) students in 2012, virtual populations rose to 24,383 in 2013 before settling back to 24,043 in 2014. However, blended learning populations have risen steadily over time. Starting with 86,257 blended students in 2012, the number of elearning students increased 17% in 2013 and 49% in 2014. In all, 74% more students are blending their learning now than were in 2012. Figure 9: Online and blended learning population totals for 2012-2014. Comparing Population Trends at Charter and Traditional Districts While the blended learning population grew 49% this year, a large percentage of that growth occurred in charter schools. Figure 10, below, and Figure Nine, above, confirm that the full-time virtual population has remained fairly flat over time, although charter schools are experiencing slight gains in virtual students. However, Figure 11 shows that much of the growth experienced this year in blended learning is due to adoption at charter schools, as confirmed by Figure Five on page seven. While blended learning in traditional districts has grown 43% since 2012, charters have experienced a 287% growth in blended learning students. Full-Time Virtual Blended Learning Figure 10: Full-time virtual population growth over time. Figure 11: Blended student population growth over time. California elearning Census: Increasing Depth and Breadth Page 11

Average & Median Populations Average numbers of elearning students increased in all categories. Online and blended summer school attendance averaged 298 students, a 126% increase; Virtual school averages rose from 139 to 256 students, an 84% increase; and the average number of blended students per district or direct-funded charter rose from 490 to 577, a 15% increase. Figure 12: Average numbers of online or blended students in summer school, virtual schools and blended learning programs. Median populations, the point where half the districts have more than the number and half have less, are often more telling. In 2013, median populations for both full-time virtual and blended populations grew 25%. This year, the virtual student population median decreased 14% to 60, while the blended population median grew to 180, an 80% increase. Figure 13: Median number of virtual or blended students during 2013-2014. California elearning Census: Increasing Depth and Breadth Page 12

Started Small but Growing Larger While the 2012 census indicated that most districts and charters were tiptoeing into online learning and that the relative numbers of students involved in each district were low, blended learning populations in districts are blossoming. Over the past three years, the percentage of districts with fewer than 20 online or blended students has steadily fallen from 24% in 2012 to 11% today (see Figure 14). When totaling all elearning students in the bottom 50 districts, we found just 542 students in 2012 while this year s total is 743 (see Figure 15). Figure 14: Percentage of districts and direct-funded charters with fewer than 20 students learning online. Figure 15: Total number of blended students learning in the bottom 50 districts and direct-funded charters. California elearning Census: Increasing Depth and Breadth Page 13

Blended Population Quartiles In order to understand how blended learning was growing in acceptance, we analyzed data by quartiles for the three years we ve conducted this census and while this chart may not look exciting, it demonstrates that this year s population is spread across a greater number of districts and charters. In 2012, 25% of the responding school districts and charters accounted for 90% of the blended student population, with the next quartile adding just 8% and the third quartile adding 2% more. In 2013, we began to see a shift with the top quartile taking 87% while the next quartile added 8%. When taken together, the top two quartiles contributed 98% in 2012, lowering to 95% in 2013. In 2014, the top quartile contributed 83% of the total, while the top two quartiles accounted for 95% of the population. The second quartile increased from 8% in 2013 to 12% in 2014. Even the third quartile, which began at 2% in 2012 grew to 4% in 2014. While the top 25% of districts and charters continue to contribute a significant proportion of the total elearning population, the trend lines show that elearning adoption is broadening among a greater percentage of schools. What has begun with some schools offering blended options for a few nonconsumers has continued to grow as acceptance increases. Increases in total population as well as the median populations in 2014 confirm this trend. Figure 16: Blended learning population, divided by quartiles. The Bottom Quartile When looking at populations through quartiles, the three-year trend shows consistent growth both in population numbers and in median populations within the bottom quartile, even though it has yet to contribute more than 1% to the overall population total. In 2014, the bottom quartile only accounted for 1352 students with a median of 20 as compared to the top quartile with its 124,713 student population and a 750 median. Figure 17 demonstrates the steady climb in both total population and median population numbers in the bottom quartile. While the overall numbers may seem insignificant as compared with the top quartile, the long-term trend, in this and in other charts, indicates that blended learning is continuing to mature and gain acceptance in California s schools. Figure 17: Bottom quartile blended population and median numbers from 2012-2014. California elearning Census: Increasing Depth and Breadth Page 14

Longitudinal Data Of the 569 districts and charters that participated in the census this year, 131 have taken the elearning census each year since 2012. While representing just 23% of the 569 participating districts this year, the longitudinal trends may be instructive. First, though, the author acknowledges that the data from these 131 districts sometimes seems random and inconsistent. Several schools noted blended learning students in one or more years followed by zero this year. When asked about the discrepancy, some districts acknowledged the mistake and provided more accurate numbers. One district listed 0, 700, and 0 as their 2012, 2013 and 2014 populations. Looking more closely, we found that the 2012 and 2014 (0,0) surveys were taken by one person and the 2013 (700 students) was completed by another. This seems the case with several districts with vastly different numbers of elearning students each year. Elementary (N=67) While 19% of elementary districts are currently implementing online or blended learning as compared with 13% in 2012, 60% have never entertained the idea. Seven districts indicate they re currently in the planning stages and three districts that were elearning in 2012 are no longer doing so. Virtual 2012: 343 2013: 147 2014: 254 Blended 2012: 1399 2013: 1305 2014: 2913 Unified & High School (N=64) 88% are currently implementing online or blended learning as compared with 61% in 2012. While eight districts are not currently elearning, five were in 2012 but then stopped. It s entirely possible, though, that they didn t stop and that the people who entered the 2014 data were uninformed. However, the virtual and blended populations from these 64 high school and unified districts are consistent with the overall trends. Virtual 2012: 13,881 2013: 11,613 2014: 13,671 Blended 2012: 11,086 2013: 26,084 2014: 46,872 Figure 18: Population trends at the 64 unified and high school districts that participated in the census each of the last three years. California elearning Census: Increasing Depth and Breadth Page 15

Blended learning instructional models Each year, we ask districts and charters to identify the blended learning models they are implementing (See Figure 19). While the predominant blended model in the 2012 census was the A La Carte, followed by Enriched Virtual, the Rotation method dominated the 2013 census, with 47%. Last year, 34% of districts and direct-funded charters reported they were utilizing more than one blended learning model, up from 31% in 2012. In 2014, A La Carte and Enriched Virtual again dominated the census. Figure 19 Distribution of blended learning models among all districts and direct-funded charters. Blended Learning Models by Grade Span When separating elementary from K-12 and grade 9-12 districts, though, we found that Rotation was again the predominate model in elementary districts, followed by 76% of districts and charters (see Figure 20). Just 14% of elementary districts indicated they were using more than one blended model. In K-12 and high school districts this year, the leading blended model was A La Carte followed by Enriched Virtual. 59% of districts and charters reported using A La Carte, followed by the Enriched Virtual, Flex, and Rotation models. 48% of these districts report using more than one blended model. Figure 20: Blended learning models in elementary districts and direct-funded charters Figure 21: Blended learning models in unified and high school districts and direct-funded charters. California elearning Census: Increasing Depth and Breadth Page 16

Blended Model Choice at K-12 and 9-12 Charters and Districts If online and blended learning are entrenched in grades 9-12, then it might be instructive to remove elementary schools from the data set and see how elearning is evolving. This chart displays data only from those charter schools and districts that are either K-12 or 9-12 institutions. While A La Carte and Enriched Virtual are the predominant models in both charters and districts, A La Carte dominates in districts while Enriched Virtual dominates in charter schools. 62% of districts utilize A La Carte compared with 54% in charters while Enriched Virtual is the preferred choice in 61% of charters compared with 49% in districts. However, Rotation and Flex seem to be popular choices in a third of traditional districts with 33% and 37% penetration. We have no explanation why Flex would be 22% more popular in districts than in charters or why Rotation is 12 points more popular in districts. 55% of traditional K12/9-12 districts utilize more than one blended model compared with 43% of K12/9-12 charter schools. Figure 22: Blended learning models of K-12 and 9-12 charter schools and districts. Factors Districts Considered Before Selecting Courseware As the census drew to a close, we sent a supplemental survey to those districts that were purchasing online or blended courses and asked about their decision making process and the factors they considered when selecting courseware. Data from this year s survey mirror s last year s results. The top four criteria in California were price (79% of the districts), comparing courses to content standards (58%), selecting from the U.C. A-G approved list (58%), and examining course outlines (54%). The vast majority of 105 districts that responded utilized at least three of the selection criteria. Figure 23: Specific actions taken before purchasing courses. California elearning Census: Increasing Depth and Breadth Page 17

Specific actions taken before courseware purchase included asking for a demo account, having the provider demonstrate the course, and having teachers sample the course. Half the respondents asked other districts about their experience with the course. Unfortunately, less than a third of districts requested student input by having them sample courses. Figure 24: Specific actions taken before purchasing courses. The supplemental survey also asked districts and direct-funded charters about the number of publishers and individual courses they investigated before committing to a purchase. This year s results indicate a significant shift in their course selection process. While last year, 24% of all districts investigated one or fewer publishers, this year only 7% have limited their search this way. 83% of districts investigated four or more course publishers this year, compared with 74% last year. Next we asked about the number of individual courses districts sampled from each publisher and here again we found a major shift in attitude. Districts are investigating more courses before they make a purchasing decision. 66% sampled three or more courses from each publisher as compared with 57% last year. On the bottom end, just 13% of districts sample zero or one course compared with 16% last year. Figure 25: Number of publishers investigated Figure 26: Number of individual courses investigated California elearning Census: Increasing Depth and Breadth Page 18

What Districts & Charters Would Do Differently In our follow-up survey to districts, we asked what they would do differently if they could start over, providing them only an open text box for their replies. 71 of the 105 districts that took the follow-up survey answered this question and their responses have lessons for every district that has yet to implement an online or blended learning program. While 33% indicated they wouldn t change anything, the majority would do things differently, including starting over. Their answers reflect five themes: More thorough planning, course creation, course selection, professional development, and other. 17% would be more systemic in their initial planning: including more stakeholders, starting earlier, performing more research, and reviewing more options. While four districts wished they had built their courses instead of purchasing them, 33% would have been much more careful in their course selection. These districts wished they had been more thorough, looking at more publishers, seeking more input, examining more courses, and negotiating harder with publishers. Finally, 11% of districts wished they had prepared their staff better for the transition into online or blended learning, providing more professional development to a larger group of people. All responses are listed below and are worth reading for their honest reflections Planning Process Parental involvement Look at more student data for credit recovery to make sure there was a good match with what students need to know and what can be completed. Start earlier. Have greater input from stakeholders. We made the best choice available at the time, but are now looking to the future with so many potential vendors (with CLRN / a-g courses). Build it into the actual school vision and plan. Figure 27: What districts would do differently. I would rigorously protect the process of using Evergreen Education Group's Quality Planning Tool: define a small group of stakeholder/lead teachers and one admin who consistently evaluate as is versus where we were and where we want to go. Include more stakeholders. Begin the process earlier so that we would have more time to talk with people already using the products. Choose teachers who are more willing to adapt to new learning models. Teacher enthusiasm, eagerness to innovate, and thinking out of the box are important to success of the blended model. Should have asked for input from students about the program. What they like and don't like. Be more student centered in the process of designing and developing the blended model. Quicker implementation of certain elective and "unique" types of courses. Spend more time reviewing the different options We would have developed an implementation guide of processes and procedures at all sites before starting the program. This guide will include roles and responsibilities of the teachers and administrators. Planning: Course Selection Would have had more of an emphasis on the ability to modify content by inserting custom content. Would have placed more emphasis on usability of teacher interface and teacher reporting capabilities. Look for UC/CSU alignment first, then ease of accessibility, then price. Trials of more vendors Aventa has been designed as an online course for individual independent study. We did not realize that the program would not always mesh well with a school-wide blended program. We have faced numerous challenges in implementing the curriculum and reporting system. I have been working with the vendor to make the program more friendly to a blended program and changes are supposed to be in the works. California elearning Census: Increasing Depth and Breadth Page 19

Allow parents to look through the courses as well. The student input is invaluable. Also, we'd check to see if the course can be added to our LMS. At what cost? How long would this process take? We'd have full access to ALL courses and do a trial run with students for 2-3 months to see if it works with our population of students. We would not pay for a large number of students without first seeing if a small number of students could be successful. Gained a deeper understanding of the difference in courses between UC A-G and credit recovery and a better understanding of what options the student learner has for self-pacing. There are many systems I would implement differently within my blended program. However, if the question is geared toward choosing an online provider, I would have engaged with my provider and demanded certain instructional and curricular expectations sooner. It is imperative to have a provider who personalizes the online curriculum for the school site rather than working with a provider who expects a school site to mold into a provider's predetermined path. Check with other districts to see how well the program is working for them. Have students sample more variations to online programs. We would examine more programs and have the students use them first to determine ease of use and suitability. Explore more options regarding vendors and start in-house development sooner. Find a cheaper, but equally good program Do a little more research on Science curriculum Cost is our biggest hurdle. This is really restricting our ability to purchase content. I would have my bosses invest more into content. Research multiple providers who are CLRN certified. Evaluate provider s a-g approval for college bound (four year university) versus credit recovery for high school graduation. Additionally, policies and procedures need to be revised to best ensure all needs are met. Possibly look at a provider for credit recovery and a different provider for a-g approved courses. Sample software and talk to reps. I would look for a program specific to K-5 and then a program specific to 6-8 as opposed to looking for one program that covered all grade levels. Our program only covers grades 3-8. Negotiate harder about the cost and have had a person with a better understanding of seat time cost versus single use and enrollment windows to inherit the programs from. I would help everyone understand that no one platform will meet all of our needs. I would also be much more specific around A-G, many vendors say they are A-G, when in truth they are not or only selected courses have been approved. Have a better understanding of the length of time that it may take a student to complete the course. Better understanding of the features within the courseware that could be modified to better meet the needs of special populations. There were many hidden costs that we did not understand at the time of the course demo. It has led to being much more expensive then we anticipated. In addition, different students need different programs. We are now identifying all of those elements to be more prudent with budget and availability of program. Planning: Build Own Courses Use technology tools to build our own blended courses instead of paying a vendor. Invested all of our time, energy, and money into developing our own content using our LMS. We tried to do both--immediate access via a commercial vendor while developing our own content, and the divided effort slowed progress on the latter. I haven't seen a commercial product that can match what our teachers have developed in terms of complete alignment to the content and rigor level of our current courses, and our own teachers can individualize and personalize better...but the effort to do this well is tremendous. Too tempting to take the easy way out with a commercial vendor. We'd go straight to something like Schoology and pay our staff to develop our own curriculum My recommendation to the board would be to develop our own courses using the National University platform. With the support of a well-known and established university I believe we could do a better job in creating meaningful activities and interactivity than the current vendor does at this time. California elearning Census: Increasing Depth and Breadth Page 20

Planning: Professional Development Greater training for counselors so they can match students appropriately with the program. We wanted to pilot a blended learning course in our CTE program. We decided to focus on a computer/technology course with the idea that the course would lend itself to a blended environment. Our challenge was finding an instructor for the course. Provide more professional development for our faculty. Look at the professional development that's provided by the publisher and the costs that are associated with it Wait until the staff had more training before fully implementing Increase the professional development component for oversight and monitoring. We would develop clearly defined procedures and protocols beyond what the provider/publisher recommends. Teachers and administration would be trained and sign an agreement to follow the protocols. Other Nothing (9 times). Nothing - we picked a provider that has great customer service and support. We felt important to be able to establish a relationship with a provider that would continually work personally with us and listen to needs. We have had that and been able to work with writers as to our needs. Nothing, it is a good start but we are looking to expand our offerings and number of online/blended options available to students. No. I am very satisfied with the program we went with. They do an excellent job communicating and developing new courses. We have been with the same provider for a while now. As our use of online/blended programming begins to grow we will hopefully re-evaluate our selection and see if other vendors provide courses that better meet our needs. This charter has a "special relationship with our vendor" I'm not sure we would do anything different. The instructor would like to create her own curriculum and content for the fall after using the vendor-provided course this spring. We were expecting this outcome and feel it is part of the evolutionary process of moving into a blended learning environment More time and more computers. Build much greater bandwidth. Stick with one. I would not change the district's online/blended model or our courses selection, which has been thorough. However, I wish the UC online course requirements weren't constantly changing and that district leaders were more aware/supportive of the positive impact of the online/blended program. I can't say we'd do something differently. We have been very methodical about this and have taken things slowly so I believe we are appropriately informed and engaged on this topic. Nothing--the changes we've made as we've grown it are appropriate and natural. Nothing. I liked our process. We are always trying to adapt to the needs of our families and update the courseware we offer yearly. As technology continues to change, we need to continuously reevaluate the effectiveness of the program we use (ie with Common Core skills and knowledge is changing - we need alignment/rigor/etc.) I don't think we would do anything differently. We have been very thoughtful in our process of selection. California elearning Census: Increasing Depth and Breadth Page 21

Course Instructors and Professional Development Last year we began asking districts and charters if their online instructors were employed by the district or by the publisher/provider and if they provided professional development to their instructors. This year, 82% of districts provided course instructors, compared with 74% last year. Just 6% of districts solely utilize provider instructors down from 14% in 2013. 12% of districts and charters utilize instructors from both. Asked whether online course instructors were provided professional development specific to teaching and learning online. 79% of districts and direct-funded charters indicated they did provide professional development to their online course instructors. Figure 28: Source of online course instructors. Figure 29: Do online instructors receive professional development specific to teaching and learning online? Professional Development Duration and Content In 2014, we expanded our Course Instructors/Professional Development question set to ask districts about the specific professional development provided to instructors as well as the length of that preparation. While a few districts indicated their training was ongoing, 212 did report specific hours ranging from one to 400. While the average training duration was 26 hours, the median was 20. When districts were asked about the type of professional development provided, census options centered around three criteria from INACOL Standards for Quality Online Courses: E5 (PD about the online course delivery system); E7 (PD about the behavioral, social, and emotional aspects of online learning); and E8 (support and use of a variety of communication modes). Of these, two are technical in nature and one is pedagogical. In addition to the above choices, districts and charters were also provided an open text box to share other professional development that instructors received. The majority (85%) of districts and charters that provide elearning professional development to their teachers focus primarily on the technical skills to maneuver through an online course, specifically how to master the learning management system, score assignments, and add content. 71% of districts provided training in the various communication modes required to stimulate student engagement. Given that teaching and learning online is dramatically different than face-to-face courses, we re concerned that fewer than half of all districts offer professional development about the pedagogical differences of teaching and learning online. California elearning Census: Increasing Depth and Breadth Page 22

Figure 30: What components were included in the online/blended teacher professional development? The districts that indicated Other were divided into two camps. Those that offered additional technical training such as tracking student hours and working with Google Docs. Other districts offered LEC Certification for online course instructors, additional Common Core or data analysis training, or instruction about how to differentiate instruction. Curricular Support Instructional materials LEC Certification (2 districts) Data and analysis Mastery Learning Standard based grading and transition to Common Core District Common Core Instructional practice - like differentiation and use of data Technical Skills How to track student hours online Google Docs Technical Support, Google Applications For Educators (GAFE) Creating a visually appealing online course California elearning Census: Increasing Depth and Breadth Page 23

Course Publishers While California s schools purchase online courses from a variety of publishers and providers, the top four players have evolved since 2013. The top four slots are now occupied by Apex, ALEKS, Fuel Education (Aventa), and K12.com. Dropping from the top four in 2014 are Cyber High and Odysseyware. We found it interesting that a substantial number of districts are creating their own courses. However, while 23% of districts purchased courses from more than one vendor in 2012 and 46% did so in 2013, 56% of districts and charters utilized multiple publishers in 2014. This dramatic increase may be the result of increased online and blended populations in each district and districts attempts to select courses for specific purposes. Please note that popularity does not always equate high quality. While several publishers in the top 10 consistently produce quality courses, great courses may be found throughout this list. We encourage all districts to fully investigate all courses, seeking input from instructors, students, and other districts. Apex!Learning! ALEKS! Aventa!(K12)! Odysseyware! Compass!Learning! K12.com! Edgenuity! Dreambox! Cyber!High! Middlebury!Interactive!Languages! BYU!Independent!Study! ST!Math! Edmentum! Accelerate! Pearson!Gradpoint! SCOUT! District!Created! Accellus! Advanced!Academics! A+!Learning! National!University!Virtual!High! Connections!Learning!(Pearson)! Florida!Virtual!School!(FLVS)! Class.com! Voyager!Learning! Currium! Udacity! University!of!Nebraska,!Lincoln! Thesys! FlipSwitch! University!of!Missouri!High!School! American!Virtual!Academy! VSCHOOLZ! 0! 10! 20! 30! 40! 50! 60! 70! 80! 90! Figure 31: Online and blended courses purchased by districts and direct-funded charters. California elearning Census: Increasing Depth and Breadth Page 24

Learning Management Systems We also asked districts about the learning management systems they are using to host online or blended courses. While most publishers or providers provide their own LMS, we found a significant number of districts continue to use Edmodo, although technically it s not a learning management system. This year, Blackboard rose to second place while CaliQity dropped from 5 th to 9 th place. Selected by one or no districts are Sakai, Edu 2.0, Desire2Learn, Angel, Course Builder, and Course Sites. Figure 32: Learning Management Systems utilized by districts and direct-funded charters. California elearning Census: Increasing Depth and Breadth Page 25

Online and Blended Learning Impact New to the census this year, suggested by the Evergreen Education Group, is a single question asking districts about the impact online and/or blended learning has had on student outcomes. Districts selected from a) Increased course completion rates; b) Improved graduation rates; c) Greater student engagement; and d) Uncertain. Districts could chose one or more impacts and could provide anecdotal information in an Other field. While 23% of districts were uncertain whether elearning had any effect on student outcomes, the majority of districts and charters were fairly positive, particularly in their comments. 58% felt that elearning resulted in greater student engagement, and 50% responded it increased course completion rates. 40% felt that elearning increased graduation rates. Those who answered that elearning increased student engagement provided the majority of the comments in the Other field. No one who answered Uncertain provided a comment. Figure 33: Online and Blended Learning Impact Other Comments Increased college readiness, self-starters, ready for independence of college learning. Teaching Staff transformed their normal classroom to more of a blended class. Individualized instruction Assist students to get caught up to their current grade level Additional options for students, outside credit Athletes are able to spend more time on the mountain Increase in 21st century skills Increased student achievement rates Met learning style needs Increased understanding of new content, increased achievement on performance standards Provides teachers with rich and timely data on student mastery Greater course offering diversity, flexible scheduling, increasing student use of 21st century skills Provided more options California elearning Census: Increasing Depth and Breadth Page 26

District elearning Leadership Given that elearning is gaining and depth and breadth, we asked this year whether districts had assigned a specific person to manage online and blended learning. While we asked districts for the name and title of the person overseeing the program, few did. So, while 57% of districts do have an employee in charge of online and/or blended learning, we are unable to ascertain whether the district or charter superintendent is that person, if elearning is part of a curriculum or technology position, or if it is a separate position. However, given that 43% of districts do not have an elearning leader is of concern at a time when blended learning is rapidly expanding. Figure 34: Districts or charters who have an assigned elearning lead. elearning Data from CBEDS Until the 2012/2013 academic year, schools in California were not required to report any data to California Department of Education regarding online or blended learning, and there were no large-scale efforts to determine the enrollments in online and blended learning. Beginning with the October 2012 California Basic Educational Data System (CBEDS) data survey, schools were asked to report the numbers of online students: 1) Online Education: One or more classes K-8; 2) Online Education: One or more classes 9-12; 3) Online Education: 50% or more classes K-8; and 4) Online Education: 50% or more classes 9-12. CBEDS data should be available during summer, 2014. California elearning Census: Increasing Depth and Breadth Page 27

Local Control Sunsets CLRN While we applaud Governor Brown s support for online learning and the Common Core State Standards, the Local Control Funding Formula (LCFF), which returns control of funding to school districts, eliminated the majority of categorical programs in the process. Although many districts will continue these program services without the attached mandates, LCFF also applies to county offices of education, which receive categorical funds for a variety of programs that serve districts, regional consortia and the state as a whole. CLRN is one of those categorical programs, a state-funded technology service that is administered from the Stanislaus County Office of Education. With the good intention of removing obstacles at the district level, the Governor has also terminated many regional and statewide programs that support his overall goals. CLRN s commitment to the State Board of Education s initiatives included reviewing all electronic resources, including online courses, open educational resources, and supplemental online resources to the Common Core, the Next Generation Science Standards and other California content standards. Online courses were also reviewed for inacol s Standards for Quality Online Courses, which helps separate high-quality, interactive, and engaging courses from simple online textbooks. CLRN s Certified courses help inform districts and charters as they select courseware for their students. Our partnership with the University of California (UCOP) guaranteed that only CLRN Certified courses received U.C. s A-G approval. Unfortunately, the Governor s budget eliminated CLRN from the Education Code, the only agency that reviewed online courses for their alignment to both content standards and to national standards for quality online courses. As of June 30, 2014, CLRN will sunset. After that, California will have lost a statewide service that supports the Governor s, the State Superintendent s and the State Board s goals. And that s ironic given online learning s explosive growth, the Governor s support for online learning and the Common Core, and districts needs for resources aligned to the Common Core and the Next Generation Science Standards. The good news, though, is that CLRN made a difference. Thanks to Christensen s and Horn s Disrupting Class, we saw the elearning revolution coming and decided to be proactive. If they were going to predict that online and blended learning were going to tip in 2012 and hit critical mass in 2019, then someone would need to inform K12 educators about course quality. Working inclusively, we designed the review process, rewrote INACOL s Standards for Quality Online Courses, and established a partnership with the University of California to Certify courses. That partnership became the fulcrum for course improvement. In order for many courses to earn CLRN Certification, publishers chose to improve their content and supplement their media offerings to better align to both the content standards and INACOL s course standards. From the start of CLRN s partnership, the CLRN Certified rate slowly rose from 25% to 70%, proving that a fair, transparent, and comprehensive review process benefits all players. Because CLRN chose to create a re-review process to provide publishers a second chance, the nation s students now have access to higher quality courses. When I became CLRN s director in 2007, my primary mission was to help CLRN continually evolve to remain relevant to its customers. We accomplished that and more. Between our reviews of digital textbooks for Governor Schwarzenneger, our foray into online course reviews, our annual California elearning Census, or our annual elearning Strategies Symposium, we broke new ground and charted a course directly into the elearning revolution. I m sorry to be the one to turn off the lights when we ve come so far and accomplished so much, but I m proud that we made a difference. California elearning Census: Increasing Depth and Breadth Page 28

Conclusion CLRN created the California elearning Census as part of its mission to inform K-12 educators and administrators about high-quality, CLRN-Certified online courses; highlight elearning s steady growth, trends and resources; and understand districts needs so that we can continue to provide relevant services to our customers. Over the past three years, we ve observed that virtual and blended learning are growing consistently as districts and charters search for new ways to serve their students. The online learning revolution, as predicted by Clayton Christensen and Michael Horn in their book Disrupting Class, has passed the tipping point and elearning is now beginning to expand throughout charter schools and traditional districts. Their prediction that 50% of all high school courses would be online by 2019 seems to be on track. This year, we discovered that online and blended learning is gaining a foothold in a greater number of districts and student populations are rapidly increasing. Currently offered in 53% of all districts and charters, 44% of unified and high school districts that are not providing elearning want to offer it to their students. Online and blended populations have grown in every category: total numbers, averages, and medians. With the total number of blended students growing 49% and the median number of blended students per district growing 80%, more districts and students see the value online and blended learning adds to K-12 education. We found also that districts are becoming more discriminating when selecting online courses. While the major publishers have begun to shift, 56% of all districts and charters utilize online courses from more than one vendor, as compared with 46% last year. While selecting by price is still the highest qualifier, administrators are also having their teachers sample multiple courses and asking other districts about their opinions before selecting courses. This seems to indicate that districts are becoming more sophisticated when selecting courseware. While most districts reported that their elearning programs are increasing student engagement and course completion, we find it instructive that two-thirds of them wished they had done things differently when they were setting up their virtual and blended programs. Their comments are instructive for all. As online and blended learning have passed the tipping point, more students and districts are participating and more blended models are being utilized. In 2014, CLRN found that online and blended learning are firmly in the zone between the tipping point and critical mass and on track to reach 50% of all courses by 2019. California elearning Census: Increasing Depth and Breadth Page 29

Appendix A: California elearning Census The third annual California elearning Census is being conducted by the California Learning Resource Network (CLRN), a state-funded technology service, in collaboration with the Evergreen Education Group. Census data will be published in the October, 2014 Keeping Pace and in a CLRN white paper this summer, which will report both elearning trends and student participation. This survey asks for the number of students who are learning online, both during the 2013-14 school year as well as during summer, 2013. Districts not currently implementing online learning, or that are in the planning stages, only need to complete page 1. FAQs and the 2013 white paper are at http://clrn.org/census. Deadline: April 1, 2014. Full-time Virtual School Students take all their courses online away from school. Students do not visit a physical campus, except to take assessments. Blended Online Learning: Online learning that typically takes place at school, where students have some control over time, place, path and/or pace. You may use this online tool to determine the models you use: http://bit.ly/blendmodels A-la-carte (formerly Self Blend): Students choose to take one or more online courses to supplement their schedules and the teacher of record is online. Enriched Virtual: Independent study students take all their online courses at home but visit a physical campus to meet with a teacher. Rotation: Students rotate, on a fixed schedule in a course, between learning online and learning from a faceto-face teacher. Rotation includes teachers who Flip their class. o To count use of supplemental and/or Internet resources as blended, students must rotate between them and a classroom on a fixed schedule within an individual course. Flex: Students take a majority of their courses online at school in an individually customized, fluid schedule and on-site teachers or paraprofessionals provide support. What is NOT virtual or blended learning: Participation in supplemental electronic activities or technology-rich activities that don t fit the definitions above. 1. Person completing the survey 2. Your Title 3. Email Address 4. County name 5. District or Direct-Funded Charter Name 6. District s grade levels: (K-5, K-6, or K-8) (9-12 or K-12) 7. Are you a direct-funded charter? 8. Do students at your charter or district participate in online learning (full-time virtual and/or blended)? If no, Is your district currently discussing or planning to implement online learning? <End Survey> California elearning Census: Increasing Depth and Breadth Page 30

9. Are teachers for your online or blended courses employed by the school/district or by the provider? a. District/school instructors <Proceed to Question 10> b. Provider instructors <Proceed to Question 13> c. Both <Proceed to Question 10> 10. Does your school or district provide professional development, specific to teaching online or blended courses, to online/blended instructors? a. Yes <Proceed to next question> b. No <Proceed to Question 14> 11. How many contact hours of professional development were provided? 12. What components were included in the online/blended teacher professional development? a. Online course delivery system (LMS) b. In the behavioral, social, and when necessary, emotional, aspects of the learning environment c. Support and use of a variety of communication modes to stimulate student engagement online. d. Other 13. Does one person manage online and blended learning for your district? a. If yes, what is that person's title and email?! 14. Which blended learning models are being utilized in your charter or district? (Check all that apply) a. Rotation Students rotate, on a fixed schedule in a course, between learning online and learning from a face-toface teacher. Rotation includes teachers who Flip their class. To count use of supplemental and/or Internet resources as blended, students must rotate b. Flex Students take a majority of their courses online at school in an individually customized, fluid schedule and on-site teachers or paraprofessionals provide support. c. A-la-carte Students choose to take one or more courses entirely online (at home or at school) to supplement their traditional schedule and the teacher of record is online. d. Enriched Virtual Independent Study or other students, who take all or most of their courses online, at home, but visit a physical campus. e. Don t know f. Other (fill in) 15. How many schools in your district/charter are implementing virtual or blended learning? 16. Students in which grade levels participate in virtual or blended learning? (Check all that apply) a. Grades K-5 b. Grades 6-8 c. Grades 9-12 17. How many students are participating in full-time virtual learning during the 2013-2014 school year? (Students take all their courses online away from school.) California elearning Census: Increasing Depth and Breadth Page 31

18. How many students are participating in blended learning during the 2013-2014 school year? Blended learning students are those who take one or more classes online AT school or who rotate between online learning and a face-to-face teacher. Use of supplemental software or Internet resources, only if students regularly rotate between computer and face-to-face learning for an individual course, IS classified blended learning. 19. How many students took virtual or blended courses during the summer of 2013? 20. If you are acquiring commercial content/courses for your school(s), from which companies are you purchasing content? (Check all that apply) a. Accelerate Education b. Advanced Academics c. ALEKS d. American Virtual Academy n. FlipSwitch o. Florida Virtual School p. K12.com q. Middlebury Interactive Languages e. Apex Learning r. National University Virtual High f. Aventa (K12 Inc.) s. Odysseyware g. BYU Independent Study t. Pearson Digital Learning h. Class.com u. Scout i. Compass Learning v. Thesys j. Connections Learning (Pearson) w. University of Missouri High School k. Cyber High x. University of Nebraska, Lincoln l. Edgenuity (formerly e2020) y. Voyager Learning m. Edmentum (formerly Plato) z. Other (fill in) 21. What learning management system(s) do you use? (Check all that apply) a. Blackboard j. Education Elements b. Brain Honey k. Haiku c. CaliQity l. Moodle (or variations like Moodle Sites) d. Course Builder e. CourseSites m. Open Class n. Sakai f. Desire2Learn o. Schoology g. District created p. Other: h. Edmodo i. Edu 2.0 22. What impact has online / blended learning had on student outcomes? a) Increased course completion rates, b) Improved graduation rates, c) Greater student engagement d) Uncertain e) None f) Other (list) The Christensen Institute, which created the blended model descriptions, invites you to share more about your program and help support the broader ecosystem of blended learning. Submit your blended profile to the Christensen Institute Blended-Learning Universe and join a community of innovative practitioners around the world. http://bit.ly/blendshare/ California elearning Census: Increasing Depth and Breadth Page 32

Appendix B: California Learning Resource Network (CLRN) Reviewing supplemental electronic learning resources and open educational resourses since 1999, the California Learning Resource Network (CLRN) launched an online course review project in 2010 to provide K-12 educators, students, and parents with detailed information about online courses. In 2011, CLRN co-chaired an inacol a committee of experts to update the inacol National Standards for Quality Online Courses. The 52 standards are organized in five areas: content, instructional design, student assessment, technology, and course evaluation and support. Each CLRN online course review includes complete information about the course's alignment to either the Common Core State Standards or California's original content standards. CLRN also applies national standards of quality for online courses that look at rigor, active engagement, higher order thinking skills, student-teacher interaction, and professional development. Besides a comprehensive review of each course's curriculum and its alignment to online course standards, CLRN's customers may participate through two separate user feedback surveys, one for educators and one for students. Each group is asked three questions which are summarized within the course review: Do you recommend the course?; Rate the degree to which the course met your overall expectations; and Rate the degree to which the course engaged and maintained your interest. Additional questions for each group are aligned with the quality course standards and are reported within each review. CLRN reviews online courses from English-language arts, history-social science, mathematics, science, visual and performing arts, and world languages. English and mathematics courses are reviewed against the Common Core and science courses, beginning next winter, will be reviewed against the Next Generation Science Standards. CLRN-Certified To earn CLRN-Certified status, online courses must address at least 80% of the course's content standards and 80% of inacol's Standards for Quality Online Courses. 15 online "Power Standards" must be among those verified by CLRN. These Power Standards include: Content: A3, A9, & A13; Instructional Design: B3, B4, B5, & B10; Student Assessment: C2, C3, & C4; Technology: D4, D10, & D11; and Course Evaluation and Support: E3, and E10. Course Certification is effective for three years from the review's posting date, after which courses must be re-submitted for review. Course publishers may utilize the CLRN Certified term and badge in association with their certified products. http://www.clrn.org/search/clrncertified.cfm UC A-G Subject Requirements With the August 2012 partnership agreement between CLRN and the University of California, CLRN- Certified courses are eligible for approval by the University of California as fulfilling the "A-G" subject requirements for freshman admission. Currently, science and visual and performing arts online courses are excluded from UC A-G approval. They may, though, be CLRN-Certified if they meet the criteria. For more information about CLRN Certification and the University of California Policy for A-G Review of Online Courses. http://www.ucop.edu/agguide/online-learning/ California elearning Census: Increasing Depth and Breadth Page 33