THE CANVAS LMS RECOMMENDATION

THE CANVAS LMS RECOMMENDATION Presented on: January 11, 2013 Prepared by: John Farquhar EXECUTIVE SUMMARY A pilot of the Canvas Learning Platform involving Western faculty, students and support staff, provided strong support for a recommendation that Western transition from Blackboard to Canvas. A summary of the results of the pilot study as well as options and timelines for the transition are presented in this report. ATUS will continue to evaluate transition options and present those recommendations by March 30, 2013. BACKGROUND Western adopted the Blackboard Learning Management System (LMS), campus-wide, early in the year 2000. Since that time, Blackboard has become the central tool for faculty at Western to deliver online content, make announcements, collect assignments, present grades, and support online collaboration. Currently, the system supports more than 1,200 courses and 40,000 enrollments each quarter. Since adopting Blackboard, ATUS has conducted two satisfaction surveys with faculty. The first was delivered in 2003 and the second in 2011. Combined, the results of the two surveys show that faculty satisfaction with Blackboard has declined over time. In 2003, more than 81% of Western faculty reported being satisfied with Blackboard. By 2011, this number had dropped to 35%. The biggest concerns expressed regarding Blackboard were its reliability and its ease of use. The 2011 survey also asked faculty to rank characteristics needed in a new LMS should we consider such a switch. The summary ranking became: 1) Easy to use without a significant investment of time 2) Improved performance with fewer issues and bugs 3) Easy to transfer existing content from Blackboard 4) Includes all features currently available in Blackboard 5) Offer additional, valuable features not currently available in Blackboard A few months after we conducted our latest satisfaction survey, the Washington State Board of Community and Technical Colleges began a review of Learning Management Systems. While involving the state's four-year institutions in their review, the principal purpose was to identify a replacement to their existing system, Angel. After a thorough review, involving input from nearly 1,000 reviewers within the state, the State Board chose Canvas which is designed and supported by Instructure. This announcement came in May of 2012. The State Board decision meant that all community and technical colleges within the state would eventually be offering online courses on the Canvas platform. The selection also enabled any of the 4-1 P a g e

year institutions within the state to take advantage of a discounted rate negotiated for all state schools. As a result of this selection, many of the 4-year public schools in the state have considered transitioning to Canvas as well. WHAT IS CANVAS? Canvas is a Learning Management System (LMS), built on a modern web framework and is offered as a fully-supported cloud-based application by the company: Instructure. The software is made available by Instructure as open source through an AGPLv3 license allowing other entities to host as well as modify the code. However, Instructure retains copyright over the Canvas project and includes features beyond the open source code in their hosted product. As the principal developer of Canvas, Instructure engages in a rapid development cycle. Consequently, updates such as bug fixes and new features are deployed by Instructure every few weeks across all environments for their hosted customers. The Canvas user-interface is well-designed for both faculty and students. And, the LMS has many options for interoperability with other systems and services through its open-source Application Programming Interface (API). For example, faculty can assign collaborative projects through Google Docs and completed Google Docs can be submitted to instructors for review. Faculty and students can link not only Google Documents, but can register into their Canvas account other social networking services and curricular content provided by publishers. THE FALL 2012 PILOT: FACULTY EXPERIENCE The fall pilot of Canvas involved 23 Western faculty and 32 courses. The pilot sought to determine whether the benefits of Canvas to the Western community outweigh the time and effort that it would take to convert content, integrate systems, and re-learn new product features. This evaluation included two formal surveys with faculty participants as well as 12 open-ended focus-group sessions where issues and concerns about Canvas were addressed. Additional input from faculty was gathered through online discussion forums and email correspondence. faculty favored Canvas over Blackboard with regard to look and feel (85% to 10%), reliability (80% to 15%), and providing new, valuable features (85% to 10%). Canvas was not rated as highly when it came to providing all of the features currently available in Blackboard with only 40% of the faculty agreeing to that statement. In a survey conducted at the conclusion of the term, 90% of the pilot faculty reported being satisfied with Canvas whereas only 25% reported being satisfied with Blackboard. Pilot 2 P a g e

On the issue of transferring content from Blackboard to Canvas, 55% of the faculty agreed that this process was easy while 30% did not. In this pilot, most faculty chose to either create new files directly in Canvas or upload source files from their local computer rather than attempt to import a course archive from Blackboard. THE FALL 2012 PILOT: STUDENT EXPERIENCE The fall pilot also included a survey to student participants. And, while the fall faculty participants are more likely to have been selfselected for their interest in finding an alternative to Blackboard, students involved in the pilot represent a far more random selection. We received 425 responses from 860 enrolled students for a response rate of 48%. Overall, 85% of the fall pilot faculty responding to the survey support a recommendation for Western to switch from Blackboard to Canvas. Fifteen percent of the faculty are neutral or have no opinion. No faculty in the survey oppose the recommendation to switch. 45%). Overall, students said that given a choice they would choose Canvas over Blackboard (73% to 17%) and would recommend that more of their instructors us e Canvas (61% to 13%). Of those responding, 91% reported being satisfied with Canvas as opposed to 67% reported being satisfied with Blackboard. In head-to-head comparisons with Blackboard, Canvas was said to have a better look-and-feel (71% to 28%) and easier to navigate (66% to THE FALL 2012 PILOT: SUPPORT STAFF EXPERIENCE Support staff within Information Technology Services (ITS) also had the task to consider how well Canvas could be integrated with existing account and information systems on campus as well as how reliable Canvas might be over time. With regard to integration, support staff were quickly able to configure Canvas for our Central Authentication Service (CAS) allowing the use of our standard university login credentials. In addition, support staff were able to create more than 20,000 accounts and generate files enrolling all fall students in short order. The final integration step will fully automate enrollment updates. This is expected to occur by the middle of February 2013. Overall, the integration of Canvas into our information systems and existing workflows has been successful without much effort. Additionally, there are several benefits to moving to the integration afforded to us by Canvas. Firstly, we will be creating a Canvas course for every Course Record Number (CRN) in our student information system. No longer will faculty need to request a course and wait for that course to be created as all courses will be created in advance. 3 P a g e

Secondly, the process of connecting different CRNs into a single course, such as cross-listing courses, is made easier by Canvas and is now under the control of the instructor rather than an activity that must be set-up by a system administrator. During the pilot period, our support staff measured the reliability of the Canvas system and the responsiveness of the companies supporting Canvas and Blackboard. Neither system experienced a full outage during the pilot period. Blackboard experienced some brief periods where new files were not being accepted. Canvas experienced some brief periods of slowdowns. Over the fall term, we escalated 11 issues to Instructure and 17 to Blackboard. Each company first responded to the issues within 24 hours in most cases. In all cases, we found the vendors responding appropriately. OTHER SCHOOL EXPERIENCES The University of Washington, Eastern Washington University and many community and technical colleges within the state began adopting Canvas prior to Western s decision to pilot the program. All schools are reporting that the adoption process is proceeding relatively smoothly and that they have been happy with the selection of Canvas. The biggest challenge that they report facing is the natural consequence of change processes. The routines that faculty, students and support-staff develop while using a specific tool need to be reconsidered when that tool is replaced. Canvas does not provide a oneto-one replacement for every familiar tool and process. Instead, Canvas has re-engineered many processes, adding new capabilities or streamlining former procedures. The change to a new system will require faculty to re-think how they use an online system for the delivery of instructional materials. ACCESSIBILITY Canvas, like Blackboard, is Gold Certified from the National Federation of the Blind. However, this certification is not universally accepted as the ultimate goal in accessibility and there remain accessibility concerns of the product. Fortunately, it appears that accessibility is important to Instructure such that they continue to listen to criticism and make frequent improvements in accessibility. Additionally, individual users can configure their notifications within Canvas to make use of their own email system for online discussions essentially bypassing the system s discussion board interface if desired. RECOMMENDATION Considering the strong support provided by our pilot faculty, students and support staff, we are recommending that Western transition from Blackboard to Canvas. This recommendation is not made lightly as we recognize the large investment that Western faculty have placed into Blackboard including the time and energy learning the interface and implementing content. Our pilot study has found that Canvas directly addresses the concerns of dissatisfaction with Blackboard as expressed by faculty in previous surveys. And, while the migration will not be trivial, there are clear benefits in the form of reliability, ease of use, and functionality provided by the new system. 4 P a g e

RISKS AND CHALLENGES A transition to Canvas will impact the university in many ways. Below, we briefly discuss some of the greatest challenges for the university in the transition. COURSE CONTENT Most faculty have valuable course content stored in the Blackboard system. Our practice has been to keep data online for a period of two years. We also create an archive of course content following each quarter. So, content which is older than two years can be recovered by restoring a course archive into the system. Course archives exported from Blackboard conform to an international standard which allows them to be imported into other compliant systems such as Canvas. Additional tools such as the desktop application bfree allow content to be extracted without importing into an LMS. Our experience during the pilot, and which is consistent with the experiences of other schools, is that faculty typically choose to re-create their course content in the new system rather than work from imported course content. Consequently, we do not recommend any large-scale exportation and importation of content from one system to another. However, we will need to keep Blackboard course archives around for several years and provide a method for faculty to access them while the transition is taking place. FEATURE DIFFERENCES Canvas does not provide a one-to-one match of all of the existing features available within Blackboard. The Grade Center, for example, in Canvas does not provide the rich features of a spreadsheet which are available within Blackboard. Additionally, there are concerns that the online testing features don t support as many delivery methods nor options for grading as Blackboard. There will be some disappointment by a minority of faculty regarding the lack of specific features that they have used in the past. In other cases, the features implemented by Canvas are presented to the end-user differently and will require faculty to unlearn practices that they have used with Blackboard. And, it is often the case that when faced with a different user-interface, users may develop inaccurate beliefs that a particular function doesn t exist. One significant feature lacking from Canvas is a plagiarism detection tool. The SafeAssign tool within Blackboard allows faculty and students to submit papers for an originality check. These papers are matched against documents on the web, documents in specific scholarly databases, and against papers already submitted to the system. This feature has been used by 33 Western faculty over the past year for up to 16 courses in any particular quarter. In order to support plagiarism detection within Canvas, we will need to adopt a third-party product. Fortunately, the product called Turnitin integrates well within Canvas. We are currently planning to pilot Turnitin during our winter quarter in order to make a recommendation about its co-adoption by the end of the quarter. 5 P a g e

Canvas also lacks the ability to make significant customizations to a course s main menu. This may be a significant concern for the handful of individuals especially those using Blackboard for department information and other collaboration projects not involving instruction. TRAINING When we consider the number of faculty impacted by this change, learning how best to use Canvas will require the largest overall investment of time within the transition project. The LMS support staff has been developing and delivering Canvas training since mid-september. Currently, we are offering training through online screencast recordings, online live webinars, in-person presentations, hands-on workshops, and one-on-one consultations. Our experience to date is that up to 30% of the faculty who have adopted Canvas at Western have become successful using the training materials available online rather than attending a live session. Western is also co-planning a regional Canvas User Group Conference to be held in Tacoma, Washington, on March 28-29, 2013. The conference will have a number of presentations for novice users of Canvas. Our faculty and staff who have more experience with Canvas are being encouraged to present at the conference. Additionally, the support team is planning a number of other faculty development efforts such as the Blended Learning Workshop being conducted over the winter term, and one or more Canvas Bootcamps to be scheduled for the summer. Assuming that we retain our current staffing level for providing faculty development, we have the capability to support training for all faculty moving to Canvas over the next 9 months. COSTS The university s overall cost for Canvas will likely be more than what we are paying for Blackboard. Currently, our overall expenses for Blackboard include a licensing fee to Blackboard, plus the purchase and maintenance of a server infrastructure as well as the support staff for this infrastructure. Our license of Blackboard includes a feature for plagiarism detection, but we do not currently license additional add-ons for mobile apps, learner analytics nor SMS messaging. These features, which we have considered adopting in the past, have additional licensing fees. An agreement with Instructure, on the other hand, covers almost all of the expenses noted above. The agreement would be for Instructure to host Canvas in the cloud, freeing us from the purchase and support of much of our own infrastructure. In addition, the Canvas product includes features for mobile, learning analytics and SMS messaging. Yet, it lacks a feature for plagiarism detection. The licensing expense for a full-featured Blackboard system, plus the expenses for supporting our own infrastructure is essentially equivalent to the fee that we would be paying Instructure for hosting. To 6 P a g e

make the feature-sets completely equivalent, we would need to license a product such as Turnitin for plagiarism detection along with Canvas. MIGRATION OPTIONS The most significant decision to be made regarding our transition from Blackboard to Canvas is to decide when we turn-off the Blackboard system. We are currently evaluating the following four options: 1. Fully conclude our licensing of Blackboard on July 1, 2013 In this option, no students, faculty, nor staff would have access to Blackboard following the spring 2013 courses. All faculty teaching summer courses would need to be fully prepared to use Canvas. This is the least expensive option. 2. Extend our licensing of Blackboard to December 31, 2013 In this option, faculty would have the summer and fall terms to prepare future courses with the fall term of 2013 being the final term where Blackboard was available to anyone. 3. License Blackboard for archival purposes for one additional year This option would not allow us to use Blackboard to teach courses after the spring of 2013, but would give faculty the ability to export individual pieces of content and refer to their Blackboard courses for a full year beginning July 1, 2013. 4. Extend our licensing of Blackboard to July 1, 2014 NEXT STEPS This option would give faculty a full year to learn Canvas and migrate content. This would be the most expensive option. In the current winter quarter, more than 100 Western faculty are using the Canvas system. In addition to supporting and training yet more faculty during the current term, the transition team will be performing the following next steps. 1. Complete Automation Currently, enrollments for Canvas require human intervention. We anticipate being able to fully automate the process by mid-february. Additional subprojects requiring programming include a replacement channel within the mywestern portal directing students to Canvas courses rather than Blackboard courses, and, a request form which will help us automate the creation of course sites not based on CRN numbers. 2. Pilot-Test Turnitin 7 P a g e

As mentioned above, we will be conducting a pilot of the plagiarism-detection software Turnitin during the winter quarter. Will Turnitin be an acceptable replacement for the SafeAssign feature within Blackboard? We will need to answer this question and note the additional costs. 3. Identify Funding At a minimum, the Canvas hosted environment will require a shift of existing resources. The overall cost for Canvas, however, is likely to be slightly more than the overall funding currently dedicated for Blackboard. New annual funds will need to be identified. Additionally, the migration project will not be successful if we do not receive continued funding for the ITS 3 position dedicated to supporting the campus LMS. 4. Determine Blackboard Retirement Date SUMMARY In order for the LMS support team to identify the best Blackboard retirement date, we will be monitoring the following factors. a) The speed that faculty adopt Canvas We anticipate reaching 50% adoption of Canvas at the beginning of spring term of 2013. If so, we believe that the remaining 50% of faculty can adopt Canvas over the spring and summer terms in order to begin using it for Fall 2013. b) The way that faculty convert content Our surveys suggest that nearly 50% of our faculty find it useful to refer to the previous Blackboard course even after converting their content for Canvas. If this trend continues in further evaluations, it would seem important to have an archive of Blackboard sites available to faculty. c) Costs At present, we are still evaluating overall costs for the different migration options. Following a successful fall pilot of the Canvas Learning Management System, we recommend that Western move forward with a plan to replace Blackboard. For the winter term, we have expanded the number of faculty using Canvas to more than 100. We will also continue our plans to fully automate enrollment and pilot Turnitin as a plagiarism-detection tool. Migration options range from turning off Blackboard as early as July 1, 2013, to turning it off one year later on July 1, 2014. Middle-of-the-road alternatives are also under consideration. While monitoring faculty adoption rates and the need for Blackboard services, we anticipate having a recommendation for the migration path by March 30, 2013. 8 P a g e