University System of Georgia Learning Management System Transition Task Force Final Report

Transcription

1 University System of Georgia Learning Management System Transition Task Force Final Report 30 April 2011 Submitted by Donna C. Llewellyn, Ph.D. Task Force Chair

2 Executive Summary The University System of Georgia Learning Management System Transition Task Force was formed in August 2010 and met from the beginning of September 2010 through to the end of April The task force was tasked with recommending the next learning management system (LMS) that will meet the 21 st century needs of the University System s students and faculty. After collecting input from the stakeholders and doing independent research into a number of potential platforms, the task force hereby makes its final recommendation for the University System of Georgia to move to Desire2Learn at the end of the current contractual period with Blackboard.

3 Background Charge: On August 18th, 2010, Dr. Susan Herbst, Executive Vice Chancellor and Chief Academic Officer for the University System of Georgia charged Dr. Donna Llewellyn to chair a task force to select a new learning management system for the System. While the memorandum with the entire charge is given in Appendix 1, it is worth stating the guiding principles here. Guiding Principles: Recommend a product that meets 21st century needs of students and faculty supporting the improvement of retention and graduation rates Recommend a product that will be used for multiple purposes (e.g. academic instruction/research/training/continuing education/economic development) Recommend a student focused minimum LMS suite to maintain affordability and increase efficiency The task force will partner with IT to recommend an enterprise solution with an architecture that provides optimal performance/stability and supports increased enrollments of 100,000 additional students by 2020 The work of the task force will be an open and transparent process to include all stakeholders Task Force Membership: Donna Llewellyn worked in collaboration with Dr. Linda Noble, Associate Vice Chancellor for Faculty Affairs, to select members of the task force with the joint goals of achieving broad representation across institutional sectors and mission, while simultaneously maintaining a small enough group that would enable efficient and effective work. A complete list of the task force members is given in Appendix 2. It should be noted that representatives from the Georgia Department of Education and the Technical College System of Georgia were invited to serve on the task force. Students in Georgia move from our K-12 schools to both our Technical Colleges and the University System institutions. Therefore, it was seen as vital that we invite these constituencies to participate in our discussions and deliberations with the intent of improving articulation and future collaborations between the different educational stakeholders of the state. These outside partners participated, but they were not voting members with respect to the choice of finalists or the final recommendations. Further, there was a sizeable representation from the central office and from ITS (Information Technology Services) on this task force. It was seen as imperative that the task force move forward with complete information that sometimes is only available at the system level, and that any final recommendations be implementable by ITS. Again, these members fully participated but did not vote on the recommendations.

4 Process Meetings: The task force had a kick-off meeting on September 2, 2010 at Macon State College. The group then met approximately every two weeks (except during holiday periods) via Wimba, ending with another face-to-face meeting on April 28, 2011 again at Macon State College. The agendas and minutes of each of the meetings were posted on the task force web site ( and the minutes are included in Appendix 3. In addition to the regularly scheduled meetings, the task force also met electronically for webinars hosted by each of the five finalist platforms and then for a debrief session after each of these. (See more details about these webinars in the Deliberations section below.) A schedule of these webinars and a summary of the discussions from the debriefing sessions are in Appendix 4. Feedback and Input: It was vital for the task force to communicate with and solicit input from the stakeholders throughout the university system. This was accomplished through a variety of means including: A web site was set up to enable widespread distribution of relevant documents along with periodic blog postings to encourage participation from the stakeholders. A summary of the feedback received from these blog postings is given in Appendix 5. The task force facilitated five campus-based regional town hall meetings (each was simultaneously available and archived via Wimba to the entire USG). o November 11, 2010 at Darton College o November 15, 2010 at Kennesaw State University o November 16, 2010 at Georgia Southern University o November 19, 2010 at Georgia State University o December 9, 2010 at Macon State University A summary document of the feedback received from these meetings is in Appendix 6. The task force presented and received feedback at various USG conferences and meetings including: o Georgia Summit o Rock Eagle o Student Advisory Council o USG Faculty Council Online surveys were administered to the faculty and students of the system. Reports of the results of these surveys were posted on the web site and are included in Appendix 7. In addition, the vista administrators of the system were surveyed to learn of any special issues from these support personnel. Sandbox environments were set up for each of the five finalist platforms. Accounts were administered at each campus and users were encouraged to complete an evaluation survey to give their feedback on each of the five platforms. Reports of those survey results are given in Appendix 8. The task force members and a few other invited participants (those most vocal during the rest of the process) were encouraged to complete a thorough rubric evaluating the sandbox environments. A report summarizing those results are given in Appendix 9.

5 Decision Deliberations Criteria: At the initial task force meeting, the group started the discussion about the nonnegotiables those things that the new generation LMS must satisfy. This list was further refined during the following months. This resulted in a Guiding Criteria document that was shared via the task force web site and was used to narrow our consideration to a set of eight platforms. This document was posted on the task force web site and is included in Appendix 10. Reference Checks: The chair of the task force then did a reference check of each of the eight platforms to check two conditions for further consideration: the responsiveness of the vendor/community and the historical track record of providing LMS functionality successfully in a large scale implementation. Following this step, the task force then further limited their consideration to five platforms: Blackboard Learn 9, Desire2Learn, Moodle, Pearson Learning Studio, and Sakai. One of the technical requirements laid out by ITS was that the recommended platform must be able to be hosted internally by the USG. It was determined during further investigations that there had been a miscommunication about one platform, Pearson Learning Studio, and that it appears that this ability is not present from this provider. Since this was learned late in the process (after it had been announced that they were a finalist), they were kept in the pool of candidates under consideration. Demonstrations and Q&A Webinars: The task force invited each finalist platform to respond to questions related to their functionality and future road map. For the open source contenders, this was directed both at appropriately selected community members. Each platform was also invited to host a webinar in order for the task force members to see a demonstration of the platform, to go over the answers to the posed questions, and to ask follow-up questions. For the open source platforms, the task force enlisted the support of commercial affiliates to host the webinars and to carry out the demonstrations, and then included community members in the Q&A portion of the webinars. The list of requested demo actions as well as preliminary questions is included in Appendix 11. Note that in parallel to this effort, ITS supplied each platform with their own list of questions related to technical requirements. Evaluation Rubric: For the final decision process, the task force agreed on an overall rubric for evaluation. This rubric is included in Appendix 12. The task force paid special attention to the issues of conflict of interest throughout their deliberations. One member of the ITS staff who had been a support member of the task force left the system to join Wimba. This person was immediately removed from all task force communications, he turned over his physical files to another ITS staff member on the task force, and he confirmed that he had deleted all of our communication and documents from his computer. It was further decided that no members of the ITS staff or the central office staff should have a vote in

6 the final recommendation. Moreover, since all vendor communication was filtered through the chair, she agreed not to have a vote in this decision. However, all task force members were encouraged to participate throughout and this included giving their input all the way through to the final deliberations. It is important to note that all perspectives were needed in this final decision, but that the USG central office staff, ITS staff, and the chair did not cast votes. Final Review: At the final meeting of the task force, all of the feedback and input data were reviewed and discussed. Each task force member present then completed the final rubric to the best of their ability individually. The group then had a lengthy discussion, reaching a consensus on the rating for each item. The rating results in each category along with a ranked order of the five platforms in each area are given in Appendix 12. One voting member of the task force had a conflict at her institution and so could not be present at the final meeting to participate in the deliberations or the vote. The chair checked in with her after the meeting and she concurred with the final recommendations.

7 Recommendations Primary Recommendation: To meet the needs of the students and faculty of the University System of Georgia in the 21 st century, the USG LMS transition task force strongly recommends adoption of the Desire2Learn platform as the next generation USG Learning Management System. This platform outranked all of the competitors in all areas of consideration other than the ease of transition. The lead in that category was Blackboard Learn 9, which is not a surprise given that the system will be moving from a Blackboard product currently in use. However, Desire2Learn was a fairly close second place contender in this area. This recommendation is a result of the task force members taking all input data into account, considering the multiple and varied contexts and needs across the system s 35 institutions, and the need to provide a fiscally responsible solution. Note that due to the necessity for ITS to negotiate the most beneficial economic terms for the implementation of any recommendation, the task force did not have final cost figures for the different platforms. However, the ITS representatives did provide their current best known figures and these were taken into consideration. Again, due to the sensitive nature of this endeavor, these specific cost estimates are not provided in this report. Instead a table outlining the relative costs in various categories across the finalist platforms is provided in Appendix 13. Especially vital is the need for ITS to be able to support the selected solution centrally while simultaneously providing the flexibility and customization that the individual institutions require, given the fiscal realities of the USG budget at this time. Moreover, this must be balanced with the need to provide our faculty and students with a usable, stable, and technologically advanced LMS platform. The task force was united in its determination that Desire2Learn was the only contender that satisfied all of these constraints. The task force also understood that due to the dire nature of the current budget situation that there is a possibility that there are zero additional dollars to invest in a new LMS solution. One potential solution to this dilemma is to wait one year for the necessary investment in infrastructure. This would enable ITS and the campuses to carry out lowcost planning during FY12 and then be ready to move forward provided that the economy has improved for FY13. This does present a potential risk, however ITS has stated that this is a feasible plan. While the task force believes that it is vital to invest in the teaching and learning environment, if it is indeed true that the System has zero new dollars available for spending on license fees during the next two years, then the task force agreed that it is acceptable to remain with Blackboard with their scaled back contract offer (current price guaranteed for up to five years) until such time as the fiscal situation improves enough to implement our primary recommendation of Desire2Learn. It should be noted that even this option does carry with it an additional cost of training and migration. Further, without a heavy investment in hardware and related infrastructure, this solution will not

8 meet the highly desired need for customization at the campus level. Please see more information about this option and all of the cost issues in Appendix 13. It should be noted that, as mentioned earlier, the Pearson Learning Studio platform can not be hosted internally by the USG. Therefore, since this was a criteria set forth by ITS, the task force does not recommend this platform under any circumstances. Related Policy Recommendations: The task force also discussed several related policy issues for the USG. They unanimously make the following recommendations: The chosen LMS provider should state their commitment to accessibility issues and if possible, this should be written into the contract. The students in the USG clearly stated this as a priority in our stakeholder survey, and it is the right and legal thing to do. There should continue to be a policy and procedure allowing institutions to opt out of the centrally provided and supported LMS. This policy should include the caveat that such a decision on the part of any institution should not result in higher costs for the other institutions in the USG or ITS. In choosing Desire2Learn, the task force gave a large weight to the flexibility that it will permit between campuses as well as on individual campuses. However, the task force understands that no one LMS solution is right for all contexts, and that therefore, if the case can be made that there exists a better LMS for a particular institutional situation then that should be respected. However, this should in no way have a negative fiscal impact on the System or the other institutions. All institutions should have one LMS for all of their faculty and students, regardless of academic discipline. The students of the USG were very clear that they strongly prefer a unified LMS platform on their respective campus. The downsides from allowing multiple LMS platforms on a single campus strongly outweigh the potential benefits of this kind of flexibility at an institution.

9 Appendices 1: Charge memorandum 2: List of task force members 3: Task force meeting minutes 4: Webinar schedule and summary of debriefing meetings 5: Summary of blog postings 6: Summary of task force town hall input 7: Faculty and student stakeholder survey reports 8: Sandbox evaluation surveys 9: Task force sandbox evaluation rubric and summary of results 10: Guiding criteria document 11: Finalist questions and demo requests 12: Final rubric form and summary of results 13. Relative costing information and considerations

10 Appendix 1 Charge Memorandum

11 DR. SUSAN HERBST CHIEF ACADEMIC OFFICER & EXECUTIVE VICE CHANCELLOR PHONE (404) WASHINGTON STREET, S.W. FAX (404) ATLANTA, GEORGIA Aug. 19, 2010 TO: Donna Llewellyn, Director, Center for the Enhancement of Teaching and Learning, Georgia Institute of Technology FROM: Susan Herbst, Executive Vice Chancellor and Chief Academic Officer cc: RE: Institutional Presidents Vice Presidents for Academic Affairs Vice Presidents for Student Affairs Chief Business/Fiscal Officers ACIT Representatives GeorgiaVIEW Institutional Administrators Learning Management System Task Force Overview Action The University System of Georgia currently hosts a centralized Learning Management System (Blackboard Learning System - Vista 8) for 32 institutions as part of the GeorgiaVIEW Service. Vista 8 operational support from Blackboard ends in January 2013 and there are currently serious issues of parity between Blackboard s Vista 8 and their newest product - Learn 9.1 (for example, lack of multi-institutional functionality, integration with the Student Information System, learning context hierarchy, lack of a clear migration path). For these reasons, we must initiate a project to bring together USG stakeholders to evaluate and select a Learning Management System that will support the strategic goals of the University System of Georgia and the needs of our institutions. Although there is a major IT component to this decision, the ultimate decision regarding next steps resides with Academic Affairs. Therefore, the University System of Georgia has identified you to chair a task force whose primary goal is to recommend a next generation Learning Management System. The task force should consist of representatives from the faculty, students, information technology, and other appropriate end-users and agencies. Guiding Principles for the Task Force Recommend a product that meets 21 st century needs of students and faculty supporting the improvement of retention and graduation rates Recommend a product that will be used for multiple purposes (e.g. academic instruction/research/training/continuing education/economic development) Recommend a student focused minimum LMS suite to maintain affordability and increase efficiency "Creating a More Educated Georgia"

12 The task force will partner with IT to recommend an enterprise solution with an architecture that provides optimal performance/stability and supports increased enrollments of 100,000 additional students by 2020 The work of the task force will be an open and transparent process to include all stakeholders Timeframe Next Steps To ensure time for a successful implementation of the new LMS and provide at least two implementation groups of campuses, the selection process must not only be thorough and inclusive of all stakeholders, but timely as well. The expectation is for the task force to submit a vendor recommendation, concluding the RFP process, by March Below are the next steps we foresee being completed as soon as possible. Doug Hyche, the director of GeorgiaVIEW (program within USG that manages the day-to-day operations of Vista 8), will organize a meeting with you to provide background and data to assist with these initial endeavors. Finalize the membership of the task force Develop a stakeholder analysis of those affected by LMS selection/implementation Develop a communication plan and strategy Establish time commitments and expectations for members Identify the selection/review process I wish to thank you and Georgia Tech for their willingness to share their expertise in guiding the selection of our system s next learning management system. The inclusion and cooperation amongst our institutional stakeholders during the selection process is critical as we prepare to select a student-centered solution that effectively supports the core mission of the University System of Georgia: teaching and learning. "Creating a More Educated Georgia"

13 Appendix 2 List of task force members

14 USG Learning Management System Transition Task Force Membership and Contact Information Member Institution Role Phone Donna Llewellyn Georgia Institute of Technology task force chair Tracy Adkins Georgia Perimeter College vista administrator Donald (Will) AllisonGeorgia Southern University student representative Jill Brady North Georgia College and State Univ registrar Christina Clayton Department of Education virtual learning, DOE Sherry Clouser University of Georgia distance education Gabby Grant University of West Georgia student representative Charles Hodges Georgia Southern University faculty representative 912/ Robert Keown Georgia Virtual Technical College virtual learning, technic Betty LaDuke Georgia Perimeter College faculty representative Barry Monk Macon State College faculty representative Darryn Ostrander Darton distance education Chris Randall Kennesaw State University faculty representative Kimberly Tanner Valdosta State University disability services Curt Carver USG CIO Virginia Michelich USG student achievement Linda Noble USG faculty affairs Barry Robinson USG project management Mike Rogers USG faculty affairs Doug Hyche USG GAView left task force 2/9/11

15 Appendix 3 Task force meeting minutes

16 2 September 2010 Kick-off Meeting of the USG LMS Transition Task Force Macon State College 10:00 am 4:00 pm Attendees: all members of the task force except Kimberly Tanner Decisions 1. Barry Robinson (ITS) will take minutes for the first meeting. The job will rotate at each meeting 2. Additional stakeholder groups identified a. Academic Advisory Committees b. GOML 3. Draft Timeline I did not record any changes to the timeline 4. Task Force will gather initial requirements from USG stakeholders to formulate criteria to narrow down to a handful of LMS choices 5. Direct all vendor communications to Donna 6. Vet all proposed surveys, official communication from task force with Donna 7. Team will meet every two weeks via Wimba Action Items 1. Task force team members will provide Donna with: a. Any contact info changes to contact list (by COB 9/3/10) 2. Mike Rogers will post SREB LMS slides to T-square (by COB 9/3/10) 3. Add to communication plan the following(barry/doug by COB 9/3/10) a. Stakeholders i. Academic Advisory Committees (see Linda Noble for breakdown) 4. Initial Requirements compiled by task force members a. Post your previously compiled initial requirements list to t-square, Resources, Initial Requirements, or to Donna (Task Force Members by 9/7/2010) b. Post GeorgiaVIEW previously compiled list of sample desires/requirements to t-square, resources, initial requirements (Doug Hyche by 9/3/2010) 5. Create t-square folder for peer institution resource material (Donna by 9/7/2010) a. Post any known peer institution/system information/resources that relate to LMS, LMS selection, or to Donna for posting (Task Force Members by 9/10/ Post Parity lists between current implementation of Blackboard Vista 8 and possible alternatives Doug Hyche by 9/7/2010) 7. Send out followup memo to USG community (similar/same listservs as charge memo) announcing the start of the project (Donna/Linda by 9/9/2010) 8. Double check ability of Task Force members to add files to t-square Donna by 9/3/ Post Ga Summit and Rockeagle info the t-square site Doug Hyche by 9/3/2010

17 10. Review communications plan and send comments/suggestions to Donna (Task force members by 9/10/2010) 11. Create list of questions for students to be used in a survey (Will / Gabby, Virginia by 9/10/2010) 12. Brainstorm what social media can be used to attract student response Gabby/Will by 9/10/ your must have requirements to Donna Task force members by 9/10/ Review polling tools Donna 15. Develop questions for faculty surveys Bettie LaDuke, Linda Noble, Doug Hyche by 9/10/ Create online surveys for different sectors on external website Donna by 9/17/ Send out list of 9/2/10 meeting action items Donna by 9/3/ Send Donna lists of LMS vendors you are aware of Task force members by 9/7/ Create survey for Vista Administrators Tracy Adkins by 9/10/2010 Informational 1. Curt Carver offered ITS resources to create sandbox environments for narrowed down LMS choices 2. Robert Keown (TCSG) has offered task force members the ability to review the Angel LMS integrations 3. Need volunteer to represent LMS Task Force at Rock Eagle conference (Sherry Clouser is possibility) 4. Let Donna know if you will be attending the Ga Summit conference (task force members by 9/10/2010) Communication Ideas/Concerns Parking Lot 1. How do we deal with vendor requests for information (for right now, forward all to Donna) 2. Specific Stakeholder interaction versus overall communication plan 3. Develop Questions / Surveys, reuse existing resources

18 16 September 2010 Meeting of the USG LMS Transition Task Force Via Wimba 2:30 pm 4:00 pm Attendees: all members of the task force except Christina Clayton, Robert Keown, Virginia Michelich, Linda Noble Secretary: Darryn Ostrander Agenda o Check in and Welcome o Call for a volunteer secretary to take meeting minutes o Volunteer: Darryn Ostrander o Review of Minutes from September 2 meeting o No comments on September 2 minutes o Update on action items o Updated Phone number o New files posted to T-Square o to USG community from Donna went out on Wednesday 9/15 copy will be sent to task force members o Darryn Ostrander agreed to help at Summit o Rock Eagle Info on T-Square October Still need volunteers confirm via o List of LMS options Went through list that has been gleaned from messages and an online list Members mentioned others and other lists o Upcoming presentations o Georgia Summit Donna and Darryn o Rock Eagle Mike Rogers, Sherry Clouser, Darryn, others? o RACRA (September 30 th ) Jill Brady o Student Advisory Council (November 6 th ) asked for volunteers, Gabby Grant volunteered, others requested via o Priority lists o Non-negotiables went over list that had been contributed. Discussed that need to clarify 508 Compliance so that this is clear to all audiences. Kimberly offered to do that.

19 o Extremely important went over the list that had been contributed. Clarified several of the items including access to source code and o Nice to Have - went over list that had been contributed. Discussion that many of these items are really features that are present in some form in Vista but that need improvement to be really useful. Bettie LaDuke offered to go through and clean up this list to make that clear. o Surveys discussed that there will be several surveys aimed at different audiences. Chuck Hodges wanted to be sure that the data would be captured by campus. Doug Hyche assured us that this will be possible. Time frame is to have the surveys administered in October. Idea is to close them in early November. Students Will Allison has started on several surveys on survey monkey and will post them Faculty Betty Laduke and Doug Hyche have gathered surveys from several other schools. Looking to edit rather recreate. o Vista Administrators Tracy has posted a survey in T-Square o Donna: Feedback has come from the website o Next Steps/Action Items/Next Meeting o Finalize Surveys aim for a close to final draft of the main three surveys by our next meeting. Student Survey Gabby Grant Will Allison Virginia Michelich Faculty Survey Bettie LaDuke Doug Hyche Vista Administrators Tracy Adkins o Start eliminating contenders from the list of candidate LMS products go through the list looking at if they would be able to satisfy our non-negotiables really a quick first pass by our next meeting. Kimberly Tanner Tracy Adkins Barry Monk Chris Randall o Start scheduling town hall meetings Donna will work with Barry Robinson, Doug Hyche, Mike Rogers, and Linda Noble to start planning and scheduling these. Let Donna know via if you wish to help with this. o Next meeting Wednesday September 29, 10:30 noon via Wimba (might need to move this afternoon)

20 USG Learning Management System Transition Task Force Meeting 29 September 2010 Via Wimba In attendance: All members except Will Allison (in class), Curt Carver, Christina Clayton, Kimberly Tanner. Guest: Hilliard Gastfriend. o Check in o Call for a volunteer secretary to take meeting minutes o Darryn volunteers to be secretary o Review of Minutes from September 16 meeting o Minutes are accepted o Feedback from the community and from Georgia Summit o Issues related to non-negotiables: o We asked, What is our current LMS missing that you want in the new one? o We asked, What do you want to go away in our next system? o We asked What tools do other LMS offer that you wish you had access to? o Upcoming Presentations/Meetings o September 30 Jill Brady at RACRA o October 5 Tracy Adkins and Bettie LaDuke will be doing presentation at GPC Faculty Development Day o October Mike Rogers, Sherry Clouser, Darryn Ostrander at Rock Eagle o November 6 Gabby Grant and Bettie LaDuke at Student Advisory Council (GCSU) o Updated Priority List o List is updated (version 2) o This will continue to be updated as we do town halls and surveys o Surveys Vista Admin Faculty Student survey o Surveys Continued Timing Process Data Analysis o Update on action items o LMS Contenders we are making progress o Town Hall Meetings o Review of timeline and where we are o Other Business o Next Steps/Action Items/Next Meeting o October 12 meeting at 2:30 via Wimba o Action Items Send feedback to the thee presenters of the surveys within a week Linda and Donna will work on the communication piece for the Surveys

21 USG Learning Management System Transition Task Force Meeting 12 October 2010 Via Wimba Minutes o Attendees: Barry Robinson, Bettie LaDuke, Charles Hodges, Chris Randall, Curt Carver, Darryn Ostrander, Donna Llewellyn, Doug Hyche, Gabby Grant, Hilliard Gastfriend, Jill Brady, Kimberly Tanner, Mike Rogers, Sherry Clouser, Tracy Adkins, Virginia Michelich, Will Allison o Secretary for today s meeting: Jill Brady o Review of Minutes from September 29 meeting o Minutes were approved as presented o This meeting is being archived o Presentations/Meetings o GaSummit presentation is archived in the location below o RACRA September 30 Jill reported that the group was receptive to the news of the LMS search, but no new feedback was gathered. o Rock Eagle - October Mike, Sherry and Darryn will present next week. The session will not be recorded. o Student Advisory Council November 5-6 Gabby and Bettie will present. o Faculty Development Day (GPC) Bettie and Tracy presented to 100 faculty (October 7) regarding the LMS search. Many faculty voiced concerns regarding the migration, but the mood was not negative. They will have additional sessions with faculty on each campus the week of November 1 (including Wimba sessions). o Update on action items o Surveys Vista Admin (Tracy) Discussion of various questions Tracy will revise this week. Students (Gabby/Will) Discussion of various questions Gabby and Will will revise this week. Faculty (Bettie) Discussion of various questions Bettie will revise this week. Final edits on the surveys will be completed this week. Surveys will be made available widely via communication with VPAAs and CIOs from Linda Noble and other student, staff and faculty lists. Surveys will be open through the second week of November. Hilliard Gastfriend will analyze the data and have initial results available in late November or early December. o LMS Contenders 508 Compliance (Kimberly)

22 Contact has been made with vendor regarding their compliance. These vendors may be divided between compliant, working towards compliance and will likely not be compliant during the timeline of the taskforce. o Town Hall Meetings Four regional meetings will be held in November Northwest (KSU), Southeast (GaSouthern), Central (GaState), and Southwest (Darton). One catch all (MaconSt )meeting will be held in early December. Simultaneous Wimba sessions will be held where possible. Suggested sites were reviewed. o Other Business o Sandboxes perhaps available during January/February (after 3(?) contenders are selected in mid-december). This will depend on ability for ITS to get these up and running in a reasonable amount of time. o Next Steps/Action Items/Next Meeting o Sessions with Vendors January/February, in parallel, with sandbox trials. Face to face if possible. By invitation - The taskforce would provide a list of questions in advance for the vendors. We would meet in Macon (perhaps) with the full task force. Issues would then be reviewed. A suggestion was made that a webinar with each vendor be held prior to the face to face meetings to review the features of the software. o Next meeting Thursday, October 28 at 11 am.

23 USG Learning Management System Transition Task Force Meeting 28 October 2010 Via Wimba Minutes Attendees: Donna Llewellyn, Tracy Adkins, Will Allison, Jill Brady, Curt Carver, Sherry Clouser, Doug Hyche, Robert Keown, Bettie LaDuke, Virginia Michelich, Barry Monk, Linda Noble, Chris Randall, Barry Robinson, Mike Rogers, Kimberly Tanner o Check in o Call for a volunteer secretary to take meeting minutes o Barry Robinson offered to do this o Review of Minutes from October 12 meeting Approved o Presentations/Meetings o Rock Eagle The idea of pilots was a theme o Clarification of Terms: o sandbox short term no teaching, just playing around without real student data, evaluate prior to recommendation o sandbox phase 2 teaching, with real student data o pilot after recommendation is made, production level support for limited teaching o early adopters first phase implementation, production level support o Discussion of issues related to pilots and sandboxes see summary under sandboxes later in the minutes. o Bettie and Tracy will be doing a series of sessions for all of their campuses next week o Update on action items o Surveys o Hilliard good response thus far, should pick up as domain announcements go out. Reports will be provided each Friday o LMS Contenders o 508 Compliance o Technical Requirements o Discussion about migration issues o Town Hall Meetings o update o Plans for Sandboxes o Type 1 Sandbox (Playing) issues related to how many users, how inclusive they can be, etc. o Type 2 Sandbox (Teaching) there are additional issues related to student data, archiving, level of service guarantee, and effect on timeline o Subgroup will study these issues and will report back to task force before next meeting o Future Meetings o Will continue biweekly wimba meetings

24 Next meeting is November 11 at 2:30pm USG Learning Management System Transition Task Force Meeting 11 November 2010 Via Wimba Minutes Attendees: Donna Llewellyn, Tracy Adkins, Sherry Clouser, Bettie LaDuke, Hilliard Gastfriend, Charles Hodges, Virginia Michelich, Linda Noble, Barry Robinson, Kimberly Tanner o o o o o o Check In Call for a volunteer to take minutes o Linda Noble offered to do this Review of minutes from October 28 meeting o Approved Presentations/Meetings o Student Advisory Council at Georgia College & State University These students validated much of what is on our requirements list Bettie LaDuke will send list of specific requests (includes better interface with Outlook/Exchange calendar, better downloading functionality, interface with mobile devices, flexibility in choice of software to complete assignments, more gradebook functionality and would like faculty to use this function more often) o Various meetings on Georgia Perimeter College campuses Much of these requests also validated our requirements list Tracy Adkins will send list of specific requests o Meetings reported other than the scheduled Town Hall Meetings where members are talking about the LMS Transition Donna Llewellyn is meeting with the Directors of Teaching and Learning Centers on November 12 Linda Noble is meeting with the University System of Georgia Faculty Council on November 19 Update on Action Items o Surveys Deadline to participate extended to November 19 Various methods of communicating the extension were determined Hilliard reported briefly on number of respondents Response rate is good from the 4 year institutions but underrepresented at research universities and two year colleges LMS Contenders o Updates were given about some of the lms s who had not responded to Kimberly s request for info about compliance

25 o o o o o o o Town Hall Meetings o Donna is attending the meeting at Kennesaw State (Nov 15) o Bettie will attend the meeting at Georgia State (Nov. 19) o Mike and Charles will attend the meeting at Georgia Southern (Nov 16) Meetings with Other Stakeholders o Donna will find a way to meet with these various groups either face to face or via Wimba o USG Librarians were added to the list of stakeholders to meet with Sandboxes o Donna reported the results of the CIO Survey on the possibility of doing the sandboxes in an actual teaching environment Results were mixed with no clear trend o It was determined that the sandboxes would not be conducted with live teaching o Donna asked for volunteers from the task force to create teaching scenarios for testing in the sandboxes o Various ways to give institutional access to the sandboxes were discussed Institutional liaisons will be needed Notice to campus CIOs will be needed because their staff/faculty will be involved o A separate testing group consisting of ITS support staff and Task Force members will also participate in the sandboxes Discussion process for choosing finalist list o All information from all sources (508 compliance, scalability, survey results, meetings, etc.) will be considered to narrow down the list Plans for vendor visits o Need for strong academic affairs presences at these meetings was expressed o BlackBoard will be at ITS in Athens on December 8 Task Force members welcome to attend Specific questions will be developed and distributed to BlackBoard by December 1 st to be addressed at Dec 8 th meeting o Meetings with other vendors will be scheduled soon these will include opportunity to meet both with ITS staff as well as task force members. All will be given questions ahead of time. Future Meetings o Next Task Force meeting is November 22 at 2:30 o Meeting Conflict on December 9 th with Town Hall at Macon State was noted Action Items will be circulated via to all task force members

26 USG Learning Management System Transition Task Force Meeting 22 November 2010 Via Wimba Minutes Attendees: Adkins, Allison, Brady, Carver, Clouser, Hodges, Hyche, LaDuke, Llewellyn, Michelich, Monk, Noble, Randall, Robinson, Rogers, Tanner o Check in o Call for a volunteer secretary to take meeting minutes Barry Robinson will take minutes o Review of Minutes from November 11 meeting Approved o Presentations/Meetings Team reported the results of the LMS Town Halls and other meetings. The archives of the town halls are posted on the website. GSU wanted more process transparency. We will try to set up a blog on the website. Will Allison requests that we have a student focused virtual town hall meeting. o Surveys 94 admins submissions 3650 faculty submissions student submissions Institutional reports will be created and distributed (process to be determined most likely sent to CIOs and VPAAs on each campus) o LMS Contenders Update on compliance issues. We now have nine lms contenders that we will concentrate on. All of the ones that have been ruled out show no signs of compliance or have been completely unresponsive to our questions about this issue. The nine that are still being considered are all aware of compliance issues and have some process in place to respond to these issues. AMAC will assist with external review as needed for finalists. Now, we will check references of each of these nine contenders to arrive at a list of 5 (or so) semi-finalists. At that point, our list will be public. Reference questions 1. size of installation how many students/faculty/courses/campuses, size of infrastructure/hosted or self-hosted? 2. how long a client/user of the platform, where did they migrate from? 3. how did the migration go when they moved to this platform? 2. level of responsiveness (support and feature requests) 4. frequency of upgrades who dictates schedule, how determined? 5. would you recommend lms (why/why not?) 6. how active is the user community 7. how hard to support faculty and student use? Does it support learning?

27 o o Others? input by to donna. Discussion process for choosing finalist list Plans for vendor visits will not meet f2f with Blackboard, will schedule virtual vendor demos for mid December - early January o Sandboxes Start with existing testing scenarios, recruit folks to do the testing (sub-committee of task force plus other stakeholders) Form a sub-committee, Vista admins as campus liaisons o o Other Business Future Meetings

28 Learning Management Selection Task Force Meeting Minutes 12/8/10 Chuck Hodges called the meeting to order at 2:00 pm as the presiding officer in lieu of Donna Lewellyn absence due to an institutional commitment. (Action items are indicated in bold.) Members present for the meeting were: Barry Monk, Chuck Hodges, Linda Noble, Mike Rogers, Barry Robinson, Bettie Laduke, Chris Randall, Darryn Ostrander, Gabby Grant, Kimberly Tanner, Tracy Adkins, Will Allison, Sherry Clouser, Curt Carver. Also present was Hilliard Gastfriend. Minutes from the previous meeting were approved as is. A revised time line was presented and discussed. Chuck indicated that completing the reference check in the prescribed time was proving to be a very difficult task due to the slowness with which vendors were submitting references. This delay impacts the due dates of all future steps. A vote was taken to approve the amended time line and it passed. Action Item: Donna will now submit to Dr. Susan Herbst the revised timeline asking for approval to proceed with the revised dates as indicated. Once it has been approved, a notice will be posted on the web site with this information. The next town hall is to take place at Macon State on 12/9/10. Members from the task force attending are Donna Lewellyn, Mike Rogers and Doug Hyche. Donna will be meeting the CIO s of the R1 institutions via conference call on 12/10/10 regarding the task force. Discussion was held regarding a student town virtual hall meeting. The decision was made to not hold such a meeting but instead create blog where students would have additional opportunities for input. Help was asked for in getting the word out to such groups as the Student Advisory Council and Student Affairs group. Action item: Linda Noble volunteered to reach out to these groups with this information. There was a discussion about several individual campus activities regarding campus based LMS selection groups. Discussion regarding the results of the surveys ensued. A time table was presented by Hilliard regarding availability of the survey results. A report of the system-wide quantitative (multiple choice questions) results will be posted on the website by the end of the month. At that time, each campus CIO will also be able to receive a data-dump of all of the campus responses. The full qualitative report will be available to the task force in early Spring. This data will be used primarily in the implementation process.

29 It was reported that Donna is checking references of the recommended vendors/communities who have submitted references. Donna is meeting with the USG legal representative concerning the RFP/sole source process and will bring information regarding legal aspects back to the LMS group when appropriate. No other business was introduced. Future meeting schedule will be posted through an from Donna. Submitted by Mike Rogers

30 USG Learning Management System Transition Task Force Meeting 18 January 2011 (postponed from January 10 due to ice storm) Via Wimba Attending: Llewellyn, Adkins, Brady, Clouser, Hodges, Hyche, LaDuke, Monk, Ostrander, Randall, Robinson, Rogers Minutes o Check in and get volunteer secretary Barry Robinson agreed to be the secretary o Review of Minutes from December 8 meeting Approved will be posted on website this afternoon o Update about survey reports Campus-filtered raw data sent to CIOs on 1/7/2011 Quantitative (close-ended results) reports will be ready for posting within the next two weeks Qualitative (open-ended results) will be ready later in the spring o Review results of reference checks Reviewed all reference checks. Two out of the eight platforms did not have references respond so they were ruled out at this point. Extensive discussion about what was learned through the reference process. Taskforce discussed how many contenders to keep in the process. Only one was eliminated due to lack of track record with large scale clients at this time. Donna will notify them ahead of this information being made public. o Next Steps: Arrange vendor/community online meetings Demos Questions (send questions one week ahead of meeting) Subgroup formed to create first list of questions for these meetings Start working on sandboxes Implementation issues Scenarios Evaluation Overall Evaluation Rubric o Future Meetings January 28 at 2:00 February 10 at 11:00 March 1 at 2:00 Will also need participation in the vendor/community online meetings o Other Business o Action Items

31 USG Learning Management System Transition Task Force Meeting 28 January 2011 Via Wimba Attending: Llewellyn, Adkins, Allison, Carver, Clouser, Hodges, Hyche, LaDuke, Michelich, Monk, Noble, Ostrander, Robinson, Rogers, Tanner Minutes Check in and get volunteer secretary Darryn Ostrander agreed to be the secretary Review of Minutes from January 18 meeting Approved Discussion with attorney about open records laws and how they impact us Daryl Grinswold spoke about opens records issues Update from Hilliard about survey reports Quantitative (close-ended results) reports are complete and in the process of being formatted Qualitative (open-ended results) is scheduled for spring delivery Update on communication with vendors/communities Next Steps: Arrange vendor/community online meetings Report out of subgroup Will run parallel process of technical questions and functional questions different expertise and audience. Each will have a rubric that will feed into the overall evaluation process Demos Questions (send questions one week ahead of meeting) discussed draft list make sure to add roadmap questions to functional list Start working on sandboxes Report out of subgroup Implementation issues Scenarios will use ecore courses migrated to each platform (if possible) Evaluation will have a rubric for anyone who uses the sandbox. These will fold into the overall evaluation procedss Cost Considerations ITS will start to build cost models for each of the remaining platforms with information from vendors and community members. Overall Evaluation Rubric will consider all four components initial technical questions, initial functional/roadmap questions, sandbox experiments, and cost considerations. Future Meetings February 10 at 11:00 March 1 at 2:00 Other Business

32 USG Learning Management System Transition Task Force Meeting 10 February 2011 Via Wimba Attending: Adkins, Carver, Clouser, Gastfriend, Grant, Hodges, LaDuke, Llewellyn, Monk, Noble, Ostrander, Randall, Robinson, Rogers Minutes Check in Chuck Hodges agreed to be the secretary Review of Minutes from January 28 meeting Approved Donna Llewellyn provided several reminders: Task Force members must notify her regarding prior or existing associations with any of our current LMS contenders. Doug Hyche has resigned from the Task Force due to leaving USG for a job with Wimba, which is a Blackboard company. Members of the Task Force representing USG will NOT vote on our final recommendation for an LMS. Donna will not vote. Donna informed the Task Force that information has been added to the Task Force website informing site visitors of the Task Force process from now to the end of the recommendation. Hilliard Gastfriend provided an update on our survey results. The analysis of the quantitative data collected with the student survey is nearly complete. The analysis of the qualitative data has begun, but there is a long way to go. Hilliard has the assistance of a graduate student from UGA for the qualitative analysis. The analysis of the quantitative data from the faculty survey is complete and will be posted on the website soon. Donna informed us that all finalists at this stage have been sent a memo regarding technical requirements, webinar and demo needs, and sandbox logistics Curt Carver informed us that all CIOs at USG institutions were asked to refuse requests from our finalists for special product promotions such as webinars.

33 Curt provided an update on contract discussions and cost calculations. The five finalists have been (or will be in the very near future) contacted regarding contract discussions/cost calculation information. This is to allow the task force to have costing information as part of the input to their final recommendation. Six points in the future negotiation of the contracts were explained. Hilliard asked if a specific length of the eventual contract has been discussed. Curt said that a few different time frames have been discussed, but no time frame has yet been discussed. Curt informed the Task Force members of a new LMS vendor has been contacting USG institution Presidents directly. He explained what the USG central office is doing to make sure that this does not interfere with our Task Force process. Linda Noble commented that our Task Force members need to do all that we can to encourage participation, as appropriate, on our campuses with the remaining parts of the selection process. Barry Robinson reported on a recent visit of ITS to the University of West Georgia to discuss ecore needs with a System-wide LMS. He urged the Task Force to provide them a way to communicate their needs to the Task Force. Linda suggested that Barry facilitate a meeting of the West Georgia personnel with Donna in the near future. Donna and Barry agreed. Donna discussed next steps regarding setting up demos/webinars from the LMS finalists and setting up sandboxes. Barry provided information regarding logistics of account management and support of the sandboxes. Bettie LaDuke showed a revised rubric to be used while the Task Force members examine the sandbox environments of the sandboxes. Donna reminded us of future meetings.

34 USG Learning Management System Transition Task Force Meeting 1 March 2011 Via Wimba Minutes Attending: Llewellyn, Brady, Hodges, LaDuke, Ostrander, Randall, Robinson, Rogers, Tanner Volunteer secretary: Barry Robinson Review of Minutes from February 10 meeting minutes approved Update on vendor communications Instructure asked to be reconsidered After a review and a discussion with several task force members, it wa would not be appropriate. Campus Cruiser some of the reference check s ended up in their junk mail, they were allowed to reope references responded so they are not being considered further. Let Donna know if you get vendor pressure Update on web meetings: o Schedule Each vendor/community partner will have two hours for demo ing their platform, answering our pre-posted then for an open Q&A. We will follow that with a meeting in our wimba site for recap and feedback. Your comments should also be posted in the appropriate resource folder in t-square. Details are posted in 3/1/11 meeting archive o Logistics The webinar portion will be hosted by the vendor/community partner you will receive log on info ahead o Details are posted in 3/1/11 meeting archive Update on sandboxes (Barry) o Schedule has been pushed back a bit but we should still have 3 weeks for evaluations. Some institutions break but all will have time to complete. o Accounts there will be faculty and student role accounts available at each institution. It will be up to the they hand these out (and if multiple users can use one account at the same time). I will send some guidelines priorities (action item). o Support - Tracy Adkins is taking the lead on this (thank you Tracy). There is a listserv set up for all the v communicate and get assistance. o Scripts These will be available through the vista admins. The feedback will be programmed in vovici to very quick turn around on the results so that we can use it in our deliberations. o Evaluation Rubric Task force members will use a more detailed evaluation rubric o Data analysis As mentioned, the stakeholders will input their feedback into vovici for ease of analysis, ta into excel. Overall evaluation rubric still in process Final meeting schedule We will meet in Macon for our final deliberations. Date to be arranged.

35 USG Learning Management System Transition Task Force Final Decision Meeting 28 April 2011 Macon State College Attendees Donna Llewellyn, Tracy Adkins, Curt Carver, Robert Keown, Bettie LaDuke, Chris Randall, Will Allison, Linda Noble, Darryn Ostrander, Barry Robinson, Mike Rogers, Kimberly Tanner, Chuck Hodges, Sherry Clouser, Gabby Grant, Barry Monk Agenda/Notes 10:00 Call to Order Introductions (matching the faces with the voices): Around the table name, institution, role Reminder of the voting process and next steps Who gets to vote Members of the task force from the system institutions excluding Central Office/ITS staff and excluding Chair Form of our recommendation to the Chief Academic Officer What will happen next Donna will submit a report to the Chief Academic Officer with the recommendation and a background of process and justification. At that time, gag order on the task force will be lifted (we advise all members to refer questions from outside your institution to Millsaps at central office). It is then up to the Chief Academic Officer of how to proceed with our recommendation. 10:20 Call for a volunteer secretary to take meeting minutes Sherry Clouser volunteered. Minutes from March 1 meeting were approved 10:25 Review of Charge to task force 10:35 Overview of the inputs to this decision process (our criteria for the decision) Review of the Rubric Teaching and Learning Environment Getting there Support and Community Moving Forward Cost of Ownership Is anything left off of the rubric? No, consider each component broadly. Overview of the data we have collected Feedback from our stakeholders: Website blog entries Town hall meetings

36 Additional Feedback from campus town hall meetings that were not included in central task force town hall meetings importance of the master course functionality, announcements, notifications, information feed, need for training Surveys Sandbox surveys How will we weight all this data? That will be up to each task force member. Reference Checks No written records for these. The references were more for preliminary data to clear that each considered platform had large scale implementations already successfully in place and to check on the responsiveness of the vendor and the community members. Webinars for the taskforce Taskforce evaluation of the sandbox environments Noon Lunch (in the cafeteria) To Kimberly - Any comments from AMAC or students? o A student who is blind was able to get through all of the systems. o No system was perfect. Any contract should include statements about addressing accessibility issues. Even features that were accessible were sometimes cumbersome. Question about whether or not the alt tags migrated if they did not, this was actually a simple fix. To the students Any further comments? o Gabby preferred D2L and Blackboard over Moodle and Sakai in the sandbox environments. o Will worked with a diverse group of students and found varied opinions regarding which was the best. Everyone liked something different. Students care that we have a good LMS, but it won t be a big deal to them which one it is. They will learn it pretty easily. To anyone - What about interaction with the vendor and/or user community? o Comments about each ITS evaluation of the technical issues related to each environment o The Pearson model is changing not enough details to completely analyze the impact of this on our decision process. o D2L offers a centrally hosted system that can be controlled by each institution. The institutions would have to work together to work out upgrade schedules. o Blackboard says they will have the ability to offer central hosting with institutional customization in January; as of right now, this is not possible without major hardware and infrastructure investments.

37 o With Sakai and Moodle, there is concern about customization for 35 different campuses. ITS investigation into the cost of ownership this is still ongoing. Some ballpark numbers were shared. Independent work on the rubric. Discussion on each line of the rubric to reach consensus. What types of recommendations can we make? Sole recommendation Ranked order of two or three Primary/Secondary Recommendation Other 4:00 Preliminary Vote 1. Sole choice is BB9 2 D2L 7 2. If cost is chief concern, go with: BB9 (3.75M) 4 BB9 (barebones) 1 3. Allow to opt to: D2L Sakai 5 Discussion about the opt-out option If the central solution is not best for a campus, should we recommend an opt-out solution? Not a specific solution, but that there be a process for a campus to opt out. If a campus opts out, it should select a single solution. Concern about saying that there is a cost issue. Would the GA citizenry blow up about that. Reference back to the chart (rubric results) D2L wins most categories. Discussion about cost issue. According to current figures, there is about a $1M difference over 5 years between D2L and Bb9 all in solutions. Therefore, given rubric and ensuing discussion, the cost consideration vote was retaken and the new result is that if the system office determines that there are ZERO new dollars to spend on an lms that

38 they should continue with the barebones BlackBoard solution (the promise of no increase from current price of ~$500,000/year) for up to five years until there is money to implement our first choice solution of the next generation LMS: Desire2Learn. 5:00 Next Steps Donna will be writing up the report. Until it has been delivered to the central office, we are to respectfully decline to answer. Then the report will also be posted to the lms task force web site. Refer questions to John Millsaps, if desired.

39 Appendix 4 Webinar schedule and summary of debriefing meetings

40 Webinar Schedule Date Time Platform March 7, am: Sakai March 7, pm: Desire2Learn March 14, pm: Blackboard Learn 9 March 15, am: Moodle March 15, pm: Pearson Learning Studio

41 Blackboard: Summary of Debriefing Sessions after Vendor Demo Webinars Prepared for April 28, 2011 Meeting of the USG LMS Transition Task Force Positive Features Faculty Already Familiar Mobiles Rubrics Negative Features Master Course Concept Not at Level Hoping For Lack of Learning Content Structure Way TAs are handled in Large Sections Desire2Learn: Positive Features Like functionality UI a little better than Sakai Has a WebCT feel to it Liked the features Much more controllable than Sakai Accessibility is not an afterthought Assess for everyone (disabilities & learning styles) All Features Requested were in the product Seem to have a lot of experience migrating content Negative Features May be a steep learning curve Why go from one companybased product to another? Moodle: Positive Features Simple Master Course function addressed what faculty wanted Students can flag questions in the quiz function Able to link external blog or use the internal one Negative Features Migration without Moodle Rooms may be impossible No rubric feature No real wow factor Nothing value added above other LMS s Does not appear to work with mobile devices unless we go through a secondary vendor Edit view has very small icons and could be difficult for some to work with Did not address access for students with disabilities 1

42 Pearson ecollege: Positive Features They handle migration of courses Definite improvement over older versions of ecollege Negative Features Instruction videos not captioned No equation editor option for students Simple, but not simplistic Cloud computing will impose some limitations They have a good philosophy regarding accessibility API concepts are nice USG central office staff have familiarity with Equella Good attention to social media (although this could also be done with the other systems) They archive everything, forever No MapleTA How non course organizations are formed Learning Edge owns Equella and that company is prone to over commit and under deliver Integration of multi media Student outcomes dashboard Exam time multiplier (good for faculty, but could be time intensive for administrators) Can alter test time for individual students Reporting features Multiple roles available and customizable Pleasing interface Good content organization as the default (although this could also be done with the other systems) 2

43 Sakai: Positive Features Did everything expected and wanted Works well at Tech for communities Negative Features Implementation Interface is very different culture shock? Will need to cluster institutions for economy of scale Will need structure for institutions to request upgrades May not be conducive to cross institutional collaboration May not be customizable enough Students may have a problem with tool bases Some good indicators regarding accessibility Identifies for non students in collaborative sites System level resources to support institutional flexibility 3

44 Appendix 5 Summary of blog postings

45 Summary of LMS Task Force Blog Postings Prepared for April 28, 2011 Meeting of the USG LMS Transition Task Force Question #1: Thoughts on Centralized and Locally Controlled Solutions 22 comments posted with multiple posts by same individuals Three responses requested central control Three responses requested central control with local flexibility & customizability One response requested local control only Not all comments responded directly to the question o Next LMS must accommodate the majority of the faculty but be flexible for innovators Question #2: Student Input Call for questions to student representatives on the Task Force resulted in zero postings Questions #3: Comments on list of 8 Finalists: BlackBoard had two posts in support and four against adoption o Negative posts included: some problems with coding errors, application deficiencies, and system design flaws too similar to WebCT Several would only use BlackBoard if required to Epsilen and Pearson each had one post against adoption Four posts recommended adoption of an open source platform and two of those recommended Moodle over Sakai o A total of 9 posts were very positive towards Moodle with some concerns, however, about its grade book function and online help support A few posts addressed desired requirements independent of an evaluation of any of the 8 finalists. These requirements included: o No Java and Flash/increased compatibility with multiple browsers o Ease of faculty interaction o More flexible content o Eliminate redundant asks when designing courses o Must work with Wimba o Better options to imbed other technologies (e.g., Turnitin) Question #4: What are the most important issues the TF should consider when considering future plans of the five finalists: Although there are several lengthy posts under this question, most are not direct responses to the specific question. The posts express concerns about the process used to recommend the next LMS, the makeup of the task force, the time line, and the evaluation of sandboxes that are not in a real teaching environment. One post addressed the question and indicated an important issue is total cost of ownership.

46 Appendix 6 Summary of task force town hall input

47 University System of Georgia Learning Management System Town Hall Meeting Comments The purpose of this document is to briefly describe the comments made by the participants in the University System of Georgia Learning Management System Town Hall Meetings. The meeting archives were reviewed and the comments of the participants were logged. A careful listening to the proceedings resulted in the development of a coding scheme that included the following categories: 1. Tools and Features 2. Integration 3. Platform 4. Migration 5. Training and Support The meetings were intended to elicit input from faculty, staff and students about the impending change of the University System of Georgia Learning Management System (LMS). As such, the proceedings were open ended and participants were encouraged to comment on any aspect of both the LMS and the selection process. A total of 145 comments were captured over the course of the five meetings with 75 relating to tools and features. There were 19 comments about integration and 12 that concerned specific LMS platforms. In addition there were nine comments about migration, and four concerning training and support. Tools and Features The most frequent topic of the Town Hall Meeting was Tools and Features. Tracking was the most frequently cited feature with eight comments. The grade book and internal features such as Wimba like, wiki s and media player were mentioned seven times each. Four mentions of were made and there were three each of assessments, editing and flexibility. Other topics that were brought up were portfolios, reporting, user control and whiteboards. Integration The integration of the prospective LMS with other technologies was a popular topic at the Town Hall meetings. There was a significant interest in proprietary technologies such as Live Text, Microsoft, and various social media sites. In addition the integration of the LMS with mobile phones and libraries was mentioned. Platform Comments about the platform ranged from questions about institutional branding, sandbox testing, and the LMS s that are under consideration. Several participants asked about the consideration of an open source LMS while others asked what specific LMS s were under consideration.

48 Migration The migration of content from the current LMS to the new system was an important topic. The participants stressed the need for an effective technology that would retain as much of their existing content as possible. Training and Support There were several comments about the importance of training and support. Respondents indicated that training well prior to the initial use of the LMS was vital and that ongoing training and support was equally important..

49 Appendix 7 Faculty and student stakeholder survey reports

50 BOARD OF REGENTS OF THE UNIVERSITY SYSTEM OF GEORGIA USG Learning Management System Taskforce Faculty Survey Results January 2011

51 Table of Contents Survey Background... 3 Comments on Survey Analysis... 3 Question 0: Please identify your primary institutional affiliation Question 1: Have you used an LMS at your primary institution within the past two years?... 5 Question 2: Please select the reasons you DO NOT use an LMS Question 3: Please select the LMS you use most at your primary institution Question 4: How do you use the LMS you use most?... 9 Question 5: If you use the LMS you use most with your face to face classes, what kinds of actions do you take with it? Question 6: Whether you are currently using an LMS or not, how would you describe your level of expertise with this type of system? Question 7: How important are the following administrative features of an LMS to you? Question 8: How important are the following instructional features of an LMS to you? Question 9: How important are the following technical features of an LMS to you? Question 10: What is your primary discipline of instruction? Question 11: What degree levels do you normally teach? Question 12: Please provide any additional comments below LMS Taskforce Faculty Survey January 2011 Page 2 of 21 pages

52 Survey Background The University System of Georgia (USG) Learning Management Transition (LMS) taskforce was charged in August 2010 with recommending an LMS to replace the current system (Vista 8) which will no longer be supported by Blackboard effective January The taskforce has a broad range of representation including students and faculty as well as other USG stakeholders. For more information about the taskforce, please visit the website: In an effort to gather input from the main users of the USG LMS users, the taskforce distributed several surveys. This document catalogues the results of the survey that was distributed to the faculty at all of the USG campuses. Comments on Survey Analysis The survey contained several categories of questions: Choose one, choose many ( select all that apply ), choose one with option to specify other, choose many with the option to specify other, yes/no, and free or open response. Because of the large number of respondents, some of the questions that allowed respondents to specify their own category of response created difficulties in the analysis. For example, question 10 was stated as follows: Q10: What is your primary discipline of instruction? This question listed 23 choices for possible answers and an Other, please specify option. We were surprised that of the approximately 2,900 responses, 238 (approximately eight percent of the sample) chose to select the Other category. We had the option of simply listing LMS Taskforce Faculty Survey January 2011 Page 3 of 21 pages

53 all 238 responses but believed a qualitative analysis of the responses would yield more accurate and more instructive information. In the aforementioned question, a new table of results was created which now contained 35 items, under the subheader Revised Table. This analysis was quite complex and time consuming and also complicated any reports we were to write, but we believed it was the best and most accurate approach. In this report, if a question offered the opportunity for respondents to specify an Other response, the revised table has been presented. The original table and the codified Other responses appear in the Appendix. The numbers labeling each response refers to a master list of all responses used by the coders in the analysis. LMS Taskforce Faculty Survey January 2011 Page 4 of 21 pages

54 Question 0: Please identify your primary institutional affiliation. Institution Count % Total Abraham Baldwin Agricultural College % Albany State University % Armstrong Atlantic State University % Atlanta Metropolitan College % Augusta State University % Bainbridge College % Clayton State University % College of Coastal Georgia % Columbus State University % Dalton State College % Darton College % East Georgia College % Fort Valley State University % Gainesville State College % Georgia College & State University % Georgia Gwinnett College % Georgia Highlands College % Georgia Institute of Technology % Georgia Perimeter College % Georgia Public Library Service % Georgia Southern University % Georgia Southwestern State University % Georgia State University % Gordon College % Kennesaw State University % Macon State College % Medical College of Georgia % Middle Georgia College % North Georgia College & State University % Savannah State University % South Georgia College % Southern Polytechnic State University % University of Georgia % LMS Taskforce Faculty Survey January 2011 Page 5 of 21 pages

55 Institution Count % Total University of West Georgia % USG System Office % Valdosta State University % Waycross College % Total 3456 Question 1: Have you used an LMS at your primary institution within the past two years? (Respondents could only choose a single response.) Response Chart Frequency Count Yes 81.7% 2790 No 18.3% 626 Not Answered 40 Valid Responses 3416 Total Responses 3456 LMS Taskforce Faculty Survey January 2011 Page 6 of 21 pages

56 Question 2: Please select the reasons you DO NOT use an LMS. (Respondents were allowed to choose multiple responses; responses sorted by count, not as they appeared in survey; Other responses placed at bottom of item list.) Revised Table Response Percentage Count I have not received adequate training.* 18.75% 138 It does not fit with my teaching style or pedagogy.* 17.66% 130 Other tools (e.g., such as my own web site, Web 2.0 tools, etc.) are more useful to me in delivering and managing my course.* 15.22% 112 It is not used at my institution or department.* 11.96% 88 There is no need for me to provide my course materials digitally/electronically. 7.07% 52 Don t know what an LMS is** 5.16% 38 Staff, administrator, or not currently teaching** 4.89% 36 It is not reliable. 4.89% 36 It does not provide adequate features.* 5.03% 37 It is too hard to learn.* 4.35% 32 New teacher or new to USG** 2.45% 18 Unclassifiable** 0.95% 7 Student preference** 0.27% 2 Time constraints** 1.09% 8 Fear of constantly changing technologies** 0.27% 2 Total 736 * Response count augmented after analyzing other responses. ** New category added after analyzing other responses. LMS Taskforce Faculty Survey January 2011 Page 7 of 21 pages

57 Question 3: Please select the LMS you use most at your primary institution. (Respondents could only choose a single response.) Response Chart Frequency Count Blackboard Vista 11.1% 302 CourseDen 4.0% 108 Distance Learning VSU Vista (BlazeVIEW) 2.4% 65 elearning@gsc 3.6% 98 elearning Commons (elc) 6.6% 178 GaVIEW Vista 5.6% 152 GeorgiaONmyLINE (GeorgiaVIEW) 0.8% 23 GeorgiaVIEW 9.7% 264 GeorgiaVIEW (Blackboard Vista) GeorgiaVIEW (WebCT Vista) 8.0% % 276 GeorgiaVIEW Vista 7.8% 212 GeorgiaVIEW Waycross College 0.4% 10 GeorgiaVIEW Vista % 291 GeorgiaVIEW@ASU 1.1% 31 GeorgiaVIEW@BC 0.2% 6 GGC Bb Vista 1.8% 49 icollege 6.7% 182 MGC Online (GeorgiaVIEW) 0.8% 23 Moodle 1.0% 27 MSC Vista (GeorgiaVIEW) 1.3% 36 Online GCSU 0.1% 2 Pirates' VIEW 0.8% 23 Sakai 0.2% 6 LMS Taskforce Faculty Survey January 2011 Page 8 of 21 pages

58 Response Chart Frequency Count SSU elearning 0.3% 7 T Square 1.8% 50 ulearn 2.7% 72 Not Answered 36 Valid Responses 2711 Total Responses 2747 Question 4: How do you use the LMS you use most? (Respondents were allowed to choose multiple responses; responses sorted by count, not as they appeared in survey; other responses placed at bottom of item list.) Revised Table Response Percentage Count To supplement my face to face classes* 43.21% 2188 To teach online classes (classes with 95% or greater activity online) To teach hybrid classes (classes with 51% 94% activity online) * 20.20% % 752 Committee work 7.56% 383 Student advising 3.48% 176 Special student interest groups or clubs 2.37% 120 Specific LMS tool** 2.41% 122 Research* 2.33% 118 Faculty training or resource** 0.91% 46 Faculty communication/common area** 0.59% 30 Unique/Unclassifiable** 0.57% 29 LMS Taskforce Faculty Survey January 2011 Page 9 of 21 pages

59 Response Percentage Count Administration training or resource** 0.41% 21 Student special projects/independent study** 0.24% 12 Student communication/common area** 0.22% 11 Unspecified communication/common area** 0.20% 10 Access to other tools** 0.12% 6 As a student** 0.10% 5 Continuing education/external courses** 0.10% 5 To view or assist other teacher's classes** 0.08% 4 Managing GTAs/student workers** 0.06% 3 Total 5064 * Response count augmented after analyzing other responses. ** New category added after analyzing other responses. LMS Taskforce Faculty Survey January 2011 Page 10 of 21 pages

60 Question 5: If you use the LMS you use most with your face to face classes, what kinds of actions do you take with it? (Respondents were allowed to choose multiple responses; responses sorted by count, not as they appeared in survey; other responses placed at bottom of item list.) Revised Table Response Frequency Count I post a copy of my syllabus % 2254 I post documents, files, and articles.* 13.65% 2231 I post assignments so students can access them electronically.* I use the grade book feature so students can access their grades.* 11.80% % 1852 I post my PowerPoint presentations % 1770 Students use it to submit assignments to me. 8.91% 1457 I set up discussions so students can collaborate virtually.* 7.39% 1209 I give quizzes and tests online.* 6.97% 1139 I post videos/audios/podcasts*** 5.97% 976 I set up small groups for assignments, discussions, and/or other reasons.* 4.31% 704 I set up chat rooms. 2.91% 476 Access outside tools/materials** 0.59% 96 I use the SafeAssign feature to check for plagiarized work. 0.48% 79 Communication/ ** 0.47% 77 Unique/unclassifiable** 0.20% 32 Calendar** 0.14% 23 General announcements** 0.14% 23 LMS Taskforce Faculty Survey January 2011 Page 11 of 21 pages

61 Response Frequency Count Surveys/feedback** 0.04% 7 Roster tool** 0.04% 7 Learning modules** 0.02% 4 Sign up tool** 0.02% 3 Track student access** 0.01% 2 Total 16,350 * Response count augmented after analyzing other responses. ** New category added after analyzing other responses. *** Original category amended Question 6: Whether you are currently using an LMS or not, how would you describe your level of expertise with this type of system? (Respondents could only choose a single response.) Response Chart Frequency Count Never used 8.9% 276 Beginner 16.1% 500 Intermediate 50.4% 1566 Advanced 24.6% 764 Not Answered 33 Valid Responses 3106 Total Responses 3139 LMS Taskforce Faculty Survey January 2011 Page 12 of 21 pages

62 Question 7: How important are the following administrative features of an LMS to you? Please indicate the importance of each feature/function by ranking them from Not Important to Very Important. Ability to automatically contact students or release information based on student performance data Ability to selectively release content to students based on assessment results or achievement towards course outcomes Ability to share documents/information (such as office hours, test questions, documents) across courses Ability to view reports on student performance and activity Ability to set permissions to prevent certain content from being downloaded Ability to administer mid term and end of course evaluations Ability to easily create formulas for calculating grades Ability to receive notification when content is updated Ability to access technical support 24/7 by telephone, e mail, or live chat LMS Taskforce Faculty Survey January 2011 Page 13 of 21 pages

63 LMS Taskforce Faculty Survey January 2011 Page 14 of 21 pages

64 Question 8: How important are the following instructional features of an LMS to you? Please indicate the importance of each feature/function by ranking them from Not Important to Very Important. Ability to create interactive learning objects, modules, or units for users with different learning styles Ability to manage and create groups for online student collaboration Ability to provide private space in the online course for students to reflect Ability to provide a space where students can develop their own eportfolios Ability to access library materials and research assistance Ability to align course content with common course outline objectives or general education competencies (Example: Provides the ability to map course content in learning modules to the same learning objectives or competencies) Ability to conduct and archive live class sessions online Ability to provide secure online testing Ability to detect plagiarism Ability to easily create and edit mathematical and scientific equations Ability to maintain a pool of test questions that are accessible for use among multiple course sections that I teach Ability to maintain a pool of test questions that can be used by other faculty members in my department LMS Taskforce Faculty Survey January 2011 Page 15 of 21 pages

65 LMS Taskforce Faculty Survey January 2011 Page 16 of 21 pages

66 Question 9: How important are the following technical features of an LMS to you? Please indicate the importance of each feature/function by ranking them from Not Important to Very Important. Ability to have online discussions among students in multiple sections of the same course Ability to include audio or video content that can be downloaded to mobile devices Ability to access course from hand held devices (Examples: iphones, Blackberry, smart phones, mobile phones) Ability to include Web 2.0 tools (blogs, wikis, RSS feeds, other new technologies) in the LMS itself Ability to integrate blogs, wikis, RSS feeds, other new technologies from outside sources with the LMS Ability to provide visual notification of the submission of new information (Examples: discussion board postings, assignment submissions, quiz and exam submissions) LMS Taskforce Faculty Survey January 2011 Page 17 of 21 pages

67 LMS Taskforce Faculty Survey January 2011 Page 18 of 21 pages

68 Question 10: What is your primary discipline of instruction? (Responses sorted by count, not as they appeared in survey. Other responses placed at bottom of item list.) Revised Table Response Percentage Count Education* 12.45% 366 Health Professions and Related Clinical Sciences* 11.90% 350 English Language and Literature/Letters* 10.14% 298 Business, Management, Marketing, and Accounting* 8.61% 253 Mathematics and Statistics 7.18% 211 Social Sciences* 6.33% 186 Computer and Information Sciences* 5.75% 169 Biological and Biomedical Sciences* 5.44% 160 Physical Sciences* 4.59% 135 Psychology 3.47% 102 History 3.44% 101 Visual and Performing Arts* 3.40% 100 Communication, Journalism, and Related Programs* 3.03% 89 Foreign languages, literatures, and Linguistics 2.65% 78 Liberal Arts and Sciences, General Studies and Humanities* 2.04% 60 Engineering 1.29% 38 Library Science 1.26% 37 Not faculty/do not teach** 0.92% 27 Physical Education, Kinesiology and Wellness** 0.82% 24 LMS Taskforce Faculty Survey January 2011 Page 19 of 21 pages

69 Response Percentage Count First Year Experience** 0.82% 24 Philosophy and Religious Studies 0.65% 19 Agriculture, Agriculture Operations, and Related Science* 0.58% 17 Legal Professions and Studies 0.44% 13 Unique/unclassifiable** 0.37% 11 Counseling and Human Development** 0.34% 10 Geography** 0.31% 9 Economics and Finance** 0.31% 9 Architecture* 0.37% 11 Earth Sciences** 0.24% 7 Natural Resources and Conservation* 0.27% 8 English for Speakers of Other Languages** 0.14% 4 Ethnic, Cultural, and Gender Studies* 0.20% 6 Continuing Education** 0.10% 3 Family and Consumer Sciences** 0.10% 3 Aviation** 0.07% 2 Total 2940 LMS Taskforce Faculty Survey January 2011 Page 20 of 21 pages

70 Question 11: What degree levels do you normally teach? (Respondents were allowed to choose multiple responses.) Response Chart Frequency Count Undergraduate 86.7% 2574 Graduate 31.0% 920 Professional 7.2% 213 Continuing Education 3.4% 100 Valid Responses 2968 Total Responses 2968 Question 12: Please provide any additional comments below. Feel free to comment on an individual question or an area that was not covered in the survey but you believe should have been. (Comments and analysis appears in qualitative report.) This report was prepared under the direction of Dr. Hilliard Gastfriend. If you have any questions, please contact him at: for clarification or corrections. LMS Taskforce Faculty Survey January 2011 Page 21 of 21 pages

71 BOARD OF REGENTS OF THE UNIVERSITY SYSTEM OF GEORGIA USG Learning Management System (LMS) Taskforce Student Survey Results January 2011

72 Table of Contents Survey Background... 4 Comments on Survey Analysis... 4 Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q LMS Taskforce Student Survey January 2011 Page 2 of 33 pages

73 Q Q Q Q Q Q Q Q Q Q Q Q LMS Taskforce Student Survey January 2011 Page 3 of 33 pages

74 Survey Background The University System of Georgia (USG) Learning Management Transition (LMS) taskforce was charged in August 2010 with recommending an LMS to replace the current system (Vista 8) which will no longer be supported by Blackboard effective January The taskforce has a broad range of representation, including students and faculty as well as other USG stakeholders. For more information about the taskforce, please visit the website: In an effort to gather input from the main users of the USG LMS users, the taskforce distributed several surveys. This document catalogues the results of the survey that was distributed to the students at all of the USG campuses. Comments on Survey Analysis The survey contained several categories of questions: Choose one, choose many ( select all that apply ), choose one with option to specify other, choose many with the option to specify other, yes/no, and free or open response. Because of the large number of respondents, some of the questions that allowed respondents to specify their own category of response created difficulties in the analysis. For example, question 14 was stated as follows: Q14: From which of the following mobile platforms would you be likely to access an LMS? LMS Taskforce Student Survey January 2011 Page 4 of 33 pages

75 This question listed 6 choices for possible answers and an Other, please specify option. We were surprised that of the 9,410 responses, 1,148 (approximately 12 percent of the sample) chose to select the Other category. We had the option of simply listing all 1,148 responses but believed a qualitative analysis of the responses would yield more accurate and more instructive information. In the aforementioned question, a new table of results was created which now contained 12 items, under the subheader Revised Table. This analysis was quite complex and time consuming and also complicated any reports we were to write, but we believed it was the best and most accurate approach. In this report, if a question offered the opportunity for respondents to specify an Other response, the revised table has been presented. Since some questions offered in the survey had conditional responses and redirection to different questions depending upon a respondent s answers, the question numbers have been renumbered in numerical order for clarity; that is, the numbers appearing in these results do not match identically with the original question numbers appearing in the survey. LMS Taskforce Student Survey January 2011 Page 5 of 33 pages

76 Q1. In which institution in the University System of Georgia are you currently enrolled? (If you are a transient student please select the university at which you have transient status.) Note: Private colleges and out of state colleges are not considered USG Institutions and should be filled in with the out of state option below. Institution Count % Total Abraham Baldwin Agricultural College % Albany State University % Armstrong Atlantic State University % Atlanta Metropolitan College % Augusta State University % Bainbridge College % Clayton State University % College of Coastal Georgia % Columbus State University % Dalton State College % Darton College % East Georgia College % Fort Valley State University % Gainesville State College % Georgia College & State University % Georgia Gwinnett College % Georgia Highlands College % Georgia Institute of Technology % Georgia Perimeter College % Georgia Public Library Service % Georgia Southern University % Georgia Southwestern State University % Georgia State University % Gordon College % Kennesaw State University % Macon State College % Medical College of Georgia % Middle Georgia College % North Georgia College & State University % Savannah State University % South Georgia College % Southern Polytechnic State University % University of Georgia % University of West Georgia % University System Office % Valdosta State University % Waycross College % Total LMS Taskforce Student Survey January 2011 Page 6 of 33 pages

77 Q2. If you have transient status at more than one USG institution, please select the institutions below: Institution Count % Total Abraham Baldwin Agricultural College % Albany State University % Armstrong Atlantic State University % Atlanta Metropolitan College % Augusta State University % Bainbridge College % Clayton State University % College of Coastal Georgia % Columbus State University % Dalton State College % Darton College % East Georgia College % Fort Valley State University % Gainesville State College % Georgia College & State University % Georgia Gwinnett College % Georgia Highlands College % Georgia Institute of Technology % Georgia Perimeter College % Georgia Public Library Service % Georgia Research Alliance % Georgia Southern University % Georgia Southwestern State University % Georgia State University % Gordon College % Kennesaw State University % Macon State College % Medical College of Georgia % Middle Georgia College % North Georgia College & State University % Savannah State University % South Georgia College % Southern Polytechnic State University % University of Georgia % University of West Georgia % USG System Office % Valdosta State University % Waycross College % Total 683 LMS Taskforce Student Survey January 2011 Page 7 of 33 pages

78 Q3. If you have transient status at more than one USG institution, please select the institutions below: Institution Count % Total Abraham Baldwin Agricultural College % Albany State University % Armstrong Atlantic State University % Atlanta Metropolitan College % Augusta State University % Bainbridge College % Clayton State University % College of Coastal Georgia % Columbus State University % Dalton State College % Darton College % East Georgia College % Fort Valley State University % Gainesville State College % Georgia College & State University % Georgia Gwinnett College % Georgia Highlands College % Georgia Institute of Technology % Georgia Perimeter College % Georgia Southern University % Georgia Southwestern State University % Georgia State University % Gordon College % Kennesaw State University % Macon State College % Medical College of Georgia % Middle Georgia College % North Georgia College & State University % Savannah State University % Skidaway Institute of Oceanography % South Georgia College % Southern Polytechnic State University % University of Georgia % University of West Georgia % Valdosta State University % Total 253 LMS Taskforce Student Survey January 2011 Page 8 of 33 pages

79 Q4. If your home institution is out of state, please type it in below: (Sorted alphabetically) Institution Count Indeterminate/ Not applicable 39 Not specified 18 Alabama 10 Florida 10 International 7 Tennessee 7 South Carolina 6 Georgia 5 Online 5 New York 4 Kansas 3 North Carolina 3 Ohio 3 California 2 Illinois 2 Iowa 2 Maryland 2 Pennsylvania 2 Virginia 2 Arizona 1 Arkansas 1 Louisiana 1 Massachusetts 1 Mississippi 1 Oklahoma 1 Texas 1 Virgin Islands 1 LMS Taskforce Student Survey January 2011 Page 9 of 33 pages

80 Q5. Please select your student classification. (Respondents could only choose a single response) Revised Table Response Percentage Count Freshman Student who has earned 0 29 college semester credit hours.* 29.46% 3551 Sophomore Student who has earned college semester credit hours.* 23.49% 2831 Junior Student who has earned college semester credit hours % 1986 Senior Student who has earned 90 or more college semester credit hours.* Graduate Student pursuing a Master s, Education Specialist, or doctoral degree (Ph.D or Ed.D.).* First Professional Student enrolled in Veterinary Medicine, Law, Medicine, Dentistry, or Doctor of Pharmacy. Post Baccalaureate Student who has a Baccalaureate degree who is taking first professional or graduate courses but who is not working toward a degree. Unclassified Undergraduate (Other) Student who has a bachelor's degree who is enrolled for credit at the undergraduate level but not pursuing another degree. Transient Student taking one or more courses which he/she has permission to transfer to his/her home institution % % % % % % 59 Medical/Dental/Veterinary Medicine Interns and Residents. 0.19% 23 Unspecified/indeterminate** 0.62% 75 2nd Bachelors** 0.29% 35 Dual/Joint Enrollment** 0.26% 31 Certificate/Add on/endorsement** 0.25% 30 LMS Taskforce Student Survey January 2011 Page 10 of 33 pages

81 Faculty/staff** 0.08% 10 Post baccalaureate, Seeking Associate** 0.02% 3 Total * Response count augmented after analyzing other responses. ** New category added after analyzing other responses. Q6. How would you describe your race/ethnicity? (Respondents could only choose a single response) Response Chart Frequency Count White or Caucasian (Non Hispanic Origin) Black or African American (Non Hispanic Origin) 60.6% % 2727 Prefer not to answer 5.7% 683 Asian or Pacific Islander 3.7% 440 Hispanic 3.6% 436 Multiracial 3.4% 409 American Indian or Alaskan Native 0.4% 44 Not Answered 118 Valid Responses Total Responses LMS Taskforce Student Survey January 2011 Page 11 of 33 pages

82 Q7. Please enter your current major of study. (Sorted by number) Response Percentage Count Nursing 16.05% 2000** Early Childhood Education ** Biology ** Psychology ** Management ** Accounting ** Computer Science ** Health* Business* Education* Political Science ** Criminology ** Mass Communications ** English ** Marketing ** General/Undecided* Information Technology* Engineering* History ** Middle Grades Education Mathematics ** Sociology ** Art ** Chemistry ** Finance ** Pharmacy* Secondary Teacher Education ** Social Work* Management Information Systems ** Athletics/Exercise Science/Kinesiology* Medicine* Agriculture* Consumer Sciences* Economics ** Physical Therapy* Law/Legal* LMS Taskforce Student Survey January 2011 Page 12 of 33 pages

83 Response Percentage Count Library Sciences* Business Teacher Education Technology Support Services ** Other* Physical Education Spanish Language & Literature ** Physics ** Theatre ** Anthropology ** Counseling* Environmental Studies ** Biology/Secondary Education Performance ** Veterinary* Dentistry* Science* Aviation* Language & Literature* Construction Management* Architecture* Speech Language Pathology ** Integrated/Interdisciplinary* Biochemistry* Geography ** Music Teacher Education Foreign Language* Occupational Therapy* Social Studies* French Language & Literature ** Teaching Field Mental Retardation Geology ** Global Studies ** International Economics ** Philosophy ** Earth Science/Secondary Education Economics, general Biotechnology* Environmental Sciences Real Estate Recreation ** LMS Taskforce Student Survey January 2011 Page 13 of 33 pages

84 Response Percentage Count Chemistry/Secondary Education Economics/Secondary Education German ** Physics/Secondary Education Theory & Composition TOTAL 12,462 *New category added after analysis of survey responses to option other **After analysis of survey responses to the option other, responses were coded and added to the counts of appropriate survey options Q8. The majority of the courses I take at my institution are: (Respondents could only choose a single response) Response Chart Frequency Count Traditional Face to Face Instruction Little or no online component Traditional Face to Face Instruction Heavy online component Online Only/Distance Learning Hybrid (50% online, 50% face to face) 59.1% % % % 1264 Not Answered 480 Valid Responses Total Responses LMS Taskforce Student Survey January 2011 Page 14 of 33 pages

85 Q9. Please select all LMS you have experience with. (Respondents were allowed to choose multiple responses) Revised Table Response Frequency Count GeorgiaView VISTA* 29.76% 7150 Blackboard 25.36% 6093 WebCT 25.13% 6037 Blackboard Learn 11.53% 2771 I do not know any Learning Management Systems. 2.48% 596 ANGEL 2.34% 563 Moodle* 1.29% 310 FirstClass 0.52% 125 T Square 0.36% 87 Misclassified/Unclassifiable** 0.30% 72 Publisher/Textbook Website** 0.27% 64 Desire2Learn 0.23% 55 Sakai 0.23% 55 Portfollio/Assessment** 0.10% 23 Other LMS** 0.09% 21 Anlon 0.02% 4 *New category added after analysis of survey responses to option other 24,026 **After analysis of survey responses to the option other, responses were coded and added to the counts of appropriate survey options LMS Taskforce Student Survey January 2011 Page 15 of 33 pages

86 Q10. Please select the primary LMS you currently use. (Respondents could only choose a single response) Response Chart Frequency Count Blackboard Vista 12.4% 1435 CourseDen 1.6% 185 Distance Learning VSU Vista (BlazeVIEW) 1.1% 128 elearning@gsc 3.8% 441 elearning Commons (elc) 8.4% 977 GaVIEW Vista 5.7% 664 GeorgiaONmyLINE (GeorgiaVIEW) 1.3% 155 GeorgiaVIEW 9.0% 1043 GeorgiaVIEW (Blackboard Vista) GeorgiaVIEW (WebCT Vista) 14.7% % 1401 GeorgiaVIEW Vista 8.0% 929 GeorgiaVIEW Waycross College 0.1% 17 GeorgiaVIEW Vista 8 4.7% 544 GeorgiaVIEW@ASU 0.3% 39 GeorgiaVIEW@BC 0.3% 30 GGC Bb Vista 0.9% 104 icollege 6.8% 793 MGC Online (GeorgiaVIEW) 1.0% 119 Moodle 0.2% 26 MSC Vista (GeorgiaVIEW) 2.5% 289 Online GCSU 0.2% 19 Pirates' VIEW 1.0% 113 Sakai 0.0% 5 SSU elearning 0.1% 7 LMS Taskforce Student Survey January 2011 Page 16 of 33 pages

87 Response Chart Frequency Count T Square 0.4% 46 ulearn 3.1% 364 Not Answered 9 Valid Responses Total Responses Q11. Approximately how many times do you log in to the primary LMS you currently use? (Respondents could only choose a single response) Response Chart Frequency Count Never 2.0% 233 Monthly 2.9% 332 Weekly 6.8% times per week 8.5% times per week 23.0% times per week 22.9% times per week 17.9% 2063 >20 times per week 16.0% 1837 Not Answered 29 Valid Responses Total Responses LMS Taskforce Student Survey January 2011 Page 17 of 33 pages

88 Q12. Please check all features you actively use in the primary LMS you currently use. (Respondents were allowed to choose multiple responses) Response Chart Frequency Count Grades/MyProgress 89.9% 8460 Course 72.6% 6831 Online Quizzes and Exams 66.3% 6238 Online Assignment Turn in 66.1% 6221 Discussion Boards 65.2% 6140 Calendar 41.3% 3889 Online Lecture (Recorded) 16.2% 1526 Task Management 15.0% 1413 Live Chat 13.4% 1257 Cloud based File Storage 7.4% 700 Online Lecture (Live) 6.4% 599 Valid Responses 9410 Total Responses 9410 LMS Taskforce Student Survey January 2011 Page 18 of 33 pages

89 Q13. Please rate the importance of each feature in an LMS. Calendar Cloud based File Storage Course Discussion Boards Grades/MyProgress Live Chat Online Assignment Turn in Online Lecture (Live) Online Lecture (Recorded) Online Quizzes and Exams Task Management LMS Taskforce Student Survey January 2011 Page 19 of 33 pages

90 LMS Taskforce Student Survey January 2011 Page 20 of 33 pages

91 Q14. From which of the following mobile platforms would you be likely to access an LMS? (Respondents were allowed to choose multiple responses) Revised Table Response Percentage Count Apple ios (e.g., iphone, ipad) 30.74% 3137 Google Android (e.g., Droid) 25.35% 2587 RIM Blackberry OS* 17.75% 1811 Windows Phone 9.14% 933 Do not have access/do not want** 5.06% 516 Palm webos* 3.75% 383 Symbian OS (e.g., Nokia phones) 2.73% 279 Specific cell/all cell model phones** 2.12% 216 Laptop/Netbook computers** 1.63% 166 Unclassifiable/Misclassified** 1.55% 158 Specific mobile/all mobile devices** 0.14% 14 e Readers** 0.05% 5 Total 10,205 *New category added after analysis of survey responses to option other **After analysis of survey responses to the option other, responses were coded and added to the counts of appropriate survey options LMS Taskforce Student Survey January 2011 Page 21 of 33 pages

92 Q15. Which features of the primary LMS you currently use would you most likely access via a mobile device? (Check all that apply.) Revised Table Response Frequency Count Grades/MyProgress 20.43% 6099 Course * 18.60% 5552 Course Announcements* 13.83% 4129 Calendar* 10.73% 3204 Discussion Boards 10.31% 3077 Online Quizzes and Exams 5.85% 1746 Online Assignment Turn in* 5.41% 1614 Task Management 4.11% 1226 Online Lecture (Recorded) 3.11% 928 Live Chat 2.32% 692 Online Lecture (Live) 2.11% 629 Cloud based File Storage 1.71% 509 Will not access/cannot access** 1.33% 396 Class content** 0.06% 17 Unclassifiable/multiple responses** 0.04% 12 Everything** 0.02% 7 Syllabus** 0.02% 5 Unique responses** 0.01% 3 Access to other web tools** 0.01% 2 LMS Taskforce Student Survey January 2011 Page 22 of 33 pages

93 Response Frequency Count Learning Modules** 0.01% 2 Total 29,849 *New category added after analysis of survey responses to option other **After analysis of survey responses to the option other, responses were coded and added to the counts of appropriate survey options Q16. Which features do you consider the most useful in the primary LMS you currently use? Please rank your top three choices below. Ignore the "Not Ranked" column. Calendar Cloud based File Storage Course Discussion Boards Grades/MyProgress Live Chat Online Assignment Turn in Online Lecture (Live) Online Lecture (Recorded) Online Quizzes and Exams Task Management LMS Taskforce Student Survey January 2011 Page 23 of 33 pages

94 LMS Taskforce Student Survey January 2011 Page 24 of 33 pages

95 Q17. For which of the following do you use the primary LMS you currently use? (Respondents were allowed to choose multiple responses) Revised Table Response Percentage Count Classroom 53.97% 8090** Studying ** Research Work ** Personal Work ** Student Organizations Social Organizations Other* None* TOTALS 14,990 *New category added after analysis of survey responses to option other **After analysis of survey responses to the option other, responses were coded and added to the counts of appropriate survey options Q18. How do you rate your expertise as a user of technology items in the following areas? Educational work (classroom, studying) Research work Personal work (outside of school) Entertainment LMS Taskforce Student Survey January 2011 Page 25 of 33 pages

96 LMS Taskforce Student Survey January 2011 Page 26 of 33 pages

97 Q19. How strongly would you agree or disagree with the following statements regarding potential barriers to your using technology in learning? Accessibility is an important issue in technology. I didn't have adequate technical support. I don't have the skills to use technology. Instructors didn't know how to implement it. Software accessibility was not an issue. Technology added extra work with little connection to the course. Technology is too complicated. Technology is too expensive. Technology takes too much time. The technology didn't work on my computer. LMS Taskforce Student Survey January 2011 Page 27 of 33 pages

98 LMS Taskforce Student Survey January 2011 Page 28 of 33 pages

99 Q20. What method(s) do you use to learn new technologies? (Respondents were allowed to choose multiple responses) Response Percentage Count Self Taught 38.13% 7560** Help from family, friends, ** colleagues Ask IT support staff ** Learn at your workplace ** Taking online computer classes Attend face to face seminar ** Course (not online)* Instructor/Classmates* Other* None* TOTALS 19,455 *New category added after analysis of survey responses to option other **After analysis of survey responses to the option other, responses were coded and added to the counts of appropriate survey options Q21. If you have a problem with technology for a class, where are you MOST likely to seek assistance first? (Respondents could only choose a single response) Response Chart Frequency Count Friends or colleagues 28.4% 2308 Course instructor 27.4% 2229 Self help website 18.7% 1523 Talk to technical help support on phone 9.9% 807 E mail technical support 7.3% 597 Use technical help support Chat Talk to technical help support in person 4.8% % 284 Not Answered 127 Valid Responses 8138 Total Responses 8265 LMS Taskforce Student Survey January 2011 Page 29 of 33 pages

100 Q22. Please rate your overall satisfaction with the primary LMS you currently use. (Respondents could only choose a single response) Response Chart Frequency Count Completely dissatisfied 3.4% 274 Could be better 33.1% 2691 Indifferent 10.5% 858 Satisfied 40.2% 3271 Completely satisfied 12.8% 1042 Not Answered 69 Mean Standard Deviation Valid Responses 8136 Total Responses 8205 Q23. How difficult is it for you to navigate the primary LMS you currently use? (Respondents could only choose a single response) Response Chart Frequency Count Very easy/no trouble 30.8% 2507 Easy 46.7% 3802 Indifferent 16.3% 1322 Difficult 5.5% 447 Very difficult 0.7% 56 Not Answered 71 Valid Responses 8134 Total Responses 8205 LMS Taskforce Student Survey January 2011 Page 30 of 33 pages

101 Q24. Please rate the usefulness of the primary LMS you currently use in improving your success in a course. (Respondents could only choose a single response) Response Chart Frequency Count Not useful at all 4.3% 352 Slightly useful 19.6% 1593 Indifferent 17.0% 1382 Mostly useful 37.0% 3007 Very useful 22.0% 1783 Not Answered 88 Valid Responses 8117 Total Responses 8205 Q25. What do you like most about the primary LMS you currently use? (Deleted, available in qualitative analysis report) Q26. What do you dislike most about the primary LMS you currently use? (Deleted, available in qualitative analysis report) LMS Taskforce Student Survey January 2011 Page 31 of 33 pages

102 Q27. In your opinion, how important is it to have ONE standard learning management system across the University System of Georgia institutions? (Respondents could only choose a single response) Response Chart Frequency Count Extremely important 36.9% 2990 Important 34.3% 2784 It does not matter 22.9% 1859 Not important 3.1% 250 Not important at all 2.8% 230 Not Answered 92 Valid Responses 8113 Total Responses 8205 Q28. Do you think you would find an interactive tutorial helpful if you were a first time user? (Respondents could only choose a single response) Response Chart Frequency Count Yes 67.6% 5439 No 13.5% 1087 Uncertain 18.9% 1525 Not Answered 102 Valid Responses 8051 Total Responses 8153 Q29. If the primary LMS you currently use had more appealing features do you think you might log on more? Please explain. (Deleted, available in qualitative analysis report) LMS Taskforce Student Survey January 2011 Page 32 of 33 pages

103 Q29. If you had comments or clarifications on an earlier question, please type them below. (Deleted, available in qualitative analysis report) Q30. If you have comments about other issues relating to LMS not covered in this survey, please type them below. (Deleted, available in qualitative analysis report) This report was prepared under the direction of Dr. Hilliard Gastfriend. If you have any questions, please contact him at: for clarification or corrections. LMS Taskforce Student Survey January 2011 Page 33 of 33 pages

104 Appendix 8 Summary of stakeholder sandbox evaluation surveys

105 USG LMS Sandbox Evaluation Survey Type: LMS Sandbox Standard Report Date: 4/18/2011 Time Zone in which Dates/Times Appear: (GMT-05:00) Eastern Time (US & Canada) Total number of responses collected: 661 1: What Institution do you represent? (Institution:) (Respondents were limited to brief text responses) Response Chart Frequency Count Georgia State University 5.3% 35 Abraham Baldwin Agricultural College 1.8% 12 Georgia Perimeter College 4.5% 30 RESA 0.3% 2 Southern Polytechnic State University 1.8% 12 Georgia Highlands College 2.4% 16 Gainesville State College 4.1% 27 Bainbridge College 2.6% 17 Waycross College 3.9% 26 University of Georgia 2.1% 14 Macon State College 4.1% 27 Georgia Health Sciences University Georgia College & State University 2.3% % 10 WebMBA 1.4% 9 Created using Vovici

106 Georgia Southwestern State University 4.5% 30 East Georgia College 2.7% 18 Gordon College 1.1% 7 University of West Georgia 2.9% 19 Darton College 3.5% 23 Augusta State University 0.9% 6 Valdosta State University 7.9% 52 Kennesaw State University 5.4% 36 USG Outreach (Best Practices) 0.5% 3 Columbus State University 1.7% 11 Middle Georgia College 2.6% 17 SREB 2.0% 13 Savannah State University 0.5% 3 Albany State University 2.4% 16 Atlanta Metropolitan College 3.5% 23 USG Training Initiatives 1.1% 7 College of Coastal Georgia 0.5% 3 Clayton State University 0.3% 2 Alternative Media Access Center (AMAC) North Georgia College & State University Armstrong Atlantic State University 0.8% 5 1.7% % 21 Created using Vovici

107 GeorgiaONmyLINE 2.4% 16 Fort Valley State University 2.0% 13 Georgia Southern University 5.7% 38 Georgia Gwinnett College 1.7% 11 South Georgia College 0.5% 3 Other Responses 0.3% 2 Valid Responses 661 Total Responses 661 Created using Vovici

108 2: Are you an Student or Instructor? (Respondents could only choose a single response) Created using Vovici

109 3: Did you evaluate Blackboard Learn 9? (Respondents could only choose a single response) Created using Vovici

110 4: Blackboard Learn 9 Instructor Evaluation (Test Results) Actions: Created using Vovici

111 Created using Vovici

112 5: Overall, do you recommend this platform (Blackboard Learn 9)for adoption? (Respondents could only choose a single response) Created using Vovici

113 6: Did you evaluate Desire2Learn? (Respondents could only choose a single response) Created using Vovici

114 7: Desire2Learn Instructor Evaluation (Test Results) Actions: Created using Vovici

115 Created using Vovici

116 8: Overall, do you recommend this platform (Desire2Learn) for adoption? (Respondents could only choose a single response) Created using Vovici

117 9: Did you evaluate Moodle? (Respondents could only choose a single response) Created using Vovici

118 10: Moodle Instructor Evaluation (Test Results) Actions: Created using Vovici

119 Created using Vovici

120 11: Overall, do you recommend this platform (Moodle) for adoption? (Respondents could only choose a single response) Created using Vovici

121 12: Did you evaluate Pearson LearningStudio? (Respondents could only choose a single response) Created using Vovici

122 13: Pearson LearningStudio Instructor Evaluation (Test Results) Actions: Created using Vovici

123 Created using Vovici

124 14: Overall, do you recommend this platform (Pearson LearningStudio) for adoption? (Respondents could only choose a single response) Created using Vovici

125 15: Did you evaluate Sakai? (Respondents could only choose a single response) Created using Vovici

126 16: Sakai Instructor Evaluation (Test Results) Actions: Created using Vovici

127 Created using Vovici

128 17: Overall, do you recommend this platform (Sakai) for adoption? (Respondents could only choose a single response) Created using Vovici

129 18: Did you evaluate Blackboard Learn 9? (Respondents could only choose a single response) Created using Vovici

130 19: Blackboard Learn 9 Student Evaluation (Test Results) Actions: Created using Vovici

131 Created using Vovici

132 20: Overall, do you recommend this platform (Blackboard Learn 9)for student adoption? (Respondents could only choose a single response) Created using Vovici

133 21: Did you evaluate Desire2Learn? (Respondents could only choose a single response) Created using Vovici

134 22: Desire2Learn Student Evaluation (Test Results) Actions: Created using Vovici

135 Created using Vovici

136 23: Overall, do you recommend this platform (Desire2Learn)for student adoption? (Respondents could only choose a single response) Created using Vovici

137 24: Did you evaluate Moodle? (Respondents could only choose a single response) Created using Vovici

138 25: Moodle Student Evaluation (Test Results) Actions: Created using Vovici

139 Created using Vovici

140 26: Overall, do you recommend this platform (Moodle)for student adoption? (Respondents could only choose a single response) Created using Vovici

141 27: Did you evaluate Pearson LearningStudio? (Respondents could only choose a single response) Created using Vovici

142 28: Pearson LearningStudio Student Evaluation (Test Results) Actions: Created using Vovici

143 Created using Vovici

144 29: Overall, do you recommend this platform (Pearson LearningStudio) for student adoption? (Respondents could only choose a single response) Created using Vovici

145 30: Did you evaluate Sakai? (Respondents could only choose a single response) Created using Vovici

146 31: Sakai Student Evaluation (Test Results) Actions: Created using Vovici

147 Created using Vovici

148 32: Overall, do you recommend this platform (Sakai) for student adoption? (Respondents could only choose a single response) Created using Vovici

149 USG LMS Sandbox Evaluation Survey Type: USG LMS Task Institutional Report Date: 4/18/2011 Time Zone in which Dates/Times Appear: (GMT-05:00) Eastern Time (US & Canada) Total number of responses collected: 661 5: Overall, do you recommend this platform (Blackboard Learn 9)for adoption? (Key: Top, Neutral, Bottom, N/A) 2 Chart Top Neutral Bottom Total Mean Student % 0.0% 0.0% % Instructor 75.3% % 11.0% 13.7% % 20: Overall, do you recommend this platform (Blackboard Learn 9)for student adoption? (Key: Top, Neutral, Bottom, N/A) 2 Chart Top Neutral Bottom Total Mean Student 64.5% 21.5% % 14.0% 21.5% % Instructor % 0.0% 0.0% 8: Overall, do you recommend this platform (Desire2Learn) for adoption? (Key: Top, Neutral, Bottom, N/A) % 2 Chart Top Neutral Bottom Total Mean Student Created using Vovici

150 0.0% 0.0% 0.0% % Instructor 67.1% % 15.3% 17.6% % 23: Overall, do you recommend this platform (Desire2Learn)for student adoption? (Key: Top, Neutral, Bottom, N/A) 2 Chart Top Neutral Bottom Total Mean Student 70.8% % 16.8% 12.4% % Instructor : Overall, do you recommend this platform (Moodle) for adoption? (Key: Top, Neutral, Bottom, N/A) 0.0% 0.0% 0.0% % 2 Chart Top Neutral Bottom Total Mean Student % 0.0% 0.0% % Instructor 28.7% 58.3% : Overall, do you recommend this platform (Moodle)for student adoption? (Key: Top, Neutral, Bottom, N/A) 28.7% 13.0% 58.3% % 2 Chart Top Neutral Bottom Total Mean Student 24.8% 57.5% % 17.7% 57.5% % Created using Vovici

151 Instructor % 0.0% 0.0% 14: Overall, do you recommend this platform (Pearson LearningStudio) for adoption? (Key: Top, Neutral, Bottom, N/A) % 2 Chart Top Neutral Bottom Total Mean Student Instructor 37.7% 19.3 % 0.0% 0.0% 0.0% % 43.0% % 19.3% 43.0% % 29: Overall, do you recommend this platform (Pearson LearningStudio) for student adoption? (Key: Top, Neutral, Bottom, N/A) 2 Chart Top Neutral Bottom Total Mean Student 51.4% 19.3 % 29.4% % 19.3% 29.4% % Instructor : Overall, do you recommend this platform (Sakai) for adoption? (Key: Top, Neutral, Bottom, N/A) 0.0% 0.0% 0.0% % 2 Chart Top Neutral Bottom Total Mean Student % 0.0% 0.0% % Instructor 34.7% % Created using Vovici

152 % 34.7% 19.1% 46.2% % 32: Overall, do you recommend this platform (Sakai) for student adoption? (Key: Top, Neutral, Bottom, N/A) 2 Chart Top Neutral Bottom Total Mean Student 52.8% 20.8% 26.4% % 20.8% 26.4% % Instructor % 0.0% 0.0% % Created using Vovici

153 Appendix 9 Task force sandbox evaluation rubric form and summary of results

154 Areas of Consideration Doesn't Meet Blackboard 9 Desire2Learn Moodle Meets Exceeds Superior Teaching and Learning Communication/Collaboration Features Calendar Announcements Course Discussions Chat Whiteboard Groups Wikis Blogs Mobile messaging Ease of Use Comments: average Doesn't Meet Meets Exceeds Superior average Doesn't Meet Meets Exceeds Superior average Content Features File types supported Ease of integrating media (a/v) Ease of linking to a Web site Creating a page File upload (one file/zipped) File Manager/folder structure File sharing between class sections Learning Modules design/flexibility Content Management Create from Master Course? Portfolios Student workspace Glossary Ease of Use Comments:

155 Areas of Consideration Doesn't Meet Blackboard 9 Desire2Learn Moodle Meets Exceeds Superior Assessment Features Quizzes, Surveys Modify by student? Database of questions Selective Release/time control Import/export Discussions for grades Group assessments Presentations Plagiarism detection tools Assignment Dropbox Ease of Use Comments: average Doesn't Meet Meets Exceeds Superior average Doesn't Meet Meets Exceeds Superior average Class Management Features Roster Tracking reports Group management Selective release Learning module/structure/flexibility Import/export/migration Design/structure flexibility Ease of Use Comments: Gradebook Features Import/export to Excel Flexibility of views Workflow Customizeability Ease of Use Comments:

156 Pearson Sakai Areas of Consideration Teaching and Learning Communication/Collaboration Features - Calendar - Announcements - Course - Discussions - Chat - Whiteboard - Groups - Wikis - Blogs - Mobile messaging Ease of Use Comments: Doesn't Meet Meets Exceeds Superior average Doesn't Meet Meets Exceeds Superior average Content Features - File types supported - Ease of integrating media (a/v) - Ease of linking to a Web site - Creating a page - File upload (one file/zipped) - File Manager/folder structure - File sharing between class sections - Learning Modules design/flexibility - Content Management --Create from Master Course? - Portfolios - Student workspace - Glossary Ease of Use Comments:

157 Pearson Sakai Areas of Consideration Assessment Features - Quizzes, Surveys --Modify by student? - Database of questions - Selective Release/time control - Import/export - Discussions for grades - Group assessments - Presentations - Plagiarism detection tools - Assignment Dropbox Ease of Use Comments: Doesn't Meet Meets Exceeds Superior average Doesn't Meet Meets Exceeds Superior average Class Management Features - Roster - Tracking reports - Group management - Selective release - Learning module/structure/flexibility - Import/export/migration - Design/structure flexibility Ease of Use Comments: Gradebook Features - Import/export to Excel - Flexibility of views - Workflow - Customizeability Ease of Use Comments:

158 Gradebook Features Class Management Features Assessment Features Content Features Communication/Collaboration Features Blackboard Desire2Learn Moodle Pearson Sakai

159 Appendix 10 Guiding criteria document

160 Guiding Criteria A. These criteria have guided our work up to this point: The platform must: Be well-established and stable operationally. Be able to be integrated with a student information system Be committed to being fully compliant with accessibility laws and recommendations Meet basic functionality requirements of our current system. B. These criteria will guide our work as we move into the next stage of our work: The platform should: Satisfy the technical requirements set forth by the university system ITS department Allow the university system to provide necessary services while being fiscally responsible Offer advanced functionality and usability to our stakeholders Allow for integration with various third party solutions that our stakeholders expect to use in conjunction with the platform Have a future roadmap of product development that ensures that while the platform stays relevant and effective, it does not inflict painful migrations along the way Have a responsive and active support network, whether this is from a vendor company or from a community of users Offer access to the source code or to APIs, etc. to allow for internal USG technical experts to provide appropriate service to the system stakeholders

161 Appendix 11 Finalist questions and demo requests

162 Functional Questions for Vendors Demo Portion: 1. How can classes/sections be created from a master course? Please demonstrate. 2. How is content created? Please demonstrate. i.e. webpage/text, assignments, quizzes 3. Please demonstrate your built in content creation tools such as the WYSWIG HTML editor 4. Please demonstrate how a quiz can be created via the following methods: a. From within the system b. Through applications such as Respondus c. Via publisher testbanks 5. How can quiz settings be modified for a single student? Please demonstrate. 6. How are quizzes completed by students and how are they graded? a. How are written response (essay) questions graded? Please demonstrate. 7. How are assignments completed by students and how are they graded? Please demonstrate 8. Please show how mathematical expressions can be input by a student. 9. How are grades calculated and displayed in the grade book? Please demonstrate. 10. How are rubrics created and used? Please demonstrate. 11. How are plagiarism detection tools accessed and used? Please demonstrate. 12. How can groups be created and used? Please demonstrate. 13. What communication tools are available? Please demonstrate any of the ones listed below and any others you wish to show us: Are chat rooms with webcam support included? Is instant messaging included? Are wikis included? Are blogs included? 14. How is student activity tracked? Please demonstrate. 15. How are publisher course cartridges loaded? Please demonstrate or illustrate. 16. Please demonstrate this platform s mobile capabilities/features.

163 Q&A Portion: 1. Explain how master/child classes/sections works. For example, suppose a master course is set up with a number of children sections. Can information (assignments/announcements/quizzes, etc.) be sent down from master to some sections but not all, and without erasing anything that has been done already in the child sections. Similarly, can information be sent from child to child section? 2. What types of content migration tools are available? 3. How do you handle audio/video content? 4. What tools are included for students? Portfolios? Workspace? 5. What does file storage/management look like from both the instructor and student perspective? 6. Are there student features for taking notes? 7. How do you integrate with the following third party solutions? a. Wimba Classroom/Voice/ Pronto b. Turnitin c. Turning Technologies d. Pearson s MyMathLab e. MapleTA f. Respondus LockDown Browser g. Respondus h. Smarthinking i. Others? 8. Please discuss browser/java requirements? What browsers/os are supported? What is your process for certifying new versions? 9. Do you allow access to an API? 10. Do you have a become user or true superuser type account available for administrators? 11. Would you describe how your LMS can/does integrate with a student information system (ie systems like Banner)? 12. Can course/student information be imported into your system? Is this a batch or realtime load? If an import is a batch process, how are students and/or other instructors added after the batch is complete? 13. Can grade/student information be exported from your system and loaded back into the student information system?

164 14. Can guest instructors or visitors/mentors be added to the course (independent of the load from the student information system)? If so, how is their access to student's personal information limited (contact information, grade information, etc.)? 15. What other roles does the platform allow (instructional designer, TA, Lead TA, etc.)? 16. Can a student (or instructor) limit what information can be seen about other students in a course (student privacy concerns)? For example, if a mass is being sent to the students on the class roster, will the addresses be hidden from others participating in the course? 17. Does the platform include a way to include pictures in the class roster? 18. Can students access a tutorial/introduction to the LMS upon initial login? 19. Is this platform compliant with Section 508c of the Rehabilitation Act? What are the accessibility features? 20. Does this platform support real time captioning? 21. Does this platform support live video feeds (remote sign language interpreting)? 22. How can users with disabilities access this platform using text to speech software (JAWS, VoiceOver), speech to text software (Dragon Naturally Speaking), and other accessibility software (refreshable Braillers, electronic notetakers, etc.)? 23. Does this platform offer text enlargement features for users with low vision? 24. Does this platform use images for technical support? Are alt tags used? 25. Does this platform s tutorial videos captioned? Do they have video descriptors? 26. Please discuss your mobile features and strategy. 27. Please discuss the philosophy of upgrades and future versions how complex is it to migrate from one version to the next and how is the timetable of upgrades determined? Is there support for past versions if an institution decides not to upgrade on your timetable? 28. Please discuss your product roadmap.

165 Appendix 12 Final overall rubric form and summary of results

166 Teaching and Learning Functionality Ease of use Accessibility Flexibility in roles Flexibility in use cases Ability to use third party plug-ins Integration with banner and other systems Mobility Browser/Java issues Getting There Migrating course content and tool functionality from Vista Migrating content from other situations Training needs Implementation set up issues Support and Community If commercial, responsiveness of vendor If commercial, availability of support/timeliness of support from vendor If commercial, other add-ons, and other product tie-ins If open-source, size and relevance of community If open source, responsiveness of community If open source, availability of commercial affiliate support Blackboard 9 Desire2Learn Moodle

167 All ability to support in-house at ITS and on campuses All infrastructure needs Moving Forward Future roadmap in terms of functionality Frequency and severity of upgrades needs for frequent downtime/migration Ease of migration to future versions Ease of migration to other products Cost of Ownership Hardware costs Human costs Implementation costs Support costs Blackboard 9 Desire2Learn Moodle

168 Teaching and Learning Functionality Ease of use Accessibility Flexibility in roles Flexibility in use cases Ability to use third party plug-ins Integration with banner and other systems Mobility Browser/Java issues Getting There Migrating course content and tool functionality from Vista Migrating content from other situations Training needs Implementation set up issues Support and Community If commercial, responsiveness of vendor If commercial, availability of support/timeliness of support from vendor If commercial, other add-ons, and other product tie-ins If open-source, size and relevance of community If open source, responsiveness of community If open source, availability of commercial affiliate support Pearson Sakai

169 All ability to support in-house at ITS and on campuses All infrastructure needs Moving Forward Future roadmap in terms of functionality Frequency and severity of upgrades needs for frequent downtime/migration Ease of migration to future versions Ease of migration to other products Cost of Ownership Hardware costs Human costs Implementation costs Support costs Pearson Sakai

170 Blackboard 9 Desire2Learn Moodle Pearson Sakai Teaching and Learning Getting There Support and Community Moving Forward Cost of Ownership First Choice Second Choice Third Choice Fourth Choice Fifth Choice Teaching and Learning Desire2Learn Sakai BlackBoard 9 Moodle Pearson Getting There BlackBoard 9 Desire2Learn Moodle/Sakai Pearson Support and Community Desire2Learn Sakai BlackBoard 9 Moodle Pearson Moving Forward Bb9/D2L/Sakai Moodle/Pearson Cost of Ownership Desire2Learn Pearson Blackboard 9 Moodle/Sakai

171 Appendix 13 Relative costing information and considerations

172 There are many components in the total cost of ownership of a learning management system. The primary ones used by ITS to provide cost estimates are: License fees these include the base fee along with some bundled services Hardware costs Personnel costs for hardware support, software support, system administration, etc. Training costs both for ITS staff and for training of the trainers at the institutions Course and content migration support Integration costs both with third party services and for customization at the different institutions Reporting costs End-user costs For the purposes of task force deliberations, four primary figures were used: License fees Infrastructure costs Support personnel costs Total cost of ownership (as calculated by ITS) Due to the ongoing nature of the negotiations and the need to keep some level of security around those negotiations until the task force results are made public, the actual dollar figures were estimated and then ranked from lowest to highest (in lieu of firm figures). The task force feels that the other (non-cost related) support in favor of the task force recommendations is strong enough that unless the actual rankings of the cost figures changes dramatically that this estimation process is sufficient. For the open source platforms, there are no license fees. However, it was recognized that there are commercial affiliates for each of these platforms and that some of the costs incurred for transition and implementation could be in the form of outsourcing rather than in-house expenses. The task force did not differentiate between these two types of expenses. For two of the commercial platforms, there were two different options presented. For the primary deliberations, in each case, the option of the most comprehensive offering from the platform was considered. However, when discussing secondary recommendations in the case of severe fiscal constraints, the more scaled back offers were considered. For Pearson Learning Studio, this involved a new offer that we have been asked not to reveal in a public document at this time. However, this is a new offering from Pearson, so it was not checked during the reference checks and there is no assurance of reliability or quality. Furthermore, it does not allow for ITS internal hosting and so it was not ruled acceptable by the task force. For Blackboard, there is an offer to keep the same price as is now in place ($590K) for an additional five years. This is a scaled back offer since the USG would still incur the extra license fees for the add-on services that are used systemwide (including for example Wimba/Elluminate, Connect, and Mobile). As noted in the report, this is only an acceptable alternative if the over-riding constraint is money for

173 licenses. Note that if this solution is chosen then there must still be an investment in the transition (migration and training), and either a heavy investment must be made in hardware, or there will not be customization available for the individual campuses, which is one of their highest priorities for the new LMS. Here are the cost comparisons for the full offerings of each platform (0 - $0 cost, 1= lowest, 5 = highest) Infrastructure Costs Support Staff Costs Total Cost of Ownership (5 years) LMS Offering License Costs Blackboard Learn Desire2Learn Moodle Pearson Learning Studio Sakai In the total cost of ownership over five years, the gap between Blackboard 9 and Desire2Learn is currently estimated at about $1.3M. The same time period gap between Desire2Learn and the open source platforms is estimated at $3.5M.