Neither Satisfied nor Dissatisfied. Dissatisfied. 1 Customizing curriculum Managing student grades

Transcription

1 Initial Report Last Modified: 12/09/ Based on your experience with the Blackboard test environment, how do you rate your satisfaction with the following LMS functions? If you did not test this functionality, please choose N/A. # Question Dissatisfied Dissatisfied Neither Satisfied nor Dissatisfied Satisfied Satisfied Mean 1 Customizing curriculum Managing student grades Tracking student progress Creating, managing and editing tests online Collaborating with students online Creating and customizing reports Communicating in multiple ways with other users Customizing curriculum Managing student grades Tracking student progress Creating, managing and editing tests online Collaborating with students online Creating and customizing reports Communicating in multiple ways with other users Min Value Max Value Mean Variance Standard Deviation

2 2. How do you rate the ease of use for the following Blackboard LMS features? If you did not test this functionality, please choose N/A. # Question Difficult Difficult Neither Difficult nor Easy Easy Easy Mean 1 Quizzes, assignments Pages, files, other content delivery tools Synchronous communication tools (e.g. chat, online meetings) Asynchronous communication tools (e.g. discussions, mail) Rubrics, learning outcomes, analytics Quizzes, assignments Pages, files, other content delivery tools Synchronous communication tools (e.g. chat, online meetings) Asynchronous communication tools (e.g. discussions, mail) Rubrics, learning outcomes, analytics Min Value Max Value Mean Variance Standard Deviation

3 3. Overall, the Blackboard LMS was intuitive and easy to use. # Answer Bar Response % 1 Strongly Disagree 7 9% 2 Disagree 13 17% 3 Neither Agree nor Disagree 12 16% 4 Agree 33 44% 5 Strongly Agree 10 13% 75 Value Min Value 1 Max Value 5 Mean 3.35 Variance 1.42 Standard Deviation

4 4. What role did you assume while testing? # Answer Bar Response % 1 Faculty 61 81% 2 Administrator 10 13% 3 Instructional designer 3 4% 4 Student 1 1% 75 Value Min Value 1 Max Value 4 Mean 1.25 Variance 0.35 Standard Deviation

5 5. Please provide any additional comments regarding the Blackboard LMS. Text Response I've been using Blackboard's content delivery and gradebook for a while so it is difficult for me to really judge how intuitive it is. My gut reaction is that the Blackboard gradebook allows a tiny bit more in the customization than Canvas (and much more than Brightspace), but you have to invest a little time to learn Blackboard's gradebook. I like the way the first few words of any announcement I post are right on the student's Dashboard, so it right there as soon as they enter Blackboard, even for a different course. Not clear to me that that is the case in the other LMS. I've used BlackBoard for years and I've never found to be user-friendly or flexible. In my opinion it is anti-creative. Blackboard is OK once you understand it. However the grading section is clunky. I found it frustrating at times trying to see student grades. found it easy to navigate and very intuitive. Also had lots of communication tools available Had used Blackboard before years ago - not much has changed. Has always been a bit clunky to use. Found myself quite frustrated trying to accomplish simple tasks. Page creation was difficult - hard to position images, errors when adding youtube link. Gradebook is horrible. Do like the course template option. The video tutorial was grueling - 30 mins long! Would have preferred seeing short clips on each aspect or feature. Overall, I am still not impressed with Blackboard at all!!! Nice clean interface and easy to customize and use. Blackboard Learning system is fine, but the dashboard is too busy and may be overwhelming to students accessing the course for the first time. It should be more organized and not have so much stuff on it. It was confusing to me also. Good features to help the student navigate the course. Very hard to navigate and manage. Not user friendly, at all. All standard LMS platforms have a ton of features and tools. It is impossible to assess and quantify the quality of the LMS based no the few questions listed within this survey. Although Bb is simple to use, ones familiarity with it helps get you started. Bb is simplistic in nature and does not offer many customization tools. Its ability to adapt to different tools in the marketplace seems limited. For example the number of LTIs that you can incorporate is limited and thus Bb very constrictive. Overall, Bb is easy to use but not the best LMS the market has to offer today. Bb is not a hosted solution and needs personnel and hardware to manage and maintain. Costs benefit analysis in each of these situations needs to be considered and assessed. Bb also does not update its system has robustly as some of the other lms options. Other LMS have a large community of users that one can engage in and this helps in learning the system. As far as students are concerned, the look and feel of all courses is pretty standard and boring. Students have repeatedly told us that some level of course customization beyond that of a header is important. Overall Bb is a decent system but not one that I would pick. Blackbaord I have been a Blackboard administrator for 12 years prior to switching to Canvas 3 years ago. I've been with Blackboard up until their 9.0 release. Having been away from Blackboard for 3 years, I am uncomfortable making any generalizations about its suitability as a tool for teaching and learning. However, I am comfortable commenting on the company and their development processes. I was unable to test uploading batches of users. However, in the past it was quite possible to do so, and I'm assuming the process of Snapshots hasn't changed to much. They were working toward a different model for loading SIS data as we were leaving in If that new model is the ONLY model for loading data today, it would represent a challenge for us. It is important that we be able to load data automatically from our SIS into Blackboard. Manually creating courses would be impossible given our scale. While they are hosting the application for us, every institution is on a different instance of Blackboard, with different versions and different compatibilities. Each school chooses for themselves when they upgrade and to what version. This results in a split where different schools are on different versions, and courses and students may encounter compatibility problems moving between them. Blackboard's upgrade model piles many large changes into less frequent "mega updates" Bug fixes are slower to release this way, and we are forced to accept bug fixes along with feature changes in single, large increments. This is very different from Canvas and D2L's model of continuous delivery, where small, changes are regularly released with reduced impact on the customers. Bugs in Canvas/D2L are also fixed faster because the system is built from the ground up to be easy to update. Updates with Blackboard are a "big deal" that must be tested, planned, coordinated with everyone. This limited them to 2-3 times a year. Blackboard is a fail in this regards. Documentation was difficult to find. I attempted to find a manual telling me about administrator functions and data loading from the SIS, but was unable to do so. This does not bode well for using this product in production. I already had familiarity with this LMS before examining additional features. I think that its help environment is not intuitive and easy to navigate. One has to dig for help. Faculty have different types of queries and concerns than do students; but the help menus do not appear to adequately bifurcate between these two different roles. Blackboard works...but that's about it. It is stale, cold, and not user friendly. Compared to the other two systems, it seems quite outdated. Blackboard collaborate is an unnecessary impediment--why not use Facetime or Skype and just directly connect with students? If LMS platforms were a physical space, Blackboard would be a dungeon. I use LMSs in two ways. In smaller classes, I simply store stuff there. Grades, files, agendas, etc. In this environment, nothing the LMS does is critical. Blackboard is fine. It does nothing really well, bur nothing bad either. For larger classes (500+ a semester), the LMS is now driving the bus -- I need it to manage the course. I have both on campus and 100% online students in the same course. I use the LMS for test delivery as well as feedback. I need to be able to control everything, and get reports about everything. First, to do anything collaborative, I need to assign students to groups based on data not in Blackboard, which means I need to upload Group Assignments. It appears to allow that. For Peer Reviews, I need to see who was assigned to Peer Review what, and if they have done that. Reporting here isn't clear. Quiz creation and deployment looks solid -- usual features. Can allow/disallow users to take exam. Lots of features for grading, partial credit, feedback, etc. Blackboard also allows for selective release easily -- I can force students to complete 1 task before the next task is available. That is helpful. Other LMSs have this, but do not make it easy to implement. Gradebook has all I need. Gradebooks are never perfect. I usually find they are structured with certain expectations of how it is used. That is a mistake. Here, the gradebook is actually a method to post anything you want a student to see individually. That is very helpful. Blackboard used to also have a Feedback feature -- non-gradebook data to upload for students to view with individual information. That feature seems to have disappeared. But I can still accomplish the same task via the gradebook. Did not test collaboration or communication tools. I don't use them, and have no comment. For me, it just the nth way for students to communicate, and that is confusing given our environment. The Retention Center tool is an excellent resource to ensure faculty are being alerted to students that are having difficulties. NB: I also tested a number of aspects in student preview. A few comments: 1) Blackboard seems to make a big deal about the Content Collection, which looks really useful, but I read several places online that this is an add-on feature not included in the regular package. That would concern me. 2) The "build a syllabus" section was very clunky and difficult to figure out. 3) In general, I found course organization fairly complex and over-compartmentalized. Using modules, my students would not be able to see an entire week's work at a glance; rather, they would have to click through a number of different pages to see the week's goals, assigned readings, and instructions for various assignments. And forget about seeing the entire course at a glance! 4) The grading systems seem robust, and the Retention Center seems a good idea, but I was really annoyed by the fact that I was required to assign "number of points possible" on all assignments. Does this mean a binary 0/1 for pass/fail assignments? The grading apparatus allowed me to designate an assignment as "pass/fail," but the assignment setup wouldn't. Overall, my impression is that Blackboard's very structured approach is useful if one's syllabus consists only of clearly-defined assignments gradable on a point scale. If one needs more flexibility, ambiguity, or just wants to tell students, "Week 1: Read the first half of _The Great Gatsby_", it's kind of a nightmare. Seemed difficult, clunky and out-dated. I found the user interface to be both bland and visually overstimulating. Considering its ubiquity, I was surprised at how clunky and difficult to use this LMS seemed to use. I did not find blackboard to be an appealing platform. It felt designed in the 1990s and most of its tools were less appealing than other systems we were testing. I was struck by how difficult the most basic operations were. The user experience was horrible. Very hard to use and unintuitive. Every page was laid out differently and made no sense compared to the previous page. Would never use or recommend Blackboard to anyone based on this. I am a constant user of blackboard at my University, so am quite familiar with Blackboard. While relatively easy to use, it can be a bit "clunky." I used Blackboard for 3 years at Florida State, and hated nearly every minute of it. While it does what it needs to do, it is neither elegant nor intuitive to use. I'm familiar with Blackboard as the main CMS currently in use at Florida State. It is fine. It is somewhat less adaptable and more "clunky" than Canvas for the types of courses I teach; it takes a little longer to "Build Content" and arrange it for the students, which is personally my primary use of an LMS. Blackboard can be frustrating at moments -- we happen to use it now -- and I hoped to be pleasantly surprised by the alternatives (Canvas and D2L). I ended up appreciating Blackboard much more. Its overall architecture works very well for the undergraduate and graduate university environment. It provides tools that the alternatives did not (for reviewing student records, uploading large sets of information, and managing files). I came to better understand a valuable tool (to flag under-performing students) included in

6 this package. It allowed me to upload reading materials, assignments, and related materials with ease. In addition, it is the only platform I reviewed which (in the Sandbox environment) included a larger 'organizations' component. This is vital for our daily university operations, and I frequently flip between these sections of Blackboard for teaching-related tasks. The absence of such 'organizations' components in Canvas and D2L -- at least what I could spot, after quite a bit of looking-around in the Sandbox versions -- was troubling (even more than the 'list your high school' box in D2L which revealed its target market). There are almost too many tools to look through on the toolbar to the left, it takes a while to find something, however, everything I was looking for was there. I really liked how you can organize your different learning modules and once you open one it shows everything available within one This version of Blackboard has a nicer landing page and I believe it will be more intuitive to users. The notifications are improved while the course tab and user profile look similar to the current version, though both of these features are currently satisfactory. I was interested in the portfolio tab and the artifacts option but, after exploring all of the LMS options, I see that this was offered in different capacities through each system. The content tab is easy to use and would make building a course a pretty simple process. I really like that each classroom is integrated with a Collaborate classroom! Searching through tabs was easy and I was intrigued by the Community section though it wasn't populated so I couldn't see whether it was similar to the commons or learning repository offered by the other LMS options. I thought it was really smart to include a link for mobile compatibility testing and consider mobile accessibility. The blackboard system was easy to navigate and access. There were multiple ways to collaborate with students. The system provided an easy way to provide grading rubrics for individual assignments. Things I didn't test: I'd have liked to meet with our office of Assessment to test rubrics and analytics thoroughly, but that wasn't going to happen in this time frame. I never make heavy use of communication systems in LMSs myself, so that's not something I'm going to be able to help evaluate well. Blackboard Learn's interface has certainly been improved over the last few years but several fundamental issues remain. * Java requirements for several user interface elements. No longer acceptable. * No readily available documentation for rich import/export of questions - what options exist, if any? * Calendar entries still simple entities with no metadata - real problem for medical schools. The reason I did not test a lot of the functionality is because I did not find Bb easy to navigate or easy to figure out how to do things. I could work with this. I like having the ability to upload test questions directly rather than through a third party product (e.g., Respondus). I also like the flexibility built into the Announcement tool. The rubrics in Blackboard are similar to those in Canvas, but they are not nearly as good because there is no speedgrader. Moreover, in Blackboard, you have to grade by student rather than by assignment (how Canvas speedgrade works), and it is more difficult and time-consuming. This seems a more sophisticated tool than Desire2Learn, but correspondingly more complex and difficult to master. BB would require a lot of training and experience to use to its fullest extent. However, a couple of nice features that Desire2Learn does not have. Not sure the complexity is needed, however, for most courses. I actually went in as a student but I was primarily faculty. I have experience with Bb 9.0 and found that Bb has is a solid useable LMS. Everything seemed to require a lot of effort to build and course tabs were overwhelming--too cluttered, not a clean look. Frustrating at times with "access denied messages" in the preloaded courses. Had access as a student, faculty and admin. First, I attempted to upload/import a zip file from my current LMS. I had to spend a long time trying to locate the menu that would allow me to do so. The process of importing the file failed miserably. I received an telling me that it was not successful. My next strategy was to try to create a course "from scratch." I spent a long time trying to figure out how to set up a course. I felt that despite the fact that there were tutorials, I kept getting lost and ended up going back to my homepage. I ended up being very frustrated! Blackboard does have many features, but the user interface is more difficult and complex than the other two candidates. This complexity couild be especially challenging to instructors who simply want to use Blackboard as a companion to a traditional course and only need to use basic features without investing large amounts of time learning to master an LMS. I feel that Blackboard Learn has the most tools with regards to a complete online experience for the user. I also like that it s the LMS with the most integrations (via installed tools directly in the LMS or via LTI configurations). It is the only LMS that has a diverse range of products as well, ranging from its own license for synchronous communication (Bb Collaborate), a whole area for use of customizable content across the institution (Bb Community Engagement), and also a whole other component dedicated to looking at big picture data (Bb Analytics). I am very happy with the product. The learning curve for understanding Blackboard is pretty steep. The average user will immediately find themsevlesconfused even with the help videos and guides. Much more time and resources will need to be devoted to support if you pick this one. Since this is the system that we currently use, I feel compelled to note that the major disadvantage is that it is often incredibley slow. This will give Blackboard an unfair disadvantage in this testing, since we cannot evaluate the speed of the other systems when they are in full deployment. Found interface intuitive, and adding content logical, step-wise. I appreciated the consistent navigation options between sections (Build content, assessment, tools, and partner content). I liked that the Portfolio feature seems well-integrated. Also liked that the Course Tools page shows an overview of available tools, arranged alphabetically with icon. BB has multiple ways to communicate with and among students. I thought the Student Preview button was a very helpful feature, allowing the faculty to view as a student without exiting and coming back in under a different login. The Blackboard LMS does provide a good set of tools and is relatively easy to use. I would rank this one as # 2 of the 3 (with Canvas being # 1). One of the drawbacks is the interface. I found myself going back to Canvas to compare differences as Canvas was so nice and the interface and user experience quite contemporary. We currently are using Blackboard, and I find it that has more costumization and feature. Blackboard's design is still very lacking in ease of use. This platform has a steep learning curve for faculty and students and has too many oddities to make its use across a system a smooth experience. Their user interfaces need significant revision to maximize the value of otherwise decent capabilities. I did watch the video, which gave me some ideas about how to use, but focused way too much on institutional administration early on. I really care about content delivery, student interaction, and some grading functions (just tracking standing--we don't use numerical/letter grades). Actually using it (and I've used a version on another campus) was less intuitive than I expected from the old war horse that's been around for ever. This seems really designed for a large university with lots of adjuncts coming and going. I did like the portfolio option, but didn't get that deep into it. I think this one's outdated, esp. compared to Canvas, from what I've read. I like D2L a bit better than Blackboard oddly, though they are considered pretty similar. I was reviewing synchronous tools. Elluminate is limited in that all uploaded documents must be in whiteboard or video format. I couldn't start a session to see how it all worked exactly, but the system seemed fairly robust, other than the limit on uploading content. I appreciated the customization. Value 47