1 st Usability Testing for the New SmartSite Gradebook NG

Size: px

Start display at page:

Download "1 st Usability Testing for the New SmartSite Gradebook NG"

Jason Cox
8 years ago
Views:

1 1 st Usability Testing for the New SmartSite Gradebook NG There are a series of short activities to perform. These are all tied to the Instructor role. Remember that we are not testing you, but rather the usability of the product. If you do not understand how to do something or cannot figure it out, it is better to say so now as it will help us improve the final release of the product. Instructions: Please think aloud as you go through the activities. If you do not understand the instructions or you cannot perform the activity, you may ask for assistance, but first look over the screen carefully to see if you have just overlooked the necessary button, graphic, or link. There is no reason to rush doing these scripts. We are listening for how easily you can follow the steps in doing certain common activities associated with a gradebook, as well as your comments about the tool as you progress through the activities. If after three attempts you have not intuited the method we will discuss the design briefly and instruct you. We do not want you to become frustrated but to identify areas where instructors might become confused. During the activities, and perhaps during the discussions, when you ask us questions we will most often rephrase the question and or simply ask you what you mean. We are doing this as a way to explore your thought process and how you interact with Gradebook NG and how you interpret its design. If you do not finish the scripts, it is not a reflection on you, but rather the software itself. This test is designed for people who have different levels of experience with gradebooks. page 1

If you do not understand how to do something or cannot figure it out, it is better to say so now as it will help us improve the final release of the product.

2 Open the New Gradebook to the default view. 1. What kind of things/actions do you think you can do from the default interface? 2. Is there anything you would expect that appears to be missing? 3. Is there anything that is not clear at this moment as to its function or purpose? 4. You will be creating 3 categories with weighting. Where will you start to set those values for your gradebook? Task 1: Creating your gradebook 1. How would you set your gradebook to have weighted categories and items w/ points? Please guess out loud as you work. 2. Proceed w/ setting up your gradebook with weighted categories and items w/ points. Questions for Task 1: 1. Do you have any questions, reactions, and/or suggestions about this first exercise? 3. Were there settings that you didn t use that you are curious about? 4. Do you feel you can create another weighted category for Homework worth 25% with two graded items "HW#1," "HW#2," each worth 25 points? page 2

Task 1: Creating your gradebook 1. How would you set your gradebook to have weighted categories and items w/ points? Please guess out loud as you work. 2.

3 Task 2: Creating a Category 1. How would you create a category with weighting in this gradebook? Please guess out loud as you work. 2. Create a new category titled Homework. This category will be worth 25% of the final grade. 3. In this category, create two graded items, 25 points each, "HW#1" and "HW#2". Questions for Task 2: 1. Do you have any questions, reactions, and/or suggestions about this first exercise? 3. Were there settings that you didn t use that you are curious about? 4. Do you feel you can create another weighted category for Quizzes worth 35% with two graded items "Q1," "Q2," each worth 100 points? page 3

Do you have any questions, reactions, and/or suggestions about this first exercise? 3. Were there settings that you didn t use that you are curious about? 4.

4 Task 3: Creating a second and third Category 1. Create a new category, Quizzes, worth 35% of the final grade. 2. Create two graded items, Q1 and Q2 each worth 100 points. Place them in the Quizzes category. 3. Create a third category titled Assignments, worth 40%. Do not add an item to it just yet. Questions for Task 3: 1. Do you have any questions, reactions, and/or suggestions about this exercise? 3. Do categories have to add up to 100% of the course grade? 4. Was anything unclear or confusing about this overall task? 5. Would you be able to go back and edit this gradebook without instruction to change the item points and category weights? page 4

Do not add an item to it just yet. Questions for Task 3: 1. Do you have any questions, reactions, and/or suggestions about this exercise? 3. Do categories have to add up to 100% of the course grade?

5 Task 4: Including Categories and Items in the Course Grade 1. Categories: a. Can you tell if the categories are now included in the Course Grade? How? b. How would you include the categories in the Course Grade? c. Include the three categories in the Course Grade. 2. Items a. Can you tell if the items are now included in the Course Grade? How? b. How would you include the items in the Course Grade? c. Include all the items in the Course Grade. Questions for Task 4: 1. Do you have any questions, reactions, and/or suggestions about this exercise? 3. Do categories have to add up to 100% of the course grade? 4. Was anything unclear or confusing about this overall task? page 5

Can you tell if the items are now included in the Course Grade? How? b. How would you include the items in the Course Grade? c. Include all the items in the Course Grade.

6 Task 5: Working with Equally Weighted Categories and Graded Items 1. Let s examine the gradebook Categories. a. Study the weight for the Quizzes and Assignments categories. b. How would you weight both categories equally? c. Proceed to weight both categories equally. d. What happens? 2. Let s examine the gradebook items. a. Study the weights for all the items in the Homework categories. b. Edit the weights on one or two graded items in Homework. c. What happens? d. Remove one graded item from the course grade calculation. e. What happens? f. Assign extra credit to one item in Homework. g. What happens? h. What else, if anything is required of you to give extra credit? i. In Homework, set drop lowest grade. j. What happened? k. Uncheck the Drop Lowest Grade option in Homework. Questions for Task 5: 1. Do you have any questions, reactions, and suggestions about this exercise? 3. What do you think would happen to a student s grade who had submitted an assignment before you made the changes? 4. How do you feel about the various terms used in these interfaces? page 6

Study the weights for all the items in the Homework categories. b. Edit the weights on one or two graded items in Homework. c. What happens? d.

7 Task 6: Working with Unequally Weighted Items 1. Let s examine the items in the Homework category. a. Study the weights for all Homework items. b. Are the weights accurate or what you expected? c. Edit the weights on one or two graded items in Homework. d. What happens? e. Add 4, 100 point items to the Assignments category, but weight each with the following values: 25, 40, 15, 20. f. After creating the items, remove one of them from the calculation for the course grade g. What happens? h. Is this what you expect? i. Add the item back to the calculation for the course grade j. Now what happens? Questions for Task 6: 1. Do you have any questions, reactions, and/or suggestions about this exercise? 3. How do you feel about the various terms used in these interfaces? page 7

llowing values: 25, 40, 15, 20. f. After creating the items, remove one of them from the calculation for the course grade g. What happens? h. Is this what you expect? i. Add the item back to the calculation for the course grade j.

8 Working with Individual Students Task 7: 1. How would you return to the default view of the gradebook. 2. Return to the default view of the gradebook. 3. Consider what you need to do to view all of the items to enter grades. 4. Make all the items visible. 5. Enter all the grades for Student 1, Sakai. 6. How might you view only Student 1, Sakai s records? If you cannot access the records in three attempts we will give you the instructions. Please remember to tell us what was confusing. 7. While examining Student 1, Sakai's records, what are the new things you can do? 8. Which require explanation? 9. Can you view the records as Student 1, Sakai? 10. Can you return to the single student s records easily? Questions for Task 7: 1. Do you have any questions, reactions, and/or suggestions about this exercise? 3. How do you feel about the various terms used in these interfaces? Please log out of SmartSite and take a moment to visit this link and give us some last thoughts: gbng.notlong.com Thank you for helping our development efforts. We may contact you again for next week's second round of testing. page 8

How might you view only Student 1, Sakai s records? If you cannot access the records in three attempts we will give you the instructions. Please remember to tell us what was confusing. 7.

Usability Test Script

Usability Test Script Session 1 [date & time] [participant s name and location] [investigator s names] Step 1. Introducing yourself & read instructions Hi, my name is [ ] and I m going to be walking you