USABILITY TESTING PLAN. Document Overview. Methodology

Transcription

1 USABILITY TESTING PLAN Dcument Overview This dcument describes a test plan fr cnducting a usability test during the develpment f new ischl website. The gals f usability testing include establishing a baseline f user perfrmance, establishing and validating user perfrmance measures, and identifying ptential design cncerns t be addressed in rder t imprve the efficiency, prductivity, and end-user satisfactin. The usability test bjectives are: T determine design incnsistencies and usability prblem areas within the user interface and cntent areas. Ptential surces f errr may include: Navigatin errrs failure t lcate functins, excessive keystrkes t cmplete a functin, failure t fllw recmmended screen flw. Presentatin errrs failure t lcate and prperly act upn desired infrmatin in screens, selectin errrs due t labeling ambiguities. Exercise the website under cntrlled test cnditins with representative users. Data will be used t access whether usability gals regarding an effective, efficient, and well-received user interface have been achieved. Establish baseline user perfrmance and user-satisfactin levels f the user interface fr future usability evaluatins. Methdlgy Scenari Style Usability Testing Rund 1- Current Site 5 peple Nt prir website visitrs Set scenaris fr prspective student tasks Usability Testing Rund 1- New Site 5 peple Nt prir website visitrs Set scenaris fr prspective student tasks The participants' respnsibilities will be t attempt t cmplete a set f representative task scenaris presented t them in as efficient and timely a manner as pssible, and t prvide feedback regarding the usability and acceptability f the user interface. The participants will be directed t prvide hnest pinins regarding the usability f the applicatin, and t participate in pst-sessin subjective questinnaires and debriefing. Fcus Grup Style Current Students Fcus grup style Cntent pieces: srt in rder

2 Cntent pieces: nmenclature questins Landing page cntent layut Faculty and Staff Landing Page Fcus Grup Fcus grup style Cntent pieces: srt in rder Cntent pieces: nmenclature questins Landing page cntent layut We wuld like t have a cnversatin with yu t discver ways we can better meet yur needs n the website. We are creating a faculty and staff landing page fr the new website and want yur input t help us ptimize this landing page and sectin. Prcedure In-Persn Testing Participants will take part in the scenari style usability test at X. The participant s interactin with the website will be mnitred by the facilitatr seated in the same ffice. Nte takers and data lgger(s) will mnitr the sessins and take ntes. The facilitatr will instruct the participant t think alud s that a verbal recrd exists f their interactin with the website. The facilitatr will bserve and enter user behavir, user cmments, and system actins. After each task, the participant will cmplete the pst-task questinnaire and elabrate n the task sessin with the facilitatr. Remte Testing Participants will take part in the usability test via remte screen-sharing technlgy. The participant will be seated at their wrkstatin in their wrk envirnment. Verbal cmmunicatin will be supprted via telephne. Rles Facilitatr Prvides verview f study t participants Defines usability and purpse f usability testing t participants Assists in cnduct f participant and bserver debriefing sessins Respnds t participant's requests fr assistance Test Observers Silent bserver Serve as nte takers Test Participants Prvides verview f study t participants 2

3 Defines usability and purpse f usability testing t participants Assists in cnduct f participant and bserver debriefing sessins Usability Tasks: Scenari Style Testing Scenari #1: Yu are interested in finding ut if the ischl has a grad prgram. Find the prgram Find ut hw much it csts Find the applicatin t bkmark fr later Scenari #2: Yu are an undergraduate student at UW and yu need t declare a majr. Smene tld yu abut the Infrmatics prgram at the ischl. Find ut what infrmatics is Find ut what kind f jbs yu can get with this degree Make a cntact t get mre infrmatin Scenari #3: Yu are an internatinal student interested in human-cmputer interactin and yu want t g t the ischl. Find ut prgram ptins Find inf n hw t apply as an internatinal student Find the prfessrs wh teach human-cmputer interactin Scenari #4: Yu have a full-time jb but want t get training n infrmatin management t bst yur career skills. Find ut prgram ptins See what classes yu might take Find the schedule t make sure it desn t cnflict with yur jb Scenari #5: Yu want t attend the ischl t becme an infrmatin architect because yur mm tld yu they make a lt f mney. Find ut what prgrams will lead yu t this Find ut the prerequisites needed Send a message t an admissins persn t ask them if it s a gd career Usability Metrics Usability metrics refers t user perfrmance measured against specific perfrmance gals necessary t satisfy usability requirements. Scenari Cmpletin The scenari is cmpleted when the participant indicates the scenari's gal has been btained r the participant requests and receives sufficient guidance as t warrant scring the scenari as a critical errr. Critical Errrs Critical errrs are deviatins frm the targets f the scenari. Independent cmpletin f the scenari is a universal gal; help btained frm the ther usability test rles is cause t scre the scenari a critical errr. 3

4 In general, critical errrs are unreslved errrs during the prcess f cmpleting the task r errrs that prduce an incrrect utcme. Nn-critical Errrs Nn-critical errrs are errrs that are recvered frm by the participant r, if nt detected, d nt result in prcessing prblems r unexpected results. These errrs may be prcedural, in which the participant des nt cmplete a scenari in the mst ptimal means r errrs f cnfusin. Subjective Evaluatins Subjective evaluatins regarding ease f use and satisfactin will be cllected via questinnaires, and during debriefing at the cnclusin f the sessin. The questinnaires will utilize free-frm respnses and rating scales. Scenari Cmpletin Time (time n task) The time t cmplete each scenari, nt including subjective evaluatin duratins, will be recrded. Usability Gals Cmpletin Rate Cmpletin rate is the percentage f test participants wh successfully cmplete the task withut critical errrs. A 100% cmpletin rate is the gal fr each task in this usability test. Errr-free rate Errr-free rate is the percentage f test participants wh cmplete the task withut any errrs (critical r nn-critical errrs). An 80% errr-free rate is the gal fr each task in this usability test. Time n Task (TOT) The time t cmplete a scenari is referred t as "time n task". It is measured frm the time the persn begins the scenari t the time he/she signals cmpletin. Subjective Measures Subjective pinins abut specific tasks, time t perfrm each task, features, and functinality will be surveyed. At the end f the test, participants will rate their satisfactin with the verall system. Cmbined with the interview/debriefing sessin, these data are used t assess attitudes f the participants. Prblem Severity T priritize recmmendatins, a methd f prblem severity classificatin will be used in the analysis f the data cllected during evaluatin activities. 4

5 Impact Impact is the ranking f the cnsequences f the prblem by defining the level f impact that the prblem has n successful task cmpletin. There are three levels f impact: High - prevents the user frm cmpleting the task (critical errr) Mderate - causes user difficulty but the task can be cmpleted (nn-critical errr) Lw - minr prblems that d nt significantly affect the task cmpletin (nncritical errr) Frequency Frequency is the percentage f participants wh experience the prblem when wrking n a task. High: 30% r mre f the participants experience the prblem Mderate: 11% - 29% f participants experience the prblem Lw: 10% r fewer f the participants experience the prblem Prblem Severity Classificatin The identified severity fr each prblem implies a general reward fr reslving it, and a general risk fr nt addressing it, in the current release. Severity 1 - High impact prblems that ften prevent a user frm crrectly cmpleting a task. Severity 2 - Mderate t high frequency prblems with mderate t lw impact are typical f errneus actins that the participant recgnizes needs t be undne. Severity 3 - Either mderate prblems with lw frequency r lw prblems with mderate frequency; these are minr annyance prblems faced by a number f participants. Severity 4 - Lw impact prblems faced by few participants; there is lw risk t nt reslving these prblems. Reprting Results The Usability Test Reprt will be prvided at the cnclusin f the usability test. It will cnsist f a reprt and/r a presentatin f the results; evaluate the usability metrics against the pre-apprved gals, subjective evaluatins, and specific usability prblems and recmmendatins fr reslutin. 5