User Interface Design

Transcription

1 User Interface Design Winter term 2005/2006 Thursdays, c.t., Raum 228 Prof. Dr. Antonio Krüger Institut für Geoinformatik Universität Münster 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 1

2 Introducing evaluation 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 2

3 The aims of this part of the lecture Discuss how developers cope with realworld constraints. Explain the concepts and terms used to discuss evaluation. Examine how different techniques are used at different stages of development. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 3

4 Two main types of evaluation Formative evaluation is done at different stages of development to check that the product meets users needs. Summative evaluation assesses the quality of a finished product. Our focus is on formative evaluation 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 4

5 What to evaluate Iterative design & evaluation is a continuous process that examines: Early ideas for conceptual model Early prototypes of the new system Later, more complete prototypes Designers need to check that they understand users requirements. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 5

6 Bruce Tognazzini on why you need to evaluate: Iterative design, with its repeating cycle of design and testing, is the only validated methodology in existence that will consistently produce successful results. If you don t have user-testing as an integral part of your design process you are going to throw buckets of money down the drain. See for discussion about design and evaluation. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 6

7 When to evaluate Throughout design From the first descriptions, sketches etc. of users needs until the final product Design proceeds through iterative cycles of design-test-redesign Evaluation is a key ingredient for a successful design. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 7

8 1984 Olympic Message System Goal: Make it possible for athletes housed in Olympic Village dorms to send and receive recorded voice messages to and from each other, to coaches and staff outside the village, and to friends and relatives in all parts of the world Project initiated in late 1983 In public voice message systems / answering machines were rare Operate for just 4 weeks; no time for a second try Failure would be broadcasted worldwide 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 8

9 1984 Olympic Message System Testing of prototypes started a few weeks into the project The early and constant availability of prototypes made extensive testing possible On-site interviews/demos Testing overseas with overseas users and telephone systems Intensive usability tests with about 100 people Local CS students were invited to come by and try to destroy the system Field test at a pre-olympic event 2800 people at IBM Yorktown Heights and 1000 in Los Angeles were connected to the system and asked to use it And the system was a success 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 9

10 1984 Olympic Message System The virtue of simplicity: many features dropped along the way, e.g. Facility to verify that a message had been received Facility to notify Olympians with waiting messages Inserting new material in the middle of a message Four alternatives in audio menus are too many Examples of problems encountered late in the project Smog, heat, and desert bugs would create hardware problems The user guide was translated to 10 additional languages Names: Order of first, middle, and last name? Differing transcription rules 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 10

11 Evaluating the 1984 OMS Early tests of printed scenarios & user guides Early simulations of telephone keypad An Olympian joined team to provide feedback Interviews & demos with Olympians outside US Overseas interface tests with friends and family. Free coffee and donut tests Usability tests with 100 participants. A try to destroy it test Pre-Olympic field-test at an international event Reliability of the system with heavy traffic 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 11

12 HutchWorld Virtual environment for cancer patients Reflect existing site (Fred Hutchinson Cancer Research Center in Seattle) Motivation: create a forum for patients and caregivers to: talk about the disease Find additional information ressources Communicate with family & friends Stay in a sterile environment 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 12

13 HutchWorld Many informal meetings with patients, caregivers & medical staff early in design Early prototype was informally tested on site Designers learned a lot, e.g. language of designers & users was different asynchronous communication was what the patients needed Redesigned to produce the portal version 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 13

14 Usability testing User tasks investigated: how users identity was represented Communication information searching entertainment User satisfaction questionnaire Triangulation to get different perspectives 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 14

15 Findings from the usability test The back button didn t always work Users didn t pay attention to navigation buttons Users expected all objects in the 3-D view to be clickable. Users did not realize that there could be others in the 3-D world with whom to chat, Users tried to chat to the participant list Other functions were triggered from this list actual chat button was somewhere else 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 15

16 Key points Evaluation & design are closely integrated in user-centered design. Some of the same techniques are used in evaluation & requirements but they are used differently (e.g., interviews & questionnaires) Triangulation involves using a combination of techniques to gain different perspectives Dealing with constraints is an important skill for evaluators to develop. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 16

17 An evaluation framework 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 17

18 The aims of this part of the lecture Explain key evaluation concepts & terms. Describe the evaluation paradigms & techniques used in interaction design. Discuss the conceptual, practical and ethical issues that must be considered when planning evaluations. Introduce the DECIDE framework. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 18

19 What are user studies? looking at how people behave either in their natural environments, or in the laboratory, both with old technologies and with new/proposed technologies. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 19

20 Overview of techniques observing users, asking users about their opinions, asking experts about their opinions, testing users performance modeling users task performance 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 20

21 What is an evaluation paradigm? Any kind of evaluation is guided explicitly or implicitly by a set of beliefs, which are often under-pined by theory. These beliefs and the methods associated with them are known as an evaluation paradigm 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 21

22 Four evaluation paradigms quick and dirty usability testing field studies predictive evaluation 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 22

23 1: Quick and dirty quick & dirty evaluation describes the common practice in which designers informally get feedback from users or consultants to confirm that their ideas are in-line with users needs and are liked. Quick & dirty evaluations are done any time. The emphasis is on fast input to the design process rather than carefully documented findings. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 23

24 2: Usability testing Usability testing involves recording typical users performance on typical tasks in controlled settings. Field observations may also be used. As the users perform these tasks they are watched & recorded on video & their key presses are logged. This data is used to calculate performance times, identify errors & help explain why the users did what they did. User satisfaction questionnaires & interviews are used to elicit users opinions. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 24

25 3: Field studies Field studies are done in natural settings The aim is to understand what users do naturally and how technology impacts them. In product design field studies can be used to: identify opportunities for new technology determine design requirements decide how best to introduce new technology evaluate technology in use 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 25

26 4: Predictive evaluation Experts apply their knowledge of typical users, often guided by heuristics, to predict usability problems. Another approach involves theoretically based models. A key feature of predictive evaluation is that users need not be present Relatively quick & inexpensive 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 26

27 DECIDE: A framework to guide evaluation Determine the goals the evaluation addresses. Explore the specific questions to be answered. Choose the evaluation paradigm and techniques to answer the questions. Identify the practical issues. Decide how to deal with the ethical issues. Evaluate, interpret and present the data. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 27

28 Determine the goals What are the high-level goals of the evaluation? Who wants it and why? The goals influence the paradigm for the study Some examples of goals: Identify the best metaphor on which to base the design. Check to ensure that the final interface is consistent. Investigate how technology affects working practices. Improve the usability of an existing product. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 28

29 Explore the questions All evaluations need goals & questions to guide them so time isn t wasted on ill-defined studies. For example, the goal of finding out why many customers prefer to purchase paper airline tickets rather than e-tickets can be broken down into sub-questions: What are customers attitudes to these new tickets? Are they concerned about security? Is the interface for obtaining them poor? What questions might you ask about the design of a cell phone? 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 29

30 Choose the evaluation paradigm & techniques The evaluation paradigm strongly influences the techniques used, how data is analyzed and presented. E.g. field studies do not involve testing or modeling 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 30

31 Identify practical issues For example, how to: select users stay on budget stay on schedule find evaluators select equipment 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 31

32 Decide on ethical issues Develop an informed consent form Participants have a right to: know the goals of the study what will happen to the findings privacy of personal information not to be quoted without their agreement leave when they wish be treated politely 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 32

33 Evaluate, interpret & present data How data is analyzed & presented depends on the paradigm and techniques used. The following also need to be considered: Reliability: can the study be replicated? Validity: is it measuring what you thought? Biases: is the process creating biases? Scope: can the findings be generalized? Ecological validity: is the environment of the study influencing it - e.g. Hawthorn effect Hawthorn Effect: Improved process data that results from process operators who know that they are observed and that just appreciate this special treatment 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 33

34 What are pilot studies? A small trial run of the main study. The aim is to make sure your plan is viable. Pilot studies check: that you can conduct the procedure that interview scripts, questionnaires, experiments, etc. work appropriately It s worth doing several to iron out problems before doing the main study. Ask colleagues if you can t spare real users. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 34

35 Key points An evaluation paradigm is an approach that is influenced by particular theories and philosophies. Five categories of techniques were identified: observing users, asking users, asking experts, user testing, modeling users. The DECIDE framework has six parts: Determine the overall goals Explore the questions that satisfy the goals Choose the paradigm and techniques Identify the practical issues Decide on the ethical issues Evaluate ways to analyze & present data Do a pilot study 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 35

36 A project for you Februar 06 IfGi Universität Münster User Interface Design A. Krüger 36

37 An evaluation project for you Read Tog s account and look at the picture of the ballot card. Make a similar ballot card for a class election and ask 10 of your friends to vote using the card. After each one has voted ask who they intended to vote for and whether the card was confusing. Note down their comments. Redesign the card and perform the same test with 10 different people. Report your findings. 20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 37