HCI Lecture 11: Rich Interaction. Barbara Webb

Transcription

1 HCI Lecture 11: Rich Interaction Barbara Webb Key points: Limitations of interaction descriptions Including embodiment Including continuous time Status-event descriptions Distributed cognition Situated cognition 1

2 Introduction So far have discussed formalisms for tasks and dialogues which tend to assume: One user interacting with one (simple) software system Making a sequence of discrete interactions, following internal plan User actions reliably change world, and user reliably knows this Most real interaction is much richer: Involves continuous close coupled loops; users are embodied Actions and events have real time courses; users deal with interleaving, interruptions, shifts in attention etc. Occurs in environmental context of rich representational artifacts Occurs in context of communication with other users Occurs in cultural/organisational context 2

3 Modes of interaction Modes (or styles) of interaction can be classified as: Instructing: user tells system what to do, by typing commands, selecting menu options, pressing keys or buttons, speaking commands Conversing: user has dialogue with system; typing questions and/or responses, or uses speech input/output Manipulation: user interacts with physical or virtual objects, e.g., holding, moving, opening, closing; object is a focus of attention Exploration: user moves through physical or virtual environment Other possibilities and higher-level classifications exist, e.g., we may interact by learning, problem solving, socializing, searching,... 3

4 Instructing Examples: Shell command line interpreters for operating systems Menu and key-driven GUI shells for OSes and applications VCRs, hi-fis, alarm clocks, vending machines, etc. Advantages: Quick and efficient Good in case of repetition or multiple objects (especially if programmable) Disadvantages: Hard to learn Seldom standardised May be overly specific 4

5 Conversing Examples: Help facilities (Microsoft s Office Assistant paper clip, Bob) Search engines ( although Jeeves has now retired) Phone services (voice recognition query answering/navigation) Virtual shopping or support assistants Advantages No special knowledge required; onus on system to understand user Disadvantages: Limited scope of understandability; may have to constrain user requests/responses Dialogue can become one-sided and cumbersome 5

6 Manipulation Shneiderman (1983) coined the term Direct Manipulation (DM). Digital objects should allow interaction analogous to how physical objects are manipulated Core DM principles: Continuous representation of objects and actions Physical actions instead of issuing commands with complex syntax Rapid reversible actions with immediate feedback on object of interest Examples: desktop files metaphor dragging and dropping also true manipulable objects: physical objects with sensors (e.g. Wii controller) 6

7 Manipulation Advantages of direct manipulation include: Novices can learn the basic functionality quickly Intermittent users can retain operational concepts over time Error messages rarely needed Users can immediately see if their actions are furthering their goals and if not do something else Users experience less anxiety; gain confidence and feel in control Disadvantages Some people take the metaphor of direct manipulation too literally Not all tasks can be described by objects and not all actions can be done directly Some tasks are better achieved through delegating rather than manipulating e.g., spell checking 7

8 Exploring Examples: 3D desktop virtual worlds where people navigate using mouse around different parts to socialize (e.g., Second Life) CAVEs (Computer Automatic Virtual Environment) where users navigate by moving whole body, arms, and head physical context-aware environments, embedded with sensors, that present digital information to users at appropriate places and times (e.g. cell phone tourism, smart home) Will discuss further in lecture on ubiquitous computing. 8

9 Continuous behaviour Many aspects of interaction involve continuous values or continuous time, e.g. position of mouse, moment of click Status-Event analysis explores these issues: Status is a value, usually changing, that can be polled Event is something happening at specific time (may be a status change) Example: agents change/poll status to communicate events time mailbox file status mailtool agent screen status user agent mail arrives event mailtool notices event change icon event user notices event 9

10 Status Event analysis Issues to explore: Granularity: what is the relevant time-scale? Events may be extended in time, and not have clear onset and offset (e.g. interacting with gestures) Action sequencing: not necessarily direct following Could be after delay At particular (external) time, or whenever you like In response to event or environmental cue (triggers) What happens in an interruption? (e.g. does memory of state persist?) Causes of error e.g. when different events cause different system status but same display status 10

11 Status Event analysis E.g. button slip problem Delete the quick brown quick brown fox 11

12 Status Event analysis E.g. button slip problem Delete the quick quick brown 12

13 Status Event analysis E.g. button slip problem HIT CLICK identical screen feedback MISS semantic feedback only closure eye moves elsewhere one solution add simulated click 13

14 Continuous behaviour Lags: May be lag between actual and perceived event or status Problems of persistence and observability e.g. visual vs. auditory May be lag in status-status mappings y= f(x) e.g. y = display, x = real state of file If an event updates x, what event updates y? Same event? Agent responsible for x? Agent responsible for y? Agent responsible for mapping? Polling; timed update; triggered when y is used; or by other hopefully related event? For some problems, best representation might be manual control theory, i.e., treat interaction as a dynamic system Lags will cause overshoot, oscillation, loss of feeling of control 14

15 Dynamical systems Cho et al. (2007) Tilt-based browsing of photos Modulate the mapping from tilt to cursor by dynamics of sticky film strip that makes it easier to settle on photo without overshoot 15

16 Distributed cognition Having the right representation of a task or problem can allow direct solution by our perceptual system Cognition is not just in the head but is distributed between internal and external representations Re-representation is important component of task achievement (e.g. sorting a hand of cards) Information required for a task or dialogue can be: Part of the task definition (e.g. use delete key to remove quick ) Remembered by the user (e.g. unix command sequence) Displayed by the system (e.g. hyperlink colour change) Put into the environment (e.g. post-it notes) Think of your own examples of using place-holders to sequence actions 16

17 Distributed cognition Example: resources model of interaction (Wright et al 2000) Six abstract information structures: Plans (sequence of actions, may have loops, branches etc.) Goals (state of world to be achieved) Possibilities (options or affordances in a particular situation) History (sequence of past actions) Action-effect relations (if x then y) States To use each as a resource, it needs to be represented Representation can be internal, external or both N.B. in e.g. GOMS everything assumed to be internal Can use to analyse different interface designs 17

18 Distributed cognition E.g. using a wizard to externalise a plan; requires the user to internalise the history Options dialogue allows task to be solved by doing afforded actions until reach match with goal, i.e. no plan. 18

19 Situated cognition Activities are situated in physical and socio-cultural situations What we do at any time may reflect moment-by-moment shaping by situation more than any rational goal-oriented plan E.g. web browsing Standard cognitive models tend to assume categories ( target, card slot ) but what we perceive as properties and events is constructed in the course of coordinated activity Clancy 1994 I.e. need physical and social experience of cards and slots to recognise a card slot Suggests understanding a task requires rich research into real system use in real settings: Purposely avoid methods such as interviews and HTA that lead subjects to make `rational reconstructions of their actions 19

20 Situated cognition: Ethnographic Studies E.g. Hughes et al (1995) 3 year study of air traffic control systems Highlighted importance of routine interactions, social and spatial factors in keeping participants informed and involved in the task Also effectiveness and multiple roles of flight progress strips (e.g. colour of pen annotation identifies controller) 20

21 Summary Embodied, distributed and situated cognition are important alternatives to the Model Human Processor view of HCI Suggest it is better to think in terms of computer mediated human activity than human computer interaction Supports development of new interface methods (e.g. ubiquitous computing) that better match human capabilities However: To date these approaches provide frameworks rather than theories: less prescriptive but also less predictive than traditional approaches Taking situated approach seriously implies we should not expect to find any general principles for successful design 21

22 References David E. Kieras and David E. Meyer (1997An overview of the EPIC architecture for cognition and performance with application to humancomputer interaction. Human-Computer Interaction 12: Hornof, A. & Halverson, T. (2003) Cognitive strategies and eye movements for searching hierarchical computer displays. ASM CHI 2003: Conference on Human Factors in Computing Systems, Cho, S-J et al (2007) Dynamics of tilt based browsing on mobile devices, Proceedings of CHI Wright, P., Fields, B. and Harrison, M. (2000) Analysing Human-Computer Interaction as Distributed Cognition: The Resources Model. Human Computer Interaction, 15:1 42 Hughes (1995) Hughes, J. et al (1995) The Role of Ethnography in Interactive Systems Design. ACM Interactions April 1995 See also: Dix et. al. sections 17.4, chapter 18 22