System Control. Virtuelle Realität Wintersemester 2007/08. Overview. Part 10: Virtuelle Realität. Prof. Bernhard Jung

Transcription

1 Part 10: System Control Virtuelle Realität Wintersemester 2007/08 Prof. Bernhard Jung Overview Definition Categorization Description of different techniques Design guidelines Myths Conclusion Further information: D. A. Bowman, E. Kruijff, J. J. LaViola, I. Poupyrev. 3D User Interfaces. Addison-Wesley Professional Chapter 8. Dachselt, R.; Hübner, A.: Survey and Taxonomy of 3D Menu Techniques; Proceedings of the 12th Eurographics Symposium on Virtual Environments (EGVE'06), Lisbon (Portugal), 8th - 10th May

2 Definition Navigation & manipulation: Interaction in/with Virtual Environment System control: a user task which a command is issued to change the mode of interaction or the system state e.g. load a scene/level, activate fly-through mode command specification involves selection of an element from a set in desktop interfaces: menus, icons - but direct transfer to VR not always effective Problem: evaluation results are sparse, foundations are not well established system control elements in VRML browser Categorization 2

3 Requirements System control should: Be fast Easy to learn Keep the user focused on the task Give useful feedback and apply strong affordances Be screen effective Graphical menus 3D equivalent of 2D menus 1-DOF menus Adapted 2D menus TULIP 3D Widgets 3

4 Graphical menus (continued) Examples Ring Menu, 1-DOF (hand-oriented) menus ring menus hand-held widgets stored at body-relative position use relative hand position for item selection e.g. Mine et. al 1997 Generalized 3D Carousel View, display of large amounts of menu items Wang et al Hand-held Widget Mark R. Mine Graphical menus (continued) Examples Adapted 2D menus function the same as in desktop environments pull-down menus, pop-up menus, flying widgets, toolbars and sliders Serra et al

5 Graphical menus (continued) Examples TULIP Uses Pinch Gloves, where menu items are assigned to different fingers and a pinch between a finger and the thumb is interpreted as a menu selection. up to 8 menu items up to 16 menu items with menu hierarchy Bowman & Wingrave, 2001 Graphical menus (continued) Examples 3D Widgets Move system control functionality into the world or onto objects Widgets are the combination of geometry and behavior (Conner et al. 1992) Functions are co-located near an object, thereby forming a highly context-sensitive menu 3D widgets (which are not based on 2D widget principles) are relatively seldom A 3D widget (co-located widget) next to several transformed 2D widgets in CDS Image courtesy by Doug Bowman 5

6 Graphical menus (continued) Dachselt & Ebert, 2001 Collapsible Cylindrical Tree Fast navigation with a cylindrical menu Submenus displayed on expanded cylinders Graphical menus (continued) Design and Implementation Issues Placement world-referenced (freely in world) object-referenced (centered to object in world) head-referenced (view centered) body-referenced device-centered 6

7 Graphical menus (continued) Selection Traditional, desktop menus: 2D selection method (mouse-based) In 3D user interfaces: 3D selection method constraints on degree of freedoms of system control interface e.g. discard all tracker data except projection on plane of the menu Representation and structure How are items represented visually? If many: How are structured? Form, size, space between items, affordance hierarchy: functional and semantic grouping, context sensitiveness, control coding 7

8 Voice (I) Access voice command is both initialization and command itself it is hands-free and natural Structure and representation invisible structure Feedback Voice output, sound cues cf telephone dialogue systems Typical examples activation of widgets, computer, give color palette" direct actions like exiting the system Voice (II) - Advantages and Disadvantages Voice is hands-free and natural, it can be well combined with other interaction styles Users need to learn voice commands, but as soon as the user has learned the commands, speech is to easy to use Voice recognition engine has limited amount of words and can have syntactic and semantic problems Continuous voice-input is tiring Speech recognition is not usable in every environment, for instance due to background noise 8

9 Gestures - Advantages and Disadvantages No menu needed - no occlusion of application Can be very powerful, for example in combination with speech Often needs considerable computations Calibration inaccuracy of glove can destroy performance: this is often caused by hand size problem User needs to learn all gestures (non self-explanatory interaction): users are easily cognitively overloaded Gloves and cables can be uncomfortable Tools (I) Physical tool - prop Duplication of input device in virtual environment: system control integrated in input device Tactile augmentation - tactile feedback Tool metaphor: tool s shape and labels indicate usage Device-per-task or device-for-all-tasks Menu can be directly connected to input device Widget-less interaction possible: no tool mode change necessary Virtual tool - toolchest Usage of proprioceptive cues to place system control at a body oriented position (belt) Ordered in menu form or as real-world like toolbox 9

10 Tools (II) Access Picking physical tool, mode change on input device, accessing virtual tool (reach/look) Functionality space versus time multiplexed (physical tool) or like graphical menu (virtual tool) Form / representation factors Form: device communicates function graphical menu related representation forms Tools (III) Examples Tablet interface users hold a large plastic tablet on which a (traditional) 2D interface is displayed in the virtual world e.g. virtual habitat Gorilla Exhibit menu Courtesy of Bowman and Hodges VRAM toolchest Regenbrecht, Kruiff Bauhaus Universität Weimar Toolchest by Toralf Sontag and Hartmut Seichter 10

11 Tools (IV) - Advantages and Disadvantages Props User does not always have to divert attention from task in application due to physical feedback of device Directness due to visibility and tactile feedback Familiarity with device can allow ease of use Form of device can give strong affordance No placement problems due to both physical feedback of device and mapping of system control on device itself, not on menu Toolchest Proprioceptive (body oriented) cues give strong placement cues and can allow quick access of system control Local stereoscopic cues allow fast(er) selection, monoscopic usage slower Using a toolchest diverts attention of actual task Proprioceptive cues can be disturbed when the head-tracker is used for placement of toolchest: different length of people gives problems Hybrid Combinations of several kinds of non-conventional control techniques also exist popular examples: put-that here interfaces combining voice and gesture, prop or gesture with graphical menu Multimodal system control might increase effectiveness of issuing commands 11

12 System control with handheld devices (brings 2D GUI on handheld to VR) work by Stefan Conrad, 2007 tracked PDA tracked stylus buttons + stylus + 6DOF tracking tablet PC Object selection (left) and property control (right) Design guidelines Don t disturb flow of action Use correct spatial reference Allow multimodal input Structure available functions Prevent mode errors by giving feedback Be creative, and test! 12

13 Myths and Reality Myth: Proven functionality of 2D GUI will always work within a Virtual Environment Reality: 2D GUIs do not always work in a VE Myth: Problems with system control are basically problems with technical limitations of input and output devices Reality: usability of system control methods will increase with technological advancements, but other factors seem much more important Myth: Different input modes are capable of transmitting comparable content Reality: one can not always exchange system control techniques without loosing some content Conclusion Evaluation testbeds are needed which test effectiveness and applicability of system control techniques Wrong application of system control interfaces can cause major performance disturbances Style guides for 3D User Interfaces? Cf Apple Human Interface Guidelines s/index.html 13

14 Appendix: 3D Widgets References R. Dachselt. Eine deklarative Komponentenarchitektur und Interaktionsbausteine für dreidimensionale multimediale Anwendungen. Der Andere Verlag Kap 3. Dachselt, R.; Hübner, A.: Survey and Taxonomy of 3D Menu Techniques; Proceedings of the 12th Eurographics Symposium on Virtual Environments (EGVE'06), Lisbon (Portugal), 8th - 10th May Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) 14

15 Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) 15

16 Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) 16

17 Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) 17

18 Prof. B. Jung Virtuelle Realität TU Bergakademie Source: Dachselt Freiberg(2004) 18