Penspective: a Perspective-Aware Game Map Editor with Natural Interaction

Transcription

1 : a Perspective-Aware Game Map Editor with Natural Interaction Alexandre M. F. de Sousa, Anderson C. M. Tavares, Carlos H. Morimoto Department of Computer Science - DCC Institute of Mathematics and Statistics - IME University of Sao Paulo - USP {alemart, acmt, hitoshi}@ime.usp.br October 15, 2013

2 Introduction Map editors; Learning curve (WIMP hotkeys, cluttered menus); pixel-hungry, indirection; mouse/keyboard impose complex interactions. Advances in Human-Computer Interaction (HCI); Post-WIMP interfaces [1]; new generation of UIs; gestures, speech... Leverage users mundane, pre-existent knowledge. reduce the gap: human goals/actions; communication in the non-digital realm.

3 Introduction Interaction: drawing sketches with pen and eraser; versatile, widely used; obvious metaphor. Visualization: illusion of depth; like magic paper. : Grid paper (little 3D blocks); Affordance of pen and eraser; 3D View without glasses. Figure: Pen and eraser.

4 Design (a) Drawing blocks. (b) Removing blocks. (c) Moving (selection phase). (d) Moving (dragging phase).

5 Design (e) Adjusting height (before). (f) Adjusting height (after).

6 System Multitouch Interaction Kinect sensor (depth + color) Detect touch events on ordinary surfaces [2]; Recognize the touching artifact [3]. (g) Detecting touch. (h) Color.

7 System Description Above-the-surface Interaction Kinect sensor (depth + color); Grab a template (grayscale); Track its position: p t = argmin (x,y) cost t (x,y) α (1 R(x,y)) + if d t (x,y) 1 (1 α) d cost t (x,y) = t (x,y) otherwise where: α [0,1],r are empirically defined; d t(x,y) = (x,y) p t 1 / r; R(x,y) [0,1] is a template matching function [4]. Depth sensor 3D position.

8 System Description Skeleton Tracking/Off-Axis Perspective another Kinect camera tracks the user in physical space; calibration phase position of the user relative to the screen; off-axis perspective projection [5]; illusion of depth. Figure: calibration phase.

9 System Description Setup: 2 Kinects, 1 LCD screen, pen, eraser. Components exchange data using VRPN [6]; Figure: System components.

10 Prototype (a) Prototype (b) Changing the viewpoint (c) Erasing (d) Drawing

11 Prototype (e) Height adjustment (f) Height adjustment (g) Height adjustment (h) moving a block

12 Conclusion More direct ways of building maps; A new form of interaction: drawing on grid paper; Working prototype to show the viability of the concept. Figure:.

13 Bibliography I A. van Dam, Post-WIMP user interfaces, Communications of the ACM, vol. 40, no. 2, pp , Feb A. D. Wilson, Using a depth camera as a touch sensor, in ACM International Conference on Interactive Tabletops and Surfaces - ITS 10, ACM. New York, New York, USA: ACM Press, 2010, p. 69. J.-Y. Lee and S. I. Yoo, An elliptical boundary model for skin color detection, in Proc. of the 2002 International Conference on Imaging Science, Systems, and Technology. Citeseer, A. Kaehler and G. Bradski, Learning OpenCV: Computer Vision with the OpenCV Library, 1st ed. O Reilly Media, 2008.

14 Bibliography II R. Kooima, Generalized Perspective Projection, [Online]. Available: http: //csc.lsu.edu/ kooima/articles/genperspective/index.html R. M. Taylor, T. C. Hudson, A. Seeger, H. Weber, J. Juliano, and A. T. Helser, VRPN: a device-independent, network-transparent VR peripheral system, in Proceedings of the ACM symposium on Virtual reality software and technology - VRST 01. New York, New York, USA: ACM Press, 2001, p. 55.