Designing natural interfaces for having more fun in a learning environment

Transcription

1 Designing natural interfaces for having more fun in a learning environment Michael Haller Media Interaction Lab Upper Austria University of Applied Sciences Hagenberg Austria

2 Hagenberg

3 Ars Electronica (Linz)

4 Upper Austria University of Applied Sciences

5 Upper Austria University of Applied Sciences 9 departments with a special focus in computer science (e.g. Mobile Computing, Media Security, Interactive Media) Interactive Media (Graduate Students), with focus in Computer Games Interactive Media Web Technologies Media Interaction Lab (

6 Natural User Interface

7 Natural User Interface

8 What is natural?

9 Eating

10

11 Sleeping

12 NUI?

13 What about gestures?

14

15 Gestures are not natural

16

17 Bedeutung? Bild von Geste, nach Bedeutung fragen

18 Ausschnitt von Studie (Video)

19 Rotate

20 Experience Design

21 What to do? Imagine You re bringing a new product to market Your #2 competitor has been in the market for over a year, selling millions of units Your #1 competitor launches the same month Your technology is slower than your competitors Your technology is older than your competitors Your last product failed in the market

22 Do you compete on Price? Do you compete on Technology? Do you compete on Features? Compete on user experience!

23 Example 1

24 Nintendo Wii Cheap - $250 Unique game play Wireless 3 DOF controller Position and orientation sensing Aiming to broaden user base Can play previous games/downloads

25 Nintendo DS $100 worth of joy Novel input Two screens Stylus Microphone Collaborative wireless

26

27

28 Sony s MOVE

29 Microsoft s Kinect

30 iphone Example 2

31 Example 3

32 But it can take longer Apple s first palm PC (Newton) failed

33 ipod (G1-G4)

34 The value of experience

35 People pay for it! 50c 20c $5.00

36 My Background Augmented Reality

37 AR Technology SIGGRAPH 2001 Courtesy of Hirokazu Kato

38 Math Education MR for math education TU Vienna, Kaufmann & Schmalstieg (2001) Teach geometric concepts Teacher sees solution, students don't

39 MagicBook Mark Billinghurst (HITLabNZ), Hirokazu Kato

40 Handheld interfaces Oliver Bimber (Weimar, Germany)

41 AMIRE (Authoring Mixed Reality) J. Zauner, M. Haller, A. Brandl, and W. Hartmann, "Authoring of a Mixed Reality Assembly Instructor for Hierarchical Structures," in ISMAR 03: Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality, Washington, DC, USA, 2003.

42 AMIRE (Authoring Mixed Reality) J. Zauner, M. Haller, A. Brandl, and W. Hartmann, "Authoring of a Mixed Reality Assembly Instructor for Hierarchical Structures," in ISMAR 03: Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality, Washington, DC, USA, 2003.

43 AHUMARI Augmented Based Human Robot Interaction Better Programming Interface for Industrial Robots

44 004

45 nteractive ables

46 Ideally ww.microsoft.com/surface

47 Interactive Tables + Gamess

48

49 KnightMage Magerkurth, C., Memisoglu, M., Engelke, T., and Streitz, N Towards the next generation of tabletop gaming experiences. In Proceedings of Graphics interface 2004 (London, Ontario, Canada, May 17-19, 2004).

50

51 Monkey Bridge Barakonyi, I., Weilguny, M., Psik, T., and Schmalstieg, D MonkeyBridge: autonomous agents in augmented reality games. In Proceedings of the 2005 ACM SIGCHI international Conference on Advances in Computer Entertainment Technology (Valencia, Spain, June 15-17, 2005)

52 Sugimoto, M., Kagotani, G., Kojima, M., Nii, H., Nakamura, A., and Inami, M Augmented coliseum: display-based computing for augmented reality inspiration computing robot. In ACM SIGGRAPH 2005 Emerging Technologies (Los Angeles, California, July 31 - August 04, 2005)

53

54

55

56 Photosensors in the bottom of the robot measure brightnes of projected marker

57

58

59

60 Tangible User Interfaces Hiroshi Ishii, MIT Media Lab

61 Motivated by the Incredible Machine

62

63

64

65

66

67

68

69

70 3DV IR-depth camera from 3DV- Systems This was before Kinect!

71 courtesy of Andy Wilson, Illustrations adapted from G. J. Iddan and G. Yahav, "3D Imaging in the Studio, SPIE, vol. 4298, pp. 48, 2001

72 Fact sheet 3DV Depth Camera RGB and Depth information available Depth information resolution: 320! 30fps 160! 60fps Adjustable region of interest 8 bit depth information

73 320!240 30fps 8-bit greyscale depth image

74

75

76

77

78 Augmentation of real objecs requires projection from the top

79

80 Takafumi Aoki

81 by Julian Oliver

82

83

84

85 Interactive Home

86

87 50 inch diagonal Couch Table Height (17 inches) Storage Areas An interactive surface that is perceived as part of t furniture but not as a horizontal computer screen.

88 Update!

89

90

91

92 Discussion: Tabletops at home Philips Entertable Microsoft s Surface Either you are a furniture company or you are a hard- or/and software company What is an interactive table? Furniture vs. Hardware Life Cycle (10 years vs. 2 years)

93 But There are companies that are successfully selling interactive tables! E.g. SMART s Table

94 004 nteractive surfaces or learning nvironments

95

96 Ideally

97 Cebit 2008

98 Presentation Display Digital Whiteboard Interactive Tabletop

99 008 ice Discussion oom

100 Oversized interactive Anoto-based whiteboard (4.5m! 1.1m) Multi-user interaction Combination with traditional interfaces (e.g. magnetic pins/post-its)

101 M. Haller, J. Leitner, T. Seifried, J. Wallace, S. Scott, C. Richter, P. Brandl, and A. Gokcezade, 2010."The NiCE Discussion Room: Integrating Paper and Digital Media to Support Co-Located Group Meetings," in CHI 10: Proceedings of the 28th international conference on Human factors in computing systems, New York, NY, USA, 2010, pp

102

103 Change this (Jakob?)

104 ANOTO

105 ANOTO

106 Digital Pen 1.5mm

107 200 X

108 200 X 1.5mm 1.5mm

109 Demo

110 Digital Pen Multi-user input Latency of 50 ms using the DP-301 pen Tracking Accuracy of 680 dpi iphone 4: display resolution of 326 dpi Scalable max. size of 60 Mio km 2 30m x 1.5m (pattern size we have) 10m x 1.5m (pattern size we can open) 6m x 1.5m (pattern size we have installed)

111 Interested? Interactive Whiteboard Package: Interactive Whiteboard Evaluation Application Interactive Whiteboard Development Kit

112 010 ice Formula ditor

113 Leitner, C. Rendl, F. Perteneder, A. Gokcezade, T. Seifried, M. Haller, R. Zeleznik, and A. Bragdon, 2010."NiCE formula editor," in SIGGRAPH 10: ACM SIGGRAPH 2010 Talks, New York, NY, USA, 2010, pp. 1-1.

114 Lecture Lab

115 Protoype with touch & pen input

116 Pressure as an Input Metaphor

117

118

119 011 ice Formula ditor #2

120 Combination with Powerpoint Sketching Board Powerpoint Board

121 GeoGebra (

122 GeoGebra (

123 GeoGebra (

124 Sketching Interface + GeoGebra

125

126 NiCE Formula Editor in action - 1

127 NiCE Formula Editor in action - 2

128 NiCE Formula Editor in action - 3

129 inal houghts

130 1 Interactive Surfaces

131

132

133 2 Be careful with multitouch (hype topics)

134

135 3 Natural User Interfaces

136

137 4 Make it simple!

138

139 Acknowledments

140 We are looking for smart PhD students! Just come and visit us

141 Questions? Michael Haller Media Interaction Lab Upper Austria University of Applied Sciences Hagenberg/Austria web:

142 Interpolating Force Sensitive Resistance (IFSR) Sensor I. Rosenberg und K. Perlin, The UnMousePad: an interpolating multi-touch force-sensing input pad, ACM SIGGRAPH 2009 papers, New Orleans, Louisiana: ACM, 2009, S. 1-9.

143 projection foil IFSR foil

144 projection foil IFSR tracking data IFSR foil

145 projection foil pen touch IFSR tracking data IFSR foil

146 pen very easy! touch projection foil IFSR tracking data IFSR foil

147 24 IFSR Prototype touch pen

148 24 IFSR Prototype touch pen

149 screen space projection foil IFSR coordinates IFSR foil

150 screen space Anoto coordinates projection Anoto pattern foil IFSR coordinates IFSR foil

151 pen touch

152 pen touch

153 Anoto Accuracy (680dpi) Multi-pen identification Scalability (60Mio km") Low latency (50ms) IFSR Accuracy (100dpi) Pressure (5g to 5kg) Shape recognition Hardware form factor