UNIVERSITTY - the course 2011, Professor Center for Interactive Spaces Department of Computer Science, University of Aarhus Email: kgronbak@cs.au.dk Course Web: https://services.brics.dk/java/courseadmin/ar/ Plan Brief presentation About the course A little AR history Concepts and Survey Quick intro to getting started with ARToolkit Intro to exercise 2 1
Augmented reality An aspect of ubiquitous/pervasive computing Focus on IT-based systems, bridging between physical and digital worlds Involves IT-research, engineering, architecture and industrial design Based on context-awareness and positioning knowledge Based on the research in Center for Interactive spaces 3 Form of the Course Lectures about prototypical systems and applications Aprox. half of the time on Thursdays Active participation Student discussion of selected papers and tools Small exercises to be accpted Evaluation: Oral exam. 20 min 4 2
Course Plan - 2011 Week 4: Introduction to the course and concepts Week 5: Surfaces and CyberCodes Week 6: AR Toolkit and examples of applications. Week 7: Mobile and Wearable AR Week 8: Position and direction based AR incl. LAYAR Week 9: Augmented Paper and Electronic Paper Week 10: Computer augmented objects, Physical Hypermedia, RFID and NFC Week 11: Tentatively: Augmented reality gaming or other option; Conclusion and evaluation of the course. 5 LOCAL EXAMPLES 6 3
SitePack mobile computing device Augmented reality: feature tracking From the WorkSpace project Integrating location based, registration, annotation and visualization 7 Visualizing hedges Nielsen, M.B., Kramp, G., and Grønbæk, K. Mobile Support for Architects based on Feature Tracking Techniques (2004). In Lecture Notes in Computer Science Volume 3038. Computational Science - ICCS 2004: 4th International Conference, Kraków, Poland, June 6-9, Part III (eds. Marian Bubak, Geert Dick van Albada, Peter M. A. Sloot, et al.). Springer-Verlag Heidelberg. pp. 921-928. 8 4
Previous EU-project WorkSPACE: Physical Hypermedia for Architects : RFID tags Visual ARToolkit tags 9 Grønbæk, K., Ørbæk, P., Kristensen, J.F., and Eriksen, M.A. Physical Hypermedia: Augmenting Physical Material with Hypermedia Structures (2003). In New Review of Hypermedia and Multimedia (NRHM). Vol 9. Taylor & Francis, Abingdon, UK (pp. 5-34) BattleBoard 3D - Game AugmentedMedia Startup company International Paper: Andersen, T.L., Kristensen, S., Nielsen, B.W. and Grønbæk, K. Designing Board Games: The BattleBoard 3D experience. In proceedings of OZCHI 2004, 22-24 November, 2004 at the University of Wollongong, Australia. 10 5
ARDressCode International Paper Kjærside, K., Kortbek, K. J., Møller, H.H., and Grønbæk, K. ARDressCode: Augmented Dressing Room with Tag-based Motion Tracking and Real-Time Clothes Simulation, In proceedings of the Central European Multimedia and Virtual Reality Conference 2005 (CEMVRC05), 8-10 June 2005, Prague, Czech Republic. 11 Aarhus Festuge Interactive Posters and Red Path with blog 12 6
Layar version of Aarhus Festival program Events within 2 km 13 Other Layar views 14 7
StarCatcher GPS game 15 UNIVERSITTY Overview of the AR field 8
Readings Azuma: Recent Advances in Mackay: CACM 93 intro Mackay: : Linking Real and Virtual Worlds. A new paradigm for interacting with computers. 17 versus Virtual Reality AR (ubicomp) Physical world as the basis Bring computer capabilities into the physical world Embodied virtuality VR Simulated artificial world is the basis Model the world in the computer 19 9
A little history Xerox around 90 Weiser, Mackay,Wellner Mackays Ariel EuroCODE project with AU Columbia University Feiner CACM July 93 Sony Rekimoto Medialab Ishii DARE konference i 2000 Helsingør 21 AZUMA ET AL. 24 10
What is AR? (Azuma et al.) Definition AR system supplement the real world with virtual (computergenerated) objects that appear to coexist in the same space as the real world. An AR system has the following properties: combines real and virtual objects in a real environment; runs interactively, and in real time; and registers (aligns) real and virtual objects with each other. 25 Enabling technologies - Displays Types of displays Head worn displays (HWD) optical see-through, video see-through, virtual retinal display Handheld displays Projection displays Problems/Challenges Quality of display Size and weight Occlusion of real objects 26 11
Enabling technologies - Tracking and sensing Indoor tracking environment prepared with tags and sensors video tracking, magnetic tracking combined techniques works best Outdoor tracking GPS, digital compass, gyroscope video-based, fixed land mark, known features 27 Tracking and sensing: challenges Low latency registration errors Environment preparation overhead in tagging Calibration overhead in preparing sensors 28 12
Interaction Beyond keyboards, mouse and menus Gestures Utilize movement sensors Tangible interaction Physical tools to grab virtual objects haptic feedback Audio/speech interfaces Audio controlled interaction 29 Visualization - challenges correct for location errors information overload in overlays filtering real time photorealistic rendering mediated reality extract real objects and replace with virtual depth perception 30 13
Perceptual problems Latency E.g. Errors in location detection Depth perception Occlusion problems Adaption Users adapt their perception to AR views and have problems when they leave the equipment Eye strain Projections on the eye, same image, different images, stereo 31 Mobile applications Real time overlay GPS and landmark labels 32 14
Sports and commercial applications Overlays with player info. computed realtime on top of video Overlays of commercials and ads on real world and tv-broadcasts. 33 Collaborative applications Examples of collaborative AR systems using use see-through handheld displays seethrough head-worn displays A problem/challenge: ensuring that the users can establish a shared understanding of the virtual space, analogous to their understanding of the physical space. In Studierstube, the designers attempt to overcome this problem by rendering virtual representations of the physical pointers, which are visible to all participants 34 15
Azuma summary/comments Very much focus on overlay and 3D techniques inspired from VR Problems/challenges are mainly graphics problems 35 MACKAY ET AL. 36 16
Wendy Mackay Focus on Interactive Paper rather than 3D/VR like augmented reality Introduces a taxonomy relating augmented reality technology to users, objects and environments Gives three central examples of interactive paper applications 37 Mackay s categories 38 17
Mackay s examples Ariel Handling of drawings and paper Video Mosaic Storyboard editor Caméléon augmented flight strips 39 Ariel Interacting with paper 40 18
Video Mosaic Storyboarding tool Paper storyboards linked to video clips 41 Caméléon Flight strips Flight strip represents one plane Corresponds to a spot on the radar Status information are written on the strips Acts as a token/depeche to be handed over to colleagues 42 19
UNIVERSITTY Discussion Q&A ARTOOLKIT (MARK BILLINGHURST ET AL.) 44 20
ARToolKIT Open source tool kit Webcam and a PC/Mac Goggles/Glasses/Monitor Supports Make your own tags Make your own models Make your own application logic Implemented on iphone e.g. by ARToolWorks http://www.artoolworks.com/ 45 http://ftp.hitl.washington.edu/research/ shared_space/download/ ARToolkit Source code in C++ Callibration and registration applications Example applications Simple, Multi, SimpleVRML 46 21
ARToolKit processing overview 47 3D models for the overlay Imported in VRML format VRML = Virtual Reality Modelling Language VRML 2.0 (VRML97) is a standard for exchange of 3D models including animations Is exported from many 3D tools, e.g. 3D Studio MAX, Alias Wavefront Maya 48 22
UNIVERSITTY AR-toolkit How to use? Demo ARToolKit refs AR Toolkit Homepage (http://www.hitl.washington.edu/research/shared_space/) HitLabNZ (http://www.hitlabnz.org/) AR Toolkit Manual (http://www.hitl.washington.edu/artoolkit/documentation) DART Designers ARToolKit integrated in Macromedia 2004 MX (http://www.gvu.gatech.edu/dart/) Sourceforge.net (http://sourceforge.net/project/showfiles.php?group_id=116280) 50 23
Using ARToolKit Important files Camera Calibration Data (camera_para.dat) Markers/Patterns (eg. Patt.hiro) Multi Marker Specification files (marker.dat) Wrl/bud_B.dat Data/Vrml_data Data/patt.hiro Wrl/bud_B.wrl 51 Demo of mk_patt.exe Demonstrates how to register your own markers. Click mouse to save pattern. 52 24
The first exercise Form groups and make a little ARToolkit prototype App Relate it to an application scenario Make a few tags Associate some objects Run the prototype Make some simple changes in the application code E.g. detect and react to proximity of tags Present in Week 6 53 Questions? 54 25