Part 21: Augmented Reality Virtuelle Realität Wintersemester 2006/07 Prof. Bernhard Jung Overview Introduction to Augmented Reality Augmented Reality Displays Examples AR Toolkit an open source software library for building AR applications Further Reading O. Bimber & R. Raskar. Spatial Augmented Reality: A Modern Approach to Augmented Reality. B & T. 2005. see also: http://www.spatial-ar.com/courses.htm http://www.augmented-reality.org/ http://www.howstuffworks.com/augmented-reality.htm 1
Mixed Reality The Reality-Virtuality Continuum P. Milgram, H.Takemura, A.Utsumi, F. Kishino: Augmented Reality: A class of Displays on the Reality-Virtuality Continuum SPIE Vol. 2351, Telemanipulator and TelepresenceTechnologies, 1994. Augmented Reality (AR) images projected over the real world aircraft head-up display semi-transparent goggles handhelds (PDA, smartphone) projecting onto a desktop types of information unrelated e.g. reading email with wearable related e.g. virtual objects interacting with world issues registration aligning virtual and real eye gaze direction Vampire Project Visual Active Memory Processes and Interactive Retrieval (U Bielefeld) 2
Applications of AR Navigation Systems Maintenance overlay instructions display schematic Museums Gaming howstuffworks.com http://www.research.ibm.com/journal/sj/ 384/spohrer.html microvision.com Augmented Interaction http://www.csl.sony.co.jp/person/rekimoto/uist95/uist95.html Goal: enhance the real world sensation with useful information Users see real and virtual objects Location-dependent additional information can be provided (display menu cards for restaurants) The real sensation can be partially disabled (replace all billboards with pictures from last holiday...) Photo-realism not necessarily first goal but in the ultimate system, people cannot decide whether parts are virtual or real 3
Augmented Reality - What do we need for the combination of read and virtual worlds? Locations and optical properties of the viewer (or camera) and the display Calibration of all devices Combination of all local coordinate systems centered on the devices and the objects in the scene in a global coordinate system Registration of models of all 3D object of interest with their counterparts in the scene Tracking of objects over time when the user moves and interacts with scene Realistic merging requires: Objects to behave in physically plausible manner when manipulated Occlusion Collision detection Shadows Detailed description of the physical scene Source: Cindy Robertson (Georgia Tech) AR Displays: Overview and Classification 4
Retinal Displays Head Mounted Displays 5
Handheld Displays Screen-based Video See-Through Displays 6
Spatial Optical See-Through Displays http://io2technology.com www.laser-magic.com Projection-based Augmentation 7
Example: DigitalDesk (Wellner, Xerox, 1992) http://www.xrce.xerox.com/competencies/past-projects/ip/digitaldesk.html Extending the desktop metaphor to the real desk Virtual (displayed) and real documents share the same space Example: Augmented Reality for New Gaming Experiences Research Project: Virtual Reality and Hypermedia in Future Telecommunication Products and Services University of Oulu, Finland http://www.infotech.oulu.fi/annual/2000/virgin.html 8
Example: Augmented Reality for New Gaming Experiences Mozzies Game on Siemens SX1 mobile phone The mosquitoes can be seen as they are placed on the live video feed from the camera. Aiming is done by moving the phone around so that the mosquitoes are at the cross hair. The Camera is used to detect the motion. http://en.wikipedia.org/wiki/siemens_sx1 Example: AR Quake http://wearables.unisa.edu.au/projects/arquake/www/ 9
Example: Steve Mann "Cyborg" http://www.eyetap.org/ Steven Mann (MIT, now Toronto University) "Steve Mann is the world s first real cyborg, a man who exists in harmony with technology." Canadian Film Institute, 2001 Started to develop Cyborg equipment in school Since the 1980s he is constantly wearing his WearComp His main research is for privacy and social impacts He noticed two main trends: The technology diminished (everything moved into sunglasses) His Cyborg -feeling has been growing over the years http://www.imdb.com/title/tt0301145/ 2007 Example: Reality Window Manager (EyeTap) http://www.eyetap.org/, http://wearcam.org/ Window manager for XWindows that overlays xterms and other windows onto planar patched in the environment Allows for real-time tracking, replacement and rendering 10
Example: Third-person perspective in augmented reality (???) Salamin, P., Thalmann, D., and Vexo, F. 2006. http://vrlab.epfl.ch/publications/pdf/salamin_thalmann_vexo_vrst_06.pdf ARToolKit an open source software library for building AR applications ARToolKit uses computer vision algorithms to solve the problem of tracking the users viewpoint calculates the camera position and orientation relative to physical markers in real time http://www.hitl.washington.edu/artoolkit/ 11