Augmented Reality Gaming

Transcription

4 Pervasive Augmented Reality Games Pervasive Games Overcoming the boundaries of conventional games Creating new enhanced gaming environments Making the real environment an intrinsic component of the overall game Bridging the gap between the virtual world of today s computer games and the real world Copyright 2007, Dr. Wolfgang Broll 4

6 Pervasive Augmented Reality Games Pervasive Augmented Reality (AR) Games Special type of pervasive games Enhancing the player s real environment by virtual objects Copyright 2007, Dr. Wolfgang Broll 6

9 Localization Requirements of AR games significantly higher than for other pervasive games Exact position and orientation required Orientation problem solved due to hybrid orientation tracking systems, e.g. Intersense InertiaCube 3 XSens MxT Copyright 2007, Dr. Wolfgang Broll 9

10 Position Tracking GPS is the tool-of-choice Blocking problem ( urban canyons ) GPS needs a line of sight to at least 3 satellites D-GPS, WAAS/EGNOS/MSAS enhance accuracy only, and WAAS/EGNOS/MSAS requires free sight towards south (geo stationary satellites above the equator) Assisted GPS (A-GPS) GPS enhanced by mobile phone cell information? Copyright 2007, Dr. Wolfgang Broll 10

11 Position Tracking Computer vision based approaches Marker based: e.g. ARToolkit, ARToolkit+, ARTag, Problem: markers usually cannot be used in public places Markerless: geometry based or feature based WiFi Problem: a database of 3D geometry and/or features and there exact position is required requires a huge amount of preparatory work Based on signal strength Even encrypted hotspots the user has not access to may be used for tracking! Signal are subject to interferences Limited suitability for AR Copyright 2007, Dr. Wolfgang Broll 11

12 Communication Pervasive AR games are distributed applications Communication between the individual participants and between participants and central components is essential Global communication (via Internet) WiFi (depending on the availability of (public) hot spots) GPRS, EDGE or 3G (up to HSDPA) Soon: WiMax Local communication WiFi (direct peer-to-peer connections without router) Bluetooth RFID Copyright 2007, Dr. Wolfgang Broll 12

13 Communication Connection typically fails in certain areas (dark areas) Games should not try to hide this, but exploit as part of the game design Requires semi-autonomous local game engines Copyright 2007, Dr. Wolfgang Broll 13

14 Communication Interoperability issues Standard data types often restricted on mobile devices Networking protocols supported often limited Consistency issues Regular distribution of the global game status required Fast and easy recovery after re-connection: server push or client pull Game design has to prevent players of having advantages over others while temporarily disconnected E.g. when collecting items Copyright 2007, Dr. Wolfgang Broll 14

15 Augmentation Superimposing the user s real environment by virtual objects Head mounted or head worn AR systems Optical or video see-through Monoscopic or stereoscopic Driven by backpacked laptop computer, micro PC, PDA Copyright 2007, Dr. Wolfgang Broll 15

16 Augmentation Handheld AR systems Video see-through only Tablet PC, UMPC/micro PC, PDA, smartphone Augmentation never correct as observer viewpoint differs from camera viewpoint Easier for groups of players Augmented video streams Projection based AR Copyright 2007, Dr. Wolfgang Broll 16

17 Devices Personal I/O devices Used by certain gamer during the game or for a certain period during the game Mounted to the players (or integrated into his clothes) or carried on by the player Public I/O devices Accessible by groups of players, all players, or even spectators Often restricted to output devices (e.g. public displays) Environmental input devices Weather sensors Web cams observing players / game area Copyright 2007, Dr. Wolfgang Broll 17

18 Game Engine Game engines for pervasive AR games do not generally differ from other game engines Receive game triggers and modify game status according to rules Game engine does need to be aware of the fact that there is a real world However: game engine needs to know all real objects as well State of real game items State and location of real players Input from environmental sensors And: game engine has be able to handle dynamic connections Copyright 2007, Dr. Wolfgang Broll 18

19 Orchestration and Surveillance Orchestration registration of the game area into real world coordinates definition of virtual representatives of real world objects (items, buildings, etc.) initialization and positioning of the game items initialization of the players including setup of the individual equipment and assembling of teams initialization of the game state Orchestration tasks often realized by configuration files or even hard-coded! Limits adaptation of games to other locations Copyright 2007, Dr. Wolfgang Broll 19

20 Orchestration and Surveillance Surveillance add, remove, or re-locate game items, or modify their state add or remove players or player equipment modify individual aspects of the game state Similar tasks as for orchestration but performed during the ongoing game Copyright 2007, Dr. Wolfgang Broll 20

22 Sample Project: NetAttack combined indoor and outdoor scavenger hunt -like game Two competing teams Indoor and outdoor players Tracking GPS + orientation tracker ARToolkit for player/player interaction WiFi communication Head-mounted stereoscopic optical see-through AR system Copyright 2007, Dr. Wolfgang Broll 22

23 Sample Project: Epidemic Menace Crossmedia game within IPerG Different devices (smartphones, PDAs, mobile AR systems, stationary PCs, ) Different media channels Two competing teams Team headquarter Virus hunters GPS + orientation tracking WiFi communication AR system using monocular optical see-through displays + backpacked laptop computers Orchestration interface Copyright 2007, Dr. Wolfgang Broll 23

26 Sample Project: TimeWarp Edutainment adventure game within IPCity project Exploring a city through time and space Head-mounted monocular optical see-through displays (temporarily) + handheld PDA GPRS / 3G communication Enhanced by Bluetooth-based player-to-player communication GPS + orientation tracker To be enhanced by CV-based mechanisms Copyright 2007, Dr. Wolfgang Broll 26

28 Boxed AR Game Boxed pervasive games off the shelf ARBox, Location Box, Ambience Box Easy to set-up Marker based for simplicity and stability Create games yourself Rule-based authoring website Goals Inspire people to make their own games Foster forming of pervasive gaming communities Show that pervasive games can be done without a lot of hassle Sample game The Alchemist Find magic ingredients Stir the cauldron Brew potent potions Copyright 2007, Dr. Wolfgang Broll 28

29 Discussion All existing pervasive AR games use GPS as primary localization mechanism for the position and an autonomous 3- DOF orientation sensor for orientation tracking Computer vision is used to enhance precision where required So far marker based CV is the tool of choice Computer vision cannot easily be used in unconstrained environments Communication mainly done by WiFi Simple tools for physical layer abstraction exist (WiFi/GPRS/3G) Roaming is still an issue Tools for network protocol abstraction (TCP/IP, UDP/IP, multicast, HTTP, CORBA, etc. ) are still missing Copyright 2007, Dr. Wolfgang Broll 29

30 Discussion Most existing prototypes focus on head-mounted optical seethrough displays But handheld systems are getting more important in the near future Games should run on any kind of AR system making use of any device available Game orchestration and surveillance showed to be essential Orchestration tool(s) required to allow for easy of setup games depending on local environment Copyright 2007, Dr. Wolfgang Broll 30

31 Current Developments AR games becoming available on real light-weight devices UMPCs, PDAs, smartphones Allows for selling boxed AR games (as no additional hardware is required) First pervasive AR game authoring tools developed (early prototypes) Flat rates for mobile Internet will increase the use of pervasive games Computer game producers have already identified AR games as future marked, e.g. EA and Sony Copyright 2007, Dr. Wolfgang Broll 31

32 Further Information - für ( e for Applied Information Technology) Department Collaborative Virtual and Augmented Environments fit.fraunhofer.de/cvae Contact: Dr. Wolfgang Broll [email protected] Copyright 2007, Dr. Wolfgang Broll 32