Considerations of Generic Framework for AR on the Web

Transcription

1 Considerations of Generic Framework for AR on the Web A Position Paper for W3C s AR on the Web Workshop Jonghong Jeon, Sunghan Kim, Seungyun Lee Standardization Research Center, ETRI, 161, Kajong-Dong, YuSong-Gu, Daejeon, Repubic of Korea { hoobit, sh-kim, sy@etri.re.kr } Background & Motivation Augmented reaity (AR) is a term for a ive direct or indirect view of a physica rea-word environment whose eements are merged with (or augmented by) computer-generated virtua imagery creating a mixed reaity. In the case of Augmented Reaity, the augmentation is conventionay in rea-time and in meaningfu context with environmenta eements. With the hep of advanced Computer Vision technoogy the information about the surrounding rea word of the user becomes interactive and digitay usabe. Augmented reaity is getting a ot of attention these days - particuary the use of AR with smartphones. The main hardware components for augmented reaity are: dispay, tracking, input devices, and computer. Combination of powerfu CPU, camera, acceerometers, GPS and soid state compass are often present in modern smartphones, which make them prospective patforms for augmented reaity [1]. GPS technoogy combined with mobie camera can do wonders in bringing augmented reaity experience. There are many appications which can interact with the video being streamed through mobie camera to the database present in the Internet about the ocation of the mobie phone. Type of Augmented Reaity Appication With growth of smartphone-based mobie AR, there are interna tensions within the traditiona AR community over whether current mobie AR quaifies as true augmented reaity. Traditiona AR community, using Miitary grade HMD (Head Mounted Dispay), and industria AR appications often understand what they are seeing. They can anayze a stream of visua data to identify that a chassis on an assemby ine is an actua chassis, or that a missie streaking toward a fighter pane is indeed a missie. Despite the ergonomic shortcomings with this, there has been success in certain appications areas, such as industria assemby, surgery training or games. However, the truy radica use of AR based on mobie technoogy that aows Anywhere Augmentation away from the desktop has not yet been reaized [2, 5]. Conversey, the current of mobie AR browsers for smartphones use ocation technoogies in order to dispay information on a video stream. The smartphone doesn t see what is in front of it; it just knows its geographic position. It uses geoocation data to pace digita content, 2-D and 3-D objects aong with inks to other information and services into the user s fied-of-vision.

2 Currenty, there are four main categories of mobie AR appications: navigation, ocation overays, geoinformation services, and gaming [2]. 1) Navigation - Augmented Reaity can be seamessy appied to existing navigation and mapping toos ike Googe Maps and MapQuest. The idea is to make these services increasingy user-friendy by dispaying arrows and basic navigation information on the screen of the mobie device whie the user is en route to the desired destination. 2) Location Overay The basic principe of this type of AR appication is to overay a specific ocation or site with information or images. The user points their phone at a target ocation such as a skyine, buiding, mountain or other geographica feature and suppementa content wi be dispayed on the screen 3) Geo-Information Services The goa of geo-information-focused AR is to take the idea of ocation overays to a new eve. By utiizing user-generated content and web sites ike Wikipedia, these apps wi dispay vast amounts of information to the users, ike digita notes, the best deas in the surrounding area, nearby andmarks or oca business reviews. 4) Gaming Extensive R&D is being invested into games utiizing AR. In their basic form, AR-based games dispay a game over a video stream of the rea word. But instead of being confined to the screen of the mobie device, AR gaming paces the user into the center of the action by expanding the game fied to incude the whoe word. Interoperabiity issue of Augmented Reaity There are sti a number of technica and practica issues to be overcome to create a seamess AR experience for the user. The mobie industry shoud address the core issues of processing speed, memory and aso the battery ife issues created by the camera being in preview mode. Deveopers shoud ensure that their appications have the abiity to compensate for the physica issues; oss of GPS or network and atent inaccuracies. If AR appications are going to be depoyed on a massive scae, there are severa key areas of technoogy that are needed [5]: (1) A ow cost patform that combines AR dispay, tracking and processing (2) Mobiity as key requirement for the patform to reaize AR in a goba space (3) Backend infrastructure for distributing of AR content and appications (4) Easy to use authoring toos for creating AR content (5) Large scae AR tracking soutions which work in rea time Among issues, Interoperabiity issue is one of major chaenges faced by the sma but fast-growing industry of Augmented Reaity. For exampe, peope cannot share the augmented information between different AR browsers and AR service environment. The Lack of interoperabiity in AR systems and services, such as 3D modeing markup, AirTag and POI description method, has been identified as a major obstace to the widespread take-up of AR appications [4, 6].

3 There are tweve interoperabiity issues between AR Provider and AR User Agent, such as figure1. 1. Composite AR Contents Markup & Format 2. AR contents Transport/Interaction method 3. Representing 3D Interactive AR/MR Contents 4. Event Scripting 5. Loca Caching method 6. Additiona functionabiity of AR User Agent 7. Device Capabiity Access APIs 8. AR Data Mashup method 9. AR Data Format POI(images, text, 3D modes, URLs), Person AR Data Service API 11. Open Marker Database 12. Security & Privacy (Figure 1: interoperabiity issues between AR Provider and AR User Agent) Interoperabiity Decreases Costs and Improves Information Access There are a number of benefits of AR interoperabiity, incuding: Improved user access: Getting users the right information they need quicky. With interoperabe AR systems, users do not need to aunch mutipe AR browsers to access AR content throughout the day. They can use a singe AR User Agent on own device and access to interoperabe AR contents, as ike Word Wide Web. Cost Reduction: Reductions in deveopment costs for AR content. AR Standards can be reducing the expensive burden of dupicated content authoring that shoud support to mutipe AR services and environment.

4 What Are the Main Benefits to Using the Web Technoogy for Augmented Reaity? Network-based Augmented Reaity refers to technoogy (mobie devices, in-windshied dispays, etc.) that can overay information from the Web on top of objects in the rea word. It is very simiar with the Word Wide Web. But In the past year, a number of mobie AR appications have been deveoped native appication. The main reason for this native app stye deveopment has been the integration of device capabiities and the abiity to access this functionaity such as the camera. One of the main weaknesses of native AR appications is the inabiity to transfer them across over mutipe devices and mutipe patforms [3]. One possibe soution to soving the fragmentation probem of native appication is to use the components of the genera Web Users Agents (e.g. web browsers) to provide cross-patform capabiities, such as figure 2. In this aways-connected age, most of the usefu additiona information for informative-ar appication you need to dispay is most ike going to reside on the web. (Figure 2: Generic Framework of the AR on the Web) Genera requirements for AR on the Web Before progressing W3C standardization activity, argument reaity services need to consider in severa perspectives, e.g. AR contents visuaization, architecture mode, privacy, context awareness, as we device resource capabiity. Recenty, many kinds of AR mobie services are emerging for mobie users. Unti now, many research and projects are basicay based on sensor technoogies, GPS information, camera and dispay devices. Besides, user s context awareness information is mixing with basic AR mechanism and it makes possibe to support more dynamic and user centric services. In market, there are aready popuar AR services, and some services are gobay providing with mobie phone.

5 For exampes, bionic eye service for searching near coffee shops, car finder service for easy finding parking ocation, red aser service for comparing prices with barcode, etc. So, it is not difficut expecting more diverse services to appear in near days. Therefore, after reviewing the service types, features and functionaities for AR standardization, and then some genera requirements can be suggested for common AR services. Though now can not define the genera requirements in detai, W3C seems necessary to consider further for the under standardization issues [7]. AR content visuaization issue ü Consideration for 2D/3D object visuaization scripts( methods ) ü Live Video Streaming in HTML5 ü Registration between Live(video) object and Virtua object. ü Interaction Mechanism with Virtua object ü Description format of Augmented Information Mark (e.g. AirTag, POI) ü Harmonization with traditiona standards with OpenGL, X3D, COLLADA, JavaScript, PNG, JPEG, etc. Genera architecture mode ü Common framework to support transport/interaction methods ü Protocos, data formats and parameters to query image, marker and ocation information and to get the resuts (Query and resut API) ü Loca caching method Privacy and Context awareness ü Protections mechanism for user s persona private information Device capabiity access ü Access to Camera, Geoocation, Gesture API ü Harmonization with DAP WG, HTML 5 WG, Web appication WG in W3C Data base issue: data structure for Object data and POI ü Key vaue and format for matching each object images/markers to another resources (eg. ISBN, UPC/EAN) ü Data structure for basic and/or detaied information reated to each object ü Key vaue and format for matching each POI to another resources (The at/ong pair is NOT an unique key!) ü Data structure for basic and/or detaied information reated to each POI ü Data structure for the POI ist for a specific oca query with at/ong/viewpoint and/or Street-view image query (eg. KML/ARML) References [1] Augmented Reaity, Wikipedia, [2] GigaOM Report, The Word Is the Desktop: Mobie Augmented Reaity, December 2009 [3] David Rogers, BONDI Augmented Reaity, Mobie AR Summit, MWC 2010 [4] Martin Lechner, Markus Tripp, ARML - An Augmented Reaity Standard, Mobie AR Summit, MWC 2010 [5] Dieter Schmastieg, Tobias Langotz, Augmented Reaity 2.0, Innovation Workshop Proceedings, ISMAR 2009 [6] 1 st AR/MR Standardization Workshop, [7] Junghee Ryu, AR standard requirements, 1 st AR/MR Standardization Workshop,