Augmented Reality Applications for Environmental Designers

Transcription

1 Augmented Reality Applications for Environmental Designers Joe Blalock and Jacob Carringer Department of Landscape Architecture College of Architecture and Planning Ball State University Muncie, IN, Abstract Augmented reality is a developing technology which combines a user s experience of the physical world with virtual reality-based video overlays. The purpose of such a medium is to enhance and/or alter an individual s experience and understanding of their surroundings. This paper describes the basic components of an augmented reality system and focuses on potential applications for augmented reality within the subfields of environmental design and environmental design education. unassociated with scientific research or education, it is hard to deny that the entertainment industry is a major driver of technological progress in the computer industry. Even though AR technology is just beginning to make its way into popular culture, the outlook for an expansion of non-entertainment AR applications is expected to continue to increase. Keywords: augmented reality, aura, Designers Augmented Reality Toolkit, head-mounted display, presence, registration error, wearable computer 1 Introduction Since the early 1990s, augmented reality (AR) has arisen as an offshoot of virtual reality (VR) technology; however, the number of potential future uses for AR may eventually far outnumber those that have been and will be proposed for VR. At this time, the only AR technology that many people are familiar with is the first and ten system used in televised NFL games, where a video overlay projects a yellow line to locate the first down on the field while players walk or run across it as though the line is actually painted on the Astroturf. Likewise, researchers have been working with first-person-perspective PC video games, such as Quake, to develop new versions which allow the player, equipped with a head-mounted display and a wearable computer, to battle virtual enemies in outdoor environments. While these applications seem 2 Augmented Reality Systems Initially, AR systems were designed to complete the following functions: 1) to allow a user to locate objects more rapidly and accurately within an environment, especially one with relatively featureless terrain, 2) to enable invisible features of an object or environment to become visible without the use of physical markers, 3) to overlay several information sources at once, 4) to allow the relationships between different objects to be determined easily, and 5) to view features of an object or environment from taskappropriate positions and orientations (Thomas). However, the number of potential general functions of an AR system will undoubtedly increase over time from this basic list. 1

2 -linuxconf-2003.pdf 2.1 System Components Typically, an AR system will consist of a headmounted display (HMD), a backpack containing a laptop with a powerful CPU, several video cameras precisely angled and mounted at strategic locations on the user GPS, inertial detection devices, and special gloves complete with brightly-colored tracking balls mounted on the thumbs. Combined, these individual items constitute what is referred to as a wearable computer, with which a user is able to use AR applications in an outdoor environment. Originally, HMDs were transparent and looked similar to goggles, utilizing a half-silvered mirror as a combiner and generating an optical overlay of virtual objects in the user s field-ofvision (FOV). Newer HMDs are opaque and contain a camera which records all of the realtime visual information which falls within the user s FOV; this data is then transmitted to the CPU (or processed by the camera itself) where it is combined with virtual data and a higherquality video overlay is generated and transmitted back to the user s HMD (Piekarski, 1). One such system has been developed as part of the Tinmith project, conducted by Drs. Wayne Piekarski and Bruce Thomas at the University of South Australia. The Tinmith Backpack houses an easily-accessible laptop within a polycarbonate frame each successive prototype continues to become smaller and more lightweight, enhancing the portability of the backpack and the mobility of the user. (Piekarski, 3) The specialized gloves are an integral part of the Tinmith-Metro modeling system. Referred to as Tinmith-Hand, the gloves allow the user to select options, such as create object, select, move/rotate, and so on, from a virtual pull-down menu each finger on the gloves is associated with a different category. Using the gloves, one can create objects by combining primitives (pyramid, sphere, union, intersection, and difference) or by constructing intersecting infinite planes (Piekarski, 5). In order to model complex shapes, such as rivers, lakes, or winding roads, the user must walk the perimeter of the object, highlighting vertices and points of interest as markers. Once the user has circled the object, a closed perimeter is formed, creating a polygon which can be extruded up or down (Piekarski, 2). The balls mounted on the thumbs of the gloves are generally colored orange, as this color has the least amount of background interference in most environments and leads least often to tracking failure. Balls are used, rather than another type of marker because their spherical shape is consistent from every angle. At least one of the several necessary cameras within an AR system is devoted to tracking these balls at all times, so that the CPU knows precisely which virtual objects the user is pointing at (Smith). Outside, the wearable computer can usually determine a user s position and orientation fairly accurately through the use of GPS tracking and inertial detectors. Inside, however, ARToolkit fiducial markers, which serve a similar purpose as the balls on the gloves, must be placed on walls, ceilings, and doors, at regular intervals, so that registration error discrepancies between the positioning of virtual objects in the video overlay in relation to the actual position of physical objects can be kept to a minimum. At least two additional tracking cameras, facing different directions at different angles, are mounted to the backpack so that a marker will be in the FOV of at least one of them at all times (Piekarski, 4). 2

3 2.2 Software The Designer s Augmented Reality Toolkit (DART) is the first AR software which takes into consideration the fact that designers have traditionally not been proficient in computer programming languages. In order to allow designers to work directly, intuitively, and effectively with DART, existing programs were utilized, such as Macromedia Director and Shockwave 3D, as components of or plug-ins to the toolkit. user s level of comfort while using a HMD is severely impacted. Due to the dynamic nature of the outdoor environment (inconsistent terrain, changing light conditions, moving objects, etc.) in combination with the significant level of satellite tracking inaccuracy at the scale of a small site, AR tracking systems operate in terms of a level of uncertainty. Generally, as the cameras send data to the laptop, the CPU calculates the average of the recorded positions of physical objects in the user s environment, in combination with incoming GPS data, to arrive at an estimated level of registration error. The CPU then applies algorithms to this information to generate fairly accurate virtual overlays. However, registration error tends to increase proportionately with the user s distance from an object (Malbezin). DART s canvas is called the stage, where all virtual content is manipulated; this content is organized into packets called sprites, or cast members which are placed onto the stage. Commands in DART are grouped into categories such as Actors, Events, Physics, and so on. Additionally, the software allows the user to create geometric shapes to represent objects in the physical world, which can be aligned with and will therefore interact more accurately with the virtual overlay (MacIntyre). The stage exists within a framework called the score, or the timeline of a given AR experience, which is dependent on a feature of the software called the DARTclock. This abstract global clock allows the user to pause, rewind, and slow down data within the score, leading to the development of higher-quality virtual overlays (Dow). 3.2 Aura and Presence Presence, in terms of AR technology, is the mental response of a user to his or her immersion within a virtual experience; the degree of presence of an experience can be quantified in terms of the user s capacity to believe in the experience. Aura, on the other hand, is even more based on the subjective experience of the user in relation to significant historical, cultural, and personal aspects of a place or object; aura is quantified in terms of the user s failure to perceive or acknowledge the role of technology in the overall virtual experience. Although AR graphics are similar to wire-frame CAD models at this time, the development of enhanced graphics and more powerful CPUs in combination with the reduction of registration error, should eventually allow the implementation of seamless virtual overlays (MacIntyre). 3.1 Registration Error Currently, the greatest obstacle to the effective use of AR technology (aside form the price of the hardware and necessary equipment) is registration error. If virtual and physical objects do not line up precisely, video overlays tend to become unintelligible or meaningless and the 4 Applications for AR Technology Because users can either operate AR systems solo (utilizing existing models as video overlays or by creating new models while in the field) or work in conjunction with another user on a remote VR system, there are numerous potential applications for this developing technology. A technician, using a portable AR system in the field could be directed by a remote specialist in a 3

4 laboratory or office thousands of miles away. The specialist would be able to intuitively interact with and guide the technician by directly augmenting the latter s perception of his surroundings objects of interest to the specialist could be immediately highlighted or annotated within the technician s FOV (Piekarski, 5). Some proposed general applications for AR include mechanical repair and maintenance, surgery, product marketing, and search and rescue operations, although this list will undoubtedly continue to expand as AR technology becomes more and more widespread. 4.1 AR and Environmental Design AR technology could be extremely advantageous for the environmental designer, whether as a tool to be used in the process of a student s education or as an integral aspect of professional practice. Site inventory and analysis are critical preliminary steps in the design process. Inventory could be completed in the field in the form of an AR model, complete with any important notes. Hidden features, such as underground power lines and sewer mains would be made visible. Even abstract site features, such as property lines, sun angles, wind direction, sources of pollution, traffic patterns, and so on could be included on different layers within the model. Combining various layers of inventory into virtual overlays would undoubtedly make the analysis process much more intuitive and time-effective. Likewise, specialists need not be present on-site at this phase of the design process. Without leaving the office, they could observe, annotate, and manipulate anything in the surroundings of an inexperienced employee or interning student equipped with a wearable computer and HMD at a remote location. On the other hand, an experienced designer using an AR system in the field could easily collaborate with engineers, contractors, and specialists from other disciplines, in the same fashion. Once the site design process is underway, the designer could meet with the client in the office, and provide a virtual flythrough of the site or fully utilize the AR experience on-site by viewing the proposed design on top of the existing. The client could suggest a wide range of ideas and options and be able to immediately see their respective influences on the developing design. Fully rendered graphics of a wide variety of hardscape materials and plants could be selected from pulldown menus and toolbars, making the system easy enough for even the most computerilliterate client to use. Once the project nears completion, the client or designer could highlight and annotate locations or features that require further attention, giving the contractor a virtual punch list through his HMD as he walks through the site. Additionally, the client could be provided with a completed AR model of the site, containing maintenance schedules and any other pertinent information. While all of the aforementioned functions would prove to be quite useful tools for students in their design studios, AR technology will also open up a wide range of other educational possibilities. For example, in a plant identification class, students could walk designated routes through campus arboretums and gardens at their own pace, using HMDs to receive a wealth of knowledge about each species that they pass along the way. Tests and quizzes could also be taken in this way, as students identify highlighted trees with choices from a pull-down menu, all without the presence of an instructor. 5 Conclusion Augmented Reality, while still in its developmental infancy, will undoubtedly become a widely-used technological tool for a number of functions in the near future. While numerous professions will benefit from the aid of AR systems, environmental designers, specifically, could greatly enhance their abilities and capacities in a variety of dimensions. Likewise, the educational process could be significantly and positively impacted by AR technology. Several potential applications for AR have been described, although the list will exponentially increase as AR technology becomes more widespread. In order for this to take place, however, the registration error in AR systems must be minimized, the generated graphics will require enhanced presence and aura through advanced rendering techniques, the integral software should be simplified considerably to promote collaboration between designers in the developmental process, and 4

5 system hardware and equipment must become more affordable. 6 Sources Dow, Steven et al. Prototyping Applications for the Physical World Using Integrated Capture/Playback Facilities. Ubicomp, Sept. 7-10, 2004, Nottingham, U.K. apers/ubicomp04-demo.html. MacIntyre, Blair et al. DART: A Toolkit for Rapid Design Exploration of Augmented Reality Experiences. Conference on User Interface Software and Technology, Oct , 2004, Santa Fe, New Mexico. apers/dart-uist04.html. Malbezin, Pierre et al. Measuring ARToolkit Accuracy in Long Distance Tracking Experiments. 1 st Int l. Augmented Reality Toolkit Workshop, Darmstadt, Germany, Sep art-2002.pdf. Piekarski (1), Wayne et al. ARQuake- Modifications and Hardware for Outdoor Augmented Reality Gaming. 4 th Australian Linux Conference, Perth, WA, Jan linuxconf-2003.pdf. Piekarski (4), Wayne et al. Integrated Head and Hand Tracking for Indoor and Outdoor Augmented Reality. IEEE Virtual Reality Conference, Chicago, Il, Mar ieee-vr-2004.pdf. Piekarski (5), Wayne et al. Unifying Augmented Reality and Virtual Reality User Interfaces. Technical Report, University of South Australia, Adelaide, SA, Jan tr-vr-2002.pdf. Smith, Ross et al. Hand Tracking for Low Powered Mobile AR User Interfaces. 6 th Australian User Interface Conference, Newcastle, NSW, Jan Thomas, Bruce et al. Using Augmented Reality to Visualize Architecture Designs in an Outdoor Environment. In Design Computing on the Net, Sydney, NSW, Nov Piekarski (2), Wayne et al. Bread Crumbs: A Technique for Modelling Large Outdoor Ground Features. Int l. Symposium on Mixed and Augmented Reality, Darmstadt, Germany, Oct ismar-2002.pdf. Piekarski (3), Wayne et al. Designing Backpacks for High Fidelity Mobile Outdoor Augmented Reality. 3 rd Int l. Symposium on Mixed and Augmented Reality, Arlington, Va, Oct ismar-bp-2004.pdf. 5