EVALUATING PRESENCE IN LOW-COST VIRTUAL REALITY DISPLAY SYSTEMS FOR UNDERGRADUATE EDUCATION * Daniel Cliburn, Stacy Rilea, Justin Charette, Ross Bennett, Daniel Fedor-Thurman, Todd Heino, and David Parsons The University of the Pacific Stockton, California 95211 (209) 946-2093 dcliburn@pacific.edu ABSTRACT Virtual reality (VR) is an exciting field that receives little attention in most undergraduate Computer Science programs. Historically, one of the barriers to teaching VR has been the cost of appropriate equipment; however, a number of economically priced systems have been proposed recently for undergraduate training and research projects in VR. In this paper, we discuss the relative advantages and disadvantages of these low-cost stereo display systems and then describe a study designed to assess the subjective ratings of presence that each system provides. Presence, a characteristic of many VR applications, is the feeling of being in a virtual world when you are physically located somewhere else. In our study, the factor that seemed to have the greatest impact on presence was the field of view provided by the system. However, we found that all the systems provided subjective presence levels significantly higher than a standard computer monitor, suggesting that all the systems could form the basis of an undergraduate VR laboratory. 1. INTRODUCTION Virtual reality (VR) is an exciting and engaging field with many contemporary applications. Unfortunately, VR receives little coverage in the curriculum of most academic institutions. An informal survey of the websites of universities worldwide, published in 2004, shows that less than 3% offer courses that teach VR [2]. The omission of VR material from the undergraduate Computer Science curriculum has been due in part * Copyright 2009 by the Consortium for Computing Sciences in Colleges. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the CCSC copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Consortium for Computing Sciences in Colleges. To copy otherwise, or to republish, requires a fee and/or specific permission. 31
JCSC 25, 2 (December 2009) to the traditionally high cost of the equipment necessary for teaching the subject. However, considerable work has been done in recent years to develop VR systems that are more economically priced [1,5,11]. What is not known is the effectiveness of these systems; particularly, which among the low-cost options is best for undergraduate education? In this paper, we describe an experiment designed to compare these low-cost VR systems. The goal of our research is to be able to provide guidelines that can be used to determine the most appropriate stereo imaging system for an undergraduate VR laboratory at practically any institution. 2. VIRTUAL REALITY Virtual Reality is an often misunderstood field that few students can accurately describe. A survey of popular textbooks reveals that virtual reality is defined by applications that are interactive and immersive [3,7]. Interaction is commonplace with most computing applications; users are generally able to manipulate the state of a computer program through input devices to accomplish some goal. The concept of immersion is not as widely understood. Immersion (or more precisely physical immersion ) is defined by Sherman and Craig [7] as synthetic stimulus of the body s senses via the use of technology. In the context of VR, the term presence refers to the degree to which users feel like they are in a virtual world when they are physically located in the real world. Both immersion and interaction are elements that enhance the sense of presence. In order to fully understand VR we feel that it is necessary for students to experience presence to a degree greater than that offered by a typical computing application. Thus, it should be the goal of VR laboratory equipment to support a heightened sense of presence. 3. THE VR LABORATORY SYSTEMS Stereo (or 3D) images have been shown to increase a user s subjective sense of presence as he or she visits a virtual world [6], and as such, the systems we evaluated in this study were all capable of generating stereo displays. 3.1 Head Mounted Displays A head mounted display (HMD) is a device that users wear on their head that provides a different display screen in front of each eye (see Figure 1). The displays correspond to the different views each eye perceives of the real world. Head mounted displays, such as the emagin z800 shown in the picture, have become relatively inexpensive in recent years [1]. However, the field of view is often small (40º with the z800) and only one user at a time can view the stereo images. 32
CCSC: Rocky Mountain Conference FIGURE 1: left - The emagin Z800 Stereo Visor; right A front projected stereo imaging system with a single silver screen. 3.2 Single Screen Front Projection Probably the easiest (and most economical) projection based stereo imaging system to construct is a front projected single screen system [11]. With this technique, stereo images are projected onto a silver screen and viewed with oppositely polarized glasses (see right of Figure 1). However, users cannot walk in front of the projectors or they cast a shadow on the display. Users must be behind (or under) the projectors causing their physical field of view of the display screen to be limited (25º in our study). 3.3 Single Screen Rear Projection With rear projection, users can walk right up to the screen without casting a shadow on the display. This also allows the physical field of view of the display area to be greater (we had the users located such that their field of view was 60º). However, rear projection often requires more physical space and the screen material must preserve the polarization of light. We have had success creating our own rear projection screens by frosting acrylic sheets [5]. The drawback is that custom screens are subject to a noticeable hot spot the glare from the projector on the display screen (see the left of Figure 2). Visual displays that encompass a greater portion of a user s field of view tend to make user s feel more immersed in a virtual world. Thus, a much larger screen that provides users with a 90º field of view (see the left side of Figure 3) may be preferable. However, the drawbacks with a system like this are that it occupies more physical space and the screens cost more. 33
JCSC 25, 2 (December 2009) FIGURE 2: left Image of a rear projection system with a custom made screen; right Image of a rear projection system with a professionally made screen. FIGURE 3: left Rear projection system with a large (8 ft. x 6 ft.) professionally made screen; right - Multiple screens provide users with a greater field of view. 3.4 Multiple Screen Rear Projection Some projection based VR systems present the user with several display screens, providing a much wider field of view than is possible with a single screen (see the right side of Figure 3). In our system [4], users are provided with a 180º view into the virtual world. However, each screen requires two additional projectors (to generate an image for each of the left and right eyes) raising the cost of the system substantially. 4. EXPERIMENTAL DESIGN Test subjects were asked to visit the same virtual world on each display system and indicate their perceived level of presence experienced through each system. Versions of two questionnaires were used to measure the level of presence for each of the systems: the Witmer, Jerome, and Singer (PQ) questionnaire [10] and Slater, Usoh, and Steed (SUS) questionnaire [8,9]. Participants were also asked to rank each of the systems and 34
CCSC: Rocky Mountain Conference explain their rankings. Due to equipment constraints (number of computers and number of projectors), it was not possible to evaluate all of the systems simultaneously. Thus, we ran two within-subject studies. The results of these studies are described next. FIGURE 4: The PQ Scores, SUS Scores, and System Rankings for the first study. 4.1 Five System Evaluation In the first study, five systems were evaluated: the HMD, the single screen front projection system, the single screen rear projection system with a custom made screen, the single screen rear projection system with a professionally made screen, and a standard desktop computer monitor (with no stereo images). Thirty-one subjects completed the experiment. An analysis of variance was used to compare the overall PQ score (the sum of the responses on the PQ questionnaire) across each of the five systems. Results revealed a significant difference across conditions, F(4, 27) = 6.09, p <.001. Post hoc analyses revealed differences between the computer monitor and all other systems, except for the single screen front projection system. Only marginally significant differences in PQ scores were observed for the other systems (see Figure 4). Similarly, an analysis of variance was used to compare the overall SUS scores (the sum of the responses on the SUS questions) across each of the five systems. Results revealed a significant difference across conditions, F(4, 27) = 19.08, p <.001. Post hoc analyses revealed a similar pattern to the scores from the PQ questionnaire. Specifically, differences between the computer monitor and all other systems were observed. Only marginally significant differences in SUS scores were observed for the other systems. Participants were also asked to rank each of the five systems in order with respect to their perceived level of presence in the virtual world (1-best and 5-worst). The Friedman Test was used to evaluate rank order for each of the five systems. Significant differences were observed among the rankings of the five systems, 2 (4, N = 31) = 27.61, p <.001. Descriptive statistics suggest that the single screen rear projection system with a professionally made screen ranked best and the computer monitor ranked worst. 35
JCSC 25, 2 (December 2009) FIGURE 5: The PQ Scores, SUS Scores, and System Rankings for the second study. 4.2 Three System Evaluation In the second study, three systems were evaluated: the single screen rear projection system with a custom made screen, the single screen rear projection system with a large professionally made screen, and the three screen rear projection system. Twenty-three subjects completed the experiment. An analysis of variance was used to compare the overall PQ score across the three systems. Results revealed a significant difference across conditions, F(2, 22) = 5.60, p <.01. Post hoc analyses revealed that the three screen system was better than the rear projection system with a custom made screen. There was no statistically significant difference between the single screen rear projection system with a large professionally made screen and either of the other two systems (see Figure 5). Similarly, an analysis of variance was used to compare the overall SUS scores across the three systems. Results revealed a significant difference across conditions, F(2, 22) = 5.18, p <.05. As with the PQ questionnaire, post hoc analyses revealed that the three screen system was better than the rear projection system with a custom made screen. There was no statistically significant difference between the single screen rear projection system with a large professionally made screen and either of the other two systems. Participants were also asked to rank each of the three systems in order with respect to their perceived level of presence in the virtual world (1-best and 3-worst). The Friedman Test was used to evaluate rank order differences for the systems. Significant differences were observed, 2 (2, N = 23) = 16.64, p <.001, with the three screen system ranked best followed by the single screen rear projection system with a large professionally made screen and then the rear projection system with a custom made screen. Thus, the pattern of participant rankings is consistent with the findings from the PQ and SUS questionnaires. 5. CONCLUSION When considering both studies, the most highly rated display system was the three screen system that provides subjects with a wide field of view. A substantially increased 36
CCSC: Rocky Mountain Conference field of view appears to enhance the sense of presence. However, the cost of this system seems prohibitive (essentially three times the cost of the others when the price of all of the projectors is considered). What should be noted is that all of the systems (with the exception of single screen front projection system) produced presence levels significantly higher than the computer monitor by all measures. What we would recommend to anyone considering the construction of an undergraduate VR laboratory is a single screen rear projection system. Depending on the available time, space, and motivation, faculty members could either make their own screen, or buy one. If space is at a real premium, then a low-cost HMD could also suffice. However, one use of our laboratory is for VR demonstrations to prospective students, and a projection screen makes it possible for several people to view the stereo images simultaneously. It is also fun for students to be able to show their virtual world applications to each other, and again, the projection screen systems are best for this purpose. 6. ACKNOWLEGEMENTS This work was funded by the National Science Foundation s Course, Curriculum, and Laboratory Improvement program under award number 0632924. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation. 7. REFERENCES [1] Adams, J., Holtrop, J., Building an Economical VR System for CS Education. Proceedings of the 13 th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE 2008), June 30-July 2, Madrid, Spain, 148-152, 2008. [2] Burdea, G., Teaching Virtual Reality: Why and How? Presence: Teleoperators & Virtual Environments, 13, (4), 463-483, 2004. [3] Burdea, G., Coiffet, P., Virtual Reality Technology, Hoboken, New Jersey: Wiley- Interscience, 2003. [4] Cliburn, D., A Virtual Reality Laboratory for Undergraduates. The Journal of Computing Sciences in Colleges, 24, (2), 57-63, 2008. [5] Cliburn, D., Stormer, K., The HIVE: Hanover Immersive Virtual Environment, The Journal of Computing Sciences in Colleges, 20, (4), 6-12, 2005. [6] Ijsselsteijn, W., Ridder, H., Freeman, J., Avons, S., Bouwhuis, D., Effects of Steroscopic Presentation, Image Motion, and Screen Size on Subjective and Objective Corroborative Measures of Presence. Presence: Teleoperators and Virtual Environments, 10, (3), 298-311, 2001. [7] Sherman, W., Craig, A., Understanding Virtual Reality: Interface, Application, and Design. San Francisco: Morgan Kaufman, 2003. 37
JCSC 25, 2 (December 2009) [8] Slater, M., Usoh, M., Steed, A., Taking steps: the influence of a walking technique on presence in virtual reality. ACM Transactions on Computer-Human Interaction, 2, (3), 201-219, 1995. [9] Usoh, M., Catena, E., Arman, S., Slater, M., Using presence questionnaires in reality. Presence: Teleoperators and Virtual Environments, 9, (5), 497-503, 2000. [10] Witmer, B., Jerome, C., Singer, M., The factor structure of the presence questionnaire. Presence: Teleoperators and Virtual Environments, 14,(3),298-312, 2005. [11] Zelle, J., Figura, C., Simple, Low-Cost Stereographics: VR for Everyone. Proceedings of the 35 th Technical Symposium on Computer Science Education (SIGCSE 2004), March 3-7, Norfolk, Virginia, 348-352, 2004. 38